We often receive questions from storage developers, operators, and energy traders around the importance and efficacy of energy price forecasts. In particular, whether Tyba’s AI-based forecasts1 outperform a persistence forecast approach2. With this blog, we dive into how Tyba thinks about forecasting performance and walk through examples of our forecasting performance in CAISO and ERCOT benchmarked against a persistence approach.
Disclaimer: Forecasts are only a portion of an overall modeling stack needed to effectively bid energy storage assets into ISO markets. Additional model components include optimization models and bidding models. This blog isolates the forecast variable to understand how different forecast approaches impact outcomes.
When evaluating the efficacy of a forecasting model/approach, we evaluate performance across a range of metrics including:
Mean Absolute Error (MAE) calculates the average of the absolute differences between the actual values (or prices) and the predicted values (or forecasted prices). It provides a measure of the model's predictive accuracy. A smaller MAE indicates a more accurate model, as it means that the predicted values are, on average, closer to the actual values.
Spearman correlation is a measure of the strength and direction of association between the ranks of two time series variables. In the case of price forecasting, we compare the forecasted vs. actual prices series for a given day.
For example, for the Day-Ahead Energy product, we would rank each operating hour in a given day from highest price to lowest price for the Tyba forecast and the actual prices. If the Tyba order perfectly matched the actual order, that would translate to a Spearman correlation of 1 for that day. If the Tyba order and actual order were perfectly inverse, the Spearman correlation would be -1 for the day.
Across a long evaluation horizon, the daily average Spearman correlation is a helpful indicator of the shape accuracy of your forecasts. For storage, having an accurate view of the shape of prices for the day helps you to set an operating/bidding strategy to sell (discharge) when prices are highest and buy (charge) when prices are lowest.
Ultimately, storage operators care about how price forecasts translate to revenue outcomes. We view this as the ultimate “task” that the forecasts + downstream optimization & bidding models are trying to achieve.
Evaluating revenue capture requires using optimization and bidding models, so we hold assumptions for those models constant to isolate for the effects of price forecast quality. Likewise, to benchmark performance, we compare the forecast approaches (Tyba and persistence) vs. a perfect case which assumes full knowledge of actual prices when setting your operating plan. While the perfect case is unrealistic, it sets the high water mark from a revenue outcome perspective.
We compared Tyba’s forecasts vs. a persistence-based forecasting approach for the Day-Ahead (DA) and Real-Time (RT) energy prices at ERCOT’s North Hub and CAISO’s SP15 Hub.
For DA Energy, Tyba generates a forecast for each hour of the next operating day prior to the close of the DA Market (so typically between 6-9am).
For RT Energy, Tyba generates an initial forecast for each hour of the next operating day prior to the close of the DA Market. We then re-forecast RT energy prices once DA prices have been posted and on a rolling basis prior to close of each operating hour.
For the MAE and Spearman correlation metrics, the case case study compares the following:
For the revenue capture analysis, we look at a bidding strategy that co-optimizes across the DA and RT Energy markets (or DART approach), with the following constraints:
While this analysis focused on applying forecasts to energy storage bidding, this can be extended to other applications including:
Working with Tyba
Tyba’s energy and ancillary services price forecasts are available through Tyba’s price forecast API service (technical document here). If interested in leveraging Tyba’s price forecasts, reach out to us over email at email@example.com or through our website.
Getting started with Tyba’s price forecast API is as simple as:
© 2022 Tyba Energy Inc. San Francisco, CA