Data Catalog 2.0: What's New in Yes Energy's Electricity Market Data Catalog

Among our core values is “we are never satisfied with the status quo,” and so we’ve been working intently to bring you our new and improved data catalog 2.0. 

Let’s explore what we’ve updated in our electricity market data catalog, including adding new Independent System Operator (ISO) data, new partner data, and new proprietary data. We’ll also show you what we’re continuing to enhance, including data observability. 

New Partner: S&P Global Commodity Insights

As of September 2023, we are channel partners with S&P Global Commodity Insights (SPGCI)! Subscribers to the America's Gas and Power Commodity Service Model data can access the data throughout Yes Energy's products (PowerSignals, QuickSignals, and DataSignals). Having day-ahead gas prices from SPGCI alongside other power market data in Yes Energy allows customers to see the correlation between volatility in gas markets and corresponding price movements in the power markets. 

Source: Yes Energy, PowerSignals product. This is one of the many places you can find S&P Global data in Yes Energy’s products. 

Additions to the Data Catalog for ERCOT

It’s been an exciting year in the Electric Reliability Council of Texas (ERCOT) market, partly because ERCOT introduced many new datasets that provide increased visibility into market dynamics. 

ERCOT Energy Storage (ESR) Data

ERCOT released the Energy Storage Resources (ESR) dashboard in December 2023. This dashboard provides visibility into the state of batteries, including charging and discharging.

We report this data at the ISO level and update it every five minutes. This is valuable because ERCOT is one of a few ISOs to provide visibility into how ESRs play a role in meeting or contributing to demand during highly volatile times. You can now access this data throughout all our products, including QuickSignals, which is showcased below. 

Source: Yes Energy, QuickSignals product. This shows sudden fluctuations in battery operations in ERCOT in a low-latency, real-time dashboard.

ERCOT Generation Outages Data

Also in December 2023, ERCOT released the Generation Outages dashboard, which shows the total megawatts (MWs) of planned and forced generation outages for renewable and dispatchable generation facilities at the ISO level. 

This data is valuable because it’s updated every five minutes, whereas other real-time generation outage data from ERCOT is updated every hour. This allows you to more quickly respond to changes in the supply stack. 

We incorporated this dataset into all of our core products, including Time Series Analysis in PowerSignals, shown below. 

Source: PowerSignals shows a volatile day in ERCOT where the real-time locational marginal price (LMP) of Hub North spiked to $400, and we could tell from this new dataset that four gigawatts (GWs) of renewable capacity were forced offline, leading to a depression in prices. 

ERCOT Contingency Reserve Service Data (ECRS)

The ECRS is a new ancillary service product that ERCOT released in June 2023 to provide frequency regulation services on the grid. (This was one of many outcomes of Winter Storm Uri to increase the grid's reliability.)

We quickly responded to the introduction of the new ancillary service product and modified all our collectors so we would immediately gather this data when available. 

So far we’ve seen that ERCOT has been calling on this service more than expected, and it’s been more lucrative than expected for eligible generators that can ramp up in 10 minutes and run for at least two hours.

Source: Yes Energy, PowerSignals product. This shows a day in ERCOT in August where we saw net load spike in the middle of the day, meaning solar and/or wind dropped off. 

Hydro Data

Moving further west, we heard from customers that they wanted more transparency into the role that hydro plays, for example in CAISO, where hydro is 7% of the supply stack. 

Historically, we just had hydro data for the Columbia River Basin, but we expanded the coverage to include 54 more hydro facilities across CAISO, MISO, SPP, and ERCOT. In total, this added around 30 GW of additional generating capacity to our products. 

This data comes from the US Army Corps of Engineers, it’s typically delayed up to seven days, and it’s now in all our core products. However, we took this a step further for those who subscribe to Live Power and nowcasted the generation data because of our understanding of the relationship between flow rate and generation. In this context, nowcasted means we brought delayed data to present (real-time).

Source: Yes Energy, Live Power product. 

Geo Data in DataSignals

Besides hydro data, customers asked for the underlying latitude and longitude associated with power plants, price nodes, transmission lines, and weather stations.

Geo Data allows you to create your own data visualization, site new assets, supplement internal geographic information systems (GIS), and use this as an input for artificial intelligence/machine learning forecasting. This add-on dataset is currently available through DataSignals Cloud and DataSignals Lake.

Source: Yes Energy, PowerSignals product

New Builds Dataset in DataSignals

Finally, our New Builds Dataset became possible through our acquisition of the C Three Group. This takes the ISO interconnection queue data and supercharges it with primary research from our team of analysts. The product includes visibility into new generation, transmission, and load centers. 

It’s valuable for siting new assets, for understanding the changing topology of the grid and how that affects FTR positions, and for long-term forecasting. 

Currently, this data is available through a new module, shown below, as well as DataSignals API and DataSignals Cloud.

Source: Yes Energy, New Builds product

Continuing Power Market Data Observability

We understand how critical our power market data is to your day-to-day responsibilities and believe you deserve control-room-level oversight of the data you count on. That’s why we have a dedicated team of data quality analysts who constantly track data completeness, freshness, and correctness across all our critical datasets. It’s all part of our commitment to bring you better data, better delivery, and better direction. 

Monitoring Data Freshness

A pillar of data observability is data freshness. We have many internal monitors set up to check for stale data 24/7. We recently implemented a new AI software, which is constantly combing through our collectors to assess anomalies in the data, including outliers and missing data. We also have internal dashboards that we monitor daily for the status of all our real-time price data collections. Currently, these dashboards and alerts are internal-facing only, but we plan to expose visibility into data freshness more in the coming months. 

Monitoring Data Completeness

Another pillar of data observability is data completeness. One new way we’ve provided visibility into this is through a new table in DataSignals Cloud that shows missing data intervals by data type, location, and timestamp. This provides greater transparency into when we are missing data from the ISOs for critical datasets like real-time or day-ahead locational marginal prices. 

This lets you write more performance queries of day-ahead, real-time prices so that you don’t need to query the entire table for missing data. You can first check this new table for which datatypes are missing data over which timeframe and when it is filled so you can requery the base tables. More performance queries will result in lower compute costs.

Conclusion

We are committed to bringing you new ways to access and analyze the data you need for better decisions. If you are a current Yes Energy customer and want to dive deeper into these new datasets, check out this help article. 

If you are not a customer and want power market data to help you Win the Day AheadTM, request a demo.