Ever wonder how accurate the EIA’s oil production numbers are?
The EIA is the Energy Information Administration, a federal government division that produces the most transparent and well researched data on oil production and storage levels—of any country in the world.
Their Weekly Petroleum Supply Report (WPSR) comes out every Wednesday at 10:30 AM Eastern time, and is the most closely watched oil report in the world. This report moves markets. With derivatives and physical trading…its likely impact is in the trillions of dollars per week.
So yeah, it’s good to know just how accurate they really are.
Well, I sent my research team to grind through the data, and they came up with two very intriguing…ok just one conclusion was intriguing.
The boring conclusion was simple— the EIA has VERY accurate data, you just have to know where to look.
Investors should understand the weekly oil production number is a fully modeled estimate, which is a fancy way of saying it actually contains no new data from the previous week. That probably surprises a few of you.
However, the monthly data has real data, and as I will show you, is incredibly accurate. Folks, you can take those monthly numbers to the bank almost every month.
I did my best to keep this simple, but my young and under-paid researcher on this story did such a good job digging up data, I’m not sure I always succeeded.
The intriguing conclusion was really the reason I asked my team to look into this story.
And that second conclusion is…the EIA has never over-estimated weekly crude oil production as much as they are right now. Look at this chart:
Figure 1: Six-Month Rolling Average of Crude Oil Production Revisions
Bars above the line means the EIA is UNDER-estimating weekly oil production; the bars below the line indicate OVER-estimating weekly production. This chart clearly shows the recent over-estimation.
Two high profile industry CEOs—Harold Hamm of Continental Resources (CLR-NYSE) and the legendary Mark Papa of Centennial Development (CDEV-NYSE)—have also been saying this.
Every month-end, the EIA releases their Petroleum Supply Monthly (PSM) report describing oil production, imports and exports, movements and inventories. Note that this report has a two month lag, has more direct data, and is therefore a lot more accurate.
After looking at previous reports, we found the EIA has revised crude oil production downwards (overestimated) eight of the last nine months in 2017. This goes against the recent trend—because actually, the EIA has been consistently under-estimating US oil production since 2012.
In fact, the most recent PSM—which reported September data—revised production upwards 131,000 bopd, but the August data reported the largest downward revision in the last decade. As you can see from the chart above—we are still in the largest period of downward revisions since 2008.
So in one sense, Hamm & Papa are correct. And it was that data-point that made me dive down this rabbit hole. To determine the accuracy of the EIA’s estimates, my team calculated the difference between the Weekly Petroleum Supply Report (WPSR) and the monthly PSM figures to find revisions.
They made an average of weekly numbers for each month—to better compare apples to apples—and then the WPSR data was subtracted from PSM to find the revisions. (Revision = Weekly – Monthly)..
Before 2012, the revisions were close to equal—up and down. But since January 2012 the revisions have been overwhelmingly upward—as weekly estimates continually underestimated crude oil production.
The only other time we saw the revisions with weekly data overestimating production was from October 2013 to February 2014.
How Accurate Is The EIA Data?
The short answer is: Very Accurate
The average historical revisions show The EIA has underestimated production by 102k b/d and 66k b/d per month since 2012 and 2008, respectively. Average revisions are seen in the third column of Figure 2.
Figure 2: EIA Accuracy Over Time
Source: EIA, Author Calculations
Figure 2 shows average revisions and average absolute revisions from 2008. In 2011 and 2013 the average revisions were low enough to conclude the EIA was really quite accurate in the weekly data–the WPSR.
But, the absolute revisions were much larger since the EIA simply overestimated and underestimated by similar amounts. These years show why absolute revisions are a better metric to evaluate accuracy than just average revisions.
The absolute revision from 2008 to present is a paltry 129,000 bopd. We are talking about millions of barrels per day over years of analysis…that 129,000 bopd is basically nothing. This small revision proves the EIA’s weekly predictions are pretty darned good.
This absolute revision is less than 2% of oil production, so it’s small, but obviously there are fluctuations of higher and lower revisions. When looking at an almost 10-year average, the true volatility may be missed.
And then we have times like the last three months, where we just had the largest downward revision on record. The weekly estimates can miss.
To assess the accuracy of the monthly PSM data, we compared it to the yearly Petroleum Supply Annual (PSA) which is the final EIA report and will be assumed to be official data. The absolute revisions to the PSM have been less than 0.5% since 2008. In other words, the PSM is very accurate.
BACKGROUND–METHODOLOGIES
After explaining all this in the most simple language we could muster, it’s worth discussing how the numbers are originally generated by the EIA. The WPSR is released each Wednesday presenting the data from the prior week (five-day lag time) providing the source for US Oil supply and storage headlines. Seven supply surveys administered by the EIA at key points along the petroleum production and supply chain collect data from a sample size of over 90% of volume by region.
Since the EIA does not survey for crude oil production data each week, the EIA uses Short-term Energy Outlook (STEO) model estimates to help describe the full crude oil puzzle. Weekly crude oil production numbers are fully modeled estimates utilizing lagged data.
At the end of each month, the Petroleum Supply Monthly is released for the reporting period two-months prior. i.e. The September production report will be released at the end of November. The EIA calculates production in the PSM by distributing the EIA-914 survey to well operators who cover 85% of oil and gas production for each state/area.
This typically equates to a coverage more than 85% of oil and gas production tending to gravitate towards 90%. The PSM provides a more accurate depiction of the current crude oil production picture, albeit two months lagged.
The EIA claims a 98-100% response rate on the monthly surveys distributed to operators. The report “Accuracy of Petroleum Supply Data” is released to describe this response rate. However, no current or historical versions have been found for reference.
Final statistics from the PSM data series are published in the Petroleum Supply Annual, which is released in August/September each year. Monthly supply data are reviewed throughout the year and some estimates may be replaced with newly available or resubmitted respondent data in the PSA. The 2016 PSA was released in September 2017. The PSA revises the data from the year prior to the most recent release which effectively means official data requires a 20-month lag time. The revised PSM data is the most accurate historical information possible. PSA data prior to January 2016 is assumed to be official.
MONTHLY CRUDE OIL PRODUCTION ACCURACY
Where can the PSM miscalculate crude oil production? The PSM is still a “best estimate” due to state reporting agency time lags for data which could take up to two years to be reported. To account for the production not surveyed and submitted, the EIA uses models based on an average of the most recently reported month with full data and the 5 months prior, or a 6-month trailing average. Figure 3 shows the minimum state lag for data recovery.
Oil and gas operators producing less than 500 barrels per day aren’t surveyed, even if the operator is more than 85% of their respective state. The 914 Methodology states, “Very dynamic events can cause the [EIA-914 model] to behave abnormally and, particularly when combined with long time lags in the state data from DI, the model may deviate from reality. For example, the rapid development of the Haynesville shale in Louisiana caused a change in the State production trend that, in turn, caused the method to overestimate for a short time late in the 2000s.”
In other words, rapid development of a shale play may lead the EIA to overestimate production as the play returns to a more sustainable production growth rate.
The PSM has proven accurate over the long-run in comparison to the PSA. The absolute revisions have been 19k b/d per month since 2012 and 21k b/d since 2008. These figures are much lower and less volatile than the PSM revisions of WPSR estimates, and are included to show the PSM data isn’t perfect.
If history proves to be accurate going forward, the PSM is a very accurate indicator of US crude oil production.
Figure 3: EIA-914 State Lag Data
Source: EIA-914 Methodology
CONCLUSION
- The average revisions seen in this report are smoothed to the point where true volatility (over a really short period of time) may be overlooked.
- Weekly oil production numbers need to be understood as a modeled estimate with absolutely no current weekly hard (actual) data.
- The first production data received by the EIA is released in the PSM reported with a two-month lag.
Having said that, the PSM has proven to be a very accurate predictor for the actual production released in the PSA.
What does this mean for the current state of US oil production? I’m not sure you can reach any Big Conclusion out of this data to answer that.
The data does show a historical skewness towards underestimating crude oil production. However, the WPSR estimates have overestimated production in seven of the eight months reported in 2017 and year-to-date 2017 data has shown the highest average overestimation in production since 2008.
The most recent (September) crude oil overestimation of 293k b/d was the largest single month downward revision since 2008. US crude oil production has been underperforming the EIA’s forecasts, but is still growing.
It is very unusual to have the EIA overestimating production this much—because overall, the EIA is very accurate.
EDITORS NOTE: That was a complex story wasn’t it? Here’s something simple–a great oil stock with the highest Free Cash Flow percentage I see, increasing production, paying down debt and increasing shareholder value. Now that OPEC has given oil stocks the green light, it’s time to look at My #1 Oil Stock – Click HERE to find out the name and symbol–RISK FREE
Keith Schaefer
