A Service of Energy CentralEnergyBlogs.com Logo

On forecaster confidence and accuracy...

As the head of a weather company, StatWeather, I often get the question, "How confident are you in that forecast?"  Techniques for determining the certainty of a forecast coming to pass can range from subjective, touchy-feely guesses to highly analytical, systematized methods.  Here is a simple checklist of 6 items that I use to evalute the degree to which I would trust a particular forecast and to attach a "risk" for a specific forecast coming to pass.  (You don't need to be a meteorologist to apply this list.)

1.  How has this forecaster been verifying lately?  No matter how "confident" a forecaster might be in their forecast, if they're almost always wrong, then that forecast is just plain risky!

2.  Does this forecast keep flip-flopping or are the model runs consistent?  If the forecast keeps changing every 6 hours, then I would wait to see some consistency before trusting that weather forecast.

3.  Do the ensemble outputs show good agreement?  In an ensemble-based forecast, if the standard deviation or the spread of the ensembles is low, then there is a high chance that, even if the current weather conditions are a little different than expected, it's not going to throw off the whole forecast going forward.  

4.  Is this forecast consistent with the forecast for nearby/surrounding locations?  If I gave you a forecast that it will be 100 degrees in New York, but every other city in the Northeast is forecasted to be 65 degrees, then I would be very careful about that New York forecast! There can indeed be micro-climate effects, but it's good to compare an individual city against the overall region just to be sure.

5.  Does this forecast agree with government models?  Assuming that government models have been reasonably trustworthy, then you can check to see if the forecast in question is lining up with other models.

6.  Does this forecast agree with top-performing forecasts (or DISAGREE with poor forecasts)?  Let's say that my competitors have been on a losing streak all winter.  If my forecast DISAGREES with them, then that should be a good sign that my forecast is more likely to be right!

If you can answer these 6 questions whenever a big decision rests on a weather forecast, you will have a very good gauge of the forecast risk and, in turn, know if it is wise to trust a particular weather forecast.

If you would like to learn more about forecast risk, probabilistic forecasting, and long-range forecasts, please e-mail service@statweather.com or go to www.statweather.com.

453 Views Comments 0 Comments Comments Add Comment Author BioAuthor Bio
ReportReport This Post as Foul/Inappropriate

Most energy professionals receive weather information stemming from government models.  The two government numerical weather prediction models most commonly employed in the U.S. energy markets...

The Quest for Speed and Accuracy - Being an energy trading floor meteorologist is one of the toughest jobs I know. I field many questions from people who would like to know exactly what is invo...

October is National Women in Business Month, and I would like to dedicate this article to Women in Energy, addressing the issues women face and how we can encourage women in this field. Energy...

I would like to take this opportunity to promote the following Risk Management and Energy Trading Course with a focus upon evaluating Weather Risk. http://www.marcusevansassets.com/HTMLEmail/VR...

Author:  Ria Persad - CEO, StatWeather   As the president of a weather company, I have compiled this list of questions that prospective customers have asked us to fulfill at some...

Author:  Ria Persad, CEO - StatWeather  (BIO) About 6 years ago, I traded commodities using my own money, because I was sick of losing money by entrusting it to 401(k) plans and broke...

Article by Ria Persad, President of StatWeather (Bio) Over the last year, I conducted a poll of approximately 3,500 weather and climate scientists on the matter of climate change. Just to clari...

Author:  Ria Persad, President of StatWeather  (Bio) When I worked as an Energy Trading Floor meteorologist, there was a period of time every morning when we would evaluate different...

Author:  Ria Persad, President of StatWeather - Bio In 1997, I had heard that there was a major shipwreck on an oil and gas exploration survey because of unanticipated weather conditions,...

Article by Ria Persad, President of StatWeather In 1997, it was challenging to forecast bad weather in the North Sea for oil and gas operations 5 days in advance. We looked at satellite al...

Article by:  Ria Persad, President, StatWeather  Occasionally I hear a long-range or seasonal forecaster say, “We are better than climate normals [climatology] 80% of the time....

Article By:  Ria Persad, President, StatWeather <click for bio> Whether you are forecasting the weather, stocks, elections, or sports, the question of when to go with a con...

My name is Ria Persad, and I am the President of StatWeather.  For the last 13 years, I have been tracking forecaster accuracy--both publicly available weather forecasters and private vendors....

 
Toolbox

Blog Editor
Search
Calendar
Recent EntriesRecent Entries
Recent CommentsRecent Comments
RSS


Sponsored Content

Copyright © 1996-2014 by CyberTech, Inc. All rights reserved.
Energy Central ® is a registered trademark of CyberTech, Incorporated.
CyberTech does not warrant that the information or services of Energy Central will meet any specific requirements; nor will it be error free or uninterrupted; nor shall CyberTech be liable for any indirect, incidental or consequential damages (including lost data, information or profits) sustained or incurred in connection with the use of, operation of, or inability to use Energy Central.
2821 S. Parker Rd. Ste 1105 Aurora, CO 80014
Contact: Phone - 303-782-5510 Fax - 303-782-5331 or service@energycentral.com.