Article by: Ria Persad, President, StatWeather
Occasionally I hear a long-range or seasonal forecaster say, “We are better than climate normals [climatology] 80% of the time.” In other words, they are saying that their forecast is more accurate than going with, say, a 30-year average or a 10-year baseline average some 80% of the time.
The fact of the matter is, this is a mathematical absurdity. Why? If a long-range or seasonal forecast can take on 3 equally probable possibilities (Below Normal, Normal, or Above Normal), then, in general, there is a 1/3 probability of the weather being “normal.” In other words, climatology will, by definition, predict the climate pattern about 33.3% of the time. If a forecaster were to call every weather pattern accurately, he or she would not be able to “beat climatology” or “beat climate normals” when the weather is actually normal, which is about 1/3 of the time. Therefore, the forecaster can only possibly “beat climate normals” the remaining 66.7% of the time when the weather is NOT normal.
If a forecaster calls the weather correctly 80% of the time, then this in general means that they are beating climatology 80% out of the 66.7% of the time that it's available or possible to beat climatology. 80% of 66.7% is 53%.
Thus, a forecaster who is accurate 80% of the time will beat climatology on average of 53% of the time.
What if you have a forecaster who decides to simply pick at random whether the climate will be “above normal”, “normal”, or “below normal”? This “random forecaster” will have a 33.3% probability of having a correct call at any given time, which means that he or she will beat climatology 33.3% out of the 66.7% of the time that it's available or possible to beat climatology. This means that the “random forecaster” will beat climatology about 22% of the time.
Thus, a random forecast will beat climatology on average of 22% of the time.
A common misconception is that if a forecaster beats climatology less than 50% of the time, then he or she is worse than flipping a coin. This would only be true if forecasts were binary (either “above normal” or “below normal”). The fact is that a forecast can also run “normal”, giving 3 possible outcomes. Therefore, if a forecaster beats climatology more than 22% of the time and also is as accurate as climatology more than 11% of the time, then that forecaster has skill! (In other words, the forecaster is worse than climatology less than 2/3 of the time, which is beating the odds.) This is a very profound result from the statistics of having 3 choices.
So, to recap, a “perfect forecaster” will beat climatology 66.7% of the time. A forecaster with 75% accuracy will beat climatology 50% of the time, and a random forecaster will beat climatology on average of 22% of the time.
Thus, if your in-house meteorologist is doing better than climate normals half of the time, that's pretty darn good.
If you would like to view StatWeather's forecasting accuracy for the Winter 2012-2013 to date, please e-mail firstname.lastname@example.org. To learn more about StatWeather, please go to www.StatWeather.com.