Thursday, November 17, 2011

Why people don't know how good forecasts are

While I'm away, I thought I would share this quote from Hockey goalie Jacques Plante:

How would you like a job where, every time you make a mistake, a big red light goes on and 18,000 people boo? 

Forecast evaluation is going back and comparing one's forecasts to what occurred. It is not necessary, but is generally considered part of a healthy operational forecasting system. Some agencies don't evaluate their forecasts for a variety of reasons, perhaps because nobody knows what actually occurred (e.g. no one collected data) or no one kept a copy of the forecast in an organized way.

You would be surprised at how often the latter happens, especially for older forecasts. At my old agency, entire seasons of forecasts from about 18 years ago were wiped out when some computer disks were erased to make more space. The total size of those files was probably smaller than the last digital photo you took of your pet.

The best resource available on the web is easily Beth Ebert's Forecast Verification FAQ

No comments:

Post a Comment