Finding the Websites

You can access the web sites for your local TV stations through the main station’s web page (for example, www.foxnews.com). If this does not work, you can attempt to do a search from an Internet search engine such as google or yahoo. If you don’t have access to the Internet, you can record the forecasts directly from the television or newspaper.

Collecting the High Temperature forecasts

On Monday, you should collect the high temperature forecast for Tuesday. On Tuesday, collect the high temperature forecast for Wednesday, and so on (you get the idea). You’ll do this Monday – Friday, collecting forecasts for Tuesday – Saturday.

Newspapers and television stations often present their forecasts in the form of a picture showing high and low temperatures for the next several days:

Some stations may substitute “tomorrow” with the day of the week.

There are different ways in which forecasts can be expressed:

1. For forecast that are numerically expressed, simply copy the number (High: 75)
2. For forecasts which are expressed in numerical ranges (High: 65 – 70), take the average of the range:
   
3. For forecast which are expressed in words (for example, “High in the Mid 70’s”), follow these examples:
• Near 70 = 70
• Lower 70’s = 72
• Low to Mid 70’s = 73
• Mid 70’s = 75
• 70’s = 75
• Mid to Upper 70’s = 77
• Upper 70’s = 78
• Near 80 = 80

Recording Your Forecasts

Organizing the forecasts you collect can be a bit tricky. It helps if you use a specially-designed worksheet. The Government has provided you with a worksheet to use while recording your forecasts.

The picture below shows a part of the worksheet. If you have Microsoft Excel spreadsheet software on your computer, click here to get the entire worksheet. If you don’t have Microsoft Excel installed on your computer, click here to display the first half of the worksheet, and click here to display the second half of the worksheet.

Verifying the High Temperature Forecasts

One easy way to verify high temperature forecasts is to use the Weather Underground website, www.wunderground.com. Go to this web site and type in the city name at the top of the page. For cities that appear in more than one state (like Miami) you may need to select the appropriate city from a list. Select your city by clicking on its name. You will be taken to a web site containing current weather conditions and forecasts for your city. At the bottom of the table containing current weather conditions there is a section called “Historical Conditions.” Use the drop-down menus to enter the proper dates and click on “Go.” The data that you want is “Max Temperature.” Do this for each day for which you collected high temperature forecasts (Tuesday – Saturday of the first two weeks).

Finding the Average Relative Forecast Error (ARE)

The first step in finding the average relative forecast error for a particular set of forecasts is to find the relative error (RE) for each forecast:

For example, if FOX forecasted a high of 65 F, and actual high was 67 F, the relative error would be -2 F. If the actual high were 63 F, the relative error would be 2 F.

The average relative forecast error (ARE) is the average of all the relative errors corresponding to a particular set of forecasts:

The formula for ARE simply divides the sum of all of the relative forecast errors by the number of forecasts made. For example, suppose a forecaster made five forecasts (n=5), and the relative errors were 3 F, 2 F, 5 F, -4 F and 0 F. Using the formula above, the average relative forecast error is

Note that the relative error, and also the average relative error, can be either positive or negative. (It could also be zero.) The implications of these possibilities are:

+ Average Relative Forecast Error:

Bias toward overprediction, indicating that the forecasts are generally too warm

– Average Relative Forecast Error:

Bias toward underprediction, indicating that the forecasts are generally too cold

So what does this tell us?

In the example given here, an average relative forecast error of 1.2 degrees F indicates that on the average, the forecasts are too high by a little more than 1 degree F.

Finding the Average Absolute Forecast Error (AAE)

The first step in finding the average absolute forecast error for a particular set of forecasts is to find the absolute error (AE) for each forecast:

You can see from the formula that the absolute error is just the absolute value of the relative error. Using the same example as before, if the forecast was for a high temperature of 65 F and the actual high temperature was 67 F, the absolute error would be 2 F (even though the relative error is -2 F). If the actual high were 63 F, the absolute error would still be 2 F. In this latter case, the absolute error would be exactly the same as the relative error. (NOTE: The absolute error can never be negative).

The average absolute forecast error (AAE) is the average of all the absolute errors corresponding to a particular set of forecasts:

The formula for AAE, just like the earlier formula for ARE, simply divides the sum of all of the absolute forecast errors by the number of forecasts made. Using our earlier example, suppose a forecaster made five forecasts (n=5), and the relative errors were 3 F, 2 F, 5 F, -4 F and 0 F. The absolute errors corresponding to these forecasts would be 3 F, 2 F, 5 F, 4 F and 0 F. Using the formula above, the average absolute forecast error is

So what does this tell us?

In the example given here, an average absolute forecast error of 2.8 degrees F indicates that on the average, the forecasted high temperature differs from the actual high temperature by 2.8 degrees F.

Comparing the ARE and AAE

Note that the average absolute error is more than twice as large as the average relative error. For this example, we can conclude that the forecaster makes an average error of 2.8 degrees F in his/her high temperature forecasts, and that the forecasts are usually 1.2 degrees F too high.

Preparing Individual and Group Reports

Examine your results, both individually and as a group, and figure out what is implied by the different data.

Example:

Average Absolute Forecast Error for FOX = 3.75 F and for NBC is 2.75 F, thus NBC was the better absolute forecaster for that two week period.

Example:

Average Relative Forecast Error for FOX = .68 F, and for NBC -.25 F, thus FOX has a bias toward forecasting temperatures that are too warm, and NBC has a bias for forecasting temperatures that are too cold. Also, NBC was the better relative forecaster since its average relative forecast error was closer to zero.

It is often useful to use graphs to display your results. The following graph shows some forecasting results for Milwaukee, Wisconsin forecasters:

This sample graph shows that the National Weather Service is the better forecaster for Milwaukee high temperatures, because their average absolute errors are at least 0.5 degrees F smaller than those of the other forecasters. This graph does not, however, show the bias in the individual forecasters. Results for average relative forecast errors are needed to provide information on bias.