Demand Forecast Evaluation

Overview

When evaluating a potential new forecast, there are several common types of analysis that may reveal any shortcomings in forecast data for individual items.

You should use a combination of numerical and graphical analyses to help identify the following common types of shortcomings in demand forecasts:

• forecast much higher/lower than corresponding recent actual demand data, without proper justification
• inconsistent trends in forecasts from one year to another
• unexplained peaks/troughs in forecast data for individual periods
• missing forecast for item(s) that require a forecast
• forecasts created for items that have been discontinued
• monthly phasing of product does not reflect underlying seasonality in demand for an item
• large, unexplained changes in new forecast data compared with corresponding data in previous forecast
• failure to adapt forecast data to reflect recent changes in actual demand trends for an item
• large, unexplained changes in split of demand data across SKUs belonging to a product group

Key Analyses

• Simple reports on monthly forecast data
• Analysis of annual growth rates
• Graphical analysis of monthly actual and forecast data
• MAT analysis of actual and forecast data
• Variances between new forecast and previous forecasts
• Comparison forecast data for next 12 months with actual data for last 12 months
• Comparison of forecast data with statistical projections 
• Comparison of monthly phasing of forecast data with that of historical actual data

Simple Reports on Monthly Forecast Data

This style of report is available via Analysis vs Time for any combination of actual and forecast files. These reports can often reveal some obvious differences in trends and patterns between new and previous forecasts, e.g. the second item below shows a dramatic change in the level of demand in the new forecast from May 2013 onwards.

Analysis of Annual Growth Rates

File Summary Analysis allows you to view annual variances and % variances between successive years and/or different forecasts. The example below uses the variance option Totals and Var, % Var vs Previous File.

It is useful to create such reports without sub-totals and to sort according to values in the final % Var. column. In sorted form you may concentrate on the top and bottom rows, which will contain the largest % variances.

Graphical Analysis of Monthly Actual/Forecast Data

This style of graph is available via Graphs vs Time for any combination of actual and forecast files. These graphs can often reveal some obvious differences in trends and patterns between new and previous forecasts, e.g. the item below shows a fairly obvious 'outlier' in May 2013. This value may in fact be perfectly reasonable if there are are unusual circumstances for that period. However, it is certainly worth checking.

Moving Annual Totals Analysis of Actual/Forecast Data

Important changes in trend in forecast data can often be difficult to detect if the data shows strong seasonality and/or large amounts of random variation. The use of Moving Annual Totals (MAT) graphs can often reveal such 'hidden' changes in trend very clearly. For example, the bulge in the MAT graph below should raise strong concerns about trend and/or seasonality in the forecast concerned.

Comparison of Forecast Data for 12 month Periods

For well established products, total actual demand for the last 12 months will be a good indicator of total forecast demand for the next 12 months.

The analysis below is obtained from Crosstabulation Analysis using a definition that computes the above 12 month totals (see Comparison of 12 Month Totals).

The report has been sorted according to values in the % Change column.

The first item here shows a dramatic (80%) increase in forecast demand compared with actual demand for the last 12 months.

The last item here shows a zero forecast for the next 12 months, even though actual demand for the last 12 months was at a high level. This may indicate that the item was overlooked when the latest set of forecasts was created.

Comparison of Forecast Data with Statistical Projections

IFP allows statistical forecasts of demand based on selected trend and seasonality models to be saved in any set of files. Hence, it is a simple matter to create reports and graphs similar to those shown above that compare your final forecasts with the values created from such statistical forecasting models.

Clearly, any significant deviations of forecast demand from these statistical forecasts should be challenged. If users have entered relevant comments available in forecast editors then many of these deviations may be easy to understand.

Monthly Phasing Comparisons

A common error in demand forecasting is to fail to phase the forecasts according to underlying seasonality. File Comparison Graphs will often show such phasing errors very clearly.

In the example below the forecasts for each year have been set at the same value for each month and do not represent the strong underlying seasonal pattern in actual data.

SKU Contribution Analysis

In normal circumstances the percentage contribution of each SKU to the total for the brand remains fairly stable.

SKU percentage contributions are best obtained using Crosstabulation Analysis with the relevant column definition (see SKU Contribution Analysis-3 Files).

The analysis below shows a significant change in SKU contributions in the latest forecast for the forecast period May - Dec 2013. This would clearly merit further investigation in practice.