Improving forecasting techniques

In very large projects in particular, pervasive misinformation about costs, benefits and risks can be a serious problem. They can lead to cost overruns, lower than expected benefits and waste. More serious still, this information gap may be partially explained by planners and promoters misrepresenting costs, benefits, and risks in order to increase the chances of projects going ahead34.

Better forecasting techniques based on a proper evidence base can reduce inaccuracy and bias in project appraisal. One innovative means to improve forecasting is to oblige planners to base their estimates on a reference class for similar projects, as opposed to taking a narrow project specific view (see Box 5 below).

Box 5 - "Reference class forecasting"

Reference class forecasting is a type of "evidence based forecasting" developed to compensate for cognitive bias in economic forecasting. It has its roots in the work of Kahneman and Tversky, two Nobel prize winning economists, on cognitive bias in decision making. Reference class forecasting consists of taking an outside view on a particular project to be forecast. The outside view is established based on information from a class of similar projects. It does not involve trying to forecast the specific uncertain events that will affect a specific project but instead involves placing the project in a statistical distribution of outcomes from this class of reference projects.

Reference class forecasting requires the following three steps for an individual project:

•  Identifying a relevant reference class of past projects. The class must be broad enough to be statistically meaningful but narrow enough to be truly comparable with the specific project;

•  Establishing a probability distribution of outcomes for the particular reference class. This necessitates having access to credible, empirical data for an adequate number of projects within the reference class to make statistically meaningful conclusions;

•  Comparing  the specific project with the reference-class distribution to establish the most likely outcome for the specific project. 

It is officially endorsed by the American Planning Association and has shown itself to be more accurate than conventional forecasting.




_________________________________________________________________________________

34 Bent Flyvbjerg, Policy and planning for large-infrastructure projects: problems, causes, cures. Environment and Planning B: Planning and Design 2007, volume 34, pp. 578- 597.