4  Calculation of Optimism Bias

This section describes how to calculate the optimism bias for the estimated project costs and time. The calculated optimism bias will be used to replace the risk element in the 6% discount rate, formerly recommended by HM Treasury (see HM Treasury's 2002 edition of its Green Book for guidance). When calculating optimism bias experienced appraisers should apply a degree of best judgement.

When carrying out project appraisals, full allowance should be given for any suspected optimism in the costs and time figures originally proposed, giving regard to the outcomes of previous projects of a similar nature. By accounting for optimism more explicitly, project options can be compared more accurately with regard to costs and time. Table 4 provides indicative figures for optimism bias. It has been prepared by taking into consideration the results of the Mott MacDonald study and reductions in optimism bias levels observed over recent years to provide an upper bound (U) for optimism bias. The lower bound (L) in Table 4 allows for improvements in practice that were evident over the review period and new procurement practices known to have been implemented in the last five years.

Table 4  Current Practice Optimism Bias

* The optimism bias for outsourcing projects is measured for operating expenditure, OPEX

The upper bound values recommended for use when calculating optimism bias represent the optimism bias level to expect for current projects without effective risk management and bad scope definition, and are the starting point for calculating optimism bias for projects. These upper bound values reflect the average historic values because the average historic values are similar to the highest values for optimism bias currently being recorded for recently completed projects that have experienced high levels of optimism in their project estimates.8

The lower bound values identified represent the optimism bias level to aim for in current projects with effective risk management by the time of contract award. Ideally by the time of contract award sufficient project risks should have been identified and effective risk management strategies developed to obtain the lower bound values for optimism bias during project appraisal. By identifying the project risks within each of the project risk areas for a project and adopting appropriate risk management strategies it is possible to gain a high level of confidence in the estimates for capital expenditure and works duration.

With the exception of outsourcing projects, the information gathered on operating expenditure and benefits shortfall was based on best judgement and was available only on a small number of projects. In addition, the information obtained on project duration was inconsistent (refer to Section 2.3). As a result this paper is unable to recommend sound upper and lower bound optimism bias levels for the operating expenditure (except for outsourcing projects), project duration and benefits shortfall for all project types. Guidance for unitary payments optimism bias for PFI projects has also not been provided as this optimism bias is affected by both capital and operating expenditure optimism bias and should be considered in this respect. Therefore the guidance in this paper is only for capital expenditure (operating expenditure for outsourcing projects) and works duration9. Optimism should, of course, be considered in respect of all parameters.

To calculate the optimism bias for project estimates during a project appraisal:

1. Decide which project type is appropriate for the project being appraised (see Section 2.1.2). Careful consideration needs to be given to the characteristics of a project when determining its project type. For example if half of a project satisfies the standard project criteria (e.g. new build on a greenfield site) and the other half satisfies the non-standard criteria (e.g. demolition and build on brownfield site, and refurbishment) it may be best to consider it as two projects under the same programme.

For ease of determining a project type for building and civil engineering projects, a project is considered "non-standard" if it satisfies any of the following conditions: (a) it is innovative and/or unique; or (b) construction involves a high degree of complexity and/or difficulty.

A PFI / PPP project that includes several project types (e.g. an element of standard building, non-standard building, standard civil engineering, outsourcing and equipment / development) should be considered as a programme with five projects.

Where standard and non-standard elements of a project are physically separate (e.g. new build on greenfield site and refurbishment of existing estate), then these should be considered as separate projects under the same programme. A project's project type should be determined by its dominant project type characteristics. However, if a building or civil engineering project has a significant amount of standard or non-standard elements (more than 35%) that are not physically separate then this type of project can be considered a combined project.

Outsourcing and equipment / development elements of a larger project should be considered as two projects within the same project programme.

2.  Use the appropriate upper bound value for optimism bias from Table 4 as the starting value for calculating its current optimism bias level (see Section 4.5 for guidance on calculating appropriate upper bound values for combine projects).

3.  Reduce this upper bound optimism bias according to the extent to which the project risk areas are managed (see Sections 4.1 to 4.4 for examples). The project risks within each project risk area can be managed. If the project risk areas for a project have only been partially mitigated then the contribution to optimism bias can be reduced proportionally to reflect the amount that each project risk area has been mitigated. When calculating optimism bias, the extent to which these risks are mitigated is measured by a mitigation factor. The mitigation factor has a value between 0.0 and 1.0. Where 0.0 means that risks in a project risk area are not mitigated, 1.0 means all risks in a project risk area are fully mitigated and decimal values between 0.0 and 1.0 represent partial mitigation of the risks within a project risk area. Ideally the optimism bias for a project should be reduced to its lower bound optimism bias before contract award if the cost of risk mitigation is less than the cost of managing the residual risk.

4.  Clear and tangible evidence must be observed, and independently verified, for the mitigation of risks in project risk areas before reductions in optimism bias should be made.

5.  If the optimism bias at the appraisal stage is appropriately low, then the project should be allowed to proceed. If the optimism bias remains high, then approval should be withheld, or given on a qualified basis, requiring further research, planning, identifying and managing project risk areas and reviewing of project scope to reduce the project risks and likely optimism bias to an acceptable level. For instance, high optimism bias may be acceptable for a strategic outline business case and very small projects (projects below £1 m in value), but would not normally be acceptable at the full business case stage for large projects.

Figure 8 summarises the procedure for calculating optimism bias. Project appraisers should review all the project risk areas that have had a negative influence on project costs, time and benefit delivery, within the appropriate project type. Table 15 to Table 17 in Appendix I show the upper bound project risk area contributions (%) to overall works duration and capital expenditure optimism bias levels for each project type.

To effectively appraise the optimism bias for a proposed project option using its business case, the proposed strategies for the mitigation of project risks and management of project risk areas should form part of the business case.

The optimism bias calculated using this guideline could be checked using one of the following:

•  An independent review of a project at key stages according to the OGC Gateway process

•  Internal audit (or other internal mechanisms)

•  Other semi-independent departmental body.

_________________________________________________________________________

7  Note that these values are indicative starting values for calculating optimism bias levels in current projects. The upper bound (U) does not represent the highest possible values for optimism bias that can result and the lower bound (L) does not represent the lowest possible values that can be achieved for optimism bias.

8  In the case of current equipment / development projects Mott MacDonald has observed a tendency to abandon these types of project when optimism bias levels have reached 150%.

9 This paper does not provide explicit terms for translating works duration delays into monetary values, however, if key financial indicators are identified for delayed benefit delivery it should be possible to calculate the financial impact due to delays in works duration.