8. Optimism bias is commonly assumed when assessing the cost of projects. Do you have a view whether it is right for assessing a public sector comparator (PSC)?
Optimism bias is discussed in chapter 5 of the Treasury's 2003 Green Book (Appraisal and Evaluation in Central Government). The report states that:
"There is a demonstrated, systematic, tendency for project appraisers to be overly optimistic. This is a worldwide phenomenon that affects both the private and public sectors. Many project parameters are affected by optimism-appraisers tend to overstate benefits, and understate timings and costs, both capital and operational".
The Treasury goes on to argue:
"To redress this tendency, appraisers should make explicit adjustments for this bias. These will take the form of increasing estimates of the costs and decreasing, and delaying the receipt of, estimated benefits. Sensitivity analysis should be used to test assumptions about operating costs and expected benefits".
From this, it follows that the any cost and timing estimates (for both the PSC and PFI alternatives) should include an allowance for optimism bias.
As the Treasury puts it:
"the adjustments should be empirically based, (eg using data from past projects or similar projects elsewhere), and adjusted for the unique characteristics of the project in hand. Cross-departmental guidance for generic project categories is available, and should be used in the absence of more specific evidence".
But all too often the optimism bias estimates have been manipulated to justify the PFI alternative-see Edwards written evidence paras 5, 6 and 7; Edwards C, June 2009 41 and 44; and Hellowell and Pollock 2009, 15. See also the written evidence of the National Audit Office where it states that the PSCs "are susceptible to manipulation and we often find problems with their implementation" (see NAO October 2009, paragraph 4.9 on page 46).
The system is open to manipulation for two reasons:
- In most cases, there has been no serious choice between the PFI project and the PSC so that that optimum bias has often been used to make the PFI a winner. An example of this was the optimism bias assumed in the Full Business Case (FBC) of the NNUH. A cost overrun of 34.22% was assumed for the PSC. Note the precision. Not 35%. Nor 34%. But 34.22%. Unsurprisingly, in 1999, the Select Committee on Health complained many times about the FBC of the NNUH. As the Chair of the Committee put it; "In other words, the full business case does not tell us the full business case" (see Edwards C, June 2009, 86).
- It is fairly easy to manipulate the comparison since the PFI contract approval goes through a number of stages. Thus the PFI price may be shown to be lower than the PSC at the Outline Business Case stage (when initial approval is given) but once approval has been given the PFI price is free to rise and has done so. As Hellowell and Pollock 2007 (page 18) have pointed out, the average cost increases for 43 "prioritised" schemes between the Outline Business Case stage and the stage at which contract were signed was 74%. The same point is made in the evidence of the National Audit Office when it says "VFM [Value for Money] is most at risk during the final stage of negotiations, when negotiation is with a single preferred (or final bidder) and competitive tension is at its weakest" (see NAO October 2009, paragraph 5.8).
This last point means that is that it is not surprising to find that "most private finance projects are built close to the agreed time, price and specification" (see NAO, October 2009, paragraph 7, page 7). If the price and completion time are expanded during the final stage of negotiations, then there is inevitably a better chance of delivering close to budget and on time.