19. The NAO believes that optimism bias is a sound theory and a good contribution to the Treasury's Green Book. Optimism Bias is a phenomenon that we frequently observe during our audit work. For example, our 2009 study on Building Schools for the Future found that the Department for Children, Schools and Families greatly underestimated how long it would take to renew England's secondary school estate, despite data from existing school projects clearly demonstrating that it would take longer than they were predicting.
20. We would like to see optimism bias reduced through better technical planning, estimating and forecasting, combined with better governance and scrutiny over business cases and plans. In particular, public authorities should reference their estimates for how long things will take to the experience of other projects.
21. But as the academic literature points out, the causes of optimism bias are not all technical. It thus matters in what context planning and forecasting is being done, and especially how the institutional incentives affect the planning decision. Applying uplifts to optimism bias is itself a subjective adjustment and susceptible to manipulation.
22. We have therefore always been professionally sceptical about optimism bias uplifts and especially in the context of Public Sector Comparators. Where optimism bias uplifts have been a material focus of one of our value for money reports we have scrutinised carefully how the optimism bias uplift was applied and its effect on the outcome. For example, the Committee of Public Accounts used our work in their report on the GCHQ building. They pointed out that GCHQ's PFI project relied solely on the highly uncertain assumption that the conventionally procured building would have over-run its budget by 24%.
23. We are also sceptical of the way optimism bias is applied to Public Sector Comparators but not to PFI projects. Although Mott MacDonald believed optimism bias in PFI projects was far lower than for conventional procurement, they believed PFI projects suffered optimism bias of around 5% after contracts were signed. However, difficulties in collecting the data meant they could not provide directly comparable figures between PFI and conventional projects.
24. Our 2009 study on construction costs showed that 31% of PFI projects were delivered late and 35% were delivered over the estimated price. Most of the price increases were from changes to the scope in the project made by the public authority or a third party. The academic literature would include such price increases in the calculation of optimism bias, on the argument that the forecasters should know the statistical likelihood of making amendments to the scope during the construction period.
25. In conclusion, our view is that is fair to consider optimism bias in estimates so long as adjustments for optimism bias are based on robust evidence. We would like to see more evidence behind the adjustments that we see. And we argue for separate disclosure of such adjustments and that public authorities make their subjective nature clear to the models' users.