7 Good decisions are based on having sufficient objective, accurate and timely information on costs, timescales, benefits and risks. Weaknesses in the quality and appropriateness of data and modelling techniques distort the information on which projects are approved and mask the risks. Over-optimism persists where these weaknesses are ignored and little effort is made to either develop robust estimates or be honest and transparent about the assumptions made on limited data.
8 There are numerous examples in our back catalogue of projects being planned and evaluated on poorly thought through data and modelling. Half of our recent reports on major government projects refer specifically to issues with estimating.
In May 2013, we identified a number of issues with the estimates in the business case for High Speed 2. At that time, the expected benefits for business travellers were based on data which was over ten years old, and the Department for Transport's model to forecast passenger demand was based on fares charged at a standard rate, rather than a premium rate. Price competition from low-cost airlines and ferry companies was one of the reasons cited for lower-than-expected passenger numbers on High Speed 1.7 An arithmetical error in the Department for Communities and Local Government's modelling for the New Homes Bonus led to a material overestimation of the impact of the project. When corrected, the model's prediction that around 140,000 new homes would be created was reduced to around 108,000.8 The Department for Work & Pensions implemented the Work Programme within 12 months, despite similar schemes taking four years. The tight timescales prevented any piloting of the approach, so the Department had no data against which to test its assumptions; for example about likely performance levels with contractors. The Department did not therefore have a good understanding of the realism and deliverability of contractors' bids.9 The Department for Communities and Local Government did not adequately test the assumptions underpinning the Mortgage Rescue Scheme business case; it failed to make enough use of available information at the outset, and the impact assessment contained very little sensitivity or scenario analysis. As a result, to the end of March 2011, the average cost of each completed rescue was much higher than anticipated: £93,000 compared to £34,000.10 |
______________________________________________________________________
7 Comptroller and Auditor General, Department for Transport: High Speed 2: A review of early programme preparation, Session 2013-14, HC 124, National Audit Office, May 2013.
8 Comptroller and Auditor General, Department for Communities and Local Government: The New Homes Bonus, Session 2012-13, HC 1047, National Audit Office, March 2013.
9 Comptroller and Auditor General, Department for Work and Pensions: The introduction of the Work Programme, Session 2010-2012, HC 1701, National Audit Office, January 2012.
10 Comptroller and Auditor General, Department for Communities and Local Government: The Mortgage Rescue Scheme, Session 2010-2012, HC 1030, National Audit Office, May 2011.