Lack of data is showing up today in insufficient provision for 'worst case' cost outcomes
One clear manifestation of the scarcity of data is that the cost estimates of current projects continue to make insufficient provision for 'worst case' outcomes.
Business cases typically include an estimate of the the median cost, or 'P50', and the worst case, or 'P90'156. In business cases produced in recent years, the difference between P50 and P90 cost estimates is generally about 7 per cent (Table 6.1 on page 40). But the experience of the past two decades has shown that the difference between the P50 and P90 costs is actually 49 per cent, on average.157
This difference between usual cost estimation practice and observed reality would be unlikely to occur were a cross-jurisdictional database of final project cost outcomes available to estimators. Such a database would indicate that large cost overruns are far more frequent than recent cost models imply.
This comparison of cost estimation practice and actual experience shows that either median cost estimates are generally too high, or - more likely - 'worst case' cost estimates are generally too low.158 Cost estimates are making insufficient allowance for unlikely events that cause large cost overruns. Part of the reason for this is likely to be that most cost estimation methods recommended in guideline documents and handbooks do not include the costs of any unknown risks not explicitly identified by the estimator (see Appendix B).
The P90 estimate for the West Gate Tunnel in the 2015 business case was $5,548 billion, 6.2% higher than the P50 estimate. Were this P90 estimate correct, it would indicate that there is only a 10 per cent probability that the eventual project costs would exceed $5,548 billion. As shown in Section 1.1.1 on page 8, the current cost estimate is $6.7 billion, and there are reports that costs could further blow out.159 Given this situation, it seems unlikely that the P90 estimate of $5,548 billion took sufficient account of unlikely adverse events.
There is a pressing need for better data.
Figure 6.1: Road costs vary widely across jurisdictions
Notes: Costs per lane kilometre exclude property acquisition and supplementary costs. BITRE randomly assigned responding jurisdictions a number between 1 and 7. Tasmania did not provide data in 2017.
Source: BITRE (2018, Table R.B3, p. 26).
Table 6.1: The difference between median and 'worst case' cost estimates is lower, on average, than experience suggests it should be
| Project | State | Cost estimate (nominal, $millions) | Difference | |
| Median | 'Worst case' | |||
| Inland Rail | National | $9,889 | $10,657 | 7.8% |
| Metro Tunnel | Vic | $10,154 | $10,837 | 6.7% |
| West Gate Tunnel | Vic | $5,226 | $5,548 | 6.2% |
| Canberra Light Rail | ACT | $759 | $806 | 6.2% |
| Bruce Highway - Cairns Southern Access Corridor (Stage 3) | QLD | $470 | $500 | 6.4% |
| Bruce Highway - Cairns Southern Access Corridor (Stage 4) | QLD | $97 | $104 | 7.2% |
| M1 Pacific Motorway - Eight Mile Plains to Daisy Hill | QLD | $713 | $747 | 4.8% |
| M1 Pacific Motorway - Varsity Lakes to Tugun | QLD | $960 | $1,017 | 5.9% |
| Townsville Eastern Access Rail Corridor | QLD | $369 | $392 | 6.2% |
|
|
| Benefit Cost Ratio |
| |
|
|
| Median | 'Worst case' | Difference |
| Beerburrum to Nambour Rail Upgrade | QLD | 1.48 | 1.35 | 9.6% |
| Bruce Highway - Deception Bay Road Interchange | QLD | 3.23 | 3.03 | 6.6% |
| Bruce Highway - Maroochydore Interchange | QLD | 3.4 | 3.2 | 6% |
| Bruce Highway - Bribie Island Road to Steve Irwin Way | QLD | 2.02 | 1.91 | 5.8% |
| Centenary Bridge Upgrade | QLD | 0.85 | 0.75 | 13% |
| Smithfield Transport Corridor Upgrade | QLD | 2.9 | 2.6 | 11% |
| Average difference of above estimates | 7.3% | |||
| Average actual difference across all projects completed in the past 19 years | 49% | |||
Notes: Public business case documents for the last six projects in the table do not explicitly include median and 'worst case' cost estimates on a comparable basis. They do, however, include benefit cost ratios estimated using median and 'worst case' cost estimates. The relevant difference between these benefit cost ratios is equal to the ratio between median and 'worst case' cost estimates, inflated and discounted, including operational costs. Operational costs are typically less than 6 per cent of capital costs for these projects.
Sources: ARTC (2015, p. 19), Victorian Department of Economic Development, Jobs, Transport and Resources (2016, p. 9), Victorian Government (2015, p. 211), Capital Metro Agency (2014, p. 86), Building Queensland (2017a, p. 4), Building Queensland (2017b, p. 3), Building Queensland (2018a, p. 3), Building Queensland (2018b, p. 3), Building Queensland (2017c, p. 131), Building Queensland (2016, pp. 12-13), Building Queensland (2018c, p. 9), Building Queensland (2018d, p. 9), Building Queensland (2018e, p. 10), Building Queensland (2019, p. 10), Building Queensland (2017d, pp. 13-14), and Grattan analysis of Deloitte Access Economics Investment Monitor.
| Recommendation 6 The government of each state should require its infrastructure minister to provide completed project data for any infrastructure project valued at $20 million or more, to that state's infrastructure advisory body. The completed project data should include: • First announced cost, contracted cost, cost estimate at the start of construction, any further significant changes to costs, and final outturn costs: - sub-divided into project management, preliminary design and investigation, property acquisition, and construction cost components. • Key physical characteristics of the infrastructure, including type of road or track, number of lane or track kilometres, and length of any tunnel. • Project location, including green- or brown-field, geology, and whether CBD, urban, or rural. • Estimated and actual construction start and completion dates. • Any material changes to scope, and the reasons and dates. • Contract type and partners. Chief Executive Officers of the state infrastructure advisory bodies should agree on the data definitions and format, to facilitate pooling of data across state lines. |
____________________________________________________________________________
156. 'P50' refers to the amount which the actual project cost will exceed in 50 per cent of cases. 'P90' refers to the amount which the actual project cost will exceed in the worst 10 per cent of cases.
157. Analysis based on comparison of final project cost to the estimated cost when first committed. Ten per cent of projects overrun by more than 49 per cent after the project is committed, while the median overrun is zero: Grattan analysis of Deloitte Access Economics Investment Monitor.
158. In fact, it is clear that relying on median cost estimates will systematically understate the cost of the portfolio of projects. This is because the distribution of cost estimates is right-skewed, and therefore the mean, or expected value, is greater than the median cost estimate. More detail on the distribution of cost estimates is in Appendix B.
159. Jacks (2020).