In the first instance, we measured cost overruns over the life of a project as the final cost in the historical database minus the first cost estimate that appeared in the quarterly pipeline (i.e. the initial cost).179 We then verified the initial cost, first cost when committed, first cost when under construction, and final cost, of the projects with the largest changes between initial and final costs (both overruns and underruns). This was to provide assurance that the largest cost changes in our analysis were not driven by erroneous datapoints or an artefact of how a project and its costs had been recorded over time in the Investment Monitor.
It was beyond our resources to research the cost histories of all 683 completed, public road and rail projects in our sample. Therefore, we restricted our checking in the first instance to the 54 projects whose final cost was at least $250 million (in $2020) higher or lower than their initial cost. This threshold was chosen somewhat arbitrarily as the level at which a tractable amount of research and checking would account for the vast majority (almost 80 per cent) of the total value of cost changes.
Having done these checks and correcting costs where appropriate, we performed a second round of checks. This involved checking costs of projects for which the change in cost as a percentage of initial cost was outside the range of percentage cost changes on projects that were looked at in the first round of checking. Such 'outliers' were checked progressively until no project in the sample had a percentage change in cost that was outside the range of those checked. This process resulted in an additional 10 projects having their costs checked.180
Of the 64 projects we reviewed in total, we revised the initial cost of 34 and the final cost of 17. These changes resulted in a 1 per cent decrease in the total initial costs of these 64 projects, and a 4 per cent decrease in the final costs.
In many cases we needed to change cost estimates because:
• the fundamental purpose of the project changed over time (see Box 2 on page 15); or
• components of the project were added or subtracted to the project record in the Investment Monitor over time (e.g. a record may have begun life in the Investment Monitor relating to an entire project, but at some point in time changed to relating to a single stage of the project only - or vice-versa).
These issues arise because we have linked into a time series a dataset (the Investment Monitor) which is intended to be looked at as a quarterly snapshot.
____________________________________________________________________________
179. 'True' overruns will be probably underestimated to the extent that projects had early cost estimates that were missed in the compilation of the Investment Monitor, including because such estimates were announced prior to commencement of the Investment Monitor in 2001.
180. Our treatment of outliers in this report differs to that in our 2016 report. In the 2016 report we simply excluded projects with percentage cost changes outside the range observed on a smaller sample of projects that we manually collated: Terrill and Danks (2016, pp. 64-65).