Conclusions/Findings - Development of the PSC

• Was the budget in the business case supported by a reasonable analysis and assessments of costs

Our view is that the (PSC) budget presented in the DBC was based on a reasonable supporting analysis and assessments of costs, given the circumstances that the Project Team were facing at that time.

That is, Waka Kotahi were asked during a very short period of time (i.e. between August and September 2012) to convert what effectively was a non-PPP project, with an existing consented scheme design, into an initial PSC estimate that could be used to inform the 2012 PPP Business Case. This situation meant Waka Kotahi had to use a design that was less developed than what they would normally include in a DBC.

Further, Waka Kotahi was also using project information that was not framed around a "PPP-lens" - for example, it is usual to see an outline of a Service Need Specification in a PPP detailed business case. Not having an outline Service Need Specification means that cost changes inevitably occur during the later procurement phase as a Service Need Specification is prepared to inform the tender documents that bidders will be asked to respond to. We believe if more time had been available to prepare an outline Service Need Specification, then this potentially would have helped to inform the costings used in the DBC. Accordingly, our recommendation is that future road PPP projects include an outline Service Need Specification in the DBC.

We note that Waka Kotahi has extensive capital works/delivery experience that would have helped inform the capital costings used in the business case (notwithstanding that only a 2011 Scheme Estimate was used) - that approach would have been very similar to the costing methodology approach used by construction members of a PPP consortium.

The one area of difference in the business case's costing methodology that we identified related to O&M costs. The review documents explain that Waka Kotahi assumed a network/marginal costing approach to recurrent O&M costs - this is fundamentally different to the approach taken by a PPP consortium, who would focus on specific recurrent costs needed to maintain the TG road section as a stand-alone project from the rest of the road network. Typically, as a first principle, the PPP consortium would have been using a full-cost approach when assessing the possible O&M costs. The PPP consortium would also be assessing their O&M risk exposure given the TGP contract has defined performance standards linked to the annual service payment made by Waka Kotahi.

We could not identify from the review documents and interviews if any PPP-benchmarking of O&M costs and estimates of Transferred Risk had been done for the DBC. We believe that Waka Kotahi should consider developing a cost and risk benchmarking database to assist managing their portfolio of road availability PPP projects. Note, we have also commented further under Other Matters about benchmarking once TG commences the services phase after September 2021 (i.e. the revised date for construction completion).

As is normally the case with any major infrastructure project, project costs and risks are reviewed during the procurement phase as more detailed information becomes available. For TG this means the PSC costs would very likely change post approval of the business case. We confirmed from the review documents that the Board was advised (in December 2012) that the PSC would be subject to a planned review in March/April 2013 - such planned reviews reflect best practice in that a PSC can change post-business case as more project details become available. In this regard, we found the first (i.e. planned) review of the PSC in March/April 2013 to be perfectly reasonable in terms of timing and outcomes (i.e. changes to the 2012 PSC estimate).

However, we think there could have been better communication to the wider community that a business case recommending a tender be conducted using PPP delivery does not ensure "cost certainty". The PPP business case's main decision is to test if there is a reasonable commercial case that justifies approaching the market with a PPP delivery option. In this regard, we recommend Te Waihanga to consider publishing additional information for stakeholders on the use of the PPP model which should address this issue.

The key issue in our mind, however, is why the first planned review of the PSC in March/April 2013 did not pick up the second round of cost changes that occurred mid-way (i.e. in August 2013) during the RFP phase. This second round of cost changes occurred when both RFP Respondents advised Waka Kotahi that they were finding it difficult to meet the AT. The main reasons for these second-round changes appear to centre on:

• Waka Kotahi-initiated changes to design requirements that were not in the consented scheme costing, mainly around an increase in the design speed to 110 km/hr (from the 100 km/hr design speed in the consented scheme estimate), road safety requirements, and revised structural requirements for bridges; and

• More detailed information coming to light as both RFP Respondents conducted their own due diligence on the seismic and geotechnical conditions to inform their technical solutions, as well as their view on how they would need to manage their consent obligations.

We note that the changes due to design speed were being flagged by the Project Team in the weeks leading up to the release for the RFP in May 2013 (e.g. as recorded in the Governance Group minutes of April and May 2013). From those minutes, it appears the Project Team were needing to confirm the revised design changes so they could be included in the RFP documents, or potentially delay the issue of the RFP documents. With this context in mind, the May 2013 minutes to the Governance Group record, "...Decision to include requirement for 110 km/hr design speed taken by Project team - PSC risk adjustment sufficient to cover this requirement.". Our reading of this fact is that the Project Team considered sufficient risk was factored into the PSC to deal with the changes in the design speed to allow the RFP to be issued - as subsequent facts showed, however, this was clearly not the case and the PSC's costs had to be increased to meet the additional design speed requirements.

Our key observation with the above information is that the record of the decision made by the Project Team could have been supported by more detailed information that showed the quantum of risk allocation that was felt to be sufficient. Further, such an important PPP decision should also have been endorsed by a higher-level governance group above the Project Team. There is a possibility that such information was prepared at the time, but we were unable to confirm that from the review documents provided or from interviews. We have commented further on the need for more transparent governance documentation in Section 5. Review Focus Area - Governance & Timeframes.

With regard to seismic and geotechnical risks, our main observation is that for a complex project like TG, potentially more preparatory work/information for bidders could have been done by Waka Kotahi prior to release of the RFP documents in May 2013 (which would have coincided with the time of the first PSC review in March/April 2013). We note that the P2W Detailed Business Case (at page 116) effectively makes the same comment when advising on the lessons-learnt from TGP, i.e.".Allow sufficient time prior to shortlisting and RFP release for the geotechnical investigations and additional design work to be carried out to support the review of the PSC, report back to the Board, and request for approval to borrow from Cabinet.".

• Appreciation of inherent and contingent risks

Our assessment is that inherent risks (i.e. uncertainties with pricing and schedule/quantities) and contingent risks (allowances for unknown, project specific risks) were taken into account during the development of the business case to derive the allocations for Transferred and Retained Risks used in the PSC. However, as noted already in this Report, the PSC was based on a Scheme Estimate and not a Detailed Estimate - this means a comprehensive assessment of some risks (such as seismic and geotechnical) would have come later as more detailed investigations were done.

The approach to calculating a project contingency for risk appears to have been done in a structured way, having regard to how Waka Kotahi would normally have prepared a contingency estimate (using SM014) and the different method used in a PPP assessment to construct the PSC. This is an important fact to consider when trying to make comparisons between these two cost estimates where contingencies are concerned.

Using the review information available for design and construction costs, which are the main cost line in any major road infrastructure project, we concluded the overall risk contingency used in both the 2011 Scheme Estimate and the 2012 PSC for both inherent and contingent risks was approximately 26 percent. Our view is this contingency range developed in 2012 is within usual benchmarks for contingencies used in a business case. As evidence, we note the following P50 and P90 contingency analysis in Australian research for contingencies used in publicly funded road and rail projects3.

Our main observation, therefore, is not about the contingency percentage per se in the 2012 PSC but more so that the contingency estimates at that time were based on a consented, non-PPP project and a Scheme Estimate design.

As a result, as the PPP project became more defined after the 2012 PPP Business Case was approved then, inevitably, the TGP's cost estimate began to increase. As the 2012 risk provisions were based on a non-PPP Scheme Estimate, then any revised costs were not able to be absorbed within those risk allocations and instead would need to be reflected in changes to the overall cost. We note that if Waka Kotahi had used a (more developed) Detailed Estimate in the DBC, then the risk adjustments would also have been more developed and could possibly have been used to manage some of the cost increase.

In summary, given the nature of the cost estimate used in the DBC, we have no material recommendations relating to the adequacy (at that time) of the risk contingency provisions. Our only recommendation in regard to inherent and contingent risks is that Waka Kotahi consider developing benchmarks, both from local and international experience, on ranges of estimated project risks for comparable motorway projects.

_____________________________________________________________________________

3 See Tan, F. and Makwasha, T., “Best Practice Cost Estimation in Land Transport Infrastructure Projects”, Australian Transport Research Forum 2010 Proceedings, page 8.