3.4 Project time periods for analysis

To overcome the limitations of earlier studies that simply compared the budget to actual project outcomes it was decided to adopt four time periods between project milestones for assessment. These four time periods were used to measure, compute and compare the relative performance of PPPs and Traditional procurement, they were:

Original Announcement means the first time the project was publicly announced.

Budget Approval means the point at which the relevant government approved the project budget prior to engaging the market.

Contractual Commitment is the time of contract execution and/or financial close.

Actual Final is the commissioning time for the infrastructure.

These four periods are shown graphically in Figure 1.

Figure 1: Comparative periods of assessment (source: Duffield 2008)

Each of these periods provides insight into different aspects of the projects and the inclusion of all periods provides the opportunity for a balanced appraisal of PPPs versus Traditional projects taking into account the different project processes between the two approaches. The rationale of the different periods is provided below:

Full Period: Original Announcement to Actual Final

The originally announced project data is based on the least robust information for the project. Frequently this data is announced prior to a detailed scope of work being defined or robust costings developed. None the less this time point becomes an important milestone in public projects as it is against this original announcement information that projects are frequently reported against by interested parties and the media. In the context of this study it is also important as it is a definitive milestone that can occur prior to the choice of a procurement strategy. It is therefore a starting point that is independent of the processes used for either PPP projects or Traditional projects.

Stage 1: Original Announcement to Contractual Commitment

The period from original announcement to contractual commitment is the period whereby a project is fully scoped and expectations and requirements of client and end users should be fully developed, understood and confirmed by the market place clarifying what is required and the price to deliver the project. In many ways a comparison based on this time period is an indicator of the accuracy and adequacy of project procurement processes within government.

Stage 2: Budget Approval to Actual Final

Project success is often internally measured and reported as performance against an agreed budget. Ideally such comparisons would be made based on linking approved budget to scope of work delivered, such as via the Earned Value Technique. This is frequently not done and in fact budgets are often adjusted as the need arises. To avoid errors in the benchmark study that would occur if current budget data was used this study adopts the final approved budget prior to going to market as the milestone.

This particular metric is the one adopted in the previous studies by Mott MacDonald, the NAO and Fitzgerald and thus it is useful as a comparator. However, it could be argued that this particular period simply gives an indicator of the accuracy of the budget estimate and that differences in the answer may be attributed to different levels of optimism at budget stage due to the rigour of the estimate. To avoid problems associated with optimism bias at budget approval it is considered comparisons based on the full period are more appropriate as optimism bias is reduced.

Stage 3: Contractual Commitment to Actual Final

Comparison of performance from contractual commitment to actual final is a measure of the robustness of the risk transfer within the contractual forms. It is also a good check on the price certainty that is obtained via either contractual approach.