We believe that the most promising approach for the Scottish Executive to improve the available evidence on the operational performance of PPPs and non-PPPs is to carry out sector specific studies. There are various approaches to carrying out such comparative studies.
The simplest approach would involve carrying out an interview or survey-based study along the lines of this research. Such studies would need to be carried out periodically to track performance over time. One option that the Scottish Executive might consider is to build on the Audit Scotland report on schools20 and identify a number of PPP and non-PPP schools projects to evaluate over time. Other sectors would of course be equally interesting, however the school sectors has the particular advantage of being a relatively mixed economy in terms of privately and publicly financed infrastructure.
The weakness with the survey-based approach relates to the potential for interview bias. A more robust approach is likely therefore to involve carrying out full comparative efficiency or 'benchmarking' studies. This analysis would aim to look at the efficiency of the overall service offered by a contractor by considering, for example the relative costs of delivering the 'final' outcomes of the contract (e.g. the availability of the asset to an agreed standard). This is the approach that utility regulators take when seeking to compare the economic efficiency of regulated companies (e.g in the water and power sectors).
A starting point for such studies would be the definition of a standard set of KPIs for PPP and non-PPP projects within the same sector. Although this is a non-trivial task - since (as noted above) there is currently a relative lack of comparable performance indicators - we believe that it could be a valuable exercise for a number of sectors.
______________________________________________________________
20 Taking the initiative: using PFI contracts to renew council schools, Audit Scotland 2002