Cost: To what extent does PPP procurement deliver overall cost savings in comparison to conventional procurement? How far can this be assessed with reference to factual data, rather than through comparisons with the assumptions used in Public Sector Comparators?
This question goes to the heart of the assessment of the relative costs and benefits of PPP. See CEPA's overall assessment in Section 1.2 below.
The Public Sector Comparator is clearly an important tool for assessing procurement options. Indeed, in all projects in our survey where a Public Sector Comparator (PSC) was prepared (84 percent), the ex-ante analysis of the proposed PPP showed a saving versus the PSC.
However, to make a full assessment based on factual evidence, it would be necessary to compare in detail a number of PPP and conventional procurements (of infrastructure and the associated operations) in the same sector over a significant length of time. It has not been possible to do this within the scope of this research. But, as identified in our conclusions on Stage 2 of this work, we do believe that is possible to carry out further work that, at least partially, overcomes these constraints.
Other evidence from our study that relates to cost is as follows. First, more than half of the authorities surveyed believed that their contract offered good or excellent value for money, and only one project was rated as poor value for money. Second, our research identified a number of areas including procurement, performance monitoring, insurance and commissioning additional works, where costs are perceived to be higher for PPP projects. Third, our research also showed that contract managers generally perceive construction and operational risks to be appropriately allocated. Each of these points is discussed in more detail below.
Competition: Is there generally a healthy level of competition for PPP contracts, and hence is the lowest market price for the PPP service being obtained?
Projects included in the survey had received, on average, nine pre-qualification bids. The minimum number of bids received was two (and this only happened in one case). In our view this represents a generally healthy level of competition and was sufficient to achieve the most economically advantageous price.
Authorities and contractors commented that the level of competition has reduced in recent years and there is some anecdotal evidence that there is now less competition for PPP contracts than for other types of contract.
Procurement and construction performance: How far does the use of PPP impact on procurement costs and timescales? To what extent does the use of PPP deliver projects more quickly and on budget in comparison to conventional procurement?
The PPP procurement process is expensive and represents a greater burden for smaller projects. The mean time taken to procure the PPP projects surveyed was 28 months, which was generally perceived to be slower than non-PPP procurement. We note, though, that the increased costs and timescales associated with procurement should be weighed against their impact on overall timing and value for money. It is also worth noting that 'bundling' of smaller projects can help to spread procurement costs across several discrete projects.
There was greater price certainty during construction in the PPP projects surveyed than has been the case historically with conventional procurement. The proportion of projects delivered on time also showed a significant improvement over historical experience.
Nevertheless, on more than half of projects, the price changed at the preferred bidder stage because of a change to the design or service specification.
Design: To what extent are PPP designs perceived to be better (or inferior), e.g. in terms of aesthetics, functionality, environmental performance, etc? Is the relevant good practice guidance being followed?
Our survey found that authorities were happy with design quality. PPP projects were perceived to perform the same as, or slightly better than, non-PPP projects with respect to aesthetics, functionality and environment.
These perceptions reflect, in part, the fact that many authorities played an active role in the design process. More objective approaches to assessing design by sector (e.g. hospitals / schools / prisons) would need to be carried out to make a more informed judgement about actual design quality.
Formal design evaluation tools were not generally used in the projects surveyed, because these projects were amongst the first to be developed and such tools were not available.
Innovation: To what extent do PPP contractors deliver genuinely innovative solutions?
There was evidence from the survey and interviews that PPP stimulated innovation, although authorities were less positive about the role played by the PPP contractors than the contractors themselves. As with design, we believe that the authorities' views reflected their level of involvement in design decisions.
Operational performance: To what extent do PPP operators deliver a better (or worse) standard of service than the public sector, e.g. in terms of availability, cleaning, catering, etc? To what extent do service levels fall short of or exceed the original expectations of the contract?
We found no evidence that PPP operators delivered a better or worse standard of service than the public sector. Authorities commented that service standards were determined by the standards specified in the contract and the budget available.
Most authorities thought that the standard of service met or exceeded expectations. Availability2 performance was rated highly, which might reflect the strong financial incentive on contractors to meet availability targets. Hard facilities management services rated relatively poorly. However, we understand that this reflects some frustration on the part of authorities over the resolution of snagging issues and minor repairs, rather than concerns about lifecycle maintenance. It is too early to draw any conclusions about the impact of PPP on longer-term maintenance. But real issues were also raised about the interaction between hard and soft FM providers in PPP structures.
Flexibility: Are PPPs seen as being more or less flexible than standard contracts, e.g. in terms of making alterations to the asset, levels of service, etc? What has been the experience of attempting to make changes to contracts?
Our survey results showed that PPP contracts are seen as less flexible than non-PPP contracts. Authorities reported that their experience of making changes has been time consuming and slow. Where changes have been implemented across a sector they have been delivered later at PPP facilities.
A number of authorities commented in interviews that:
• The costs associated with making contract changes meant that they often sought to wrap up a number of changes over time into a single negotiation.
• Changes were rarely incorporated within the unitary charge mechanism because of the complication of agreeing the financial impact. But also, for an authority with grant available to spend on a specific change, it made sense to make a one-off payment rather than to incur financing costs.
• Works costs for contract changes were perceived to be expensive in the absence of the ability to tender them. Further work would be required to assess whether costs quoted by PPP partners reflected an allowance for maintenance and renewals.
Clearly it is important for authorities to get the scope of their projects correct when they are preparing Business Cases and ITNs. But it is inevitable that, over the life of a 30 year contract, changes will be required to support, amongst other things, new work practices and different user groups.
Contractual Relationships: Are relationships between customers and contractors perceived to be based on a "partnership" approach or an "adversarial" approach? What has been the use and impact of penalty payments/bonuses and/or the threat of termination of the contract?
Survey evidence showed that the majority of relationships between authorities and contractors are good. In interviews it was clear that both parties recognised the benefits of developing a long-term partnership.
Payment mechanism deductions are being implemented on the majority of projects. Authorities rated availability performance highly, and cited cleaning and minor repairs as the areas where performance was least satisfactory.
Budgetary Issues: How valid is the argument that the use of PPP delivers benefits due to budget restrictions in the public sector (particularly capital budgets)? For example, is it valid to argue that PPP procurement brings forward investment and/or ensures that optimal maintenance strategies are followed?
Many survey respondents commented that, at the time their project was procured, PPP was the only option given the authority's capital constraints. (See Section 1.2 for commentary on this).
It is too early to compare maintenance strategies between PPP projects and contemporary non-PPP projects. However, our survey confirmed the observation that PPP does ensure that funds for maintenance of particular assets are effectively ring-fenced over the life of the contract, whereas non-PPP projects compete with all of an authority's non-PPP assets for a share of the maintenance / renewals budget. Indeed the overall maintenance budget itself can be subject to pressures from other priorities.
Risk transfer: To what extent is risk transferred in practice? Is it always clear where risk lies? Is there any evidence of contractors or customers seeking to shift risk onto the other party after signing the contract?
Survey respondents believed that risks were, for the most part, allocated appropriately.
In the case of operational risks, authorities noted that ambiguities in the contract drafting sometimes made it unclear where risks lie and presented opportunities for risks to shift. We would expect this to become less of an issue as the lessons learnt from early projects are incorporated into standard contracts. More issues around where risk lies were raised in sectors with significant interfaces between the PPP contractor and public sector staff, users and other contractors. This was most notable in the health sector where the interactions between hard and soft facilities management and the risks associated with the needs of different user groups were not always clearly defined in the contract.
______________________________________________________________
2 In most PPP contracts, a significant proportion of the payment to the contractor is linked to the asset being available for use by the public sector at the contracted standard.