Overall performance of PFI contracts
Most PFI projects are performing satisfactorily or better
2.1 The Department's devolved delivery model means it does not systematically monitor the performance of its PFI projects. Furthermore, the systems used by Trusts do not allow for easy comparison either between contracts or with non-PFI services. We have drawn on a range of evidence to conclude that most PFI contracts are meeting the expectations of Trusts and the Department, and appear to be performing to the standards set in the contracts:
a The Performance Management Systems report that performance typically meets the contractual specification. These are used by Trusts to monitor performance and charge payment deductions for failure to meet contractual standards (Figure 3). Fifty-three per cent of Trusts did not charge any deductions in 2008-09. Of those that charged deductions, the amount ranged from £200 to £651,000 a year, but always represented a reasonably small percentage (0.01-5.18 per cent) of the unitary charge (Figure 4 on page 18).
b Most PFI providers are meeting Trusts' expectations. We asked Trusts to rate the performance of their ProjectCo in managing the contract, and their subcontractors in delivering the contractual services. Sixty-seven per cent of Trusts rated all services as at least satisfactory. Of the 72 Trusts that provided data, 11 (15 per cent) rated all their services as better than satisfactory (Figure 5 on page 18). These are self-assessments, normally provided by the contract managers on behalf of the Trust. Contract managers may have an incentive to claim that they are achieving value for money, but are normally the best-informed observers.
c There is no difference between PFI hospitals and the rest of the NHS in assessments of environment and catering. Patient Environment Action Team scores show that PFI hospitals are not performing significantly better or worse than other hospitals (Figures 6 and 7 on page 19).
d The majority of Trusts report consistent or improved performance over the course of the contract. The majority of Trusts said service delivery had either stayed the same or improved. For cleaning and catering services the majority of Trusts have seen an improvement (Figure 8 on page 20).
Figure 3 PFI contracts require contractors to report their activity against a set of performance indicators which together form a Performance Management System. This would, for example, record a report of a blocked toilet, when it was mended, and if it was mended within the time specified by the contract. Performance indicators are negotiated during the bidding process and indicators thus vary significantly between contracts. Trusts can charge financial deductions to the unitary payment for failure to meet performance indicators. Trusts can also charge deductions if parts of the building are not available for use, e.g. a leak preventing the use of patient rooms. Trusts' use of the Performance Management System is covered in more detail in Part Three. Trusts charging sustained high deductions are unlikely to be achieving value for money Trusts should use deductions to penalise poor performance and encourage improvement. A sustained high level of deductions, however, is an indication that there are more systemic issues with the Trust, contract, relationship or service provider. The deductions are not intended to be sufficient compensation for persistent poor performance. For instance, one of our case studies, King's College, has made substantial deductions over a number of years. It is not happy with the level of service and has attempted to improve performance through enforcement of its contractual entitlements. In the Trust's view these deductions have forced the ProjectCo to address the poor performance of their subcontractor, and to include the subcontractor at quarterly performance meetings. We informed the Trust that, in our opinion, it would not achieve value for money until performance improves, but its poor relationship with its contractors inhibited their ability to work together to achieve this. The subcontractor was replaced via market testing in late 2009 and the Trust reports that performance has subsequently improved. Source: National Audit Office |
Figure 4 | ||||
| Number who charged deductions (proportion of the Population) | Lowest annual Deduction by a Trust | Highest annual Deduction by a Trust | Mean |
Maintenance services | 27 (36%) | 17 | 651,000 | 46,000 |
Hotel services | 21 (28%) | 4 | 436,000 | 31,000 |
All services | 35 (47%) | 200 | 651,000 | 54,000 |
Source: National Audit Office Survey |
|
|
| |
Figure 5 Most Trusts assess performance as satisfactory or better
Source: National Audit Office Survey |
Figure 6 Patient Environment Action Teams: • Are annual assessments of healthcare sites in England. They are carried out by NHS staff and patient representatives chosen by the Trust, with an element of peer review. • Use objective criteria to assess the quality of environment, catering and patient dignity. The proportion of available points achieved by a hospital in each area translates into a score between 1 (unacceptable) and 5 (excellent) for each category. • Are conducted at a hospital, rather than Trust level. • Can be used to see if there is a systematic difference in the standard of the environment (a proxy for the standard of cleaning) and catering in PFI hospitals. Source: National Patient Safety Agency |
Figure 7 There is no significant difference in scores Percentage
Source: Patient Environment Action Team Database |
Figure 8 Performance is generally improving or staying the same Service
Source: National Audit Office Survey |
2.2 Ostensibly, low deductions should provide assurance that the contractual specifications have been met. In part three we set out our findings on how Trusts are monitoring and driving performance using their Performance Management Systems. Not all Trusts are monitoring these systems sufficiently, and many could be strengthened as a tool for improving performance. On the other hand, 77 per cent of Trusts say their systems are effective in reporting performance and our review of systems at our case studies found that Trusts are generally able to audit and monitor performance. In our judgement, the main reason there are very few deductions and reported problems is that performance is normally meeting the contractual specification. But poor monitoring of performance may explain the low level of deductions at a minority of Trusts.
However, a significant minority of Trusts report problems with the performance of their contractors
2.3 Twenty-four Trusts (33 per cent) rated at least one service as below satisfactory (Figure 9 on page 22) and no Trust rated all services as excellent. Maintenance services and the ProjectCo's performance in managing the contract were the areas most frequently cited as underperforming. But the issues that Trusts raised varied (Figure 10 overleaf) and our analysis did not identify a clear set of causes for dissatisfaction. The age of the contract, the subcontractors providing services, or the level of resource invested in contract management are not correlated with the level of satisfaction.
2.4 No Trust reported problems with performance as a result of the current economic downturn.
PFI services are generally well integrated with clinical activity
2.5 Our case studies reported that PFI services had not had an adverse impact on clinical activity. They felt the staff delivering the contractual services were well integrated into the hospital and were prepared to work flexibly in order to ensure patient services are not disrupted.
2.6 Only 9 per cent of Trusts agreed that outsourcing staff to the PFI contractor had a negative impact on relationships, and 20 per cent that it inhibited operational flexibility. Of our case studies, Oxford Radcliffe, King's College and University College London Hospitals told us that reliance on temporary domestic staff impedes team integration. Such high turnover is common for inner city hospitals, including clinical staff as well.
Figure 10
NOTE 1 Because some Trusts said more than one service was unsatisfactory, these numbers are not cumulative. Source: National Audit Office Survey |
Figure 10 • King's College Hospital was dissatisfied with lift maintenance. Broken lifts meant patients often share lifts with visitors to get to operating theatre. This is an ongoing issue yet to be resolved. • Oxford Radcliffe Hospitals was dissatisfied with cleaning standards and developed a collaborative working agreement with its contractor to improve performance for the benefit of the Trust and its patients. • Dudley Group of Hospitals believes the portering function is understaffed and patients are not always moved within the required time, delaying treatment. The Trust worked in partnership with the contractor to improve efficiency by introducing Lean principles. • Hull and East Yorkshire experienced poor performance on some maintenance work. A high level of involvement from matrons has since ensured that clinical and maintenance services run smoothly together. • St Helens and Knowsley Hospitals' contractors initially provided inappropriate patient menus and communicated poorly with those on elderly wards. The Trust worked with contractors to resolve issues quickly and now rates its catering highly. • Buckinghamshire Hospitals NHS Trust believes that change requests often take too long to process and lead to an increase in payments. This can inhibit change, so reducing operational flexibility. It performed a lessons learnt exercise to improve the efficiency of future changes. • University College London Hospitals stated that the contractor's inconsistent helpdesk service made Trust staff reluctant to use it, which meant problems were unreported. Both parties worked together to develop a plan to feedback regularly on the progress of jobs to build confidence in the contractor's helpdesk processes. Source: National Audit Office case studies |