The risk of incorrect performance

2.35  In PFI projects the recording of performance is usually undertaken by the contractor's staff to enable them to remedy service problems and to reduce the burden of performance recording on the public sector. As a control, the public sector should normally obtain user feedback, and discuss this and the contractor's performance data with the contractor on a regular basis. These methods of monitoring performance normally work well. In the case of the Main Building Redevelopment project the Department and the contractor have not yet found the right balance between self-reporting by the contractor and audit by the Department and are working on a solution. As we noted in our earlier report on this project (Ministry of Defence - Redevelopment of MOD Main Building, HC 748 2001-02) it is appropriate for the Department to keep performance reporting systems under review to ensure the integrity of the performance information they receive.

 

16

Performance mechanisms' effectiveness at incentivising the contractor

 

The performance mechanisms we have in place incentivises the contractor to provide the service required by the contract.

Source: National Audit Office census of all Ministry of Defence PFI projects

 

2.36  These methods of reporting performance also carry a risk that private sector staff might not record lapses in performance where the lapse is unlikely to be detected by the public sector but, if recorded, would result in payment deductions.

2.37  In the Defence Fixed Telecommunications System project, we found a specific instance of the private sector fraudulently recording performance in such circumstances. The contractor, BT, was required to meet targets for the percentage of calls answered within specified time limits and they would be liable for payment deductions if the targets were not met. BT staff at a call centre in Kettering, now closed, artificially inflated the number of calls being answered within the required time limit by calling each other.

2.38  This fraudulent recording of performance was not detected through the performance management system and only came to light subsequently within BT. BT senior management have informed us that they were not initially aware of the error in performance reporting which they consider arose from the misconduct of a small number of staff working at the call centre. As the inflation of calls being answered did not result in an adverse experience for the Department's users of the telecommunication services there were no complaints or adverse comments in customer feedback surveys to trigger an investigation. This type of fraud can only be identified by audits of the performance monitoring systems and spot checks.

 

2.39  The Department and BT reviewed the problems and have agreed the compensation due to the Department. As a result BT has made retrospective service payments to the Department of £1,021,000 and has reimbursed the Department for the external cost incurred by the investigation totalling £122,000 excluding VAT and for the value of the overstated calls totalling £197,000 excluding VAT.

2.40  Following the fraud the Department and BT have imposed a new management structure to the Defence Fixed Telecommunications project including new governance arrangements. BT is now required to provide more detailed reporting and the Department will carry out regular detailed audits of the new BT reporting system to ensure its integrity. This will be supplemented by regular service audits by the Department. Certain BT staff involved in the activities of Kettering call centre have been released by BT.

 

17

Case study projects' performance mechanisms

 

Project

Comment

 

 

Medium Support Helicopter Aircrew Training Facility

Field Electrical Power Supplies

The payment regime appears effective but the performance and payment mechanisms are very complex.

 

 

Defence Fixed Telecommunications System

There were specific problems regarding the reporting of performance on the Defence Fixed Telecommunications System project with the private sector fraudulently recording performance.

 

 

Main Building Redevelopment

The payment regime generally appears effective. However the Department believes that the penalties incurred by the contractor to date for the occasional service lapses that have occurred were too small. The Department also recognised that its involvement in service monitoring caused antagonism between the public and private sector. The Department therefore suspended its service audits in September 2006. Both the contractor and the Department are determined to maintain and improve performance standards and to introduce a more effective audit methodology. This had not been implemented by the time of our audit in autumn 2007 although the Department and the contractor are continuing to work together on this issue.

 

 

Heavy Equipment Transporter

Tidworth Water and Sewerage

Few performance deductions, if any, have been necessary to date. Payment mechanisms have some indicators, such as those governing reliability and availability of the asset, which will become more challenging as the contract progresses and the assets age.

 

 

Defence Animal Centre

The performance mechanism was poorly designed. There are a large number of potential performance indicators, not all of which are relevant to the delivery of the core service. In addition, deductions for the asset not being available are set at a low level and are much less than the financial cost to the authority of service failure. As a result, the payment and performance mechanism lacks credibility and has been the cause of dispute between the authority and the contractor.

 

Source: National Audit Office analysis of case study projects