The Public Sector Comparators have been carefully prepared
16 The process of constructing the London Underground Public Sector Comparators involved three stages:
■ establishing a methodology;
■ producing an unbiased estimate of the costs to meet the required level of performance over 30 years; and
■ estimating the range of risks that would face a public sector infrastructure operation.
17 London Underground established a clearly documented methodology, which they published in March 2000. This methodology stated that the Comparators would estimate the costs of the public sector meeting the PPP performance specification. The methodology also stated that London Underground would estimate base costs, the risk margin to add to those base costs as a result of uncertainty and the likely efficiencies London Underground could achieve over time. Its intention was to ensure that risks for public sector procurement were fully and appropriately quantified, without either managerial bias towards optimism or an automatic assumption that any past shortcomings in managing risk will continue.
18 The base costs to meet the PPP performance specification have been carefully estimated, on the basis of London Underground's existing costings for individual assets, after attempting to strip out any contingency or margin on
1 |
| The Process leading to the deal |
those costings. This approach involved considering the scope of physical works designed to meet the performance specification to produce a Long Term Investment Plan. It appears to us that London Underground's approach to estimating the capital spending programme has been a reasonable one, although lack of past knowledge about asset conditions means that margins for uncertainty are inevitably substantial.
19 These base costs have been adjusted to reflect possible risks and efficiencies that could arise for a public sector operation. The risks were estimated on the basis of an analysis by engineering experts of London Underground's historic record of cost overruns. For example, on design and construction risk this involved consideration of some 242 individual projects, with an average overrun of 20 per cent for various reasons including client changes of scope and funding constraints. Engineers judged that some of the more extreme overruns observed on projects were genuinely one-off and could not be taken as indicative of likely future achievement. Other overruns were judged to be irrelevant, such as the costs of new tunnelling on the Jubilee Line Extension since no new tunnelling work is envisaged in the PPP requirement.
20 Engineers also estimated the likely efficiencies that London Underground could reasonably expect to achieve over the next 30 years. They concluded that some moderate cost savings could be anticipated, especially for investment in the station infrastructure. However, in other areas, such as investment in new rolling stock, they assumed that little improvement was possible because of the extensive use of PPP-style contracts in those areas already.
21 One of London Underground's key judgements was about its ability to meet performance targets. London Underground concluded that in the past, it has failed to meet the performance improvements expected from new investments, because of its failure to integrate systems effectively. It further concluded that these performance failures would occur in future investment projects, because learning from one project was not easily transferred to the next. As a result London Underground concluded that the Comparator should include the notional additional costs that such failures to meet the performance specification would impose on London's passengers, using its standard cost benefit techniques. This adjustment also applies to bidders.
22 Figure 2 summarises the risk and efficiency categories used by London Underground.
2 |
| The risk and efficiency categories used by London Underground |
Factor | Description | Impact |
Relative price effect | Reflects the uncertainty surrounding the impact of inflation on different categories of cost | Small increase in costs |
Unit cost uncertainty | Reflects the uncertainty surrounding the estimates of unit costs within the basic cost estimates | Small increase in costs |
Investment cost uncertainty | Reflects uncertainty over the proposed scope of investment projects, which, in London Underground's experience, always increases as the project progresses | Large increase in costs of more than 10 per cent |
Maintenance uncertainty | Reflects uncertainty over the proposed scope of maintenance projects | Increase in costs |
Performance uncertainty | Reflects the risk that London Underground's investment projects fail to deliver expected improvements, leading to a failure to meet the PPP performance specification | Large increase in costs, mostly estimated as the economic cost of delays to passengers |
Efficiency uncertainty | Reflects efficiencies that London Underground could reasonably expect to achieve over the next thirty years. | Reduction in costs |
Source: National Audit Office | ||
23 London Underground did not quantify all of the key risks that it identified. Instead, it summarised them in a commentary on the Comparators provided to decision-makers. For example:
■ the risk that legal disputes may arise under the PPP approach has not been quantified formally because it would be very difficult to estimate reliably.
■ the potential positive impact of the introduction of the PPP structure on the efficiency of the operating company retained in the public sector has also not been quantified.
24 London Underground has updated the Comparators since March 2000. Revisions have been well documented, and revising Comparators as more information becomes available complies with Treasury guidance and best practice. As a result of these revisions, the value of the Comparators has increased significantly between the shortlisting stage and the Best and Final Offer stage. The main reasons for the increase have been:
■ net increases in the operating base costs (around 20 per cent of the increase) as a result of more up-to-date information on the costs of the infrastructure services and revised estimates of administrative costs, for example relating to existing PFI contracts for power, ticketing and communications;
■ changes in the scope of the PPP specification that bidders are being asked to meet (around 20 per cent of the increase); and
■ bringing the method of quantifying performance risk in line with that proposed for the evaluation of bids (around 60 per cent of the increase).
25 London Underground is obtaining independent assurance on the preparation of the Comparators from several sources. London Underground was supported by its engineering consultants, Ove Arup, who stated that they consider the Comparators represent a fair estimate of the costs of meeting the output specification, and by its financial advisers, PricewaterhouseCoopers. KPMG, London Underground's statutory financial auditors4, concluded that the methodology for the Comparators complied with relevant Treasury guidance and is expected to report that the final Comparators represent a reasonable basis for estimating public sector costs. Finally, London Underground has also acted on our own comments about its approach to the financial analysis.
_____________________________________________________________________________
4 KPMG are also auditors of London Transport and its successor, Transport for London.