Chapter 5. The National Audit Office (NAO) reports on PFI projects in the health sector - the with-and-without test for VFM

Apart from setting up a model through which it could test for the sensitivity of the PFI projects to the different costs of capital, the Andersen report looked at a number of reports on PFI projects carried out by the National Audit Office (NAO). In trying to assess Value for Money, the NAO approach has been to do a with-and-without comparison - that is a comparison of the project with and without the PFI. The big unknown in this comparison is the 'without', namely the Public Sector Comparator (PSC) and the big question is; what would the project's capital cost have been if financed and contracted directly by the public sector? As the Andersen report puts it;

"The process does place considerable weight upon the PSC in making a value for money judgement. The credibility of these judgements (about the process for comparing public and private sector projects) would be further enhanced if the process were made more transparent to the public" (Andersen 2000, 5).

But the process has been far from transparent - neither in the Andersen report nor in the subsequent NAO reports.

The Andersen report pointed out that the NAO reports on PFI projects had generally been pro-PFI (Andersen 2000, 6). Andersen pointed to an average capital cost saving of 20% across the seven projects examined by the NAO up to December 1999 for which a Public Sector Comparator (PSC) was produced. In other words the PSC was 25% higher on average than the PFI construction cost. These seven projects were across a range of sectors, but unfortunately no details were given as to how the PSCs were arrived at.

Only one of the seven projects was in the health sector. This was the construction of the Dartford and Gravesham hospital. The Andersen report points out that the Dartford and Gravesham hospital became operational considerably faster than most publicly-financed hospitals and that, compared to the PSC, it saved about £5 million (Andersen 2000, 18). But note that this saving was less than 3% of the PSC cost and as Jon Sussex has put it;

"Such a small amount must lie in the margin of error for estimating the cost differences between the PFI option and the public sector comparator" (Sussex 2001, 82).

Furthermore it is also worth noting that although the PSC figure included an allowance for risk, this allowance was not itemised and it is also worth noting that in the Dartford and Gravesham case, the PSC was set by the Trust at £194 million and then reduced by £12 million after errors were discovered by the NAO (see Andersen 2000, 18). It was after this adjustment was made that the margin of advantage came down to only about £5 mn.

The Andersen report itself did a review of 29 PFI projects (across a range of sectors but none of them in health) for which a PSC was available. The report claimed that there was a cost saving of about one-sixth by going down the PFI track but, once again, few details were given of how the PSCs were arrived at and what allowances were made for risk. The report does tell us that risk transfer valuations were available for only 17 of the 29 projects and that for 6 of these 17 projects, the estimate that the project would deliver value for money was entirely dependent on the risk transfer valuation (Andersen 2000, 54). But the details as to how the allowances for risk were derived are not given. Furthermore Andersen did not use an unbiased selection process since the sample was not random (Sussex 2001, 81).

There are three problems to watch for in making VFM comparisons.

The first is that the comparison needs to be made at the same stage, namely outline business case or full business case. As Hellowell and Pollock have shown for 43 large PFI hospital schemes, the capital costs at the final contract stage were an average of 74% above the estimated costs at the outline business case stage. The escalation for the NNUH was 28.5%; from £122.9 mn at the OBC stage to £158 mn at the final close (Hellowell and Pollock 2007, table 6).

The second is that the PFI capital cost should include financing during the period of construction. Table 1 of Gaffney et al 1999 compared the construction cost with and without financing costs (during the period of construction) for a sample of seven PFI hospitals. The table showed that the average difference as a percentage of the base construction cost was 36%. The percentage for the Norfolk and Norwich University Hospital (NNUH) was 49% with the base construction cost being given as £143.5 mn compared with a final construction cost (including financing costs during the period of construction) given as £214 mn. But as can be seen from table 5.1 below, some of this increase of £70.5 mn consisted of highways (£5 mn), of IT hardware and systems (£8 mn) and of catering and other equipment (£4 mn). Excluding these three items which total £17 mn, the increase at the NNUH was about £53.5 mn or about 37% of £143.5 mn comparable with the average for the group in the study by Gaffney et al 1999

The third problem is that the comparison should be over the life of the contract so that the higher interest rates associated with the PFI are taken into account. Chapter 4 of this report showed that, using the Andersen model, PFI construction costs need to be about a half lower than those of the Public Sector Comparator in order to compensate for the higher interest costs of the private sector (based on 3.5% pa discount rate over 39 years).

If all three problems are taken into account, it is highly unlikely that the PFI will be superior and yet, in practice, the comparisons invariably have been fiddled in such a way to show that the PFI alternative is superior.

Such fiddles are shown in table 4 of the paper by Gaffney et al 1999 in which the comparative figures are given for five PFI projects (at Calderdale, Carlisle, Dartford, Durham, and Wellhouse) in the health sector. The figures are from the original Full Business Cases. In all five cases except Wellhouse, the PSC cost was lower than the PFI cost, before any adjustment for risk was made. The addition for risk made to the PSC for these five hospitals averaged £38.8 mn. compared to an average base construction cost of the private sector (calculated from Table 1 of Gaffney et al) of £68.8 mn. Therefore the addition for risk averaged 56% of the base construction cost of the PFI. That is the PSC cost had to be 56% higher than the PFI cost to compensate for the higher interest cost of the private sector.

Since the discount rate used was 6% pa, this is what we would expect from our earlier analysis from table 4.3 using the Andersen model. The calculations of that table showed that the PSC cost needs to be about 61% higher than the PFI alternative for the latter to be superior and provide value for money.

The paper by Gaffney et al 1999 (see table 4) pointed out that only after 56% was added on to the PSC construction cost were the PFI alternatives superior. Gaffney et al went on to point out that the 56% addition for risk compares with an average increase (in the 1990s) in cost over approved tender sums for NHS capital projects of between 6.3% and 8.4%.

Similar points are made in a paper by Pollock et al 2002 which looked at six health sector projects. In all of these six cases, the margin of advantage was in favour of the PSC before any adjustment for risk, but the risk adjustment was sufficient in all cases to tip the balance in favour of the PFI version (see Pollock et al 2002, table 4)

All of the above comparisons were based on a 6% pa discount rate. If the Treasury's post-2003 discount rate of 3.5% pa had been used, the PSC would almost certainly have been the preferred alternative. It is likely then that at the 3.5% pa rate of discount recommended by the Treasury since 2003, all of the schemes referred to above would have had a margin of advantage in favour of the publicly-financed alternative. This was a point noted by the Select Committee on Health in 2002 when it stated that;

"The VFM margin between PFI projects and the PSC is relatively slim, and according to the Department [of Health] averages out at 1.7%. Therefore, if the discount rate is revised downwards by a couple of points it could make the PFI route the more expensive option - all other things being equal, a lower net discount rate favours public procurement over PFI" (UK Parliament 2002, paragraph 88)

Similarly Thomas Blaiklock, a PFI/PPP consultant writing in the British Medical Journal, noted that the PFI version of the West Middlesex Hospital which had a margin of advantage of about 4% at a 6% pa discount rate would have been more expensive by about 14% at a 3.5% pa discount rate (Blaiklock 2003). Interestingly, before the risk adjustment, the discounted PFI capital cost at the West Middlesex was estimated to be 50% above the PSC - similar to the excess derived from the Andersen model.

Generally it is far from clear how the PSC has been arrived at in the Business Cases and it is also far from clear how the risk premium has been arrived at. There are basically two issues concerning risk. First, does the private sector carry the risk? Second, what is the risk being eliminated? In other words, what is the likely construction cost overrun (or what is often called 'optimism bias', the optimism referring to the public sector coming in at cost and on time) for a project financed by the public sector? The next chapter looks at these issues.