(i) Project sampling
The NAO sample was restricted to the first projects to close. As such, the NAO's sample may not represent the full picture of the procurement. Looking only at the projects that were most successful in terms of the procurement process makes it hard to assess how well LIFT projects are progressing generally. The report might have been more representative had it included analysis of LIFTs where organised opposition had started to develop.
The NAO did receive letters from people complaining about their local LIFT schemes but the report did not include these cases in its evaluation.
(ii) Selection of informants
The NAO's choice of informants could lead to biased results. In the two national surveys, it only contacted private sector bidders and LIFT project directors. The experiences and views of staff and service users were not incorporated. Both private sector bidders and project directors have an obvious interest in providing the NHS with a positive account of their projects. This is also, of course, the case with PFI and indeed public procurement processes. However, unlike PFI and public procurement, LIFT involves the promise of future projects, and it may therefore be particularly difficult for those involved to provide objective evidence.
(iii) The contracting out of the financial analysis
The Committee may question the legitimacy of outsourcing the review of the financial models to Operis, a consultancy company which advises private sector bidders and banks involved in PPPs.
(iv) Evaluation of performance
The NAO notes that all of the LIFT schemes it studied have failed to conduct proper post-project evaluations. Of the six LIFT projects examined by the NAO, just one had developed a post-project evaluation plan. However, the NAO fails to point out that these LIFT schemes-and the DoH itself-are consequently in breach of published guidance. The NAO seems not to be aware of this guidance. Of evaluation, it says there is "no clear guidance recommending either its nature or timing" (page 30, para 3.8).
But this is wrong. In January 2002, the DoH published guidance to assist NHS bodies involved in capital schemes in the process of evaluating their completed projects. This guidance, The Good Practice Guide: Learning Lessons from Post-Project Evaluation, states that such evaluations are "an essential aid to improving project performance, achieving best value for money from public resources, improving decision-making and learning lessons" (page 1).
This guidance sets out a four-stage process of evaluation and a number of technical considerations. In addition, it advises NHS bodies to carry out an initial post-project evaluation of project outcomes six months after the facility has been commissioned.
It would appear that the LIFT schemes studied in this report are in breach of this guidance, since in most cases some buildings have been operational for more than six months. In addition, since all LIFTs involve Partnership for Health-a representative of the DoH-arguably the DoH is itself in breach of its own published guidance.
While the details of this guidance have the status of "advice" to NHS bodies, the requirement to evaluate and learn from projects is in fact mandatory for all DoH projects with a cost in excess of £1 million. Further, guidance specific to LIFT states that LIFTCos should "regularly monitor and report the standard of performance" of the services they provide (Standard Strategic Partnering Agreement, version 4, Section 2).
(v) Value for money comparison
The NAO does not produce direct quantitative comparisons with public sector finance or with GP-managed developments. The NAO takes the DoH line that value for money can be demonstrated through the running of a competitive procurement, in addition to some benchmarking and an assessment of rents by a district valuer.
We would question this proposition, which runs against the process operating under PFI, in which a public sector comparator is used to test the value of the PFI proposal against a theoretical publicly financed scheme.
It could be argued that the PSC system is not appropriate for LIFT schemes, since public finance is rarely available for large-scale capital investment in primary care. Instead, investment has primarily been through debt-financing in the form of interest-free loans from the General Practice Finance Corporation (GPFC). These loans are paid back by the NHS in the form of a number of different types of subsidies to GPs.
However, there is no reason why the government could not produce comparators, based either on public financing or financing through the GPFC in order to provide information about base costs. The production of a fully costed "theoretical" publicly financed project has taken place within the mainstream PFI programme since the initiative's conception.
In addition, it seems the NAO is unaware of recent examples of public sector funded health centres, such as those built with London Implementation Zone grants. Anecdotal evidence suggests that these projects have been very successful. For example, Greenwich's Fairfield Grove Health Centre was highly commended by the Commission for Architecture and the Build Environment (CABE 2002).
(vi) Risk transfer and the rate of return
The NAO presents little data with which we might make inferences about the value for money of LIFT projects. This is despite the fact that local NHS bodies were sceptical. For example, some of the Strategic Health Authority representatives in the NAO's case study areas expressed concerns that initial business cases did not sufficiently explore the risks of LIFT, and that it was hard to have complete assurance about value for money for an untried initiative.
Meanwhile, the quantum of risk transfer is not explored by the NAO, despite the higher rates of return in LIFT (case study range is from 14.3% to 15.9%) compared to the average in PFI schemes (12.5% to 15% in the NAO's comparator PFI schemes). Despite this, the NAO agrees with the government that LIFT is "clearly value for money".
(vii) Affordability
Affordability refers to the income required from the public sector to pay the unitary charge to the LIFTCo. The NAO Report does not produce any affordability calculations, despite the fact that GPs have expressed concerns that smaller GP premises may lose funding because the higher lease costs of LIFT schemes within the locality are tying up resources (Pulse magazine, 30 April 2005).
In addition, representatives from the National Pharmaceutical Association, the British Dental Association and local authorities told the NAO that they had concerns about rental costs. Indeed, the NAO comments that "there is a common perception from these groups of prospective tenants that the higher cost of LIFT, compared to current rent payments, outweighs the benefits of new, purpose built premises" (page 21, para 2.14). The Report makes no final conclusion on affordability.
(viii) Conflicts of interest
The NAO makes some criticisms in this area and recommends stronger management arrangements. Its report expresses concern about the potential conflicts of interest, in particular the issue of PCT directors sitting on the LIFTCo Board. It comments (page 32): "if the LIFTCo was in financial diffculties, as a LIFTCo director a Primary Care Trust employee might have conflicting pressures between helping the LIFTCo and protecting the interests of the Primary Care Trust" (para 3.13).
The NAO points out (page 32, para 3.13): "the public sector director, in the role as a LIFCo Board Member, has a fiduciary duty to act in the interests of the LIFTCo and not for the Primary Care Trust." The NAO provides no evaluation of how these diffculties have impacted on issues such as accountability, transparency and the avoidance of conflict in the governance procedures of the six LIFT areas studies.