14. The National Audit Office's methodology was set out in detail in the Report to allow scrutiny. The CIPHP main criticisms are that the Report relies on surveys of interested parties, it looks at only the first six projects completed and that the Report is not structured around the four main high level questions it identified. The independent quality review of the Report commissioned by the National Audit Office concluded that a range of appropriate methodologies had been used and gave the Report a good mark- an average of 4.1 out of five. Further information on methodology is set out below.
15. Questionnaires were issued to three main interest groups, principally to gain an understanding of the adequacy of the standardised procurement process devised by Partnerships for Health. These were: all LIFT Project Directors; shortlisted bidders across the 42 LIFT schemes extant at that time; and the Local Pharmaceutical Committees in each case study areas.
16. The National Audit Office aimed to obtain a balanced view of the scheme by consulting widely across parties with differing views on LIFT, although not all agreed to speak to the National Audit Office team. As is normal in National Audit Office studies, the team had access to a panel of experts who were invited to comment on the audit plan and the structure and draft of the Report, thereby providing quality assurance. The panel for the LIFT Report included amongst others a dental practitioner, a general practitioner and the Chief Executive of a PCT.
17. A key element of the evidence was provided from analysis of six case studies (Appendix 1, paragraph 8). The first six schemes to have completed the procurement process were chosen for practical reasons and because they represented all three waves of LIFT. These case studies were used to identify best practice and lessons for future LIFT developments. A wide range of stakeholders were interviewed in the local health economy for each case study and key documents were reviewed. The National Audit Office was aware of concern in some areas about LIFT schemes. It flagged this up in the Report (paragraph 2.4) but the projects were not advanced enough to be analysed as case studies.
18. The CIPHP note suggests that the National Audit Office team was unaware of Department of Health guidance "Learning from Post Project Evaluation" although this guidance is referred to in paragraph 3.8 of the Report. This paragraph of the National Audit Office Report went on to say "there is no clear guidance recommending either its nature or its timing" in relation to the post project evaluation of LIFT. This latter reference relates to the National Audit Office's concern that the Department and Partnerships for Health should issue detailed guidance-as referred to in recommendation 9 of the Report.