4.3 Quality control and quality assurance are crucial for any evaluation. Without these, the methods and results from the evaluation cannot be guaranteed to be of sufficiently high standard or fit for purpose. This means the resulting evidence is not robust enough to provide answers to the questions the evaluation was designed to resolve or to reliably inform the decision making process. Quality control can be described as follows:
• quality control ensures that the evaluation design, planning and delivery are properly conducted, conform to professional standards (such as ethical assurance), and that minimum analytical standards are adhered to;
• quality control will be informed by the governance community (e.g. a steering group), other stakeholders, the evaluation team, the manager of the evaluation within the commissioning body, external reviewers, and the commissioned research team where appropriate; and
• quality control will ensure consistency in data collection, methodology, reporting and interpretation of findings.
4.4 Without good quality control, the conclusions of an evaluation cannot be relied upon. Quality control and assurance should therefore be built into an evaluation. This will mean that any weaknesses in methodology, design, data collection and so on can be identified and understood early enough for changes to be made and adverse effects on results or reliability avoided or reduced. This can be achieved by applying existing departmental quality criteria and processes for research and evaluation, and working closely with government analytical and evaluation specialists. The manager of the evaluation within the commissioning body should take responsibility for applying quality control criteria. The use of external assessors and/or peer review can also be useful and is often standard practice.
4.5 Four particular issues are often critical in managing an evaluation in a way that satisfies quality principles and criteria - ensuring independence, inclusivity, transparency and robustness:
• researcher independence and objectivity are essential for any evaluation. However, this does not automatically necessitate the use of external contractors or keeping the evaluation team at arm's length. This is because close interaction between the research team and policy colleagues while retaining independence and objectivity is important in delivering an effective evaluation;
• inclusion of recipients, delivery bodies or stakeholders - through a steering group, for example - enhances the potential learning from an evaluation and acceptance of its results, but it has to be actively managed as a continuous process of communication and engagement. This is likely to involve: improving awareness of the evaluation; obtaining feedback on research design; and communicating scoping, interim and final conclusions;
• transparency must be a feature of any evaluation but especially for a high-risk or innovative policy intervention. An evaluation plan can set out the evaluation objectives and questions, how the evaluation will be conducted, the timescale and how the findings will be acted upon. In turn, this will facilitate stakeholder engagement, allow the issues and risks to be identified and managed, and the delivery outputs and milestones to be agreed and documented. Evaluation reports should be published and contain sufficient technical detail for others to judge for themselves the robustness of the findings; and
• robustness in research plans and/or the final report is assessed against required analytical standards so that there is an assessment of a) whether the planned research is likely to provide robust evidence to answer the research questions and/or b) that the research findings and conclusions are presented and reported accurately and clearly.