4.6 Process evaluation is often able to identify when a novel policy is encountering initial difficulties in implementation, and so can be useful in ironing out these types of problems. This might mean that it is desirable for an impact evaluation to occur after a process evaluation, as analysts and policy makers can be more confident that the impact evaluation is measuring the policy itself, rather than the effects of delivery problems. However, this is likely to lead to a longer overall evaluation period. Some process and impact evaluations which follow a new policy as it develops can take years to complete, although useful results will usually be obtained throughout the study as well.
4.7 The timing of the evaluation will also be affected by the outcomes affected by the policy and of particular interest to the evaluation. Some impacts might take some considerable time (e.g. years) to appear, and it might be unfeasibly costly to incorporate these into an intensive process evaluation. An impact evaluation, undertaken some considerable time after the policy was implemented, might be the only feasible option for measuring these impacts, but might then be of less value in affecting the way the policy is implemented or rolled out.
4.8 Retrospective impact evaluations using existing data sources, will not generally suffer from the effects of implementation problems, and can sometimes be undertaken in a matter of weeks. However, the tendency to rely on administrative data will generally limit such an evaluation's ability to provide a rounded explanation of why and how any estimated impact actually occurred. Additionally, the timing of an evaluation might need to be aligned with specific requirements for review. Timetabling is particularly important where the evaluation is intended to inform a Sunset Review as it will need to be completed in time for any renewal or amendment legislation to be enacted (otherwise the legislation will automatically expire).1
_______________________________________________________________________________________
1 Further guidance is provided in Sunsetting Regulation: Guidance, HM Government, 2011 http://www.bus.gov.uk