3.2 As discussed in Chapter 2, one of the keys to good impact evaluation is obtaining a reliable estimate of the counterfactual: what would have occurred in the absence of the policy. This is frequently a significantly challenging part of impact evaluation, because of the often very large number of factors, other than a policy itself, which drive the kinds of outcome measures relevant to public policy (e.g. increased employment, falling crime, reduced prevalence of obesity). There are various approaches to impact evaluation (sometimes termed research designs) which can be used to attempt to isolate the impact of the policy from all these other drivers. The success of these approaches largely depends on their ability to establish a counterfactual through obtaining what are called "comparison (or control) groups". This in turn is critically affected by the way the policy is "allocated", that is, who or where receives the policy and when.
3.3 In other words, the design and implementation of a policy affects how reliably it can be evaluated, and even quite minor adjustments to the way a policy is implemented can make the difference between being able to produce a reliable evaluation of impact and not being able to produce any meaningful evidence of impact at all. This chapter briefly explains the role of comparison groups in improving how well a policy can be evaluated, and then provides some simple examples of how minor policy adjustments can improve the chances of a reliable evaluation. It finishes with a consideration of the factors which might be taken into account when deciding whether such adjustments might be appropriate