2.13 The preceding discussion has suggested a number of factors which should be considered when deciding what type of evaluation is appropriate for any given intervention. The first is the type of information required about the policy intervention, that is, the questions the evaluation needs to answer. Process and impact evaluations can sometimes consider similar issues and questions - a process outcome (e.g. the number of job interviews following a training scheme) can also be an "impact" outcome (e.g. the overall increase in the number of job interviews for the trainee group).
2.14 There is then the additional consideration of what sort of answers process and impact evaluations can provide. This chapter has portrayed the answers from process evaluations as more descriptive, and the answers from impact evaluations as more definite and in some sense "robust". This is because good impact evaluations attempt to control for all the other factors which could generate an observed outcome (that is, they attempt to estimate the counterfactual). But again, the distinction between the two is not as simple as this suggests. Chapter 3 provides more information about impact evaluations.
2.15 This is because the importance of controlling for these other factors depends on how many there are and how likely they are to affect the result of interest. If the relationship being examined between the policy and the desired outcome is a simple and direct one, there might be few intervening factors and the need to take account of them by estimating the counterfactual with some form of control group might be slight. In these cases, the more descriptive assessment provided by a process evaluation might be sufficient to give a robust answer about whether the policy delivered its desired outcome. However, if the relationship is complex, with many factors potentially affecting the outcome(s) of interest, a more descriptive approach is unlikely to be able to account for all these factors reliably, and a more formal attempt to estimate the counterfactual will be necessary.