3.4 Research designs usually estimate the counterfactual by ensuring that there are some individuals, groups or geographical areas not exposed to the policy at some point during its implementation. A comparison can then be made between those who have been exposed to the policy and those who have not. A simple example of this is a medical drugs trial where one group of participants (the "treatment" group) receives a new drug and the other (the "comparison" or "control" group) receives a placebo. Who actually receives the drug or the placebo is decided by chance, through a formal randomisation process. Then, so long as the treatment and control groups are similar in all other relevant respects, they can act as comparisons for one another. If there is then any difference in observed outcomes between the two, it can reasonably be assumed (under certain technical assumptions) that the difference is due to the policy (treatment).
3.5 There are two obvious difficulties with applying this simple scenario to the public policy context. First, those areas or individuals who receive policy "treatment" in practice do tend to be different from those that do not in quite obvious and relevant ways. Crime reduction policies tend to be implemented more often and intensely in areas with higher crime rates. Individuals who enrol on employment assistance programmes tend to be those who have lower work skills, lower educational achievement and live in areas with poorer economic performance and prospects. Those who choose to stay in treatment for drug misuse tend to be those who are more motivated to improve their lives and reduce the costs of their drug problems. Then, the difference between the treatment and control groups will not just be that one received the intervention and one did not, but all of the other differences in underlying characteristics. The comparison will be between "apples and pears", and it will not be possible to tell whether differences in observed outcomes between the two groups are due to the intervention or something else.
3.6 Second, social policy interventions do not tend to be administered to the policy target group randomly, with no regard to perceived need, justification and so on. So there is not generally a group of untreated subjects who could have been eligible for the intervention but were purposely denied it. Those that do not receive an intervention tend to be those for whom it is deemed unsuitable, and will therefore be systematically different from those who are. So there is unlikely to be a readily available comparison group of non-treated individuals who are similar to those who do receive treatment.