6.12 A policy might have wider impacts, such as knock-on or multiplier effects6 in the local economy.7 Developing the logic model of the intervention and considering the various stages in which it is intended to operate provides an opportunity to consider the wider or additional effects of the activity. These can then be incorporated into the evaluation as appropriate.
6.13 There might also be effects which are recognised as possible but not definite outcomes of the policy, and which evaluations will also need to look for. They could be harmful or beneficial and might be generated amongst those directly targeted by an intervention or more widely for others indirectly affected by the intervention. Table 6.A presents examples of potential unintended effects.
Table 6.A: Examples of potential unintended effects
Effect | Definition | Example |
Displacement | Positive outcomes promoted by government policy are offset by a negative outcome of the same policy elsewhere. | The displacement of crime from one area, where a crime reduction policy is being implemented, to a bordering area |
Substitution | The effects of an intervention on a particular individual, group or area are only realised at the expense of other individuals, groups or areas. | An employer appointing a jobless person from a government scheme, rather than a standard applicant, in order to secure a recruitment subsidy. |
Leakage | The policy benefits others outside the target area or group | Jobs generated in a target area are taken by those who live outside it. |
Deadweight | The policy supports outcomes which would have occurred anyway. | An employer receives a subsidy to take on workers who were going to be appointed anyway. |
6.14 A policy might also result in other effects that are completely unanticipated, generally termed "unintended consequences". These often result from perverse incentives which are established as a result of interaction between the way the policy works and existing processes. Box 6.E provides examples of unintended consequences and sources of further information.
Box 6.E: Examples of unintended consequences The effects of licence plate rationing in Mexico Modal shift in travel patterns was primarily to taxis and small buses rather than to subways, offsetting any improvements likely to be achieved by reductions in car travel. Demand for petrol went up after two months of implementation, and Mexico City became a net importer rather than net exporter of used vehicles from the rest of the country. The inference was drawn that residents evaded the restrictions by becoming multi-vehicle households (with variably coded licence plates) and acquiring older (and less fuel efficient and more polluting) vehicles from the countryside. Source: Cambridge Systematics (2007)8 The impact of funding incentives on fire prevention Source: Department for Communities and Local Government (2008)9 Further examples are available from: Additionality Guide, English Partnerships, (http://www.thesroinetwork.org/component/option,com_docman/task,doc_view/gid,30/), Research to improve the assessment of additionality, Department for Business, Innovation and Skills, 2009 (http://www.bis.gov.uk/) and Wider economic benefits in transport appraisal, Department for Transport, (http://www.dft.gov.uk/) |
_______________________________________________________________________
5 For more information on theory-based evaluations, see Guidance for transport impact evaluations: choosing an evaluation approach to achieve better attribution, The Tavistock Institute for the Department for Transport, 2010, http://www.dft.gov.uk/
6 Further economic activity (jobs, expenditure or income) associated with additional local income and local supplier purchases as a result of the intervention.
7 For more information see: Additionality Guide, English Partnerships http://www.thesroinetwork.org/component/option,com_docman/task,doc_view/gid,30 /, Research to improve the assessment of additionality, Department for Business, Innovation and Skills, October 2009, http:/www.bis.gov.uk/; wider economic benefits in transport appraisal, Department for Transport http://www.dft.gov.uk/ ; and http://www.hm-treasury.gov.uk/green_book_guidance_regeneration.htm
8 Congestion Mitigation Commission Technical Analysis: License Plate Rationing Evaluation for the New York City Economic Development Corporation and New York City Department of Transportation, Cambridge Systematics, 2007
9 Safer Houses: Celebrating 20 years of fire prevention in the home, Department for Communities and Local Government, 2008