Introduction

6.1 Establishing a framework for the evaluation provides a consistent and systematic means to designing the evaluation, collating and analysing the existing evidence and the new data created, and generating and interpreting the results. It can be used to understand what existing evidence tells us and to identify those gaps in the evidence base which the evaluation should focus on. As suggested in Chapter 5, the evaluation framework is most likely to be based on some form of logic model. This chapter provides more detail on logic models, and how they can be used and developed into a theory-based approach. It also considers some of the techniques which can be used to review and evaluate existing evidence.

6.2 Even if there was a significant body of evidence and experience on which to draw, the rationale for an intervention will have been based to some extent, and in certain aspects, on assumptions about how the inputs will cause the intended outcomes and impacts and what other contextual factors will influence this. These assumptions and the evidence on which they were based can be set out formally in a structured "logic model", which can provide the framework within which the impacts of the intervention can be evaluated and (if appropriate) quantified.

6.3 A logic model describes the causal pathways underlying the rationale for a policy. It does this by linking the intended outcomes (both short and long-term) with the policy inputs, activities, processes and theoretical assumptions. Box 6.A presents the simple Kellogg Foundation logic model, and provides definitions (with examples) of its various components.

Box 6.A: Logic models and the terms they use

A logic model describes the theory, assumptions and evidence underlying the rationale for the programme . . . "it links outcomes (both short and long-term) with programme activities/processes and the theoretical assumptions/principles of the programme." Source: WK Kellogg Foundation (2004)1

KelloggFoundation Logic Model

Term	Definition	Example
Inputs	Public sector resources required to achieve the policy objectives	Resources used to deliver the policy
Activities	What is delivered on behalf of the public sector to the recipient	Provision of seminars, training events, consultations etc.
Outputs	What the recipient does with the resources, advice/ training received, or intervention relevant to them	The number of completed training courses
Intermediate outcomes	The intermediate outcomes of the policy produced by the recipient	Jobs created, turnover, reduced costs or training opportunities provided
Impacts	Wider economic and social outcomes	The change in personal incomes and, ultimately, wellbeing

6.4 Developing the logic model can be done as a desk exercise, based on a review of policy documentation such as the Impact Assessment, business case or project initiation documents. It may also be developed using previous evaluations and evidence about particular aspects of the inputs, activities, outputs, outcomes and impacts. It might draw on relevant theoretical and empirical frameworks describing the (different links of the) model's impact pathways. Examples could include economic models of individual market behaviour, bio-physical models of the impact of air pollution on the environment, and inter-disciplinary models of how changes in health status affect social and psychological wellbeing. Some examples of logic models are provided in Box 6.B which demonstrates that they can be formulated in different ways, albeit around the same basic structure.

_______________________________________________________________________

1 Logic Model Development Guide, WK Kellogg Foundation, 2004