7.19 Box 25 sets out the differences between undertaking an assessment at the outset, in support of government intervention - appraisal - and undertaking an assessment to evaluate how successful such action has been - evaluation.
BOX 25: COMPARISON OF APPRAISAL AND EVALUATION
| ||
Aim | Ex ante assessment of whether action is worthwhile and impacts | Ex post assessment of whether action was worthwhile and impacts |
Use of Output | Project procurement, policy and programme design | Feedback for: (a) future procurement, project management, (b) wider policy debate, and (c) future programme management. |
Application | Projects, policies and programmes | Projects, policies and programmes |
Timing | Always prior to implementation | ❑ During implementation ('formative') ❑ After implementation ('summative') |
Data | Forecasted | Historic and current, estimated and actual. Estimates of counterfactuals |
Method | Comparison of options against 'do nothing' option | Comparison of results against 'do nothing' option Assessment of risks that did or did not materialise |
Analytical Techniques | Cost Benefit/ Effectiveness Analysis Discounted cash flow analysis Multi-criteria analysis Other statistical analysis | Cost Benefit/ Effectiveness Analysis Discounted cash flow analysis Multi-criteria analysis Other statistical analysis - e.g.: analysis of performance indicators |
Decision Criteria | Comparison of NPV, NPC for different options Non quantifiable factors may be included if quantification impossible | Consideration of whether correct criteria were used |
Audit and Enforcement | Public Accounts Committee (PAC), NAO, HMT, OGC Gateways 0, 1 Departmental arrangements |
BOX 26: EXAMPLE 'EXPANDING VOCATIONAL TRAINING' - QUESTIONS FOR EVALUATION:
❑ To what extent did the anticipated costs and benefits match the actual outcome ('benefits realisation')? ❑ In the light of experience with the target group of trainees, would better results have been achieved if this group had been more tightly defined, e.g. the alternative option of focusing purely on low or unskilled workers? ❑ Has any new information about the impact of vocational training come to light since the policy was implemented? (i.e. how effective is it in meeting objectives)? ❑ Were the risks assumed for completion of the training course justified or did they understate/exaggerate the true risk? ❑ Control group - how does the productivity of those individuals who undertook training compare to the productivity of workers of similar skill who were not offered training? |