5.76 Costs and benefits that have not been valued should also be appraised; they should not be ignored simply because they cannot easily be valued. All costs and benefits must therefore be clearly described in an appraisal, and should be quantified where this is possible and meaningful.
5.77 Research may need to be undertaken to determine the best unit of measurement. Alternative non-monetary measures might be considered most appropriate (See Box 17). For example, one of the benefits arising from a transport improvement is likely to be 'time saved'. These savings must be measured before attaching an aggregate monetary value. In many cases, more than one measure will need to be included to capture the different impacts of the proposal, and the different dimensions of those impacts. For example, there are a number of quantitative indices based on loudness, duration and variability of noise levels. Valuation techniques for use in these circumstances and examples of their application are set out in Annex 2.13
BOX 17: EXAMPLE OF NON-MONETARY QUANTIFICATION: DESIGN QUALITY INDICATORS
| The Design Quality Indicator (DQI) is a method for assessing the design quality of buildings, which can be used by stakeholders involved in the production and use of buildings, including building users and visitors, and practitioners engaged in the commissioning, design, planning, production and management of the built environment. The DQI can be used at any stage in the development process, from setting the brief, evaluating design proposals, during construction, and when a building is complete, to set and check that the intentions for the quality of the building are being met.14 |
5.78 The most common technique used to compare both unvalued costs and benefits is weighting and scoring (sometimes called multi-criteria analysis).The basic approach to weighting and scoring involves assigning weights to criteria, and then scoring options in terms of how well they perform against those weighted criteria. The weighted scores are then summed, and these sums can be used to rank options. An even simpler method is to list the required performance criteria (sometimes called 'critical success factors'), and assess options in terms of whether they meet them or not.
5.79 In practice, the weight to give to factors that are thought to be important by key players cannot be decided by 'experts'. They inevitably incorporate the judgments of stakeholders and decision makers. The risk that they are weighted towards acceptance of more expensive solutions by those who would enjoy the potential benefits should be tempered by at least one stakeholder representing the opportunities that an expensive solution would be foregone elsewhere. There are other pitfalls to avoid in carrying out this type of analysis, and reference should be made to guidance on multi-criteria analysis.15
BOX 18: EXAMPLE - WEIGHTING AND SCORING
| In order to support the introduction of a new training programme, and other departmental objectives, a new IT system is required. A budget of £900,000 is available. The project team discussed with managers and staff the relative importance of the unvalued benefits required of the new system, and submitted proposed weights to be used in the evaluation to the project board, which approved them. For the purposes of this example, only two of the benefits are shown.
Three options, based on different systems, were being appraised. Each member of the project user group provided a score for the user friendliness of the systems. Managers provided scores for the management information the systems provided. These were averaged to provide the following scores:
Option A has the highest score, but costs 25% more than option B, and 67% more than Option C, and is 11% greater than the available budget. Should it be accepted? On a further analysis, a conservative estimate was that the time saving for staff from the user friendliness of the system, where Option A scores most highly, would come near to, or equal, the 67% extra cost over Option C. On the other hand, the additional management information - Option C's strong point - could not be substantiated as leading to general improvements in performance. Option A also retained the flexibility for additional management information tools, which could be considered later as part of a separate, smaller business case. On this basis, the Finance Director decided to fund Option A. |
_____________________________________________________________________________
13 Reference can be made to the website of the Office of the Deputy Prime Minister: http://www.odpm.gov.uk/.
14 For further information refer to http://dqi.org.uk, http://www.cic.org.uk, or http://www.cabe.org.uk
15 An introduction to multi-criteria decision analysis-weighting and scoring - is given in Multi-Criteria Analysis. A manual available from the ODPM website: http://www.odpm.gov.uk (see DTLR archive)