4.2.2 Develop an analytical framework

Once responsibilities have been assigned and the aim of the ex post evaluation study has been defined, it will be necessary to decide which analytical framework will be most appropriate for achieving the aims of the study. Guidance 115 This implies defining:

• the evaluation criteria and expected outcomes of the project; and

• the appropriate alternative (i.e. what would have happened if the project had not been implemented as a PPP?).

• A well-designed PPP contract should provide for sufficient information, collected during the monitoring phase, to support this evaluation exercise.

PPP projects will normally be defined in terms of value for money. This implies identifying both the benefits derived from project outputs and the cost of delivering those outputs (benefits and costs being both monetary and in terms of timing). However, more qualitative benefits and costs, such as service quality, contract design and risk allocation, also need to be considered in the evaluation.

In addition to examining the benefits and costs, the evaluation will need to identify which alternatives should have been looked at. These can be alternative procurement models to PPPs or different project delivery and implementation procedures. It is common practice to use the public sector comparator as a relevant alternative. Guidance 116, 117

CHECKLIST: Ex post evaluation

		An ex post evaluation of a PPP project can focus on many aspects including, for example, the design and performance of a PPP contract or an audit of the procurement process. Examples of the type of questions to address when designing and undertaking an ex post evaluation are:

		Has agreement been reached on the primary focus of the ex post evaluation? Is it the behaviour and effectiveness of the Authority; the effectiveness of the procurement method chosen (including negotiations and contract award); the effectiveness in the performance and management of the PPP contract; or all of the above?

		Have the information needs for ex post evaluation been identified and included in the PPP contract to enable adequate information to be gathered during the performance phase?

		Have the necessary instructions been given, resources made available and high level support obtained to motivate the contract management team to acquire the necessary information for an ex post evaluation assessment?

		Has a timetable for ex post evaluation been developed and approved, balancing the need to obtain useful information quickly in order to inform current processes and to obtain meaningful data on performance?