Evaluation Criteria Weighting

Identifying a systematic procurement evaluation tool requires identification, measurement and weighting of each of these procurement characteristics. In the final analysis, this process is a matter of judgement and the selected weightings will be influenced by subjective views about what is important and what isn't. For these purposes, the weightings used in this comparison were sourced from a review of empirical data that employed procurement performance reviews conducted by state government audit offices and independent procurement reviews commissioned by government and industry in the four countries from which the sample was sourced (See Diagram 1). Additionally, the analysis was tested using both weighted and non-weighted evaluation criteria.

The weightings selected for testing were value for money (60%), delivery performance (15%), quality service outcomes (10%) and process management (15%). The components of each category are set out in Diagram 1 and Appendix A.

The weighted comparison indicates the superior procurement performance of the non-adversarial contracting forms whereby design is integrated with construction, the private contractor's incentives to meet performance benchmarks were structured in a collaborative rather than an adversarial context and the contract was delivered to a full or significantly output-based specification. PPP was the found to be the most effective procurement mechanism, followed by the build own operate transfer (BOOT) method and outsourcing. Each of these procurement methods has a significantly greater value for money score and is their economics are built around full lifecycle costing. However, the BOOT model was the outlier here because it employs an input specification, it has an asset rather than a service delivery focus, contractors are selected by lowest-price tender and there are adversarial aspects to contract formation and administration. Nevertheless, the strong incentive characteristics, contractor design input and the intervention of new technologies and innovation were drivers of its better performance. BOOT and similar arrangements are frequently described as PPPs in Asia and Europe and the formal distinctions between these two procurement methods is rapidly disappearing. Alliance contracting was also a more effective procurement method than traditional lowest price tender models and in-house provision.

Diagram 1 Procurement Efficiency Weightings