Development of the Output-Based Specification

18. There are five distinct phases to the development of the OBS along with the supplementary project and contract documents through the acquisition process.

19. Stage 1 - Strategic Level

• The strategic level of the OBS should be developed at the same time as the business case for the down-selection of the Procurement Strategy. The focus should be on the development of the top-level outputs required from the Contractor based upon the Key User Requirements.

• It should define for each top-level output 'what' is required, and its function and purpose from the user's perspective. The level of detail does not need to be any greater than the Key User Requirements that it is intended to address. It must however, clearly identify the outputs that the eventual Contractor is required to deliver, the interface points with any other Authority assets/equipment/infrastructure, and any strategic constraints and assumptions.

• It is also essential that the strategic level of the OBS defines the 'service boundary' for the contract; the lines of responsibility between the Authority and the Contractor. A pictorial summary of a service boundary is shown at Figure 3 below.

• It is important that the strategic level of the OBS is sufficiently matured and agreed with the User before embarking on any further development of the OBS.

• Once the strategic level of the OBS is matured and agreed it can form the basis of early engagement with industry in the form of market soundings and industry days.

Figure 3 - Pictorial Summary of a Service Boundary

20. Stage 2 - Definition of Output Performance Standards and Service Levels

• This stage of development of the OBS should be carried out in conjunction with the development of the PPM.

• This stage involves the definition of the functional performance and service levels of each output that crosses the service boundary. This must include volume requirements and quality standards, for example:

- how many/how much of the output is required;

- how often/when is it required;

- what are the minimum performance/standards of the output.

• For both the OBS and the PPM, the key questions that will need to be answered are:

- is the level of performance specified in the OBS sufficient to enable the user to achieve the desired outcome;

- what level of poor performance would cause the user inconvenience;

- what level of poor performance would cause the user to incur additional costs (i.e. in waiting time/queuing/finding alternative sources of supply);

- what level of poor performance would constitute non-delivery of the output when required by the Authority;

- what level of poor performance would constitute the non-availability of the service when required by the Authority;

- what level of poor performance would constitute a critical failure to the achievement of the user's desired outcome.

- how will poor performance be identified/detected;

- how will the levels of performance be monitored and measured;

- does the OBS specify the level of performance and standard of service in an unambiguous and objective manner;

- is the method of monitoring and measurement, and the mechanics of the PPM capable of drawing a distinction between the differing levels of poor performance;

- are the outputs (or the failure to deliver the outputs) incentivised in accordance with their impact on the user and the achievement of the customer's desired outcome.

• It is particularly important at this stage to consider how the user will make use each output.

- For some outputs, the Authority will need to make a specific demand for the delivery of the output, such as the use of a simulator sortie or an air-to-air refuelling sortie.

- Other outputs will be used on a largely continuous or ad hoc basis, such as access to accommodation facilities or computer-based training.

• The performance and service level requirements will need to be tailored to the method use of the output:

- whether the output was delivered as requested; and

- whether the outputs were available for their intended use when required.

21. Stage 3 - Definition of the Technical Evaluation Criteria

• Once the OBS is sufficiently developed the acquisition team will need to consider the tender evaluation criteria for the bidders' proposals and prepare the technical aspects of the tender evaluation plan.

• The evaluation of the bidders' proposals will need to consider:

- whether the bidder proposes to fully or partially meet the requirements of the OBS;

- whether the bidder's proposed inputs are likely to satisfy the requirements of the OBS; and

- whether the bidder is likely to be able to deliver the proposed inputs.

• The evaluation will therefore need to include an evaluation of:

- the impact on the user of any stated non-compliance and its effect on the achievement of the desired outcome;

- the level of confidence that the proposed inputs will deliver the required outputs;

- the level of confidence that the bidder can project manage and carry out the design, construction and commissioning of the proposed assets, equipment and infrastructure;

- The level of confidence that the bidder can manage, administer, and integrate the proposed inputs in order to deliver the outputs.

• The evaluation criteria will therefore need to consider not only the inputs proposed by the bidder, but also the bidders' competence to deliver and implement them. The key test as to whether the evaluation criteria is appropriate, will be whether it appropriately assesses and differentiate between circumstances such as:

- The bidder proposes to meet the OBS, but the inputs are evaluated as being unlikely to deliver the outputs;

- The bidder proposes to meet the OBS with inputs that are likely to deliver the outputs, but there are flaws in the bidder's project management planning and/or plan for the management, administration, and integration of the proposed solution.

22. Stage 4 - The Definition of the Scope and Content of the Bidder's Proposal

• The principal purpose of the bidders' proposal is to describe how they will meet the OBS and define the inputs that will deliver the outputs.

• However, based on the evaluation criteria defined in Stage 3, and given that the bidder's proposal is intended to be incorporated into the Contract it is sometimes useful to require that bidders submit their proposals in two parts:

- the first to define the solution and provide a functional specification of the inputs to satisfy the OBS that will form part of the Contract (hereafter referred to as the "Input Specification"); and

- secondly, the evidence of the bidder's competence, skill and experience to deliver and implement the proposed solution and inputs that will be used for the purposes of evaluation only (hereafter referred to as the "Supporting Evidence").

• The Input Specification should as a minimum require that bidders propose:

- An OBS Compliance Matrix that confirms whether or not their proposal meets the performance standards and service levels for each output defined in the OBS, and identifies any shortfalls or limitations in their achievement.

- A top-level description of the inputs that are proposed and how they contribute to the delivery of the outputs. This should include a clear picture of what inputs contribute to the delivery of which outputs;

- A technical specification of each input that:

■ describes the physical (qualitative and quantitative) performance characteristics of the input;

■ describes how the physical and performance characteristics of the input enables the input to contribute to the delivery of the output.

- A description as to how any constraints specified in the OBS will be complied with.

- A specification of any Government Furnished Assets (GFA) that is required to be provided by the Authority (either directly or from other Authority Contractors) including:

■ how many/how much of the GFA is required;

■ how often/when is it required

■ what is the minimum performance/quality standards of the GFA.

- For the bidder's proposed solution (as a whole) bidders should also provide details of the processes, tools and deliverables that:

■ they would use to manage, administer and integrate the inputs to deliver the outputs;

■ the user of the outputs would be expected to interface with in order to book and schedule the use of the outputs;

■ they would use to enable the monitoring and measurement of service delivery and performance;

■ they would use to demonstrate compliance with the OBS and obligations of the contract.

• The Supporting Evidence required from the bidders might include:

- The bidder's requirement analysis showing how they derived their solution to the OBS and the physical and performance characteristics of the input in order to ascertain the validity of the bidders solution and their understanding of the requirements of the OBS1;

- The bidder's project management plan for the design, development, construction and certification of the inputs;

- The bidder's risk management plan and risk register for both the Pre-Operational and Operational phases of the Contract with particular attention to risk mitigation and fallback plans;

- The bidder's logistic support analysis of the assets, equipment and infrastructure and how the design and the logistic support of the equipment and infrastructure will deliver the planned level of availability;

- a Technology Readiness/System Integration Readiness Level Assessment for any solution or input that includes new technology, new products or new systems;

- evidence that the bidder (or its proposed sub-contractors) has the experience or competence to deliver and implement the proposed solution2

23. Stage 5 - Finalising the OBS and the Bidder's Proposal for Contract Award

• Prior to Contract Award the OBS will need to be updated to reflect the bidder's proposal in respect of any non-compliance, limitation, or restriction on the achievement of the performance standard and service levels. Any such changes must only be accepted with extreme care and acquisition teams need to consider:

- Whether the change constitutes a material change to the Authority's requirement such that they are outside the scope of the original procurement adverts and therefore will need to re-competed;

- Whether the change to the OBS is solution specific (i.e. is specific to one bidders' proprietary solution) or is a change that should be made to the OBS for all bidders in order to maintain and level playing field between the bidders;

- The impact of the change on the end-user and the achievement of the Statement of User Need or Key User Requirements;

• The Input Specification in the Bidders' Proposal should be amended to reflect the outcome of any clarification or negotiation. Acquisition teams should also confirm that the description of the inputs fully reflects the basis on which the Authority has made its evaluation decision. For example, if the Authority regards bidder A's solution to be more environmentally sustainable because he has proposed to use photovoltaic panels to supplement mains electricity, then this must be reflected in the Input Specification that will go on to be included in the Contract.

• The Supplementary Evidence in the Bidders' Proposal will only need to be updated to ensure that the evidence on which the Authority makes its evaluation decision is documented and auditable.

• Acquisition teams should also prepare, as part of the final bid evaluation reports, a requirements traceability matrix that shows how the requirements in the Statement of User Need or Key User Requirement are reflected in the OBS and are then satisfied by the Input Specification in the Bidders' Proposal.

______________________________________________________________________________________________

1 For construction and works project this might include evidence to support the Design Excellence Evaluation Process

2 This would be more detailed and solution specific when compared with the technical evaluation carried out as part of the Pre-Qualification Questionnaire