As measuring performance can be both time-consuming and costly, measures should be considered carefully, taking into account the costs and benefits involved. Not all aspects of performance will need to be measured or assessed with the same frequency. There may be some measures that will require daily measurement while others may only require assessment at longer intervals, such as quarterly or yearly. Other factors to consider when establishing performance measures include how the data to allow measurement will be collected and the potential burden on clients of that collection, the costs of collecting and analysing it and what assurance the acquiring entity has in regard to its accuracy.

In establishing a performance regime it is important to consider the costs and benefits of it. An overly complex set of measures can result in an increase in the contract price that outweighs the potential benefits.

In establishing a performance regime it is important that the potential impact on the price of the goods or services be considered.

In addition to the costs of collecting performance data, consideration also needs to be given to the level and type of resources that will be needed to analyse data to determine whether performance is satisfactory or not.

Where the contract deliverables are of a technical nature, relevant technical knowledge may be required to assess whether the deliverables meet the required standard. Where the required expertise does not exist within the acquiring entity, external expert advice may need to be engaged to obtain the necessary level of assurance that performance standards have been met.

The periodic independent testing or certification of performance reports provided by contractors can also be a useful means of obtaining additional assurance to test the accuracy of performance reports submitted by the contractor.

The following case studies discuss the development of performance indicators.

Case Study: Specifying cost-effective performance indicators

Frequency: IT services may be assessed daily through measurement of the time when the system is not available, or responsiveness of the helpdesk. Client satisfaction with IT services, which is more difficult and costly to measure, may only be assessed on an annual basis.

Where a product is being developed, the acquiring entity may wish to test components critical to success at key points rather than waiting until a final product is delivered.

Cost: A client satisfaction survey may be useful in determining whether services being provided are appropriate, but conducting surveys can be costly and impose a burden on the end-user or client.

Case Study: Standards for helpdesk services

For ongoing, complex services there can be pages of detailed descriptions of service standards, specifying minimum standards. For example, for a telephone help desk, a simplified set of performance standards might include:

Call answering

All calls will have records kept of call time, and elapsed time till the call is answered or terminated. A monthly analysis is to be provided to the contract manager.

For all answered calls a job number will be recorded.

Calls will be categorised by priority according to the protocols specified in Schedule X.

80% of calls to be answered within 60 seconds, and 95% within 180 seconds, on working days 8:00 AM to 6:00 PM.

Any day where this service level is not met is a service shortfall day.

Call resolution

90% of Priority 1 calls should be resolved within 1 hour. Any Priority 1 calls not resolved in 2 hours time shall be escalated to the contract manager.

90% of Priority 2 calls should be resolved within 4 hours. Any Priority 2 calls not resolved in 1 working day shall be escalated to the contract manager.

90% of Priority 3 calls should be resolved within 2 working days. Any Priority 3 calls not resolved in 5 working days shall be escalated to the contract manager.

Service Credits

If in any calendar month there are more than 4 service shortfall days, service credits will apply as specified in Schedule Y.

Comment: This example uses a few simple performance measures, such as speed of call answering and speed of call resolution. Each measure can then have several performance standards. For example, the timeliness of call answering has one standard to be met for 80 per cent of calls, and another lower standard to be met for 95 per cent of calls. Having a graduated set of performance standards allows a practical approach to performance management, recognising that perfect service may not be practically achievable or cost-effective.

Good Practice Tip: You get what you measure-so measure the things that are important.

In one (perhaps legendary) case, a key performance measure was the time taken to answer phone calls. The target was met by the contractor simply picking up the phone and hanging up again. What was really wanted was an indicator of how long it took to have a client's actual enquiry answered.

Ensure performance indicators are designed to motivate the contractor to focus on your true requirements.