| Relationship, capability and performance assessment | |||||||||
| Domain | Key Question | Supporting Questions | Minimum information to collect | Judgement questions
(non exhaustive) | Evidence sources
(non exhaustive) | Indicators of good practice
(non exhaustive) | Level of Attainment | ||
| 1. Commercial strategy | Is there an overarching commercial strategy, with a clear rationale for the approach being taken? | 1.1. Is there a clear and consistently held view of what the contract is producing, the type of commercial relationship desired, the basic contract structure and how it will be managed? | 1.1.1 Is there a clear commercial case as part of the outline business case and the full business case? | 1.1.4 How well did the commercial case set out a realistic set of options, and describe a sound options appraisal? | - Board papers and minutes for the outline and full business cases - Commercial strategy - Risk strategy | - There is a documented need for the contract - which has been signed off by senior management - There is clear support and commitment from the client, evidenced in the business case with sustainable funding - The client has a good understanding of market capability - Lessons learnt from previous contracts and commercial relationships are incorporated into the commercial case and overarching commercial strategy - There is a comprehensive options appraisal - Contract ownership is clear, with the budget holder, SRO and contract manager clearly defined | There is no clear commercial strategy; the client has moved to market without fully considering its options | There is a clear commercial strategy but the reasons for it are not consistently or coherently held. External factors or expectations may have undue influence on approach. | The client has a good, shared understanding of the commercial arrangement underpinning the service. SROs and contract managers consistently articulate the reasons why the commercial strategy supports organisational strategy and wider value for money |
| 1.1.2 Does the commercial case set out the nature of the product, the desired commercial relationship and supporting contract and the preferred management arrangements | 1.1.5 Has the client effectively responded to changes between the ideal and actual commercial relationships and contract (e.g. where they have diverged from outline to full business case)? | ||||||||
| 1.1.3 Is the business case approved at senior management level? | 1.1.6 Does senior management consistently understand the business case? | ||||||||
| 1.2 Has there been an assessment of strategic drivers, including policy drivers, and the internal and external environment? | 1.2.1 Has an environmental assessment been undertaken? | 1.2.3 How well did the environmental assessment consider risk, uncertainty and capability? | - Strategy papers - Strategic business case - Outline business case - Environmental assessment | - There is a clear assessment of strategic drivers, policy imperatives and the internal and external environment - Senior management has reviewed the assessment - The assessment has been undertaken by an organisation/person that is independent of the programme delivery team | There has not been a formal environmental assessment or other assessment of strategic drivers | An assessment has been undertaken but it is incomplete, has not followed a recognised method, or has not effectively considered uncertainty, risk or capability | A comprehensive environmental and strategic assessment has been undertaken and the assessment has followed a recognised approach | ||
| 1.2.2 How was the environmental assessment undertaken? | 1.2.4 [not used] | ||||||||
| 1.3 Has the commercial strategy been based upon the assessment of strategic drivers and the internal and external environment? | 1.3.1 Has the commercial strategy assessed both market and client capability and risk? | 1.3.2 Is there a coherent link between the commercial strategy, the strategic drivers and the environmental assessment? | - Business case - Risk assessments | - The client understands the risk and value of the contract - The client has a good understanding of market capability - Lessons learned from previous contracts and commercial relationships are incorporated into the commercial case and overarching commercial strategy | There is no linkage between the commercial strategy and the client's assessment of policy and strategic drivers or the environment | There is a linkage between the commercial strategy and the client's assessment of the policy and strategic environment but key weaknesses have not been addressed by the commercial strategy | The environmental and strategic assessment and the chosen commercial strategy, are clearly linked and issues in the assessment (e.g. capability weaknesses) are addressed by the strategy | ||
| 2. Capability & governance | Does the client have the capability needed to manage the contract and is it developing capability for the future? | 2.1 Does the client have the necessary capability, skills and systems? | 2.1.1 Who manages the contract at organisation and service level? | 2.1.4 Does the SRO and the contract management team have the necessary skills and authority to engage with the supplier? | - Organisational chart and job descriptions - Data management systems/ Management information systems - Contract management systems - Policies and procedures for monitoring and evaluation - Commercial capability assessment/s and associated plans | - The necessary support systems are in place to manage the contract at organisation and supplier level - The client has access to and manages performance reviews from the supplier - Both the supplier and the client use the same contract version, potentially using a shared information system - The client and supplier use whistleblowing and user feedback to solve issues effectively and efficiently - Contract management is adequately resourced, in proportion to the importance of the contract | Commercial capability levels are unknown, with key roles performed by staff new to the function, or newly designated from longer term administrative functions. There are no or inadequate systems for contract management | Commercial workforce is known, with skills and other capabilities identified, but the overall capability is assessed as lacking by self or others | Commercial capability is externally validated and assessed as high. Systems for contract management are fit for purpose and mature. |
| 2.1.2 How is information about the performance of the supplier used by the client? | 2.1.5 Does the SRO and contract management team have access to the necessary information to make a judgement on the contract? | ||||||||
| 2.1.3 What systems are in place to manage the contract at organisation and service level? | 2.1.6 How are the systems used to manage the contract? | ||||||||
| 2.2 Does the client understand its future needs and is it working towards meeting them? | 2.2.1 What are the perceived future needs of the client? | 2.2.4 Is the rationale for the perceived needs coherent and complete? | - Service level plan or strategy for short, medium and longer term - Capability assessment at organisation level - Commercial capability development plans - Organisational capability development plans | - There is a strategic and operational plan to develop commercial capability - A senior governance committee considers and acts on the commercial capability development plan - Lessons learned from previous contracts are being implemented across the client | Capability development is undertaken on an ad hoc basis or in response to crises | Commercial workforce planning is undertaken as part of normal organisational and workforce planning | Capability needs are linked to longer term organisational needs; commercial requirements are known in detail and capability is planned and managed across the organisation | ||
| 2.2.2 How are the needs assessed? | 2.2.5 How well does the client assess its needs? | ||||||||
| 2.2.3 How will needs be met? | 2.2.6 Is the choice of strategy to fulfil the needs appropriate? | ||||||||
| 2.3 Has the client deployed its capability in a balanced way across the lifecycle and is commercial capability effectively integrated with the business? | 2.3.1 What specific capabilities and people have been deployed at each stage of the contract lifecycle? | 2.3.3 How well has the client balanced the need to deploy high quality capabilities in the pre-procurement, procurement, contract negotiation, contract management and closure stages? | - Roles and responsibilities of the commercial function throughout the contract lifecycle - Number and level of specialists deployed on the contract throughout the contract lifecycle | - There is a clear matching of resources to risk profiles across the stage of pre-contract, contract, implementation, termination and end of contract - The client is actively managing the tendency for high quality resources to be only deployed in procurement and contract stages | High quality commercial capability is only used during procurement and contract formation. There are no, or very limited, links between the commercial function and wider core business | The client deploys commercial capability throughout the contract lifecycle but without clear reference to value creation | Commercial capability is deployed in accordance with clear and consistent understanding of when value can be created and lost. Commercial capability is integral to governing and managing strategic investments | ||
| 2.3.2 How does the commercial function develop and maintain its insight into, and form connections with, the core business? | 2.3.4 How has the client integrated its commercial capability into its sourcing and contract management, and wider core business? | ||||||||
| 3. Market management & sourcing | Has sourcing supported the commercial strategy and followed recognised good practice to optimise VFM? | 3.1 Has market management driven long term value for money? | 3.1.1 Has the client undertaken activities to manage the market? | 3.1.3 If the client is a significant procurer in the market, has it shaped the market in line with its requirements? | - Summary of market intelligence - Minutes from market sounding day/s - Public statements by major suppliers - Press coverage - PQQ - Supplier briefings | - The list of strategic suppliers is consistent with the market view - The client has a good understanding of the market, including its limitations - The market provides goods/services which are a near fit, or fit, to the client's requirements | There has been minimal market management activity and the client's requirements are not met | Market management is limited to procurement. Even though the client is a significant procurer in the market, its requirements are not adequately met | There is a known and actively managed supplier market. Market management is undertaken beyond procurement. The market offers services and products which fully meets the client's requirements |
| 3.1.2 Is the market clear about the client's requirements? | 3.1.4 [not used] | ||||||||
| 3.2 Was there a defensible process that resulted in the selection of a capable supplier? | 3.2.1 Has the procurement process complied with competition law, procurement directives and procurement principles? | 3.2.3 If the provider is not well known for its capability, has the procurement process adequately considered the associated risks? | - Legal advice and assurance reports | - The procurement process ran effectively and efficiently - The contract negotiation was done effectively and efficiently | A potential supplier successfully challenged the procurement process and/or the chosen supplier has proven to be incapable of delivery | Procurement process was challenged but the government successfully responded to this. Procurement and service delivery have worked together to select a capable supplier. | There was no procurement challenge. Procurement and service delivery worked together to select a capable supplier. | ||
| 3.2.2 Has the procurement process included appropriate due diligence activities to ensure a capable supplier? | 3.2.4 [not used] | ||||||||
| 3.3 Was there optimum use of competitive pressure? | 3.3.1 Have capable bidders responded to the government's requirements? | 3.3.3 Where a sole supply situation exists or where only 1 or 2 competitive bids were received, has this reflected a reasonable view of market capability and diversity? | - Pre Qualification Questionnaire and Invitation to tender - Respondents to PQQ - Long list of bidding suppliers - Down select documentation - Public statements by major suppliers - Press coverage | - There are no discrepancies between cost estimates of all bidders that cannot be explained - There are no discrepancies between activities and objectives of all bidders that cannot be explained - Sector-specific criteria were used to evaluate the bids - The selected bid shows the highest VfM compared to all bidders | Sole supply arrangements were approved, or very limited number of competitive bids were received, despite there being an active market of suppliers | A restricted market of suppliers has responded to the requirement, representing only a small sub-section of potential suppliers | An active market of suppliers responds to the requirement. Where sole supply arrangements exist, the rationale for sole supply is understood by industry and prospective competitors | ||
| 3.3.2 Have bidder/s dropped out of competition, leaving only 1 supplier bidding? | 3.3.4 [not used] | ||||||||
| 4. Contract Approach | Does the balance of risk and reward encourage service improvement, minimise perverse incentives and promote good relationships? | 4.1. Is there an appropriate allocation of risk between client and supplier? | 4.1.1 What are the main risks? | 4.1.6 Does the allocation reflect the client's risk appetite? | - Financial model - Obligations schedule - Planned and actual volumes - Contract, including termination and payment clauses - Business case - Risk transfer strategy and matrix - Statement of requirements | - A risk assessment was carried out prior to contracting - There is a risk management strategy that addresses at least financial, operational, political and social risk - The risk register reflects a good assessment of the risk associated with the contract and with all stakeholders - The risk register is regularly updated and incorporates changes in the environment - Risk is controlled at organisation level, service level and supplier level - Risk and reward are well balanced, reflecting innovation and investment - The supplier is incentivised to perform well in terms of payment and sanctions | The client has an immature understanding of the risks in the service or project. It has allocated the risks and rewards in the contract without a clear rationale of which party should bear or manage risk | The client understands most risks in the service or project, and has put in place effective mitigations. However, these risks are unclearly allocated and managed through the contract | The client has a good understanding of the risk in the service. This understanding is shared with the supplier, and has been allocated on the basis of the client's and the supplier's ability to manage it. |
| 4.1.2 Who have they been allocated to? | 4.1.7 Does the risk allocation reflect the supplier's ability to manage risks? | ||||||||
| 4.1.3 How has risk transfer/ risk allocation been achieved? | 4.1.8 Does the risk allocation reflect who is best able to bear or manage risks? | ||||||||
| 4.1.4 What are the mitigations of the main risks? | 4.1.9 Is the risk understanding comprehensive? | ||||||||
| 4.1.5 [not used] | 4.1.10 Have risks been appropriately mitigated where they can be? | ||||||||
| 4.2. Are there incentives to encourage the supplier to act in the interest of the client? | 4.2.1 What are the incentives and penalties in the contract? Are they known by the client? What constitutes a breach of contract? | 4.2.3 Have the incentives and penalties been used consistently and appropriately? | - Contract showing KPIs and terms of payment - Contract performance reports - Penalties and incentives report | - The chosen contract (structure) encourages the desired commercial relationship - The contract includes penalties for non-delivery and incentives for performance above target - Gainshare and clawback arrangements are used - The relationship between client and supplier is balanced, or favours the client | The incentive regime is immaterial and/or is not used by the client | The incentive regime is material but is inconsistently used by the client | The incentive regime works to encourage service delivery in line with the contract intent. Incentives are both material and are effective at the upper and lower boundaries of performance | ||
| 4.2.2 Are the incentives and penalties in the contract material to the contract? Are they material to the supplier? | 4.2.4 Are the incentives and penalties effectively linked to performance and KPIs? | ||||||||
| 4.3. Are suitable mechanisms established to drive the desired relationship? | 4.3.1 What are the mechanisms set out in the contract to support the desired relationship? | 4.3.2 Have the mechanisms in the contract been used to create and drive the desired relationship between supplier and client? | - Relationship indices - Governance meeting minutes - Dispute resolutions | - Where partnership relationship is desired, regular formal measurement occurs and both supplier and client act on the results - Target behaviours set out in the contract (e.g. use of competition and benchmarking) are exhibited - Each contracting party understands the other's goals and how to quantify and achieve them | The desired relationship is not set out in the contract, and neither supplier nor client is consistently clear on the nature of the relationship | The contract sets out the desired relationship, and both parties consistently understand this. However, progress towards creating the relationship is inconsistent. | There is a productive relationship. Clear mechanisms to manage the relationship are set out in the contract, with incentives and penalties to encourage productive behaviours. | ||
| 5. Contract management | Is the service being managed well, with costs and benefits being realised as expected? | 5.1 Do the client and the supplier have comprehensive knowledge of service performance? | 5.1.1 What are the KPIs in the contract? | 5.1.4 Are the KPIs well defined? | - Audit policy from supplier and audit feedback - Open book accounting info - Progress reports - Whistleblowing and user feedback - Contract schedules and KPI definitions | - Supplier performance (as defined by KPIs and other issues) is shared with the client - Contractual performance information includes subsupplier performance and user feedback - The KPIs are complete and 'smart' | Contract documentation is uncontrolled. Wider corporate processes cannot provide assurance on delivery | Contract documentation systems are assured by third parties. Corporate processes such as Open Book provide audited evidence of probity | Contractual performance is comprehensively and unambiguously known. Systems and processes provide 'traceability' of delivery back to a well-controlled contract |
| 5.1.2 Are the supplier's obligations clearly set out? | 5.1.5 Are the KPIs comprehensive, providing a rounded view of performance? | ||||||||
| 5.1.3 What assurance processes are undertaken on performance reporting? |
| ||||||||
| 5.2. Is the supplier delivering in accordance with the contract and are they actively managed by the client to meet or exceed requirements? | 5.2.1 Are the KPIs assessed in a timely manner? | 5.2.4 What is the level of contract performance against requirements? | - Contract showing KPI's and terms of payment - Payments - Similar contracts - Progress reports - Interviews with client - Interviews with supplier | - There are formal performance reviews with suppliers, with documented improvement plans if necessary - Senior management undertakes management reviews of the contract, showing their control and responsibility over the contract - Contract management staff are incentivised through objectives and performance management to continuously improve contracted services - Payments have been made according to contract, payment schedule and achievement of KPIs - Progress is made according to agreed KPIs - There are regular meetings to monitor and evaluate the progress made against KPIs | Service performance is not effectively monitored. The supplier's performance is essentially unknown | Contract management processes, including the incentive regime, are effectively deployed. Service performance is less than target and the target remains an acceptable proxy for good performance | Contract management processes, including the incentive regime, are effectively deployed. Service performance meets or exceeds target | ||
| 5.2.2. If the level of performance is not in accordance with requirements, are penalty clauses used? | 5.2.5 Do the KPIs continue to reflect target performance over time? Is the need for innovation and continuous improvement over time adequately met? | ||||||||
| 5.2.3 [not used] | 5.2.6 Does the client triangulate supplier performance reports with other information (such as user feedback, site visits) and use this to actively manage performance? | ||||||||
| 5.3 Is the client meeting its obligations? | 5.3.1 Are the client's obligations clearly set out? | 5.3.2 How well does the client know its performance against its obligations? How does the supplier consider the client is performing against its obligations? | - Governance reports - Relationship indices - Interviews with client - Interviews with supplier | - The client reports its performance against its obligations to SRO and senior management | There is no monitoring of client performance against obligations | There is ad hoc or planned monitoring of client performance. Performance is below target | The client regularly monitors its own performance against obligations. Performance is in line with target or better | ||
| 6. Contract lifecycle | Will the service continue to demonstrate VfM through its lifecycle? | 6.1. Does the contract continue to support the client's strategic intent? | 6.1.1 Has the business need for the service changed? | 6.1.3 Has the contract changed to reflect changes to the business need or the wider strategic environment? | - Governance reports - Client strategic and operational plan - CCS plan and strategies - Sector specific plans and strategies | - The need for the contract is checked at various intervals - The original strategic intent is still supported by the contract | The need for the contract is no longer important to the client | The contract continues to meet a clear business need but is inconsistent with wider strategic environment | The contract clearly meets a business need throughout its life and remains consistent with the wider strategic environment |
| 6.1.2 How often is the business need for the service reassessed? | 6.1.3 How much flexibility is built into the contract? e.g. break clauses | ||||||||
| 6.2. Are VFM mechanisms used to ensure the contract continues to deliver VFM over its life? | 6.2.1 What are the VFM mechanisms (often set out in a contract schedule)? | 6.2.4 Does the contract continue to reflect a reasonable view of VFM? | - Benchmarking reports - Market test exercises | - Contract terms are consistent with market practices - VFM mechanisms are used to test price and service - Price and service standards are re-negotiated in line with movement in the market | There are no VFM mechanisms in the contract, or they are not used, and frequent competition is not possible. Lifetime costs are not tracked or managed | VFM mechanisms are used but results are inconclusive and are seen to lack 'teeth' | There is an active programme of VFM testing which is externally reviewed and shows the contract offers VFM or leads to changes to improve VFM. The client understands lifetime costs and the value created | ||
| 6.2.2 How have the VFM mechanisms been used? What activities does the client undertake to maintain competitive pressure on the supplier? | 6.2.5 Does the client continue to retain the technical knowledge required to manage and challenge the supplier? | ||||||||
| 6.2.3 How do lifetime costs compare to original bid and department financial model? | 6.2.6 Is the supplier motivated and challenged to increase performance year on year? | ||||||||
| 6.3. Is change controlled and well managed and does the contract remain current? | 6.3.1 What is the change control process? | 6.3.4 Do governance and assurance processes give confidence that change is well managed? | - Contract amendments - Minutes and documentation from dispute proceedings - Change control board documents - Sample of Change Control Notices | - A current comprehensive contract can be produced - Both the supplier and the client use the same contract version, potentially using a shared information system - Amendments to the contract are legally backed up - Change to the plan is justifiable - Issues are escalated correctly | Governance processes for change during contract lifecycle are not specified with sufficient detail. Change governance is ad hoc | There are clear governance processes for change but these are not followed or are followed in an ad hoc manner | There are clear change governance processes which are documented and followed | ||
| 6.3.2 Are agreed changes formally written into the contract? | 6.3.5 Are major changes (e.g. contract termination) under consideration and if so, is appropriate contingency planning and preparation underway? | ||||||||
| 6.3.3 What are the break clauses in the contract? | 6.3.6 [not used] | ||||||||
| 7. Transition & termination | Is the client ready for the end of the contract? | 7.1 Has market management been undertaken to support new contracts? | 7.1.1 Has the client undertaken activities to manage the market? | 7.1.3 Has the client managed the risk that the incumbent may be disadvantaged or preferenced? | - Lessons learned log at supplier and client level - Summary of market intelligence - Minutes from market sounding events - Public statements by major suppliers - Press coverage | - Lessons learnt have been captured and shared throughout the client - Market intelligence is used to maintain an understanding of the market and of alternative suppliers | The market views the incumbent supplier as the only possible supplier for the new contract | There is an active market of suppliers however the incumbent is seen to be preferenced or disadvantaged | There is an active market of suppliers able to bid for the new contract |
| 7.1.2 Is the market clear about the client's requirements? | 7.1.4 [not used] | ||||||||
| 7.2 Has the end of the contract been managed effectively to allow re-bid or handover? | 7.2.1 How was the end of the contract managed - at client and supplier level? | 7.2.3 Has the client managed risks of a degradation of service performance? Have appropriate due diligence actions been undertaken (e.g. checks on asset condition)? | - Organisational chart and role and description of different clients - End of contract clauses - Lessons learned log - Exit and transition plans | - There are mechanisms for identifying contract 'trigger points' such as notice periods, contract closure etc - There is an exit, transition or end-of-contract plan - There may be dedicated governance and resources to manage the end of contract - The correct authorisations have been used - If used, termination clauses are legally sound and used appropriately | Performance degrades as the contract end approaches. The client considers there are substantial risks in appointing a new supplier or in-sourcing the activity | Performance of the incumbent remains adequate or better, however key skills and capabilities could not be transferred to new supplier or the client | The performance of the incumbent supplier has not degraded and key skills and capabilities have been transferred to the new supplier or in-sourced | ||
| 7.2.2 What transitional and/or exit arrangements were instigated? | 7.2.4 How effectively have capabilities been transferred from the supplier to the client? | ||||||||
| 7.3 Are insights from the operation of the contract brought to bear in developing the new contract? | 7.3.1 What are the lessons learned on the operation of the contract, the outcomes of the contract and the performance of the supplier? | 7.3.3 Have the lessons learned been integrated into needs assessments, the commercial strategy and the list of strategic suppliers? | - Lessons learned log - Minutes from CCS/other stakeholders setting out performance of the supplier - List of strategic suppliers - Needs assessment - Strategic plan | - The client has reflected upon the performance of the supplier and adjusted (if needed) its list of strategic suppliers - The client worked across government to share intelligence on the performance of the supplier | There is no documented learning from the contract | Lessons are known but are either not documented or not shared across the client and across government | There is documented learning from the contract (including supplier and client performance) which is used to inform new contracts, strategic supplier management, and the client's commercial capability | ||
| 7.3.2 How are the lessons learned shared within the client, and with wider stakeholders? | 7.3.4 [not used] | ||||||||