● Assurance should help Accounting Officers discharge their financial responsibilities.
● Reviewers must report directly and independently to the top of the organisation.
● Assurance must be designed as a framework not a process to give rigour to reviewers but also the ability to adapt the approach as required.
● Reviewers must use all existing information to form their opinions, building on credible sources.
● The benefits delivered by projects should be assured; not just the successful delivery (completion) of the project.
● Assurance should protect the reputation and financial position of HM Treasury and departments.
● Reviewers should identify and document lessons learned as part of the review.
● Evidence needs to be quantitative (for example the cost of non-compliance) as well as qualitative (for example informed opinion).
● Assurance findings should include clear responsibility for action.
| Figure 4 Perform assurance | |||||
| Ideal state | Gap analysis | Suggested priority actions | Measures | ||
| Required element Provide existing project information. | Purpose Provides reviewers with all the information required to perform assurance: ● Key contacts & schedule. ● Existing information on project (for example internal audit, previous external assurance etc.). ● Information on review team. | Statement of current position There are logistical problems in organising access to key stakeholders in the project. Information from other sources is not always known about or provided. | Evidence for judgement Projects state there are sometimes difficulties in arranging meetings with key stakeholders, which in the case of MPRG can include Accounting Officers and Ministers. Assurance does not tend to build upon the work of others. Some departments are beginning to integrate various forms of assurance applied to projects but this is not widespread and does not include central providers such as HM Treasury and OGC. Forty five per cent of stakeholders cite poor integration across the different assurance mechanisms leading to duplication of effort. | Implement an approach that consistently provides all reviewers with existing project information; for example other forms of assurance, internal audit, NAO reports. Difficulty: Easy | Approach implemented (Indicators - Percentage of occasions information received/used and rating of how useful each source. Reduction in project team effort particularly in providing information for reviewers). |
| Perform assurance (all types). | Assurance has different functions, dependent on stage in the project lifecycle, but should support the effective management of projects as highlighted in PRINCE 2's key themes. | PRINCE 2 identifies explicit objectives for assurance. | Business Case ● Assist in the development of the business case. ● Verify and monitor the business case against external events and project progress including any changes made. ● Ensure the project fits against the overall organisational strategy. ● Monitor project finance. ● Verify and monitor the benefits plan. | The functions of assurance as highlighted in PRINCE 2 should be considered as part of any refresh of the current assurance mechanisms. Difficulty: Easy | Guidance on assurance mechanisms reflects PRINCE 2. |
| Organisation ● Advise on selection of project team members. ● Advise on stakeholder engagement. ● Ensure communication to stakeholders takes place. | |||||
| Risk ● Review risk management practices to ensure they are performed in line with strategy. | |||||
| Change ● Advise on examining and resolving issues. | |||||
| Progress ● Verify and monitor the business case against external events and project progress including any changes made. ● Verify and monitor changes against the project plan. ● Confirm stage and project progress against agreed tolerances. | |||||
| Perform assurance (Point in time). | To help Accounting Officer's assure their projects against the common causes of failure and provide an informed understanding of the status of the project against profile agreed in the business case (time, cost and quality). | Stakeholders value point in time assurance. | Ninety six per cent of respondents say that, overall, assurance adds value above the assurance offered by departments. Eighty five per cent of those interviewed feel that MPRG/ Gateway recommendations are specific enough to act upon Reviewers are usually judged to have the necessary skills and experience. MPRG is particularly valued for the access to unbiased commercial expertise. Although relatively new Assurance of Action Plans have been cited by some stakeholders as an effective way of ensuring projects respond to a Gateway review. Starting Gate is regarded as a timely challenge to the deliverability of policy objectives. | ||
| Assurance should explicitly help Accounting Officer's discharge their financial responsibilities to assure their projects (for example extant DAO letters and Managing Public Money). | Accounting Officers are required to assure their projects against the Common causes of project failure1. The presence of the common causes of failure is not explicitly tested in any of the current mechanisms. Our sample analysis indicates that in some instances Gateway reviews did not detect them. 68 per cent of stakeholders feel that assurance did not pick up on issues present in the project in a timely manner. | The financial responsibilities of Accounting Officers should be considered as part of any refresh of the current assurance mechanisms. Difficulty: Easy
| Reference in assurance mechanisms and processes to how assurance helps Accounting Officers meet their financial responsibilities. | ||
| Point in time assurance does not include a sufficient element of quantitative analysis; including calculating the cost versus benefit of assurance itself. | Gateway reviews are limited to five days and tend to be opinion based. Quantifying the total costs versus benefit of assurance is not part of the review process meaning an overall return on investment calculation is difficult. | Recommendations should include hard evidence and aim to close the variance between status of the project against the profile agreed in the business case (time, cost and quality). If variance against profile ● highlight this to stakeholders external to the project ● make recommendations on how the project can get back on track ● decide to recommend further interventions If no variance against profile ● recognise reasons for success, capture lessons learned and communicate external to the project. Difficulty: Hard | Reviews involve an independent quantitative assessment of project status - variance between time, cost, quality profile and that agreed in business case. | ||
| Provides a check on action resulting from the recommendations occurring from point in time assurance. To highlight any concerns which occur between planned point in time assurance and to act as a resource for projects to draw upon. | Assurance tends to be point in time. Examples of the deployment of continuous assurance are usually at the instigation of the project and involve professional services companies. | All assurance mechanisms deployed are point in time. There is currently no mechanism in place to assign a full time assurance specialist to projects. | Develop the capacity to embed continuous assurance in the most high risk projects. The staff involved should be independent of the project and have the authority to: ● investigate any issues raised, or not raised, in the formal point in time assurance and report on status (open/ closed); and ● routinely report directly to Project Director, Senior Responsible Owner and where necessary external to the project. Difficulty: Hard | Evidence of closer tracking of actions, interventions or cancellations in those projects with continuous assurance experts. Increased demand for continuous assurance experts. | |
| Produce assurance report. | To capture the results of the assurance activity in a formal way, assigning responsibility for resolving actions to help tracking the implementation of recommendations. | ||||
| Gather lessons learned. | To document lessons at the point of learning regarding the project and the use of the assurance process itself. | The gathering of lessons from assurance (including reviewer performance) is not real time. | The lessons learned bulletins are performed retrospectively by the centre. There is a wealth of knowledge held by reviewers themselves which is not being capitalised on. | Amend the review templates to include a key lessons section for completion in real time by reviewers. Difficulty: Easy | Template for lessons produced and implemented (Indicator - more systemic identification of themes/ lessons at departmental and portfolio level). |
| Feedback on reviewer performance. | To provide a real-time assessment of the standard of reviewers versus the project requirement. This informs training requirements and assurance planning. | Reviewer performance is reported on as part of current point in time assurance but more could be done. | There is little evidence that reviewer performance is actively managed, or that it leads to targeted training or regular recertification. | ||
| Source: National Audit Office NOTE 1 HM Treasury, Dear Accounting Officer letter Gen 04/07, 2004-05. | |||||