Information collected increases visibility of projects, but there have been problems with data quality
2.2 The Government Major Project Portfolio is a big step forward in terms of government's understanding of its project portfolio. It comprises all central government funded projects and programmes that require approval by HM Treasury during their life cycle. All projects within the portfolio provide quarterly returns to the Authority. There have been three returns since the Authority was created in March 2011. The returns include project costs and benefits (budget, actual and forecast estimates), key milestones and latest assessments of delivery confidence and risk level.
2.3 The reporting process provides regular information about projects and gives greater visibility over the current portfolio. The Authority feeds information from the data returns into portfolio and department dashboards. These give a good overview of the current portfolio size and spend profile, and how indicators, for example, actual and forecasted spend, align over the spending review period.
2.4 There are 205 projects in the latest portfolio with a total value of £376 billion. The projects include the introduction of large IT systems, the construction of defence equipment such as ships and helicopters, and the implementation of major changes to how services are delivered by government. Thirty-nine of those projects have a delivery confidence rating of 'red' or 'amber/red' (Figure 6). Ninety-one per cent of the whole-life costs in the government major project portfolio are attributable to projects from five departments (Figure 7 on page 22).
| Figure 6 Total whole-life cost of projects (£ bn) and delivery confidence | ||
| Delivery | Number of | Whole-life costs |
|
| 42 | 84 |
|
| 53 | 66 |
|
| 71 | 135 |
|
| 33 | 89 |
|
| 6 | 1 |
| Source: National Audit Office analysis of 2011-12 quarter three government major project portfolio returns | ||
2.5 The Authority and departments are now more satisfied that the information being collected is an appropriate data set for the Authority's purposes. Both the process for returning submissions and the information requested have evolved over the three quarters. Also, the Authority has responded to issues with the data and to issues raised by departments. The Authority amended forms and guidance developed in a pilot phase following the first returns.
2.6 The Authority aims to collect meaningful and consistent data without placing undue burden on departments. Placing restrictions on the form of data increases the burden on departments if their internal reporting systems differ, and may result in non-completion. Since the first quarter, the returns guidance acknowledges and allows for departmental differences. In our interviews with department and project staff, most did not find the reporting process burdensome (beyond any initial set-up). Departments' compliance with the process is now near 100 per cent.
2.7 This variability in departments' data returns has contributed to some quality issues. The Authority considers that complete accuracy of data and comparability between projects is less significant than overall movements in the data and trends. The Authority may be able to better use information it collects to determine and quantify the scale and likely impact of any inconsistencies between departments. For example, in our analysis of the quarter three 2011-12 report in January 2012, 25 projects indicated that they had not provided full project costs in their return, while 26 had understated the level of benefits in some way.
| Figure 7 Major project spending by department (£bn) |
|
|
| NOTES 1 Excludes four Department for Transport rail refranchising projects, which have negative whole-life costs. 2 For some departments the whole-life costs of projects reported in their quarterly returns may not reflect the entire cost of those projects - for example, the return from the Department for Culture, Media and Sport for the 2012 Olympic and Paralympic Games includes costs incurred by that Department but not by other organisations. 3 Departments supply additional explanatory information to the Major Projects Authority in their returns, for example, to signal estimates that were currently under review or to indicate additional costs that were not included in the return. Source: National Audit Office analysis of 2011-12 quarter three government major project portfolio returns |
2.8 While these inconsistencies are of less importance in department and project-level monitoring, they limit opportunities to use data for decision-making across the portfolio. The Authority expects both the portfolio and department dashboards to evolve as data quality improves. It also intends to identify specific areas that require regular monitoring and conduct more robust trend analysis.
2.9 Some lack of clarity remains on the purpose of the portfolio reporting process, mainly because data issues have limited the use of portfolio data. Departments told us that they are getting little feedback from the exercise, and would appreciate more understanding of how the data are used.
2.10 There are some potentially useful data that are not collected. The Government Major Project Portfolio does not record outcomes. There is no explicit coverage of changes in quality or scope of projects, information on projects' definitive start and end dates, or data on project types. Collecting these data would strengthen the analysis that the Authority could carry out.