Utilising data from a number of sources has required a high level of quality control to ensure consistency throughout. This has been considered from two different angles: firstly ensuring that the data is factually correct and secondly checking that for individual projects it is appropriate to assess performance in a standard manner; excluding those projects where is not deemed appropriate. (For a full list of projects and in which samples they appear refer to Section F.15.)
For the top 20 Major Projects covered by the NAO it can be assumed that this data is correct as it has undergone a rigorous audit process and it can be used to verify the current and historical CMIS data. However, for projects that do not appear in the NAO it has been necessary to verify any uncertainties where possible with DE&S. This process has undergone a number of iterations and the following corrections have been made to errors arising from data entry issues in CMIS:
• corrections to Main Gate approval dates;
• removal of duplicate projects; and
• corrections to forecasts.
The second aspect of ensuring that appropriate comparisons are being made was considered for both the Assessment phase and Demonstration & Manufacture phase separately.
Some projects are still loosely defined when they receive Initial Gate approval and as a result the scope can change significantly during the Assessment phase, thus for such projects IG to MG comparisons are relatively meaningless. These significant changes in scope have been identified through IPT interviews or by identifying outliers in the analysis and subsequently verifying that there has been a change in scope.
Moreover a number of projects have been excluded from the analysis of Performance during the Demonstration & Manufacture phase:
• three projects excluded due to missing or inconsistent data:
- unknown Main Gate date;
- no ISD as 'support style' project to deliver ongoing capability; or
- no Demonstration & Manufacture stage as capability met by UOR.
• three NAO Major Projects excluded as they do not appear in CMIS and dropped out of the NAO Major Project Reports before reaching completion.