7.2.1. Sample of projects under analysis

Data on individual project performance has been sourced directly from DE&S' CMIS system as at February 2009, with support from DE&S' CAPR team, and supplemented with data from:

•  projects tracked through the NAO Major Projects Reports but not present in the CMIS system;

•  business cases / review notes / approval notes held by Sec(EC); and

•  c.50 interviews with DE&S' IPTs.

This gives rise to a total sample of projects under consideration of c.170, which is more comprehensive than either those tracked as part of the Departmental Strategic Objective (40 - 50) or those monitored historically by the NAO (i.e., largest 20 post Main Gate projects, 10 pre-Main Gate projects in each year).

As noted, information is generally presented for "mature" projects only (i.e., those where more than 75% of time has elapsed between Main Gate approval and forecast in-service date) and the sample of projects changes between analyses as a consequence of data availability.

Furthermore, comparisons have generally been made using forecasts at the 50% level at Initial Gate, Main Gate and latest forecast.

There are two key phases of the CADMID64 cycle which this Review has considered separately in order to maximise the sample of projects available for analysis (see Figure 7-1):

•  1 - Assessment phase, taken here as the period between Initial Gate and Main Gate approvals

•  2 - Demonstration and Manufacture phase, taken here as the period between Main Gate approval and delivery of a defined level of operating capability at the in-service date ("ISD")65

Development of estimates through project

Figure 7-1: Development of estimates through project lifecycle

The presentation of the information on cost in the analysis reflects two key analytical decisions:

a)  costs are presented on a "per unit" basis (e.g., cost per aircraft); and

b)  adjustments have also been made to reflect "changed requirements" compared to the approved business case66 (other than unit volume related changes, which are captured above).

For further detail on the sample and methodologies used to undertake the analyses presented in this section, please refer to Appendix F.

__________________________________________________________________________________________________________________________________

64 Concept, Assessment, Demonstration, Manufacture, In-service, Disposal cycle

65  The Review has analysed the period from Main Gate to ISD for duration (i.e., the date at which the Front Line Command has a defined level of operational capability) due to the Departmental focus on this dimension and it should be noted that further dynamics may be evident in the period post-ISD, pre-Full Operating Capability ("FOC") which may not be clear in this analysis. The Review has analysed the full Demonstration and Manufacture period to FOC for cost.

66  In aggregate these adjustments are relatively minor having both positive and negative effects on individual projects.