A number of other countries publish data relating to the performance of their major defence acquisition programmes, which allows some comparisons of relative performance with the UK system to be drawn. Table 10-2 provides an overview of those audit reports used during the course of this Review.
Country | Body | Report | Reporting since | Number of projects in latest report | |
|
|
|
| Pre main investment | Post main investment decision |
UK | Major Projects Report | 1983* | 10 | 20 | |
USA | Assessments of Selected Weapons Programs | 2003 | 96 |
| |
| Selected Acquisition Reports | 1969 | 91 |
| |
Australia | DMO / ANAO | Major Projects Report | 2007 | - | 9 |
Note: * Was known as the Major Projects Statement prior to 1993
Source: NAO; US DoD; US GAO; Australian DMO
Table 10-2: Overview of publically available audit reports from different nations
Note: France and Canada produce only un-audited data
Arguably the most interesting and relevant benchmark is the US. Since 2003 the U.S. Government Accountability Office (GAO) has conducted an annual assessment of selected Department of Defense ("DoD") weapon programmes. The DoD also published cost, schedule and performance
details of major defence acquisition programmes in Selected Acquisition Reports since 1969.
The Australian Defence Material Organisation has produced two Major Projects Reports, most recently in 2008, based upon the model of NAO's MPR120.
The reviews of major defence programmes across the different countries identify common outcomes for projects past their main investment decision: significant cost overruns from initial estimates and significant delays to expected in-service dates, as is set out in Table 10-3.
Country | Number of active projects | Total cost overrun post main investment decision to date | % Change in total acquisition cost from main investment decision estimate | Straight average time slippage to date (months) | % Change in expected timescales since the main investment decision | % cost overrun post main investment decision in 2008 | Straight average time slippage in 2008 |
UK (MPR) | 20 | £3bn | 12% | 25 | 24% | 0.8% | 6.0 |
UK (DSO) | 41 | £3bn | 10% | 11 | 18% | 0.8% | 4.0 |
UK (CMIS) | 101** | £6bn | 8% | 13 | 27% | 0.4% | 1.9 |
USA | 96 | $296bn | 25% | 22 | 25% | n/a | n/a |
Australia | 16 | n/a | n/a | 30* | n/a | n/a | n/a |
France | 80 (total) | n/a | n/a | n/a | n/a | 0.2% | 1.5 |
Note: All costs are unadjusted and calculations based on weighted averages unless otherwise stated; * Slippage to FOC for 9 projects covered in the DMO Major Projects Report; ** Includes projects that have achieved ISD for which cost slippage is possible, but duration slippage is not
Source: NAO Major Projects Report; CMIS (February 2009); GAO Assessment of Selected Weapon programs (March 2009); DMO Major Projects Report 2007-08; DGA Annual Report
Table 10-3: Summary of acquisition process performance in 2008
As shown in Table 10-3 (using the more complete data from CMIS) in terms of overall cost121 overrun the UK outperforms the US with an average overrun of 8% compared to 25%. The US, however, outperforms the UK on delivering major projects on expected timescales with delays of 25% compared to 32%.
Compared to Australia, the UK appears to be delivering projects with a lower average delay122, although French performance of an average delay of 1.5 months per year appears better than the 6.0 months per year observed in the UK123.
_______________________________________________________________________________________________
120 Major Projects Report, Defence Materiel Organisation 2007-08 (Nov 2008)
121 Both the as-published UK and US cost figures represent aggregate cost and therefore do not reflect changes made to adjust capability.
122 The DMO Major Projects Report argues that the average of the nine projects in the DMO Major Projects is "not representative of the other 217 major projects [costing more than AU$20m] currently managed in the DMO". The sample of nine projects consists of, for example, the HF Mod project with a delay of 127 months.
123 The total in-year slippage of 96 months observed in 2008 by the NAO was for 16 projects. 124 DoD Directive 5000.1, 'The Defense Acquisition System' (Oct 2000)