2.21 Defence capability is defined as the ability to achieve a particular operational effect.47 An operational effect is achieved by combining fundamental inputs to capability such as personnel, training, supplies, facilities and equipment platforms and systems (for example, ships, aircraft and electronic systems).
2.22 In acquiring Defence platforms and systems, a range of documentation (including operational concept documents, functional and performance specification and test concept documents) is developed and sets out the detailed requirements/performance attributes to be achieved. In the case of a ship, for example, this would include elements such as its range and speed, handling characteristics, level of self-protection, certification plans, supportability and the compliance with Navy regulations. A deal of this information is classified for national security reasons and so is not publicly available.
2.23 In the acquisition phase, the scope of each project is primarily defined through a Materiel Acquisition Agreement (MAA) between Defence's Capability Development Group (as the purchaser) and the DMO (as the supplier). MAAs usually incorporate a number of MOEs, which at a strategic level are designed to set out the key capability performance attributes of the system to be delivered by the DMO. 48 Where key attributes are not achieved, this could be expected to have a significant effect on a system's suitability for planned operational service.
2.24 In general, MOEs focus on the achievement of certain technical, regulatory or operational requirements. Budgetary and schedule requirements are less likely to be included as an MOE (and are instead covered in other areas of an MAA). Given this approach, MOEs generally reflect the expected overall technical status of the project when complete, and provide insight into a system's likely suitability for planned operational release.
2.25 There is considerable diversity across the projects in the number, level of specification and focus of MOEs. While MOEs need to capture the key capability result areas for the project, and a rudimentary template assists the DMO to do this, the ANAO notes that there is not a clear underlying consistency in the identification and articulation of MOEs in the MAAs. The development of more robust key performance indicators to address the current deficiencies with MOEs was highlighted by the DMO in its section of last year's report as an area for attention.49 The DMO has since advised that MOEs are unlikely to be reported in their current form in 2010-11.
2.26 The level of confidence in delivering each MOE (using traffic light indicators) is assessed by DMO's project management, and reported monthly within DMO and Defence.50
2.27 In this regard, as the MOE data concerns forecasting future achievements, it has been excluded from the scope of the ANAO's formal review.51 However, the ANAO has been provided with data from the DMO reporting systems in order to examine the accuracy of the MOE data in the PDSSs.
2.28 Due to national security considerations, only the overall status from each project's assessment of the likelihood of delivering the required MOEs is contained in the 2009-10 MPR and earlier reports. Figures 10 and 11 present DMO's assessment of the percentage of MOEs that:
● it has a high level of confidence will be delivered (green);
● are under threat but still considered manageable (amber); and
● at this stage are unlikely to be met (red).
2.29 By way of illustration, Figure 11 shows that in relation to the FFG Upgrade, the project's own assessment as at 30 June 2010 is that it has a high level of confidence in delivering the requirements of half the MOEs contained in the project's MAA, while the project has assessed that the delivery of the remaining MOEs as under threat but this threat is considered manageable.
__________________________________________________________________
47 Department of Defence, Defence Capability Development Manual 2006, p.4.
48 There are on average around 10 individual MOEs within each MAA for projects covered by the 2009-10 MPR. Over time, the number of MOEs for a project may vary, due to amendments to the capability to be delivered.
49 Australian National Audit Office, 2008-09 Major Projects Report, Part 2, p.121.
50 An issue identified by the DMO is support for project managers to guide assessments and reduce the level of subjectivity in assessments. See ANAO, op cit.
51 See paragraph 1.9 on PDSS items out of scope for the ANAO's assurance review.