1.31 DMO's assessment of the likelihood of delivering the key capabilities (also called Measures of Effectiveness - MOEs) for major capital acquisitions, is specified publicly at the project level for the first time in the 2009-10 MPR. This data is set out as a 'traffic light' percentage pie chart in Table 3.5 of each project's PDSS. It is important to emphasise that MOEs predominately focus on the expected future achievement of certain technical, regulatory and operational requirements. Budgetary and schedule requirements are generally not included as an MOE. For example, the eight current MOEs for the FFG Upgrade project solely focus on the forecast achievement of certain technical capabilities.
1.32 As at 30 June 2010, while some MOEs are historical (for example, that a certain capability is satisfied by a date which has passed), the majority of other MOEs require DMO project managers to assess the likelihood of delivering in the future the aspects of the key capability for which the DMO is responsible.
1.33 Necessarily, this data usually involves making certain assumptions in forecasting achievements and is therefore subjective in approach (noted by the DMO in the 2008-09 MPR).36 In light of this subjectivity and inherent uncertainty, this information is excluded from the scope of the ANAO's review. Nevertheless, noting this caveat, the ANAO has included some analysis of the DMO's data relating to its capability forecasts in addition to our analysis of projects' performance in regard to budgeted cost and schedule in Chapter 2.
1.34 The ANAO's examination of MOEs, which are drawn from the Materiel Acquisition Agreements (MAAs) between Defence's Capability Development Group (as the purchaser) and the DMO (as the supplier), noted that the MOE framework is not sufficiently developed to ensure consistency in the level and scope of MOEs across projects.
1.35 DMO has recently advised that it is unlikely that MOEs will be reported in their current form in the 2010-11 MPR as DMO established a new MAA template at the beginning of 2010 for implementation in 2010-11. The new template does not include MOEs, but instead requires the specification of completion criteria for the achievement of materiel release to the ADF. In this context, the issue of key capability measures is likely to be a matter for consideration by the JCPAA and further examination by the ANAO in the 2010-11 MPR.
_____________________________________________________________________
36 Australian National Audit Office, 2008-09 Major Projects Report, Part 2, p.121.