Project snapshot-Time elapsed and project maturity progress

2.26  The DMO has acknowledged that schedule remains its biggest concern in delivering and sustaining equipment for the Australian Defence Force (ADF).67

2.27  The third snapshot, Figure 8, sets out each project's percentage of scheduled time elapsed against the project's maturity score progress.68 This highlights a variance between some projects' scheduled time elapsed and the maturity scores, which is a result of projects receiving 50 per cent of the total maturity score at Second Pass Approval. By benchmarking the maturity score progress against the scheduled time elapsed, any indication of potentially overly optimistic assessments of a project's maturity can be assessed.69

2.28  While both the project maturity score progress and a project's scheduled time elapsed are indicators of a project's overall performance, the ANAO's analysis indicates that the application of a preset benchmark score for all types of projects may not depict a project's progress to FOC as accurately as an objective assessment.

2.29  Lead times and developmental risks vary considerably between the various types of procurement within the Australian Defence Organisation, i.e. between projects classified from Military-Off-The-Shelf (MOTS) to Developmental. In developmental projects a greater period of time is required for arriving at the Capability Managers' agreed requirements and specifications than for Off-The-Shelf acquisitions, and also for the production and integration of elements of the project.

Figure 8
Project snapshot-Time elapsed and project maturity progress (percentage)

2.30  Figure 9 re-orders the projects by their procurement types, as reported by the DMO i.e. MOTS, Australianised MOTS (AMOTS) and Developmental70, and presents the variance between the maturity score progress and the scheduled time elapsed for each project. This allows for a comparison between project types.

2.31  In the case of Additional Chinook and 155mm Howitzer, which the DMO classifies as MOTS, a preset benchmark of 50 per cent of maturity for Second Pass Approval provides a significant variance between the reported maturity score progress and the scheduled time elapsed which, as indicated above, is an indication of a potentially overly optimistic assessment of project maturity. For example, the variance between the maturity score progress percentage and scheduled time elapsed percentage for Additional Chinook is 44 per cent and in the case of 155mm Howitzer is 37 per cent.71

2.32  This is similar for projects classified by the DMO as AMOTS. For example, the variance between the maturity score progress percentage and scheduled time elapsed percentage for AWD Ships is 41 per cent and in the case of LHD Ships is 29 per cent.

2.33  For the majority of projects classified by the DMO as Developmental, the variance between the maturity score progress percentage and scheduled time elapsed percentage are much closer as a comparative indicator of the progress of the procurement process than for MOTS projects. For example, the variance between the maturity score progress percentage and scheduled time elapsed percentage for Air to Air Refuel is nine per cent and in the case of HF Modernisation is 10 per cent.

Figure 9
Selected project snapshot-Variance between maturity score progress and time elapsed (percentage)

__________________________________________________________________________________

67  Joint Committee of Public Accounts and Audit, Report 422, Review of the 2009-10 Defence Materiel Organisation Major Projects Report, Submission No.3: Materiel Organisation.

68  The DMO have advised that the project maturity score concept was not designed for a strictly linear representation of a project's progress. However, the ANAO's analysis has utilised a linear representation for comparative purposes in Figures 3, 8, and 12.

69  Source 1: Second Pass Approval is the point at which the government approves a project proceeding to the acquisition phase. Responsibility, authority and accountability for management of the acquisition phase of the materiel life cycle are vested in the DMO's line management, the focal point of which is the designated Project Manager for an acquisition project (DMO Acquisition and Sustainment Manual, p. 65).

Source 2: The maturity score is defined by the DMO as the maturity of a project by way of an objective score based on the project managers' judgement at defined milestones in its capability development and acquisition phases. Australian National Audit Office, 2010-11 Major Projects Report, p. 524. The more accurate the maturity score then the more reliable the estimate of a project's progress and the greater the confidence the project team would have in pursuing outstanding requirements.

70  Australian National Audit Office, 2010-11 Major Projects Report, Part 2, Table 2.3, p. 146.

71  Variances for Figure 9 have been derived from the data in Figure 8, i.e. for Additional Chinook the maturity score progress percentage (63 per cent) has had the scheduled time elapsed percentage (19 per cent) subtracted from it to give a total variance of 44 per cent.