A number of governance mechanisms have been recommended over the years to deal with this problem. The most notable of these were the Gateway Reviews introduced by OGC in 2001 following Peter Gershon's review of procurement in central government. The planning and execution of major procurements were to be challenged at different stages, by a committee chaired by a person with no stake in the outcome of the project.107
Gateway Reviews have made an important contribution in challenging unrealistic assumptions, and they have since been adopted throughout most of the English-speaking world. They are similar to the 'black hat' processes employed by some private firms prior to making major investment decisions, which deliberately employ a devil's advocate to challenge the proponents' assumptions. However, there is evidence that 'black hats', internal due diligence processes and investment committees can fail even in the private sector, if a culture of robust challenge is not maintained. Governance solutions such as these cannot be relied upon if they are largely dependent on support from outstanding individuals in senior positions. Gateway Reviews have sometimes failed for the same reason.
The establishment of strong organisational boundaries between the policy and regulatory functions of government and the delivery agents has sometimes been used as a structural solution to this problem. This was implicit in the creation of distinct executive agencies throughout the late 1980s and 1990s.108 The 'Next Steps' initiative quickly delivered some significant improvements, and contributed to the later development of Public Service Agreements. But as the Institute for Government has pointed out, over time, 'Next Steps' became business as usual, the quality of leadership declined and the discipline was lost. 'By the mid-1990s, it became clear that the framework agreements for individual agencies were not necessarily incentivising improvement in performance'.109
In defence procurement, there has been an attempt to address this problem of unrealistic expectations through the erection of a clear boundary between the customer and the supplier functions. In Australia, the Kinnaird and the Mortimer Reviews recommended the functional separation of the Defence Materiel Organisation from the department, with its establishment as an Executive Agency of government.110 Here in the UK, the Gray Review proposed the separation and clarification of roles and accountabilities between the MoD centre and Defence Equipment & Support, with the suggestion that DE&S be established as a GOCO operated by a private firm.111 Lord Levene approved of a strong customer-supplier divide in his 2011 report on defence reform.112 In neither the UK or Australia were these recommendations fully adopted.
If it is organised properly, commissioning will do this same work of challenging the systemic self-deception. however as noted elsewhere in this report, commissioning is both a lever and a place to stand - commissioners must have the authority and the incentives to challenge as well as the capabilities. This is not to say that there always needs to be a separate commissioning agency, but it is fundamental that there is a distinct function (and hopefully, a recognised discipline).
If commissioners are well-trained and they are respected for being able to make a significant contribution to the ongoing relationship between policy to delivery, then they will be accorded the authority to challenge.
One model for what is being suggested here might be the Corps des Ponts et Chaussées, established by the French national government in 1716 to manage the concessionaires employed by government to construct its public works. With an associated school to train the engineers who would staff it, the Corps established a formidable reputation for professionalism and independence. The US Army Corps of Engineers was directly modelled on it, and the origins of cost-benefit analysis (and the marginal revolution in economics) lie in the work of the Corps des Ponts.113
In this case, a strong reputation for technical expertise endowed the engineers and proto-economists of the Corps des Ponts et Chaussées with the authority to challenge unrealistic demands made by the individuals and agencies responsible for policy and funding.