8 After the Directive in 1991, it took a considerable time to develop measurable standards and to appraise options capable of meeting those standards. As the Directive does not specify thresholds for compliance, the Department endorsed standards which the Environment Agency and Thames Water developed. Between 2000 and 2005, the Thames Tideway Strategic Study Steering Group analysed the impact of overflows, proposed environmental objectives, and criteria to define satisfactory performance against those objectives. This resulted in threshold-based standards for dissolved oxygen levels, and rules defining unsatisfactory CSOs along the Tideway which solutions would have to address. With the Department's agreement, Thames Water used these criteria in its appraisal of potential solutions, which concluded in 2010 (Figure 2 and paragraphs 1.3, 2.2 to 2.4 and 2.12 to 2.14).
9 The Department's objectives were broader than simply complying with the Directive, and this is reflected in the criteria used to appraise potential solutions. We have two observations on these criteria:
a The evidence was more robust for some criteria than others. We found that the dissolved oxygen standards were supported by a rigorous scientific approach, which was favourably peer-reviewed by an independent academic and comparable to other English standards for environmental regulation. Thresholds for identifying unsatisfactory CSOs were based primarily on the Environment Agency's judgement which it only reviewed internally (paragraphs 2.7 and 2.8).
b There was a degree of contingency in the criteria used to test whether the options considered complied with environmental standards. The Environment Agency agreed with Thames Water that a maximum of four spill events in a modelled 'typical year' would deliver satisfactory environmental performance and compliance with the Directive. The Tunnel is designed to achieve this 'four spills' criterion. Data from 2016 suggests that the number of spills can exceed this threshold without any breaches of the dissolved oxygen standards, and some European Union member states have set less conservative maximum spill thresholds. The Department considers that inferences should not be made from a single year of data, and that thresholds adopted by other member states to demonstrate compliance are not relevant comparators, because of differences in the nature of overflows and water areas affected. The Department considers that the 'four spills' level of protection reflects the Government's aim to achieve its environmental and legal objectives well into the future, having regard for the likely impacts of population growth and climate change (paragraphs 1.7, 2.4, 2.11 and 2.16).
10 In 2007, the Department endorsed the Tunnel based on Thames Water's assessment that it was the lowest-cost option capable of achieving its objectives by 2020. We found options were appraised primarily based on their ability to achieve the dissolved oxygen standards. Our review found a wide range of options had been considered although, after the Department's 2007 decision, Thames Water's analysis of alternatives was less detailed; its costing of alternatives was not independently scrutinised; and combinations of alternative technologies were not appraised. The Department reviewed its position and concluded in 2014 that delaying the Tunnel to consider alternatives further would likely increase the risk of fines for breaching the Directive (paragraphs 2.12, 2.15, 2.16 and 2.19).
11 The Department and the Environment Agency did not fully explore uncertainty in the modelling before endorsing the full tunnel option. Models to forecast spills and dissolved oxygen levels played a key role in eliminating alternatives to a full-length tunnel (Figure 8, page 24). They were used to conclude that all alternatives except in-sewer separation would fail the dissolved oxygen standards and to set the 'four spills' criterion itself. The Environment Agency could not provide us with evidence that it had sufficiently understood the impact of uncertainty on the outputs from the models. The Environment Agency's consultants in 2007 reported that the models could predict dissolved oxygen levels which were overly pessimistic when compared to measurements in practice, and made recommendations to refine the modelling in 2009. The Environment Agency partially adopted these recommendations but has not carried out another validation exercise since 2007. The Environment Agency told us that, although it would have been possible to increase confidence in the model results by obtaining more extensive data sets, it did not consider that any of the areas of uncertainty with the results were sufficient to justify the costs necessary to obtain any improved data (paragraphs 2.17 and 2.20).
12 Correcting for inaccurate predictions could have resulted in a smaller, lower cost tunnel. More accurate modelling is unlikely to have affected the choice of a tunnel as the strategic approach, given assessments that alternatives would either fail to meet all key objectives or do so at significantly higher cost. However, it may have resulted in a smaller, lower cost design of the preferred 'Full Tunnel' option. Refinements to Thames Water's sewer model after 2007 indicated that the planned capacity of the Tunnel would considerably outperform the 'four spills' threshold. This allowed Thames Water to reduce the length of the reference design by 9 kilometres in 2009, saving £646 million, while still achieving the 'four spills' threshold. Further refinements to the modelling could have identified the potential for further capacity reductions, albeit through reducing the diameter of the tunnel, which estimates suggest reduces costs relatively less than reducing the tunnel's length. The Department considers that a tunnel of smaller diameter would not have cost significantly less, based on Thames Water estimates from 2006. It considers that a smaller diameter tunnel would have carried a greater risk of non-compliance and fines, and that, following the 2012 ruling, the European Commission would have known it was possible to capture more spills with minor cost increase. The Department considers that the cost of rectifying a tunnel with inadequate capacity would be prohibitive, and that the Tunnel chosen offers greater certainty that the tunnel will be "future-proof" (paragraphs 2.18 and 2.21 to 2.24).