Risk, Robustness and Water Resources Planning Under Uncertainty

Risk‐based water resources planning is based on the premise that water managers should invest up to the point where the marginal benefit of risk reduction equals the marginal cost of achieving that benefit. However, this cost‐benefit approach may not guarantee robustness under uncertain future conditions, for instance under climatic changes. In this paper, we expand risk‐based decision analysis to explore possible ways of enhancing robustness in engineered water resources systems under different risk attitudes. Risk is measured as the expected annual cost of water use restrictions, while robustness is interpreted in the decision‐theoretic sense as the ability of a water resource system to maintain performance—expressed as a tolerable risk of water use restrictions—under a wide range of possible future conditions. Linking risk attitudes with robustness allows stakeholders to explicitly trade‐off incremental increases in robustness with investment costs for a given level of risk. We illustrate the framework through a case study of London's water supply system using state‐of‐the ‐art regional climate simulations to inform the estimation of risk and robustness.


Introduction
The 21st century water manager faces significant challenges. Climate change, population growth, urbanization, and governance failures are putting pressure on limited and variable resources (AghaKouchak et al., 2015;Liu et al., 2017). These factors, alongside changing societal expectations with respect to the aquatic environment, frame a complex and rapidly evolving decision context in which water investments need to be evaluated (Grafton, 2017).
In most modern societies decisions with respect to water resources and, more broadly, water security are still largely based on cost-benefit analysis (CBA) (Stakhiv, 2011). There are many potential benefits of investments in water security, for agricultural, industrial, and domestic water users. However, these water services are subject to risks, in particular where water resources have been over-exploited. In the face of climatic changes, increasing demand for water and recognition of the need to restore the aquatic environment, a growing amount of decision-making relates to managing the risks to and from water (Alfieri et al., 2017;Gallagher et al., 2016;Grafton et al., 2016;Hall and Borgomeo, 2013;Veldkamp et al., 2016): risks from flooding, water scarcity, and harmful impacts on water quality. Managing water becomes a risk-based decision analysis problem (Garrick & Hall, 2014), whereby the benefits of risk reduction are compared to the value of the marginal reduction in risk achieved by an intervention with the marginal cost of that intervention.
Analyzing the risk of harmful outcomes, which is an integral part of this version of CBA, involves estimating the probability and consequences of risks (like flood, droughts, or pollution incidents) with and without the proposed intervention-which might entail investments in supply or storage infrastructure, measures to reduce leakage, or interventions to promote more efficient use of water. Hydrological frequency analysis of observed conditions provides the evidence to estimate these probabilities of harmful outcomes, alongside extensive stochastic simulations of water systems to sample spatial and temporal variability.
The risk-based decision process leads to a water resources plan that balances residual risks with the cost of the plan, subject to constraints, such as maximum allowed abstraction or minimum reservoir releases that guarantee ecological flows. Most modern water resources planning is based on these tenets of operational hydrology and rational planning introduced by the Harvard Water Program (Fiering, 1967;Maass et al., 1962;Matalas, 1967).
This approach hinges heavily on the validity of the data and models used as input to the hydrological frequency analysis. Anthropogenic influences on the water environment, from climate change to land use change, mean that this validity has come to be questioned (Brown 2010). Water investments now need to be evaluated in a moving nonstationary frame (Milly et al., 2008). This moving frame means that decision analysis methods based on probabilistic risk analysis of historical hydrological observations alone are no longer adequate to identify a water investment capable of meeting a service standard. In this context, the applicability of CBA based on probabilistic characterizations of uncertainty has been disputed as an approach to ranking decision alternatives in the face of large uncertainties, especially those associated with climate change (Dessai & Hulme, 2004;Lempert et al., 1996;Tol, 2003). For CBA to provide decision support under these circumstances, it needs to be reframed to account for the large uncertainties facing many types of decision challenges (Lempert, 2014).
The recognition of nonstationarity and need to expand the tenets of operational hydrology and traditional water system analysis has given rise to a range of methods for decision-making under uncertainty, such as Robust-Decision Making, decision scaling, vulnerability analysis, and dynamic adaptive policy pathways (for a review, see Maier et al., 2016). These approaches are based on a shared set of principles. First, they identify conditions under which the performance of the water investment becomes unacceptable prior to assigning probabilities to these conditions Groves & Lempert, 2007;Lempert et al., 2006;Nazemi et al., 2013;Prudhomme et al., 2010;Turner et al., 2014). Second, they emphasize robustness to uncertainty, broadly defined as a decision that performs acceptably well under a wide range of plausible future conditions Hine & Hall, 2010;Matrosov et al., 2013;Moody & Brown, 2013;Trindade et al., 2017). Third, these approaches highlight the importance of flexibility in water investments, that is, the ability to switch or change a decision depending on what outcomes materialize (Groves et al., 2015;Haasnoot et al., 2013;Hino & Hall, 2017;Jeuland & Whittington, 2014;Kwakkel et al., 2015;Woodward et al., 2011;Zeff et al., 2016). Fourth, they strongly emphasize the multiobjective nature of all water investments and the need to explore the trade-offs between these multiple objectives under uncertainty (Barbour et al., 2016;Giuliani et al., 2014;Huskova et al., 2016;Kasprzyk et al., 2013;Matrosov et al., 2015;Mortazavi et al., 2012;Paton et al., 2014;Reed et al., 2013).
Pursuing robust and flexible water investments may well make a lot of sense when water managers are operating systems and planning investments under "deep" uncertainties and face the possibility of extreme events. However, robustness is seldom cost-free (Ben Haim, 2006). There is usually a trade-off between the performance achieved by a system under ideal conditions and the robustness that system has across a wide range of uncertain, and possibly unforeseen, futures. As also observed in the socioecological systems literature, there is "no free lunch for robustness," meaning that it is fundamental to assess performance and robustness trade-offs associated with decisions (Janssen & Anderies, 2007;Muneepeerakul & Anderies, 2017). The notion of no-regret (or low-regret) decisions is attractive, but practically all decisions involve some cost-including opportunity costs of options and decision that may be foregone by deploying resources to implement a chosen plan. To justify a project, water managers therefore need to address the question of the value of robustness. How much should be paid for a given level of robustness? What is the 10.1002/2017EF000730 incremental cost of robustness? How does robustness relate to observable consequences of the selected water investment?
We investigate these questions from the perspective of a water resources master planning problem. Our approach aims to bridge the traditional risk-based approach to water planning with the emerging literature on robustness and decision-making under uncertainty. It is based on the idea that a water resources management decision under uncertainty is undertaken with a sound understanding of (1) its consequences, in terms of the costs of these investments (how much do proposed plans cost? how much will this cause water bills to increase?) and in terms of the benefits (how many fewer people will experience shortages? what will the avoided costs be?) and (2) its robustness, where we recognize the multiple definitions of robustness and argue that metrics suitable to the decision at hand should be selected. Faced with the need to develop long-term strategies, water resources managers seek decisions that achieve robustness over a long period of time and will do so by building flexibility in their water investments.
Building on these theoretical considerations, this paper presents two methodological innovations for risk-based decision analysis applied to water systems. First, it demonstrates how to trade-off robustness with the costs of water investments for different risk attitudes. Second, it estimates a risk metric that includes duration, severity, and frequency of water use restrictions using a super-ensemble (tens of thousands of elements) of weather sequences obtained from a state-of-the-art regional climate modeling experiment, called weather@home2 (Guillod et al., 2017a). weather@home 2 is part of the climateprediction.net project, which leverages the computing power of thousands of volunteers around the world to generate these very large ensembles (Allen, 1999). Weather sequences from weather@home contain synthetic drought events whose severity and frequency goes beyond the historical record, allowing for extensive stress testing of the decisions. Weather sequences from weather@home have been successfully applied to study the impact of climate change on the occurrence of extreme weather events (Haustein et al., 2016), flood-related property damage , and heat-related mortality (Mitchell et al., 2016), but never to a water resources planning problem, which is the focus and contribution of this study.
To demonstrate how risk-based decision analysis can be extended to explore trade-offs between risk, cost, and robustness, we use a single actor decision analysis problem of an existing water supply system in the United Kingdom. The water supply system is located in Thames river basin and serves more than 8 million water users, including the city of London. It presents the characteristics of many large urban water systems, faced with challenging investment choices to ensure supply security in the context of population growth, climate change, and uncertain regulatory constraints.

The Framework
The framework to identify a decision which robustly meets a tolerable level of risk follows four steps. The first step involves problem framing, including definition of objectives, inputs, decision alternatives, and system model construction. In the second step, risk analysis using synthetic climatic sequences is conducted to estimate a risk metric comprising the three dimensions of duration, severity, and intensity. In this second step, a classical CBA is carried out to explore the trade-off between risk and cost and inform the identification of a tolerable level of risk. In the third step, robustness to a wide range of futures is integrated in the analysis to explore trade-offs between robustness and cost for a given level of tolerable risk. In the final step, decision-makers select an option that robustly meets their risk attitude and monitor outcomes against their objectives to inform future decision evaluations. The steps are discussed in detail below and summarized in Figure 1.

Problem Framing
As in any decision problem, the analysis starts with stakeholders defining the problem at hand. This involves identifying water-related outcomes whose occurrence has economic, social, or environmental consequences. These water-related outcomes should be observable states of the water system. Examples of these outcomes include a water use restriction for domestic users, an environmental flows shortage, a reservoir level, or a water level in a canal. In this paper, we do not discuss the methodological choices available to quantify the economic, social, and environmental consequences linked to these outcomes, though we recognize that they make-up a fundamental component of the decision problem. Alongside outcomes, stakeholders also identify decision options. These alternative decision options can be searched using multiobjective optimization or predefined at the start of the analysis.
Following problem definition, a system model is constructed to assess the impact of changing conditions and alternative decisions on the occurrence of the outcomes. In the field of water resources, these would either be existing system models, for instance those commonly available in platforms such as Water Evaluation And Planning (WEAP) (Yates et al., 2005), Interactive River-Aquifer Simulation (IRAS) (Matrosov et al., 2011), WATHNET (Kuczera, 1992), or Source (Welsh et al., 2013), or models built with stakeholders to specifically analyze the problem at hand (Basco-Carrera et al., 2017) in different river basins around the world. The significant aspects of these models are that they are simulation models, so can simulate the frequency, severity, and duration of events of interest to water resources managers. Simple models based on bulk water supply/balance estimates are not informative about the observable outcomes of interest to decision makers. Where water quality is a potential constraint on abstractions, in particular during droughts, then it can be explicitly included through rainfall-runoff and river water quality simulation (Bussi et al., 2016a).
For initial assessment and screening of options, fast integrated models or metamodels are particularly useful instead of computationally expensive simulation models. Examples of these include Haasnoot et al. (2014), who used a fast model to screen and rank adaptation policies in the Rhine Delta, Hall et al. (2016), who developed a national-scale system model for integrated infrastructure planning and Beh et al. (2017), who used a metamodel to estimate robustness of water infrastructure investments. This can be followed by a more carefully constructed system model incorporating the full operational complexities to assess the performance in terms of risk-cost-robustness of a smaller set of options.

Risk Analysis
The second step uses extensive simulations of the system model to evaluate the consequences of observable water-related outcomes under uncertain climate (and other) conditions. In a traditional risk-based decision-making sense, assessing the consequences of a decision begins with quantifying the probability of not meeting a desired standard for that particular decision or condition. In water resources management, this has been interpreted as the probability of not meeting a desired frequency of water use restrictions of different levels of severity . This probability is estimated by counting the frequency of water use restrictions and their severity over a range of future time horizons simulated with a water system model (Borgomeo et al., 2014. However, counting restrictions is not a complete metric of risk to water users because, while it considers the frequency and severity of restrictions, it neither considers the duration nor does it explicitly consider the consequences of restriction. To address these limitations, we propose that the metric of system risk 10.1002/2017EF000730 should invariably combine the frequency and duration of restrictions of different severity-which encompasses all three of Hashimoto et al.'s (1982) criteria for evaluating the reliability, resiliency, and vulnerability of water resource system performance-and should be quantified in terms of the expected cost of water use restrictions, to explicitly consider the consequences of these water-related outcomes. Although metrics of expected costs of water use restrictions have been used in the past to evaluate water resources system performance (see for instance Zeff et al., 2014), to our knowledge they have never been quantified using a single risk indicator that combines information on restriction frequency, duration, and severity, as done in this study.
Given statistical estimates of the distribution of frequency and duration of restrictions, it is straightforward to apply economic metrics of consequence, for example, on the basis of water users' stated willingness to pay to avoid a day of restriction of given severity. We do this with caution, however, as willingness to pay surveys yield a wide range of valuations (Hensher et al., 2006) and have not, as far as we are aware, addressed willingness to pay to avoid prolonged restrictions-even though the impact of restrictions will not necessarily scale linearly with duration. The economic impacts of restrictions on multiple sectors can also be quantified using economic system models such as input-output models or agricultural production system simulators for agricultural water users.
Reporting metrics of risk requires extensive and explicit simulation of hydrological variability, extending well beyond historic droughts to include worse than observed conditions. The methodologies are available to do this either via stochastic streamflow (Borgomeo et al., 2015;Herman et al., 2016) and groundwater (Mackay et al., 2014;Prudhomme et al., 2013) simulation, or via regional climate simulations (Guillod et al., 2017a) or weather generators (Glenis et al., 2015;Steinschneider & Brown, 2013) coupled with rainfall-runoff and groundwater models. For spatially extensive systems this will involve consideration of spatial variability in the simulations (Serinaldi, 2009).
Based on the simulation outputs, stakeholders are able to identify a tolerable level of risk. This involves a classical risk-based decision, whereby the cost of achieving an incremental reduction in risk is compared to the benefits achieved by that reduction, in terms of reduced costs of water use restrictions. In practical terms, this involves identifying a level of risk which stakeholders deem tolerable .
Recent advances in climate modeling are of particular relevance to the risk analysis step. Novel approaches to represent high-resolution climatic processes combined with advances in the way in which climate models are run allow for the generation of superensembles of climatic conditions and extreme events at high resolution (Massey et al., 2015). These climate modeling platforms run global and regional climate models on distributed networks of volunteers' home computers, allowing for tens of thousands of weather sequences to be generated. As shown in this paper, these superensembles are an important element to inform the estimation of a risk metric which includes frequency, duration, and severity.

Robustness Analysis
The identification of a tolerable level of risk in step 2 does not formally deal with the robustness of the preferred plan. This makes the decision potentially optimal only under the conditions represented in the model input and parameters and motivates the introduction of a third robustness objective in the decision problem. Robustness has been considered as a decision objective in several studies (Beh et al., 2017;Mortazavi-Naeini et al., 2015;Ray et al., 2013), yet it has never been explicitly traded-off against cost and a risk metric comprising duration, frequency, and severity dimensions as done here.
Linking risk attitudes-expressed in the selection of a tolerable level of expected annual restriction costs-with robustness allows stakeholders to explicitly trade-off incremental increases in robustness with investment costs for a given level of risk. For a given level of risk, water managers will have to invest more to achieve more robustness. On the other hand, for a given plan cost, water managers will be able to achieve a tolerable level of risk at low robustness or, if they are prepared to tolerate a higher risk, achieve a higher robustness for the same level of investment. This optimization problem will involve at a minimum three objectives, though more objectives may be added if decision-makers want to compare alternatives based on multiple robustness measures. Evolutionary algorithms allow to solve this multiobjective optimization problem and find the Pareto-approximate set of alternatives .
Similar to recent applications of many-objective evolutionary algorithms to water resources system planning (Trindade et al., 2017;Watson & Kasprzyk, 2017) based on concepts from robust optimization (Hamarat et al., 2014;Mortazavi-Naeini et al., 2015), this step directly embeds a range of uncertain factors in the simulation-optimization process. Embedding uncertain factors in the decision search aims to generate water resources management plans that are robust to a range of future scenarios much wider than the one considered in step 2. This differs from other studies (such as Kasprzyk et al., 2013;Herman et al., 2014) where decision alternatives are generated and then their robustness to uncertainty is evaluated a posteriori, to identify combinations of uncertain factors that influence their ability to meet a required performance standard.
Multiple robustness measures exist, and the decision on which measure to adopt is itself an uncertain decision, because different definitions can lead to potentially different choices and outcomes McPhail et al., 2018;Mens et al., 2011). As discussed in Lempert and Collins (2007) and Herman et al. (2015), water managers often aim for decisions that minimize the cost of choosing incorrectly (regret measures) and that meet a minimum acceptable performance or service standard under the widest possible range of planning scenarios considered (satisficing measures), yet the choice of which robustness measure to include will depend on the decision context and stakeholder objectives. Maximin (select alternative with the least worst performance), maximax (select alternative that maximizes the best possible performance), minimax regret (select alternative that minimizes the worst-case regret) are examples of robustness measures that could be included as an objective in the multiobjective optimization.
This step seeks to address a major limitation of all robustness-based assessments; namely, the lack of clarity over how plans to achieve robustness may be altered under different risk attitudes. By explicitly framing the decision problem as a problem of trade-offs between plan cost and robustness under different levels of risk, this step helps water managers relate risk attitude (defined on the basis of water users' willingness to pay to avoid water shortages of different levels of severity) with choices over a preferred level of robustness.

Option Selection
Building on the analysis of the risk-robustness-costs trade-offs, stakeholders can then select and implement an option which robustly meets their tolerable level of risk. The combinatorial problem of option selection requires identifying an option (leakage reduction, wastewater reuse facility, etc.) or a portfolio of options and their sequencing. In this step, water managers will examine the ability of water resources management plans (either comprising one option or a portfolio of options) to achieve robustness over the long term. This can be achieved by integrating considerations of flexibility in the analysis using approaches such as real options analysis (Jeuland & Whittington, 2014) or dynamic adaptive policy pathways (Haasnoot et al., 2013;Kingsborough et al., 2016). In developing water resources management plans, water managers should also carefully examine the multiobjective optimization results to search for options that are consistently selected by the optimizer.
Implementation of the preferred decision takes place within a monitoring and evaluation framework. This is meant to inform future decisions and track the performance of the decision with respect to the level of tolerable risk over time and as conditions change. As discussed in the climate change and decision-making under uncertainty literature (Haasnoot et al., 2013;Rosenzweig et al., 2011), adaptation under uncertainty requires continuous monitoring to understand if risk tolerability thresholds have been surpassed and action is required. A significant increase (decrease) in the frequency, duration, or severity of the observable water-related outcomes identified in step 1 would lead to an acceleration (or delay) of a planning decision (Reeder & Ranger, 2011).

Problem Framing: Regional Climate Ensembles
Risk and robustness analysis of water resources systems require a much larger (of the order of tens of thousands) ensemble of sequences of weather patterns, including extreme weather events, than is otherwise available using the historical record alone. Climate model-based approaches allow for the generation of very large ensembles, called superensembles, of weather sequences over specified regions of the world to very high resolutions (25 km). Using these superensembles allows for the extensive sampling of spatial and temporal variability than has hitherto been possible in standard hydrological frequency and reliability analysis.
In this study, we use the large sets of daily weather sequences over the Thames basin from the weather@home2 platform (Guillod et al., 2017b). The climate modeling platform of weather@home2 is based on a Global Circulation Model, HadAM3P, downscaled with the Regional Climate Model HadRM3P from the UK Met Office Hadley Centre model (Gordon et al., 2000), both with a series of improvements in lower-resolution climate processes introduced by Massey et al. (2015) and in land-surface processes introduced by Guillod et al. (2017a). These improvements now allow for the generation of high-resolution 25 km RCM weather sequences over Europe, which have been shown to achieve a good representation of extreme events (Guillod et al., 2017a). Of particular relevance to our case study is the spatially coherent nature of the sequences, which allows for the spatial coherence of droughts to be taken into account when analyzing risks.
The synthetic weather sequences reproduce well historical weather observations in the Thames basin, as shown in Figure 2. Here we show results for precipitation only, a thorough validation and description of the bias-correction approach can be found in Guillod et al. (2017a and. Figure 2 displays the return values of low precipitation accumulated over 1-4 hydrological years in the weather@home2 ensemble (boxplots, years 1900-2006) and in the historical weather observations (points, years 1900-2006, data from the CEH-GEAR data set described in Keller et al., 2015). For one hydrological year, weather@home2 tends to overestimate precipitation drought for low return period (e.g., 5 years) but to underestimate droughts for rare events with greater return periods. Nonetheless, the observed values are always included within the range of return values from the 100 baseline simulation time series . For an accumulation time of 2 years, the return values are very well represented by the weather@home2 time series, while for longer accumulation times the model tends to slightly overestimate drought severity.
To simulate future climate conditions, weather@home2 was run to generate sets of 100 sequences, each of a length of 30 years. Two sets of projections are used in this study, both assuming a high-emission climate scenario (RCP8.5). The first set, referred to as Near Future (NF), represents climate for the 2020-2050 period and is interpreted as the best-estimate set of scenarios used to estimate the risk metric. The second set, referred to as Far Future (FF), projects climate variables for the 2070-2100 period and is intended to represent a wider range of futures for robustness testing. The 100 NF scenarios have been generated from multiple realizations of the same climate model, so are intended to represent multiple samples of climatic variability. They may therefore be taken to be equiprobable. The additional 100 scenarios, referred to as FF, represent hypotheses of possible climatic conditions to which the water resource system may be exposed, so provide the basis for testing the robustness of the system to more extreme future conditions. Figure 3 displays the return values of low precipitation accumulated over 1-4 hydrological years for the simulated baseline, NF, and FF scenarios. In all scenarios, the risk of low precipitation accumulated over 1-4 hydrological years increases, with a larger increase in droughts for the FF scenario. Figure 3. Return values of low precipitation accumulated over 1-4 hydrological years (columns) in the 100 time series for each scenario (boxplot color), for return times of 5-20 years (rows). Values are spatially averaged over the Thames basin. In all cases, NF and FF scenarios have statistically significantly different mean return value compared to the baseline (BS;years 1975-2004 at the 95% level according to a two-sided t-test.

Problem Framing: Simulation Framework and Decision Alternatives
The INCA hydrological model was used to reproduce the rainfall-runoff dynamics of the Thames basin. The INCA model is a semidistributed process-based rainfall-runoff model, which takes as inputs daily time series of rainfall and temperature. INCA uses a temperature-based method to compute evapotranspiration, and it computes soil moisture through a balance between net rainfall, evapotranspiration, infiltration, percolation, and subsuperficial flow (Futter et al., 2014;Whitehead et al., 2016). The validation and calibration of the model is explained in Bussi et al. (2016aBussi et al. ( , 2016b. Changes in groundwater availability in response to weather@home2 sequences are not modeled in this simulation framework, and are taken to be constant at the dry year annual average (a year in which unrestricted demand can only just be met by available supplies) following the method adopted by the water utility in the basin (Thames Water, 2014).
The runoff values generated with the rainfall-runoff model, assumptions about groundwater availability and data on domestic water demands (Thames Water, 2014) are used as inputs to the WATHNET model. WATHNET is a generalized node-arc simulation model using network linear programming to allocate water (Kuczera, 1992). Within WATHNET, nodes represent a supply source, demand centers or transfer points while arcs represent rivers or pipes/channels to transfer water between nodes.
WATHNET was selected because of (1) its efficient computation time and capability of running on parallel nodes, (2) the scripting feature which facilitates introducing any rules or constraints, and (3) its architecture that facilitates the implementation of multiobjective optimization and handling optionality. WATHNET has been successfully used in many water resources planning applications (Mortazavi et al., 2012(Mortazavi et al., , 2014(Mortazavi et al., , 2015. The decision alternatives considered in the case study are taken from the supply-side options list developed by the water managers in the basin and described in their Water Resources Management Plan (Thames Water, 2014Water, , 2017. The options considered are listed in Table 1. Additional information on these options including their capacity, operational, and capital costs are listed in the Supporting Information S1. For each Table 1.  (2017) option, constraints related to the earliest possible start date and mutual exclusivity with other options are applied. In this case study we limit the analysis to the options considered feasible by the water utility and the regulator in the basin. Other options, including changing system operating rules, were not considered because they are tightly constrained by environmental flow regulations. Demand management options are built into the model based on the implementation plan developed by London's water utility (Thames Water, 2014).

Supply-side Options Considered as Decision Alternatives in the Multiobjective Optimization
Population in the Thames basin is expected to increase in the coming decades and so is domestic water demand. In this study, three different water demand scenarios are employed-referred to as low, medium, and high-based on population growth projections and per capita consumption estimated by the basin's water utility (Thames Water, 2014). Under each scenario, demand increases by about 0.25%, 0.5%, and 0.75% per year as shown in the Supporting Information S1. Demand is assumed to be constant throughout the year and the impacts of climate change on demand, such as increasing per capita water consumption due to higher temperatures, are not modeled, following studies that suggest low sensitivity of water demands to climate conditions in the Thames basin (HR Wallingford, 2012).

Risk and Robustness Metrics
The existing water system regulatory rules, which provide limits on surface water withdrawals, are implemented in the model and used as the basis to compute risk and robustness metrics. In the Thames basin, water utilities impose different levels of restrictions on water use depending on the amount of water available in the system's pumped-storage reservoirs. The total volume of water stored in the reservoirs located in the lower part of the Thames basin in each month determines whether or not restrictions on use are imposed and to what extent. Water managers compare the volume of water stored with the reservoir thresholds defined in Figure 4 and implement a water use restriction of a given severity if the observed reservoir levels fall below one of the thresholds.
Four different reservoir thresholds exist, each implying a water use restriction L of severity i. Each level of restriction is associated with progressively more stringent measures to control domestic water use, shown in Table 2. For each level of restriction, water managers in the basin have also specified an expected frequency of occurrence and expected demand reduction listed in Table 2.   (2014) The economic consequences of water use restrictions are particularly challenging to quantify, as discussed in Section 2.2. In the Thames basin, estimates have been developed by the water utility to be 6.8 MGBP for the level 3 restriction and 282 MGBP for the level 4 restriction (Lambert, 2015). Estimates of the economic consequences of Levels 1 and 2 restrictions are not available and thus excluded from the analysis, though we note that they are expected to be orders of magnitude lower than the economic costs of Levels 3 and 4 restrictions (Lambert, 2015). Based on these system rules, the severity, frequency, and duration of water use restrictions can be estimated and combined with economic estimates of the consequences of restrictions (also listed in Table 2) to generate our proposed risk metric. The severity i of a restriction L is represented using the four levels of restrictions listed in Table 2. The frequency of restrictions is defined as the annual frequency f of a water use restriction of severity i, calculated as the number of years in the simulation where a restriction occurs over the total number of years in the simulation. Finally, duration is estimated as the length of time d the restriction remains in place (i.e., length of time simulated reservoir levels remain below one of the four thresholds shown in Figure 4).
Running the system model under the weather@home2 NF sequences allows for the joint probability density function of the frequency and duration of water use restrictions of a given level of severity to be constructed. The joint probability of the frequency and duration of water use restrictions of a given level of severity p i (d, f ) is combined with estimates C i of the economic losses associated with water use restrictions to estimate a 10.1002/2017EF000730 risk metric measured as the expected annual cost of a water use restriction L of severity i: In general terms, this risk metric essentially expresses the most likely annual losses due to water use restrictions. It condenses information on the probability of occurrence of a harmful event (the probability density function of frequency and duration) of a given severity and its consequences (the economic losses associated with the water use restriction). This risk metric is calculated under the NF climate projections from the weather@home2 superensemble and a medium population scenario. This represents a typical CBA approach whereby frequency analysis of the best-estimate hydrological conditions informs the estimation of the expected costs and benefits of different decision alternatives.
The minimax criterion was employed to measure robustness under a range of scenarios wider than that considered to estimate the risk metric in Equation (1). This criterion focuses on the worst possible performance among all the scenarios considered (Wald, 1950), selecting the alternative a* that minimizes the maximum expected annual cost of water use restrictions: where the expected annual cost of water use restrictions of severity 3 and 4 depends on the state of the world w Ω that materializes. In this case, the set Ω of possible states of the world contains all weather@home2 sequences and all demand scenarios described in Section 3.1, for a total of 600 scenarios (|Ω| = 600). This metric, which explores an extreme worst case, was selected to reflect the risk-averse nature of water managers who are responsible for public water supplies in a city of global economic significance. Alternative formulations of this robustness metric could be adopted, for instance by minimizing the worst first percentile across the ensemble of scenarios, as done in Quinn et al. (2017), which would be a more stable statistical quantity to focus upon.

Multiobjective Optimization
Based on the risk and robustness metrics described in the previous section, multiobjective optimization is employed to explore trade-offs between the expected annual costs of water use restrictions, the costs of the plan, and the robustness metric (defined as the worst-case cost). The three objectives optimized in the multiobjective optimization are described below.

Plan Cost
Minimizing the expected total present worth cost, including capital and operational expenditures linked with the plan, averaged across the near-future scenarios: where CPX i and OPX i represent the capital and operational expenditures associated with a plan for year t in the simulation and T is the total number of years in the simulation (30 in this case). N is the total number of scenarios, which is equal to 100. r is the discount rate, set to 4.5% in this study.

Expected Restriction Cost
Minimizing the expected restriction cost of Levels 3 and 4 restrictions averaged across the near-future scenarios: where E[C i ] is the expected annual cost of a water use restriction L of severity i imposed in year t, calculated with Equation (1). C i is the restriction cost of restrictions of severity 3 and 4 imposed in year t.

Worst-Case Cost
Minimizing the maximum restriction cost over all climate (NF and FF) and demand scenarios: where Ω is the set of 600 scenarios resulting from all possible combinations of weather@home2 and demand scenarios.
The first two objectives are evaluated under the NF weather@home2 sequences and a medium population growth scenario (100 total scenarios). The third objective is evaluated under all possible weather@home2 and population scenario combinations (600 total scenarios). This choice was dictated by the need to evaluate robustness over a much wider range of plausible conditions than the best-estimate NF scenarios used for the estimation of the risk metric.
The decision variables considered in this study include the supply-side options listed in Table 1 and their times of implementation, for a total of 32 decision variables. Decision variables related to operational or drought contingency measures were not considered, as these are subject to strict environmental regulations. Demand management options are implemented following London's Water Resources Management Plan (Thames Water, 2014).
Multiobjective optimization was employed in order to evaluate the trade-offs between the objectives for sets of options. An -dominance multiobjective evolutionary algorithm ( MOEA) was used, as this algorithm has been successfully applied to solve water resources planning problems Mortazavi-Naeini et al., 2012, 2015 and has been demonstrated to have consistently high levels of performance compared to other MOEA for water resources management applications (Zatarain Salazar et al., 2016). MOEA uses the -dominance concept to divide the objective space into hyperboxes of size and allows only one nondominated solution to reside in each box (Laumanns et al., 2002). Inclusion of this concept in an evolutionary algorithm produces a method capable of maintaining a diverse and well-distributed set of solutions with a small algorithmic computational cost (Deb et al., 2003).
Based on previous applications of MOEA to water resources optimization studies (Mortazavi-Naeini et al., 2012, 2015 the following MOEA parameters were set: (1) probability of crossover = 1.0, (2) probability of mutation = .001, and (3) probability of inversion = .001. The maximum number of iterations was set to 20,000, again based on previous applications of the algorithm to water resource system planning problems (Mortazavi-Naeini et al., 2012, 2015. The MOEA epsilon values were set to 100 for the first objective (capital and operational energy costs) and 0.01 for the other objectives to be sufficiently small to ensure high resolution. The termination condition was defined as either reaching the maximum number of iterations or no changes in the Pareto frontier for 100 iterations. The multiobjective optimization was performed using 20 nodes, each with 16 cores, for a maximum run-time of 80 h. More details on the MOEA can be found in the Supporting Information S1.

Risk-Cost Trade-Offs
The Pareto-optimal plans identified under a single set of scenarios (NF climate scenarios from the weather@home2 superensemble and medium population growth scenarios) are shown in Figure 5. These are the results obtained using a classical CBA approach informed by the analysis of risks of water use restrictions under a set of best-estimate hydrological conditions. Each point in Figure 5 represents a different 30 years long plan. For each plan, the optimizer estimates the total plan cost (capital and operational expenditures, shown on the horizontal axis) and the expected annual cost of water use restrictions (shown on the vertical axis). As expected, minimizing the expected restriction cost, which means minimizing the risk of having to impose costly water use restrictions, comes at an increasingly higher investment cost.
These results are helpful to focus stakeholders' attention upon the level of risk they are willing to tolerate and pay for in their water resource system. Based on their risk attitude, water managers may identify a tolerable risk threshold, shown as a dotted line in Figure 5. In principle, the selection of the tolerable risk threshold hinges upon a classical risk-based decision-making process whereby the plan located on the point where the slope of the cost-risk curve is 45 ∘ is selected (triangle in Figure 5). This plan ensures "optimal" risk conditions, that is, conditions where the marginal benefit of an increment in risk reduction (in terms of avoided restriction costs) is equal to the cost of that increment incurred in the water resources plan. In this typical risk-based decision where robustness is not considered, the plan that achieves the "optimal" risk reduction would involve (beyond the demand management options detailed in Thames Water, 2014) the conveyance of treated wastewater just downstream of London's water supply abstraction point in the River Thames to allow for more abstraction (called "Teddington Direct River Abstraction") activated in 2039 and a small transfer (15 ML/day of additional supply) from the River Mythe activated in 2044.
Water managers with a conservative risk attitude would select plans located in the bottom-right of Figure 5, where higher investment costs are incurred because in addition to direct river abstraction and a small transfer, the optimizer also selects a desalination plant to augment supplies and reduce the expected annual restriction cost.

Robustness Trade-Offs
In practice, the plan selected in Figure 5 may not be robust to uncertainty. Identifying plans which meet a level of tolerable risk and are robust to uncertainty requires explicitly trading-off robustness with risk and cost as part of step 3 in our framework. The trade-offs between risk, cost, and robustness are shown in Figure 6. Water managers would prefer a plan (a point in Figure 6) that reduces all three costs (plan cost, expected annual cost, and worst-case cost of water use restrictions) to zero. However, this may not possible because reducing the expected and worst-case costs of water use restrictions comes at an increasingly higher plan cost. Water managers may then seek plans which keep the expected cost of restrictions at a tolerable level, and also minimize the worst-case restriction costs, thus ensuring robustness to a wide range of futures under a given risk attitude.
Plans capable of minimizing the expected annual restriction and worst-case costs are located in the bottom right in Figure 6. The starker trade-offs are between plan cost and the risk reduction and robustness objectives, implying as expected that risk reduction and enhanced robustness come at increasingly higher costs. However, a closer examination demonstrates how under a given tolerable risk threshold, water managers may be able to attain significantly different levels of robustness. The parallel coordinates plot in Figure 7 shows how plans that have a similar risk profile (i.e., plans with the same expected annual cost of restrictions) can achieve different levels of robustness. In Figure 7, the horizontal axis shows the three objectives. The performance of each plan with respect to these three objectives is represented as a line, with the direction of preference always being downward for all objectives. Two sets of plans are highlighted in Figure 7, each representing a different risk attitude. The same plans are also circled in yellow and red in Figure 6.
Under the first set of plans (points circled in yellow in Figure 6 and yellow lines in Figure 7), water managers have a moderate risk attitude. The two plans selected achieve approximately the same level of expected annual restriction cost; however, they have different abilities to minimize worst-case costs. An increase in plan cost (about 30%) would lead to a 55% increase in robustness for the same level of risk. In this case, the plans differ in the time of implementation of option 4, which is implemented earlier on in the planning period for the more robust plan.
The level of robustness that can be attained under a more risk averse attitude is shown by the points circled in red in Figure 6 and red lines in Figure 7. For this level of risk and a 15% difference in plan costs, two significantly different levels of robustness may be attained, with one solution having a much greater ability (about 35% more) of minimizing the worst-case cost, thus achieving a much greater robustness.
Inclusion of risk and robustness metrics in the same multi-objective problem allows evaluation of the value of robustness, showing stakeholders how the preferred plan and the costs to achieve robustness change under different risk attitudes. The results show that risk reduction and robustness are not conflicting objectives; however, they also illustrate that under a given level of risk different levels of robustness may be attained. This suggests that water managers should expect starker trade-offs between plan cost and the risk reduction objectives, but that they should also closely examine risk reduction-robustness trade-offs. This helps define their willingness to pay for incremental increases in robustness under a tolerable level of risk or, alternatively, their willingness to accept a higher risk (i.e., a higher expected annual cost of water use restriction) to achieve greater robustness for the same plan cost (i.e., a lower worst case cost). To show how different levels of robustness may be attained depending on the options and times of implementation selected, results from the optimization are further scrutinized in Figure 8. Figure 8 shows the minimum worst-case cost attained by the options listed in Table 1 and related times of implementation selected by the optimizer. Figure 8 essentially shows the options and times of implementation that achieve the minimum worst-case restriction cost and so better enhance plan robustness. It does not show all possible combinations of all options and times of implementation, focusing instead on the individual options and times of implementation that attain the minimum worst-case cost.
Options that are consistently selected include direct river abstraction (with treated wastewater effluent being put into the river just downstream of the abstraction point) (option 4 and 15), desalination (option 5), a small capacity water transfer (Mythe option 14). Other options, such as the three large water transfers (options 1, 2, and 3), one of the reverse osmosis desalination options (option 7) and the new reservoir (option 6) are never selected by the optimizer. This could have to do with their excessive costs compared to other options. Early implementation of options 4, 8, and 15 is associated with low worst-case costs and greater plan robustness. Robustness of most options progressively deteriorates after 2040, suggesting that in order to achieve robustness water managers need to act in the next 5-10 years.

Discussion and Conclusions
Water management decisions are set in a rapidly evolving, complex, and uncertain context, which makes classic approaches based on risk analysis of historical hydrological variables no longer adequate. This paper has presented a framework for water resources management under uncertainty that links risk-based decision-making and robustness analysis. The framework takes as its starting points the challenges of climate nonstationarity and the importance of robustness to uncertainty to suggest that water management requires a new approach to decisions based on the analysis of trade-offs between robustness and cost under different risk attitudes. We identify this as an alternative to traditional water management approaches based on CBA using hydrological frequency analysis of historical conditions. The development of the framework was further motivated by the recognition of the costs that come with increasing robustness and the impossibility of being robust to all uncertainty. Explicit understanding of the trade-offs between robustness and costs for a given level of tolerable risk becomes then a core element to all water management decisions. In practice this means that water managers will need to search for options that work well under a wide range of possible futures, but they will also need to explicitly trade-off incremental increases in robustness with investment costs for a given level of risk.
Through the application of the framework to a case study, we have shown that different decisions may be reached, and therefore different levels of robustness attained, when risk attitudes (expressed as willingness to pay to avoid a water use restriction of a given severity) are explicitly considered and traded-off with robustness and costs. The framework accommodates best-estimate climate projections to drive risk analysis, yet it also considers robustness to a wider range of futures (called here far futures).
Although the issue related to flexibility is not explicitly considered in this paper, we acknowledge that it is an essential principle for water management under uncertainty. Flexibility considers the dynamic nature of decision-making and the human impact on the water environment, which can influence the occurrence of water-related outcomes. Insights from the application of methods for dynamic adaptive planning (see Kingsborough et al., 2016 for an application to the case study area) and for the modeling the human influences on water-related outcomes (e.g., Hale et al., 2015) will inform future work seeking to integrate flexibility in the analytical framework presented here. Future work will also seek to trade-off different robustness measures, as decision outcomes may be under or overestimated and alternatives may be obscured if a single metric of robustness is adopted .
Both sides of the risk equation-the probability of occurrence of harmful outcomes and the valuation of their consequences-can be subject to incomplete and problematic knowledge. The method proposed here goes beyond traditional risk analysis by explicitly including robustness to uncertainty, which helps to account for our incomplete knowledge about probabilities of future outcomes. On the valuation of the outcomes, the method was applied to an engineered system with known conditions and with known evidence of the consequences of water-related outcomes. In open systems with contested framings and disagreements about consequences, it may be more difficult to apply such methods given ignorance or problematic knowledge about water-related outcomes and their economic and social consequences (Stirling, 2007). Under these conditions, sensitivity to uncertain valuation of outcomes should be explored.
For many water management decisions under uncertainty and where a risk framing is appropriate, setting a tolerable risk threshold will be far more challenging than it appears in the case study application described here. In some instances, the level of tolerable risk might be defined by regulation (as it is often the case for flood protection standards). Under other circumstances, the level of tolerable risk may be identified using participatory risk management techniques (Döll & Romero-Lankao, 2017).
The most general conclusion from this study is that risk-based decisions or robustness-based decisions alone will not tell water managers all they need to know and that linking the two-within an adaptive and learning planning approach-warrants the possibility of a "resilient" decision. 2016 Food Energy Environment and Water (FE 2 W) network meeting in Bellagio for interesting discussions on some of the issues explored in this paper. The authors would like to thank Gianbattista Bussi for providing the INCA river flows. Chris Lambert's guidance and discussions on the Thames' system characteristics are gratefully acknowledged. Skillful information design advice from Brian Baldassarre is acknowledged. The data used are listed in the references, and the weather@home2 sequences can be downloaded from the Center for Environmental Data repository http://catalogue.ceda.ac.uk/uuid/ 0cea8d7aca57427fae92241348ae9b03. Two anonymous reviewers provided helpful comments and suggestions on this paper. This work was undertaken within the MaRIUS project: Managing the Risks, Impacts and Uncertainties of droughts and water Scarcity, funded by the Natural Environment Research Council (NERC), and undertaken by researchers from the University of Oxford (NE/L010364/1). The authors would like to acknowledge the use of the University of Oxford Advanced Research Computing (ARC) facility in carrying out this work. https://doi.org/ 10.5281/zenodo.22558