Uncertainty in Seasonal Runoff Forecasts Affects Water Management Decisions

Heterogeneous snow accumulation in the mountains introduces uncertainty to water-supply forecasting in much of the world. Water managers’ awareness of the challenge may account for forecast errors in management decisions. We assess the impact of uncertainty in seasonal-water-supply forecasts on reservoir management using the western slope of the Sierra Nevada of California as a case study. We find that higher forecast uncertainty decreases the volume of water released from reservoirs between April and July, suggesting that water managers hedge against the possibility of lower-than-expected runoff. We modeled April-July water releases as a function of corresponding runoff forecasts, their reported uncertainty, and available storage capacity. An unbalanced (n=416) panel data model with fixed effects suggests that if uncertainty goes up by 10 units, water managers reduce releases by about 6 units, even holding the mean forecast constant. The forecast volume, its uncertainty, available storage capacity, and the interaction between forecasted volume and uncertainty were all statistically significant predictors (p < 0.005) of releases. Increased forecast uncertainty and increased available storage were significantly and inversely associated with April-July release volume, whereas forecast volume and the interaction between forecast uncertainty and forecast volume were significantly and positively associated with release volume. These results support the hypothesis that water managers behave as if they are risk-averse with respect to the possibility of less runoff than forecasted. Thus, reducing operational forecast uncertainty may result in more water being released, without the need for direct coordination with water managers.


Abstract 1
Heterogeneous snow accumulation in the mountains introduces uncertainty to water-supply 2 forecasting in much of the world. Water managers' awareness of the challenge may account for 3 forecast errors in management decisions. We assess the impact of uncertainty in seasonal-water-4 supply forecasts on reservoir management using the western slope of the Sierra Nevada of 5 California as a case study. We find that higher forecast uncertainty decreases the volume of 6 water released from reservoirs between April and July, suggesting that water managers hedge 7 against the possibility of lower-than-expected runoff. We modeled April-July water releases as a 8 function of corresponding runoff forecasts, their reported uncertainty, and available storage 9 capacity. An unbalanced (n=416) panel data model with fixed effects suggests that if uncertainty 10 goes up by 10 units, water managers reduce releases by about 6 units, even holding the mean 11 forecast constant. The forecast volume, its uncertainty, available storage capacity, and the 12 interaction between forecasted volume and uncertainty were all statistically significant predictors 13 (p < 0.005) of releases. Increased forecast uncertainty and increased available storage were 14 significantly and inversely associated with April-July release volume, whereas forecast volume 15 and the interaction between forecast uncertainty and forecast volume were significantly and 16 positively associated with release volume. These results support the hypothesis that water 17 managers behave as if they are risk-averse with respect to the possibility of less runoff than 18 forecasted. Thus, reducing operational forecast uncertainty may result in more water being 19 released, without the need for direct coordination with water managers. 20

Plain Language Summary 21
Over a billion people around the world rely on snow and ice for their water supply, and in many 22 areas, reservoirs store the water once the snow melts. Deciding when to let water out of the 23 reservoirs depends on forecasts of how much more rain and snowmelt will flow into the 24 reservoirs. Often these forecasts express uncertainty, reporting a wide range of possible flows. 25 As a case study, we use the historical record of past forecasts and water releases in California's 26 Sierra Nevada to examine how the people responsible for releasing water from reservoirs 27 respond to forecasts. Results suggest these water managers hedge their bets against the 28 possibility of less water than forecasted. Greater uncertainty in a forecast was significantly 29 associated with a reduced amount of released water. Reducing forecast uncertainty can support 30 data-driven decisions in water-resource management by increasing water managers' confidence 31 in upcoming inflows. Our results suggest that reduced uncertainty could allow more water to be 32 released from reservoirs earlier in the year. 33

Introduction 39
Mountain snowmelt is the primary water supply for over a billion people, and recent 40 estimates place the annual value of snow globally at $1 trillion (Sturm et al., 2017). Despite 41 snow's economic significance, the harsh weather and terrain of the mountain environment 42 constrain our understanding of the volume of water in the snowpack each year. This crucial 43 water supply varies from year to year and place to place, which adds uncertainty to water 44 resource management because decisions rely on predictions of future snowmelt runoff. Forecasts 45 of runoff, derived from measurements, models, and climatology, are uncertain and often wrong. 46 Through years of experience, water managers know that these forecasts contain errors. A 47 pressing question arises, applicable to both the gauged and ungauged regions of the world: How 48 consequentially does the uncertainty of forecasts affect water management? The traditional 49 approach to answering these questions involves running hydrologic and decision-optimization 50 models, tuning parameters, and then observing modeled changes in operations (Ajami et al.,51 2008). In theory, accurate forecasts have value. In practice, forecasts have operational value only 52 if water managers respond to them. In this paper we empirically examine how water managers 53 respond to forecasts by analyzing a historical record of operational water supply forecasts and 54 the corresponding water management decisions. 55 To optimally manage reservoirs in headwater basins during the snowmelt season, water 56 managers must balance the benefits of filling reservoirs with water for drier months with the 57 opportunity cost of not sending water down the river for other uses. Forecasts help water 58 managers decide when and how much water to release. The tradeoff between the value of an 59 early forecast versus the value of a later but more robust and accurate forecast has been explored 60 within optimal stopping theory known as the "commitment" problem under forecast uncertainty: 61 decisions on allocations of water and ensuring available storage space for incoming runoff must 62 be made before the runoff has occurred (Krzysztofowicz, 1986 decision-making environment for water operators during the spring runoff season is marked by 77 the need to fulfill water contracts by making decisions with imperfect hydrologic information. 78 Our empirical analysis of the historical record examines how water managers respond to 79 operational forecasts. A first principle, which we test, is that water managers will release more 80 water if they anticipate higher runoff, all else equal. This hypothesis seems uncontroversial, and 81 indeed we confirm it. A more nuanced question, and the one on which we focus, is how forecast 82 uncertainty affects water releases. Holding the expected forecast constant, does a more uncertain 83 forecast cause managers to release more, or less ? Or does the degree of forecast uncertainty have  84  no effect? To test these hypotheses, a fixed-effects panel regression model is used in a case study  85  of basins on the western slope of the Sierra Nevada to determine the association between April-86 through-July runoff forecasts, their uncertainty, and volumes released from the reservoirs. 87 2 Methods and Data 88

Study Area 89
California's Sierra Nevada provides an ideal case study location due to both the 90 economic importance of water in the state and the extensive historical record of publicly 91 available water data to model the relationship between operational forecasts and water 92 management decisions. Figure 1 portrays  California has some of the best-instrumented rivers and monitored watersheds in the 131 world. The Sierra Nevada are safe and accessible, water in the state is such a valuable good with 132 no substitutes, and there is limited storage space available, all suggesting a high value of 133 information from forecasts. California has a century-long history of observing and measuring the 134 snowpack in the Sierra Nevada (Church, 1914 spring runoff based on estimated year-to-date rain and snow and expected precipitation (Huber & 139 Robertson, 1982). Snow courses, linear transects of measured snow water equivalent, provide the 140 oldest form of field data used for the forecasts (Church, 1933). Eventually, the field data were 141 expanded to include snow pillows, 123 telemetered stations scattered around the Sierra Nevada 142 in largely flat clearings at mid elevations. California has consistently invested in the effort 143 needed to collect snow data for spring flood and water supply forecasts: in 2019 inflation-144 adjusted US dollars, $220,000 per year in 1930, over $560,000 per year by 1954 (Strauss, 1954), 145 to the latest published estimate in 2003 of $5.5 million (Roos, 2003). These forecasts comprise 146 the baseline of information available to water managers for making informed decisions on 147 storage and releases from reservoirs in preparation for the upcoming summer. 148 The end, errors can never be worse than -100%. On the positive end, one forecast in ten shows errors 166 of +50% or more, and one forecast in fifty shows errors of +100% or more. Negative errors 167 indicate an under forecast, there was more flow than expected; positive errors indicate an over 168 forecast, less flow than expected. 169 Reservoir managers rely on a combination of operational forecasts, predetermined 170 allocations of water, and flood risk management rules for guiding operations and determining 171 water releases. In the Central Valley of California, below the Sierra Nevada, as is common 172 among many water projects, delivering contracted water to customers mainly drives decision 173 making. The largest water systems in the state, California's state water project (SWP) and the 174 U.S. Bureau of Reclamations Central Valley Project (CVP), provide water to 279 contractors 175 which supply water to more than 30 million people and over 6 million hectares of farmland. 176 Delivering allocated water supplies drive these major water operations in the spring, summer, 177 and fall. 178 Water management decisions should relate to operational forecasts of water supply. 179 Measured accumulated precipitation and spring runoff forecasts in the Bulletin 120 likely 180 influence the decisions to increase or decrease the annual allocations of water within these major 181 water systems. Prior to the accumulation of winter precipitation, SWP receives requests for 182 annual water allocations from the 29 contractors it supplies. In November, still prior to any 183 significant annual precipitation, SWP determines the fraction of each contractor's requested 184 water to be allocated for the coming water year. These allocations are updated in public notices 185 to SWP contractors, up to a few times a year, usually before 01 April (CADWR, 1963(CADWR, -2019 Aside from delivering water to customers, surface reservoirs also provide protection 198 against floods, reducing flooding in the communities and irrigated lands below these 199 mountainous watersheds. During the winter months, maximum stored volumes in each reservoir 200 are independent of forecasted flows, allocated water supplies, or anticipated environmental 201 releases required in the summer. Maximum stored volumes through the end of March are fixed 202 lower than the available space to store water in each reservoir to reduce the risk of flooding from 203 rain during large winter storms. After the risk of flooding from winter rain subsides, flood-pool 204 storage is reduced, and is available to store additional late season runoff, and the focus of water 205 managers shifts to delivering the contracted allocations of water, generating hydroelectricity 206 when possible, and meeting environmental flow requirements. Within the constraints of these 207 predetermined allocations of water and available winter storage, water managers have the 208 freedom to make subjective decisions about when to release water and how much to release. 209

Data 210
To assist water supply management, California has an expansive network of field 211 measurements and a record of all operational forecasts. Sensors monitor rain and snow in the 212 mountains, stream gauges monitor runoff, and recordings monitor the volume of water stored in 213 each reservoir. Not only has the state invested in measuring snow, rain, runoff, and reservoir 214 storage, it has devoted resources to the care of statewide hydrologic data and prioritized public 215 access to these comprehensive hydrologic records. CDEC, the California Data Exchange Center 216 (http://cdec.water.ca.gov/) houses these public data as an example of databases essential to the 217 "Fourth Paradigm" of environmental informatics (Frew & Dozier, 2012;Hey et al., 2009). Using 218 these data, we assembled a panel dataset of 34 years of water data from the 14 high elevation 219 basins. Figure 3 gives an example from the Tuolumne River in 2011 of the data used in the panel 220 data model, and Figure 4 displays a conceptual overview of actions within the watershed that 221 lead to monthly release volumes. Each component of the system in the conceptual model is 222 explained in detail in the following subsections. Table 1

Monthly Full Natural Flow 247
The calculated Full Natural Flow (FNF) represents the unimpaired flow in a river. 248 Measured flow is adjusted to FNF by accounting for six ways that human actions and natural 249 processes alter that unimpaired flow as Figure 4 shows: changes in reservoir storage, reservoir 250 evaporation, exports to other basins, imports from other basins, and diversions minus return 251 flows from consumptive uses within the basin. We use the monthly values. The California 252 Department of Water Resources, water operators on each river, and the US Army Core of 253 Engineers are responsible for the calculations. Monthly FNF data are from the same stations for 254 which the seasonal runoff forecasts are produced for each of the 14 basins in Figure 1. These 255 monthly FNF data are used to calculate the forecast error of the CADWR April through July 256 runoff forecasts in Figure 2. 257

California Department of Water Resources April 1 st Forecast and Uncertainty 258
The California Department of Water Resources publishes the April through July runoff 259 forecasts in Bulletin 120 each month from February through May. The forecasts of the monthly 260 Full Natural Flow for these 14 Sierra Nevada Basins are the longest running forecasts in 261 California of water resources from the Sierra Nevada, in some watersheds dating back to the 262 1930s. The forecast of spring runoff used in this study is the Bulletin 120 seasonal volume 263 forecast, which includes a confidence range (90% to 10% exceedance probabilities, as Figure 3  264 shows). These forecasts are based on regression analyses from measured snow water equivalent, 265 rain, and streamflow. We use the forecasts made on 01 April because they are the final, and most 266 accurate, forecasts available before the start of the forecasted time period so they represent the 267 information available to water managers at the start of spring runoff. The median (50% 268 probability) forecast is used as the forecast value, and the uncertainty in the forecast is calculated 269 as the 10% exceedance forecast minus the 90% exceedance forecast (Figure 3). This measure 270 represents the level of uncertainty available to water managers at the time of the forecast and 271 varies from year to year as seen in the individual box plots for each basin's uncertainty range in 272 Panel b of Figure 5. 273

Monthly Basin Releases 274
The high elevation basins in the Sierra Nevada offer a diverse mix of water management 275 conditions and management actions determine the amount of water released April through July. 276 Basin releases-defined as the measured monthly flow below the terminal reservoir for the 14 277 basins in Figure 1-are the volumes of water released by water managers. Panel a in Figure 5  278 shows the variability in the relationships between the basin releases April through July and the 279 Full Natural Flow from the basins during the same time period across all study years and basins. black whiskers extend to data up to 1.5 times the spread between the 25th and 75th percentiles. 289 Outliers are not plotted. 290

Available Space to Store Runoff at the End of March 291
Our measure of the real time conditions water managers make decisions within is their 292 total available reservoir space to store incoming flows. Each year there are variable amounts of 293 available space to store runoff in relation to how much runoff is expected in the forecasted 294 period. Panel c in Figure 5 shows the variability in available space to store runoff at the end of 295 March in relationship to the forecasted volume of runoff to enter the basin during the spring melt 296 season. For our panel data model, all reservoirs in a basin are treated as a single large reservoir 297 because we are interested in the total available reservoir space to store incoming flows above the 298 basin release point. The is the release of water from basin i in year t, which we take to be the manager's decision, 327 is the forecasted flow, is the available reservoir space to store runoff the day before the 328 forecast arrives, is the uncertainty in the forecast, and interacts the forecast with its 329 uncertainty. This interaction term allows for the possibility that a manager reacts differently to a 330 forecast depending on its uncertainty and its magnitude. The fixed-effect accounts for basin-331 specific unobservable variables that do not change over time. The error term has a mean of 332 zero and no autocorrelation:  Table 2 summarizes the results of the fixed-effects panel data model in equation (1),  338 which tests the association of basin releases and water supply forecasts. The model coefficients 339 estimate the marginal change in basin releases from a unit change in each of the predictors, 340 controlling for the heterogeneity between groups. By analyzing associations at the basin scale, 341 the preferred model captures 70% of the variability in the outcome of April-July basin releases 342 (adjusted R 2 = 0.70, F-Test, p ≪ 0.001). The 01 April forecast, forecast range, available storage 343 capacity, and the interaction between forecasted volume and uncertainty are all statistically 344 significant predictors (p ≪ 0.005) of basin releases. The sign of each estimated coefficient is 345 consistent with first principles. Increased forecast uncertainty and increases in available storage 346 are both negatively associated with April-July basin release volume, whereas forecast volume 347 and the interaction between forecast uncertainty and forecast volume are both positively 348 associated with release volume. The estimated interaction term, 1 , is positive, which requires 349 interpretation. Its sign implies that the effect of uncertainty on releases also hinges on the 350 magnitude of the forecast. The fact that 1 > 0 implies that when flows are forecast to be higher, 351 the net effect of uncertainty is weakened (the marginal effect of uncertainty on releases is 352 2 + 1 , and 2 < 0) . When forecasted flows are high enough, the influence of uncertainty 353 on releases disappears, above about 5 km 3 . Table 3 lists the individual fixed-effects, from 354 equation (1), for each basin in the study. These are estimates for the unobserved time invariant 355 variables specific to each basin. The fixed effects estimates account for heterogeneity that does 356 not vary with time, so can be interpreted as baseline flow levels for each basin. 357 Our main result is that all else being equal, basins with larger forecast uncertainty on 01 358 April release less water. 359 Water managers have significant discretion about how much water to release, and we find 370 evidence that they make decisions based on the forecasts. When higher flows are forecasted and 371 when reservoirs are already quite full, water managers release substantially more water. The 372 more nuanced question is: Do these operators also respond to the uncertainty in forecasts? 373 Holding the forecast volume fixed, a risk-averse water manager facing a more uncertain 374 forecast releases less water, hedging against the possibility of lower-than-expected inflows. 375 When the forecast is more precise, water managers have higher confidence that additional inflow 376 will refill reservoirs, so they release more water earlier in the year. These early releases may 377 enable additional beneficial uses of the water not available had it been released later. By 378 including an interaction in the analysis between forecast volume and forecast uncertainty, we 379 find that the effect of uncertainty on releases is mediated as the forecast volume goes up: 380 Hedging one's bets becomes less important. 381 While our analysis focuses on snowmelt runoff in California, the diversity of operational 382 structures, geology, hydrology, and climatology among the 14 analyzed basins suggests that our 383 findings can inform water management in other locations. Principally, reducing uncertainty in 384 runoff forecasts can beneficially affect water management. 385

Water Operations and their Relationship to Hydrology 386
Our results empirically confirm water manger behavior predicted by reservoir-operation 387 models. Models predict forecasts to return the most value to water managers when reservoir 388 storage capacity is between 25-100% of the mean annual flow of the river and that managers will 389 not respond to improved forecasts if storage capacity is significantly greater than the mean 390 annual flow (Barnard, 1989;Ødegård et al., 2019). Consistent with but distinct from previous 391 literature, we show that releases across 14 Sierra Nevada basins, which have reservoir storage 392 capacity similar to annual flow volumes, are sensitive to uncertainty in runoff forecasts. 393 We find that uncertainty plays the largest role when forecast volume is low. This effect 394 decreases, and eventually disappears when there is more water expected than room to store it. 395 The forecasts. These methods provide operational products in only a few basins. We provide 405 evidence to support the connection between improved forecasts and actual water supply 406 operations. 407 As a legacy network, snow pillows and snow courses provide statistical power to 408 forecasts from their long historical record. But with climate change, as the past becomes less 409 representative of the future, the power of these forecast may be reduced as their uncertainty 410 increases. Current work on improved sensor networks that accurately measure the spatial and 411 temporal variability of precipitation, snow depth, wind, temperature, and humidity ( resources (Dettinger, 2013). As the season progresses, the forecast fundamentally changes to an 439 estimation of likely runoff based on knowledge of the snowpack and other hydrologic conditions 440 (Cox et al., 1978) combined with expected precipitation to come. When communicating forecasts 441 to water managers, it may be valuable to separate forecast uncertainty into these two categories: 442 measurement uncertainty and prediction uncertainty. Knowing where the confidence lies in the 443 predictions of upcoming runoff may give water mangers more confidence in making a decision 444 that would be too risky if the uncertainty were distributed differently between these two sources. 445

Strengths and Limitations of the Analysis 446
We focused on releases that occurred in the Spring, not on the actions of the water 447 managers from August through March. Although our results show that less water is released 448 during the spring melt season when forecast uncertainty is greater, we did not identify if these 449 releases are delayed outside the April-July timeframe. We focused on April through July because 450 most of the annual runoff occurs then, forecasts are available at the start of the time period for 451 the expected flows, and water managers are actively controlling the volumes and timings of 452 basin water releases. 453 In principle, it is possible to test our hypotheses at a finer resolution by breaking the 454 analysis down to the individual operating agencies or to the individual reservoirs themselves. We 455 analyze at the basin level because doing so integrates behavior across all actors in a region and 456 most of the data are at that scale. The significant findings at this coarse resolution give merit to a 457 more detailed look at operators or individual reservoirs. While the hydrologic conditions each 458 year significantly affect how much water they release from each basin, no variables perfectly 459 correlate with releases. Some constraints on water supply operations are independent of river 460 flow; flood control, environmental flows, and water rights dictate specific actions each year that 461 may reduce the explanatory power of a forecast on actual basin releases. 462 These significant findings in California bolster the case to look at similar systems in other 463 states or parts of the world where water management and the timing of decisions are different 464 and may confirm our findings or lead to different outcomes. 465

Conclusion 466
Reducing uncertainty in runoff forecasts is a tractable objective that can drive forecast 467 improvement, would not require reducing forecast error, and could lead to changes in amount 468 and timing of water released from reservoirs. Our analysis shows that water releases across 14 469 Sierra Nevada basins are sensitive to the magnitude and uncertainty of runoff forecasts. Water 470 managers effectively hedge against the possibility of lower-than-expected spring runoff. 471 Increased forecast uncertainty and increases in available storage are both negatively associated 472 with April-through-July basin releases, whereas forecast volume and the interaction between 473 forecast uncertainty and forecast volume are both positively associated with releases. Results 474 suggest that all else being equal, basins with larger forecast uncertainty on 01 April release less 475 water from April through July. As forecasted flows increase, the effects of uncertainty are muted. 476 Narrowing the uncertainty of the forecast, independent of improving the forecast itself, 477 could improve water operations. Moreover, changing the presentation of forecast uncertainty to a 478 format that communicates uncertainty arising from the measurement of snow on the ground 479 separately from the uncertainty in expected precipitation could guide water operations and open 480 new opportunities for using surface water. 481