The Sacramento-San Joaquin Delta is undergoing continual and often rapid change. This poses challenges in predicting and preparing for the future, as past data and models are no longer sufficient to anticipate future conditions. This uncertainty cannot be addressed by collecting more data, and decision-making becomes complex when stakeholders have differing views on the consequences of actions. To effectively manage the Delta, managers need new methods for anticipating the future.
The Delta Independent Science Board is conducting a review of the Decision Making Under Deep Uncertainty, an interdisciplinary approach that provides decision-makers with new tools and processes to make better-informed decisions despite the challenges they face. The Board is focusing on scenarios as a useful tool within the toolbox.
The diagram below illustrates what is different about the DMDU approach. The red cone to the left shows the historic range of variability; that is what has happened before and what may happen again. But conditions are changing, increasing the uncertainty into the future. 
So what scenarios try to do under DMDU is to not just look at the most probable outcomes (our best guess of the future) represented by the green center but also consider a broader spectrum of plausible conditions to understand how the management strategies will perform.
The Delta ISB is reviewing DMDU to provide support to help prepare for extreme and unpredicted events and to build an understanding of scientific tools and concepts that provide a structure for anticipating and managing uncertainty. The effort will include a webinar series, a survey of how scenarios are currently being used in the Delta, and discussions with stakeholders to understand how scenarios are applied to policy. The end product will be a report summarizing what has been learned and useful insights that agencies and others in the science enterprise can apply to the Delta.
The first webinar, Decision-making under deep uncertainty: What is it and why is it useful?, was held in spring of 2023 and is summarized here: DELTA ISB: Decision-making under deep uncertainty: What is it and why is it useful?
The second webinar of the series featured Dr. Robert Lempert, principal researcher at the RAND Corporation and director of the Frederick S. Pardee Center for Longer Range Global Policy and the Future Human Condition, and Andrew Schwarz, the State Water Project climate action coordinator for the CA Department of Water Resources. The presenters discussed successful applications of tools that apply the decision-making under deep uncertainty approach.
Tools for Decision Making Under Deep Uncertainty: Policy-relevant scenarios and adaptive strategies
Dr. Robert Lempert began the webinar with an overview of Decision Making Under Deep Uncertainty (DMDU) and how policy-relevant scenarios and adaptive strategies can be applied.
So how do we use science to inform important policy decisions when the world is hard to predict? One thing we know about the future is we’re likely to be surprised, he said. We also know that we won’t manage it very well if we’re not using good science and evidence. So how do we reconcile those things?
One way is to use quantitative analysis to inform policy by making predictions: this will happen, or there’s a certain likelihood of something happening. But while predictions are a core principle of the scientific method and useful in a lot of cases, in many instances, they can complicate the use of quantitative information, particularly when uncertainty has the characteristic of being deep, which can lead to disagreement among stakeholders. Fortunately, there’s a better way: Decision making under deep uncertainty (DMDU).
A powerful set of risk management policy and analytic tools already exist, which work well when the uncertainty is limited or well characterized. While various tools are available, they tend to follow a pattern Dr. Lempert called “Predict and Act.” It begins with a consensus understanding of future conditions, a ranking of the decision options, and perhaps a sensitivity analysis to see how sensitive the ranking is to uncertainties.
“Basically, they’re designed to require a consensus understanding of the future actually to do the ranking,” he said. “Predict and act works great in a variety of circumstances. For example, you wouldn’t get on an airplane if it didn’t work very well. But if anybody doubts whether the airplane will perform exactly like it’s supposed to, you generally don’t fly. So you can turn the problem off when you’re not quite sure how it will work. But for a lot of situations, particularly when you’re managing large societal environmental systems, we don’t get to turn off climate change for 50 years until we figure it out. We have to manage it, even though we don’t totally understand it.”
“In situations where there is this characteristic of deep uncertainty, there can be great pressure to underestimate uncertainty so that you can make a strong policy statement,” he continued. “Competing analyses can lead to gridlock, so if a policy is predicated on a forecast and you don’t like the policy, you attack the forecast because the forecasts may have more holes than the policy. The whole exercise of spending our entire analytic effort on getting better forecasts may distract from the activity of using what we know to form solutions which may be successful over a range of uncertainties or are robust.”
A definition often used for deep uncertainty is that it occurs when the parties to a decision do not know or do not agree on the likelihood of alternative futures or how actions are related to consequences; usually, it involves disagreement, imprecision, or lack of any good probabilistic information on future states and some uncertainty in the model.
Dr. Lempert said in those situations, it’s often useful to turn the analytics around and conduct the analysis backward. Start with goals and plans: what are we trying to achieve? What are our plans to get there? That’s often how agencies think: they have a set of goals they’re trying to achieve and are proposing some plans; you can then use the analytics to stress test plans and ask, under what sorts of futures is this plan consistent with the goals? And what sets of futures is the plan inconsistent with the goals? In many cases, you can get higher confidence answers to that question rather than what’s the likelihood that the plan will succeed.
“For example, if you got a structure by the coast, you can figure out what amount of sea level rise will make that structure no longer serve its purpose with high confidence, even if we don’t know what the trajectory of sea level rise is over the coming decades,” he said. “Then we can use the results of the stress tests to try to identify new or revised plans that are more robust and less sensitive to the things we don’t know. And the set of methods that follow that approach is called decision making under deep uncertainty.”
Policy-relevant scenarios
One of the tools of the DMDU approach is policy-relevant scenarios, which are used to develop adaptive strategies and inform decisions.
The benefits of scenarios include:
- Scenarios can be very powerful; they can help decision-makers expand their mental models of how the world works and think about plausible situations and their implications.
- Scenarios can reduce overconfidence because it’s considering a wider range of futures.
- Scenarios can facilitate buy-in and collaboration because people with different assumptions about how the world will work can have scenarios that resonate with them and so they will buy in to the same analysis.
However, there are challenges:
- Scenarios may be disconnected from decisions.
- Participants can have different interpretations of scenarios.
- They can fail to explore the most relevant futures.
The DMDU approach uses stress tests to help identify unambiguous reproducible policy-relevant scenarios, which have a clear meaning and relevance to policy, with an audit trail that shows how they were developed.
Dr. Lempert gave an example of a water quality implementation plan for a sub-watershed of the Los Angeles River. The city of LA had devised an implementation plan to meet the federal water quality requirements that didn’t include climate change. The plan was developed using hydrological and optimization models that determined the optimum set of best management practices, which was submitted for the regulatory assurance analysis. However, they hadn’t considered climate change and uncertainties in land use.
“So we basically took the same models they had used for the regulatory assurance analysis and just ran a whole bunch of different climate projections over a whole bunch of different assumptions about land use,” said Dr. Lempert. “So you get this big cloud which was, in this case, a hundred or so different futures. And you can look at each one and say, Does this meet the regulatory goals? And if it is, it’s a blue dot on this picture; if not, it gets a red dot. So then, what to do with this big mass of data?”
“One thing you can do is then ask a question using various types of classification algorithms and say, draw me the picture that does the best job of separating the blue and the red dots,” he continued, noting that the graph only uses two axes: 24-hour rainfall intensity changes and impervious area. Each line represents a different land use scenario, with the green X being the land use in LA at the time they did the study.
“The green X is the planning case, and, not surprisingly, it is within the big cluster of blue dots where the plan meets goals. … then the algorithm draws that line through it. Everything on the left of the line, the plan meets the goals; everything on the right, the plan misses goals. The line divides the blue and red dots pretty well, though it’s not perfect. But this is the best line you can draw through any two-dimensional space that separates those dots. So this gives us two policy-relevant scenarios with clear axes, definitions, and an audit trail of why we’re using those in the analysis. This then becomes a powerful tool to use in policy discussions.
Adaptive strategies
Another tool in the DMDU toolbox is adaptive strategies. One way to make strategies that are robust and insensitive to deep uncertainties is to make them adaptive so they evolve over time. The components of an adaptive strategy are near-term actions, monitoring, and contingent actions depending on the particular way the system is evolving.
“The policy-relevant scenario suggests that you monitor impervious areas and the frequency of severe storms,” he said. “In that study, it turns out that it’s a lot easier to measure the impervious area of the city of LA than it is to get the climate scientists to tell with any certainty what the storm frequency will do. So you can see exactly how that map would let you build an adaptive strategy.”
The figure on the slide is what he called an ‘adaptive pathways subway map,’ which works well for thinking about adaptive strategies. In this example, there is a system by the sea. As sea level rise increases, the structure meets the goals for the system – up until some point, and then a threshold where the policy fails. So how long is it good for? We don’t know. It might take 25 years, or it might take a decade.
“So we might have an action A and a point where you can transfer to this new policy, which might be costly to do in the near term,” he said. “You might do an action B, which is good for 50 years if sea level rise is slow, or 25 if it’s fast, or you could do C, which would take you out further and allow you to jump to another level. So you can work these out and get a plan which is robust over a range of sea level rise scenarios. And, you can explain this to stakeholders.”
The figure on the slide below is from the Thames River estuary plan that used this approach.
Informing decisions
The chart below is from the recent IPCC Intergovernmental Panel on Climate Change Assessment. It reported its sea level rise projections in working group one to support various adaptive pathway maps in working group two on adaptation.
“The key thing here is that the sea level rise projections have probabilistic projections for the well-understood component of sea level rise so that people can use that part in their planning. And then the high-end storylines, which essentially are various ways in which the Greenland and Antarctic ice sheets could collapse that give you much higher rates … they don’t have probabilities because we don’t understand it well enough to assign probabilities. But they have physical descriptions of the things that would need to happen to get you there, which is meant to provide at least a signpost for decision-makers to say, if I’m really worried about those things happening quickly, what signs do I need to monitor? What warning might I get? So it’s designed to support that sort of planning.”
Equity is another important piece. The graph is from work done by a colleague considering adaptation in the Mekong Delta. Adaption strategies were mixes of subsidies and building dikes across 23 districts. He ranked adaptation strategies using different definitions of what’s fair: who gets what, who decides, and respect for other points of view, as well as a variety of ethical frameworks. Looking over the wide range of scenarios, where do the ethical frameworks give you the same ranking of adaptation strategies, and where are they different?
“This is a way to bring the scientific information into some of these ethically charged debates,” said Dr. Lempert. “DMDU might be good at this because it avoids premature aggregation over expectations and values, makes the trade-offs more explicit, avoids embedding ethical assumptions into the model, and can identify strategies that perform well with high confidence across multiple expectations of the future.”
In conclusion …
“Deep uncertainty sounds like the off-putting and gumming things up,” said Dr. Lempert. “But in many ways, embracing it can empower decision-makers because it helps you identify the low-regrets strategies, adaptive strategies, and diversified solutions, and gives you a very clear and concrete way of making the case of why those are ways to approach these difficult problems.”
Applications of Decision-making under Deep Uncertainty at DWR
Andrew Schwarz began by reflecting that there is a focus on low probability, high consequence events, which he acknowledged are important. “I think we sometimes get myopically focused on these very extreme ‘Black Swan’ events; they make headlines, and they make really good media articles. But we don’t necessarily make investments to protect ourselves from these rare events. And whether we do or not largely depends on our perception of what low probability means and what the consequence of those events might be.”
Resource managers are constantly balancing the need to provide protection against undesirable conditions, such as flooding or drought, with the willingness and ability to pay for these increasing levels of protection. He pointed out that we do this ourselves with cars: how many of us drive around without the latest safety technology? We probably all have seatbelts and airbags, but how many don’t have electronic stability control or blind spot detection? We understand the consequences of being in a crash, but we use our limited resources to spread them across all the priorities that we have in our lives.
The graphic on the slide illustrates how California’s annual variability in precipitation is higher than anywhere else in the United States. “That high variability and uncertainty force us to gamble, essentially leaving risk unaddressed in our system because the uncertainties are so huge and the variability is so large,” said Mr. Schwarz. “We can prepare for extremes, but we ultimately have to play the odds.”
Mr. Schwarz then gave examples of how DWR is using DMDU techniques. These approaches are used to understand a wider range of possible outcomes, allowing them to more accurately assess the level of concern and the solutions we should invest in. He said that scanning across the range of plausible outcomes can help lead us to adaptation strategies that address key vulnerabilities and are robust to a wide range of conditions, even if they may not address the most extreme events.
The blue bubble represents the probability density function of climate model outcomes across California’s major watershed area that contributes to the State Water Project and other major water projects in California. Change in average precipitation is on the horizontal axis, and change in temperature is on the vertical axis.
“You can see that there’s a lot of uncertainty in both,” said Mr. Schwarz. “Certainly, the temperature is going up; essentially, by the middle to the end of the century, there is no probability that we won’t see major temperature increases, but the change in average precipitation is all over the place. And the models are largely split over whether we might get wetter or drier.”
This is the problem that managers and planners have to deal with. The hot, dry end of the bubble will have a very different set of adaptation strategies than the cool, wet end. If there is 15-20% less precipitation, that limits adaptation strategies for water supply reliability considerably, making options like recycling and desalination more attractive despite the cost rather than Flood MAR or other strategies designed to take advantage of precipitation.
Example 1: DWR Climate Change Vulnerability Assessment (2018)
The first example is from the DWR’s 2018 Climate Change Vulnerability Assessment. The approach used is called decision scaling; the figure is the results of a system stress test for State Water Project deliveries. Orange colors mean worse system performance, blue colors mean better system performance, and the black line is the point that maintains existing performance.
“The stress test explores all of these diverse conditions and future changes that we might have to deal with, but it’s agnostic with respect to time,” Mr. Schwarz said. “It doesn’t matter when these changes occur; it says, if these changes occur, this is how your system is likely to respond. So this is a very powerful tool for understanding how our system responds … there are 1100 years of natural variability under each point in that graph. We can even plot that bubble over the top of this response surface and start to understand, at each period in the future, where might I be? What range of conditions might I have to deal with based on the probability of climate models?”
That information can be compiled, different conditions weighted by the likelihood of the climate, and the probability distribution functions plotted for what the system might look like and the uncertainty ranges. In this hypothetical example, the gray shadow is the historical uncertainty of the system. The red line is a future condition in 2050 if no action is taken; generally, there is lower performance as the line shifts to the left. The purple line is a generic adaptation strategy that, if implemented, improves performance.
“But you don’t recover that uncertainty; you’re still dealing with a much wider range of conditions,” he said. “So this would be an attractive adaptation strategy as it certainly works over a wide range of conditions. So that’s one of the ways we can test these adaptation strategies using the stress test approach.”
California Water Plan 2018
Another approach used with the California Water plan was robust decision-making. The figure shows information for the San Joaquin and Tulare Lake Basin. Agricultural water supply reliability is on the vertical axis; urban water supply is on the horizontal axis. The comets show how different adaptation strategies move the needle in different directions. Each comet is a different future, a different climate, and a population combination that the Water Plan was considering.
“This is another way to communicate the idea of, is the diversification working for urban water supply reliability if it’s moving to the right or agricultural water supply reliability if that comet is moving up?” said Mr. Schwarz. “If it’s moving up and to the right diagonally, that’s a really good one because you’re getting both out of that one.”
Delta Stewardship Council Delta Adapts 2022
The figure is from the Delta Adapts project and illustrates flood risk in the Delta. The Delta is a difficult place to model hydrologically with inflow from five different inflow tributaries, the ocean boundary, different channel capacities, and other things that complicate modeling.
The process to develop these maps used a Monte Carlo approach where each boundary condition had a distribution function that was then combined into a million scenarios that were then used probabilistically to make the maps that show what flood risk would be at different periods in the future from overtopping of levees.
“I think this was a really powerful activity for planning that is still being carried forward,” he said.
2022 Central Valley Flood Protection Plan
The Central Valley Flood Protection Plan from 2022 utilized many different climate change scenarios. Mr. Schwarz noted a lot of modeling went into this project, which is usually the case with DMDU, which is then used to develop graphical representations to communicate ideas.
“I just like this one a lot because it’s so simple,” he said. “It shows you where our risk is right now and where our risk range might be in the future, depending on what future we get. And then, if you take this investment strategy, how much that risk is reduced.”
“But it shows you that even if you make this huge investment, you’re still not getting back to where we are right now. It’s very clear that there are limitations to what we can do with adaptation under this changing climate. And I think that’s sometimes an important message to convey. We’re not superheroes.”
California Aqueduct
This example, from the California Aqueduct Subsidence Program, is considering subsidence issues in the San Joaquin Valley in the vicinity of the California Aqueduct, which degraded the carrying capacity of the Aqueduct over the last several decades.
The program is dealing with compounding uncertainty from climate change, drought occurrence, whether SGMA gets implemented as expected or if it’s delayed, and if what is planned gets implemented as it’s been outlined, and how that would affect State Water Project deliveries.
“They’ve been able to compound all of these uncertainties together in a number of scenarios and look at different futures and what to do in terms of a fix for the Aqueduct under these compounding uncertainties,” said Mr. Schwarz. “So that’s just another way that built on much of the other DMDU work that had been done with the stress testing I showed in the first few slides.”
What DWR has learned
Mr. Schwarz then turned to what the Department has learned from the DMDU approach.
Regarding the State Water Project, they have determined that the system is not equally sensitive to changes in temperature, precipitation, and sea level, and each has a different effect on the system. The graphs show the various impacts, independent of any other impacts.
“We can simulate the system with just a temperature increase or just the change in precipitation and see how it responds,” said Mr. Schwarz. “The one that was most elucidating to us is the graph on the lower right, which is precipitation variability. So we kept the average precipitation the same and then just played with the standard deviation of annual precipitation, getting more wet and dry years. And we found that you don’t benefit from those wet years because, with current infrastructure and operations, you don’t have a place to store that water. So it really hurts you on the dry end. So with no change in average precipitation, we can get very much more significant severe drought impacts with just that disparity.”
In the analysis for Delta Adapts, the map shows which areas of the Delta are more vulnerable to flooding from the tides from sea level rise or flooding from rivers flowing into the Delta.
“It’s not surprising that the Western Delta would be more vulnerable to sea level rise, and the far corners of the Delta along the rivers would be more sensitive to the riverine flooding,” said Mr. Schwarz. “But teasing out where this boundary is a boon to adaptation because you’re going to need different adaptation strategies for these different stressors. Sea level rise is an inexorable incremental change, whereas the riverine flooding will come in episodic events and then recede.”
They have learned that stress testing is often too computationally demanding for small projects and even for large projects with lots of integrated modeling. “When you have fish modeling, temperature modeling, and all kinds of downstream modeling, you just can’t do all of this stress testing – it’s too much.”
“We’ve also used a few scenarios with book ends, and then when we provide those without quantified risks of particular to those extreme scenarios, people ignore them,” said Mr. Schwarz. “They plan to the middle. And so probabilities – at least a relative uncertainty quantification on these scenarios is really important.”
Plans for continued integration of DMDU approaches
To address some of those problems, one of the strategies for the State Water Project Delivery Capability Report in 2023 is to hybridize a traditional scenario analysis approach with what they have learned from DMDU.
“This will feature a limited array of scenarios, but each of those scenarios will come with a probabilistic level of concern,” said Mr. Schwarz. “And those levels of concern really give you a sense of how extreme the scenario is relative to the other scenarios. How much risk am I hedging by using this scenario to plan to? We think that’ll be really helpful for local water agencies. And they’re planning to understand those risks and that uncertainty.”
The Department is doing watershed studies in the San Joaquin Valley. The slide shows draft graphics of three different levels of Flood MAR implementation and what risk can be brought down in each of these watersheds with these approaches.
So, in closing, DMDU approaches that use wide-field uncertainty evaluations have been critical to helping the Department gain understanding. “All models are wrong, we know that, but we can get a lot of information and understanding about our system by using these wide-field approaches,” said Mr. Schwarz. “However, it may not be appropriate for every application. And when we talk about low-probability events, we need to understand how low and for what purposes. That’s just not always appropriate for how we’re planning and what we can do.”
QUESTIONS & ANSWERS
QUESTION (Deirdre Des Jardins): Andrew Schwarz’s approach assumes model democracy – you assume each model is an equally certain projection. But the CMIP 5 models project generally increasing precipitation, but that’s not what we’ve seen in California since around 1999 or 2000. There is some uncertainty that some very basic processes in the models are not fully capturing the dynamics of the sea surface temperatures in the Pacific Ocean. And so there’s sort of a drier future. Then there’s sort of a wetter future with the climate models.
ANDREW SCHWARZ: “I think Dierdre is raising a basic tenet of DMDU, which is people disagree on the probabilities of these futures. So one of the key benefits of stress testing is that the climate uncertainty is much later in the modeling process, allowing you to test those assumptions, but the stress test stays the same. You can then extract where you might be in the future based on a different probability distribution if you have a different set of models that you want to use. That’s relatively simple and quick to do. And so that is one of the features of what we’re doing with the Delivery Capability Report scenarios is we would be able to change the quantified levels of concern that you would attribute to each of those scenarios; the scenarios will be the same, but the quantified probability or the level of concern that you would associate with each of those would change based on whether you’re using democratic models or some selection of models. We could do that. We’ve already planned that that could be something we can evaluate if desired by stakeholders.”
QUESTION: How do you see the DMDU resulting in different decisions for different organizations?
ANDREW SCHWARZ: “The Delivery Capability Report is issued by the State Water Project every two years, and it reports on the current delivery capability of the system, what the reliability is, and then gives a projection 20 years out into the future. This is used extensively by our water contractors for their local planning, urban water management plans, ag water management plans, integrated regional water management plans, and even SGMA plans as well. So it’s an important piece of information for local planning for agencies to use.”
“Historically, it has provided a single scenario of what the future might look like. But we realized that different folks have different risk tolerances and different dependencies on the State Water Project. Some contractors have multiple water supplies that they can shift around; they have a lot of flexibility. Others are more dependent, have more limited supplies, and really require a certain amount every year to get by. So this range of scenario severities would fit in with the level of concern or risk tolerance that each of these folks has. So it has a built-in way to play into their planning.”
“It’s very difficult at a statewide level for us to do an adaptation strategy because a lot of those adaptation strategies happen at the local level. And so we’re adapting our DMDU approach to make it work for this local planning option.”
QUESTION: You talked about how the temperature will go up and that it may be slightly wetter precipitation conditions for the state; however, from a water management perspective, especially for the DWR, it’s the streamflow that’s available, and that might be more important than the general precipitation conditions. The historical data on the Colorado River recently published found that the one degree Celsius temperature increase will result in about 10% or higher reductions in the Colorado River stream flows. … How will the information regarding the temperature change reducing the stream flow and water quality be implemented or used for future planning?
DR. ROBERT LEMPERT: When doing planning studies with water agencies in California or the West, you look both locally and within the region for the local supplies, and then you look across a range of State Water Project scenarios and Colorado River scenarios. Then ideally, you look at the correlations among those … so there will be general scenarios where you get drying across the entire Southwest, which affects everybody, but other cases where there may be more localized drying in your region but more water elsewhere, and you’re planning against that full set.”
ANDREW SCHWARZ: Streamflow is ultimately the thing that goes into most of our models. So it is extremely important. And recognizing that when you go from meteorology to streamflow, you add another layer of uncertainty from your rainfall runoff model, the routing, and what you’re doing with ET. These are all compounding uncertainties that we want to build into this uncertainty space to make sure that what adaptation strategies or response policy responses we put in place would be robust to these different levels of uncertainty. So it is built into the process chain already, but it is something that we’re thinking about, and it’s a potentially another dimension of that stress test.
QUESTION: A lot of infrastructure for delivering water was built 50 years ago, and aging infrastructure varies across these different utilities. Concrete doesn’t fix itself; that’s not uncertain. It’s represented as if there’s no uncertainty in the current state. But a lot of the infrastructure is vulnerable … so how does that figure in here?
ANDREW SCHWARZ: “For the State Water Project, asset management is one of our key focus areas right now. We’re trying to understand exactly how to optimize the repair, refurbishment, and replacement of these facilities to balance cost-effectiveness with readiness and flexibility and have the facilities online, ready to move water when available, and not have these unplanned outages caused by extreme events. It is certainly something that we’re focused on and trying to implement and incorporate new climate metrics into this.
This is an ongoing science that has been developing for years but hasn’t necessarily thought too much about how climate change figures into that. So we’re adding that to our asset management plans, thinking about how our pumping plants have to cycle more frequently. They’re not 50 years old because they’ve been renewed and maintained, but they continuously wear out. And when you cycle them more frequently because of more extreme conditions and big events, we want to run it as hard as we can for a few days to get that water into storage. And then tail off when the hydrograph declines so that we’re not having impacts on fisheries. That’s harder on your equipment, so all of that goes into managing these systems.”
“With our aging infrastructure, as we think about replacement and refurbishment of all of these different components, it’s also an opportunity to look at different ways that it might be able to function or that we might be able to operate these things in the future. And that opens the door to rethinking possibilities and different project alternatives.”
QUESTION: Rob, you said initially that DMDU is used when uncertainties are deep or people disagree. So given the cognitive biases that we’re all walking around with, how do we know when uncertainty is deep? And which ones should we be focusing on to decide to use DMDU? What’s the initial process?
DR. ROBERT LEMPERT: “We have a graphic that I didn’t show, which is when to use DMDU, and in particular, when to use quantitative value. It has three axes. One is how well characterized is the uncertainty. Another axis is the richness of the decision space; if you have a binary decision – you jump or you don’t- you get your best estimate of probability and go. But if the decision is rich, such that you can take actions over time or have a rich portfolio of things you can do, there are more opportunities for developing strategies that are robust to the uncertainty. So in some sense, how deep does uncertainty have to be before you think of this as a deeply uncertain problem versus one of well-characterized uncertainty? It has to do with the nature of the uncertainty and the decision options. And then the other axis is complexity, which is more of a heuristic axis for if you were just got a bunch of smart people sitting around and come up with three scenarios, how much can you just intuit what the right scenarios are? And how much do you actually get from this process of running these big clouds of points and doing cluster analysis on them? So those are the axes.”
“On any practical application, DMDU is often more complicated than traditional analyses. So is it worth that? And as people sit down and try to write guidance documents, like when to use DMDU, you come up with screening processes, which, at the lowest level, you can tell that something is already resilient to climate, so you can do something simple. Then, the next level up, if you can’t just do something very simple, then you kick into the more complicated analysis. So depending on what sort of application you’re talking about you, you might want some sort of screening process as well.”
ANDREW SCHWARZ: “I think the way that we should be planning, we do some top-down where we’re looking in a big area of vulnerability assessments, adaptation strategies, and strategic planning where we have a rich possibility of different possibilities, I think that’s where DMDU is going to be really useful. And then also from the bottom up where now we’ve identified a project, and we’re going to evaluate that project. A lot of that analysis gets done in environmental impact reports and other things that are far more constrained. I think it would be very difficult to deploy a DMDU approach within that bottom up piece, but I think what you’re trying to do with DMDU is ensure that the projects you’re selecting really work.
QUESTION: In the context of the Delta Conveyance Project, we’re investing huge amounts of money for long periods. How do you look at those types of rare extremes in this context? And how do you deal with those? The models will probably break at some point in those scenarios.
ANDREW SCHWARZ: It’s not that we don’t want to look at those “Black Swan” events – I think they are important, most importantly, for tabletop exercises for emergency response. What would you do if this confluence of very rare events or these things happened? The problem is when it’s presented as we should be preparing, we should have infrastructure that is fully adaptive to the Arc Storm 2.0 or something like that … I can’t remember what the price tag on the Central Valley Flood Protection Plan was, but it’s multiple billions of dollars. Much of that is being allocated and implemented, and it will be done over time. But there’s a limit. We’re not putting 10,000-year protection on every city. I don’t think any of us want to pay for that in our property taxes. But ‘Black Swan’ events are still important; we still have to have a response to that. It just may not be in the infrastructure.”
QUESTION: In your subway map scenario, how do you deal with making sure that the resources you need for other pathways will be there when you need them when you switch your train station to another subway line? If you need land to go along another path, how do you ensure those resources are available for that alternate pathway you might choose?
DR. ROBERT LEMPERT: When you use these tools as part of a deliberative process with agencies and stakeholders, you’ll adjust your views of the uncertainties and strategies and often adjust your goals as you understand what’s possible on the upper or lower end. … There was a slide that showed the current performance in the system with no adaptation and then with lots of adaptation, and the point was made that it doesn’t get you back to where we are, right? So if your goal is to maintain current performance, there may be no adaptations you could do. … So in an agency planning context, where they went in with certain reliability goals, you might reasonably conclude that we can’t maintain our goals under all cases, but we can do better. So even in these black swan cases, it’s useful to look at what could we reasonably do.”
“Then there are the political challenges. It may be hard now to say what the newer reliability target would be in these very bad situations. But from an analytic point of view, it’s useful to know, are the things we’re doing for the 95th percentile scenario also helpful for the 99th? Or do they pull in different directions? If they’re consistent, okay, good, we’ll do them all … if they pull in different directions, you might think about if there is something different we would do. … With sea level rise, if you have a facility and you’re armoring it, you’re not going to move it for the very extreme ones unless we get a lot more information, but if it’s equal, I could put it here, I could put it here. And all other things equal, maybe I’ll put it here because that’s good for extreme cases.
“How can we be sure that we’ll take the jump onto the new subway line in the future? So that’s a challenge with these adaptive strategies. And in some debates, you’ll see some groups pushing back against the idea of an adaptive strategy because they’ve got the political capital right now to lock things in, and they may be afraid that they won’t down the line.”
“But in the water quality example I showed, the adaptive strategy does suggest that you would hang on to the land you might need for some of these larger projects for an extra five years before you allow development … so the near-term action is designed to hold open the option to switch in the future.”
QUESTION: That graph that shows there’s just not going to be as much water as there used to be, in all likelihood, is a pretty scary graph. And if I were a water contractor, I’d think I would start taking action. Have you seen any response from these outreach efforts and specific actions that either the Department is taking or the contractors? Did the idea of covering the Aqueduct with solar panels to reduce ET losses spin out of this in any way?
ANDREW SCHWARZ: “Yes, I think this has had some influence on what the contractors are doing. If you look at particularly what’s been going on in Southern California for the last two decades, it’s been pretty incredible what they’ve done in terms of investments in water management and storage and the degree to which they weathered the last drought, which was pretty extreme is pretty amazing. So they’ve been doing this for a long time, and these new tools are going to push that even further and help them more.
“There already is a lot more interest in capturing these flood flows, so nothing like a recent event to remind us that it’s not always going to be dry … all of the projections have suggested that this is the future that we were going toward longer droughts punctuated by these wet events. And we’ve just seen it in the last four years in a big way. And so trying to store these floodwaters in every bucket, bowl, and spoon we have for the dry times, and the reuse of water and water treatment, I think, will be important.
DR. ROBERT LEMPERT: “We’ve been working in Southern California for a long time using these approaches. I think it’s hard to point to a decision that would not have been made had they not been doing these sorts of things. But at the very least, it has provided them with a framework to understand and explain what they’re doing more clearly and to say this set of actions works. A lot of the water capture and recycling works; no matter what happens with the climate, it is a good thing to do. Things like exploratory desal, we’re just trying to figure out how this works in case we need it … “
“There is a quote I repeat because I like it. One of the Met planners, during one of the public outreach meetings where he was being challenged on the climate models, was asked, do you really believe these models? And he said, I’m not sure we believe the models, but we believe in the plan.”
