Designing a solution to enable agency-academic scientific collaboration for disasters

. As large-scale environmental disasters become increasingly frequent and more severe globally, people and organizations that prepare for and respond to these crises need efficient and effective ways to integrate sound science into their decision making. Experience has shown that integrating nongovernmental scientific expertise into disaster decision making can improve the quality of the response, and is most effective if the integration occurs before, during, and after a crisis, not just during a crisis. However, collaboration between academic, government, and industry scientists, decision makers, and responders is frequently difficult because of cultural differences, misaligned incentives, time pressures, and legal constraints. Our study addressed this challenge by using the Deep Change Method, a design methodology developed by Stanford ChangeLabs, which combines human-centered design, systems analysis, and behavioral psychology. We investigated underlying needs and motivations of government agency staff and academic scientists, mapped the root causes underlying the relationship failures between these two communities based on their experiences, and identified leverage points for shifting deeply rooted perceptions that impede collaboration. We found that building trust and creating mutual value between multiple stakeholders before crises occur is likely to increase the effectiveness of problem solving. We propose a solution, the Science Action Network, which is designed to address barriers to scientific collaboration by providing new mechanisms to build and improve trust and communication between government administrators and scientists, industry representatives, and academic scientists. The Science Action Network has the potential to ensure cross-disaster preparedness and science-based decision making through novel partnerships and scientific coordination.


INTRODUCTION
Environmental disasters are becoming increasingly more frequent and severe globally (IPCC 2012).Although those who respond to disasters ("responders") continue to improve their preparation for and response to crises, integrating the best available scientific information and expertise is vital for efficient and effective decision making (Lubchenco et al. 2012).Yet how scientific information and expertise is incorporated into decision-making processes in the United States is challenging because industry and governmental decision makers, often from multiple agencies, operate on different systems of rules and rewards across agencies and compared with academic institutions (Janssen et al. 2010).Our results indicate these disparities must be addressed to improve effective collaboration, and response should be informed by an understanding of the complex human behaviors that are present during crises.
Human-centered design is a problem-solving methodology that focuses on the needs and behaviors of stakeholders who are either driving the problem or have the authority to enact interventions (Giacomin 2014).By comparison, systems thinking is a set of synergistic analytic skills used to improve the capability of identifying and understanding systems as sets of dynamic interdependent relationships, predicting their behaviors over time, and devising modifications to them in order to produce desired effects (Arnold and Wade 2015).The systems thinking approach contrasts with traditional analysis, which studies systems by breaking them down into their separate elements.When humancentered design is strategically integrated with systems thinking, it is uniquely suited to address complex problems.Importantly, this integrated design process is not a science, but rather a method of iteratively framing, understanding, and deconstructing human-system challenges (Brown 2008).We present a methodology for applying human-centered design and systems thinking to crisis preparedness and response, demonstrate how this methodology is applicable to other social and environmental challenges, and propose a solution, the Science Action Network.
Despite the increasing popularity of human-centered design, or "design thinking," across a number of fields, its methodology remains a relatively neglected area of research as applied to environmental challenges (Razzouk and Shute 2012, Santo et al. 2015, Sorice and Donlan 2015).We explore the convergence of human-centered design and systems thinking (henceforth called the Deep Change Method) as tools for tackling complex problems rooted in human behavior (Brown and Wyatt 2010).Also referred to as systemic design, this intersection "brings human-centered design to complex, multi-stakeholder service systems" (Jones 2014:93).Both human-centered design and systems thinking have received attention as potent tools for addressing current societal problems, but literature on the coapplication of these methods is scarce (Jones 2014).The Deep Change Method, created by Stanford ChangeLabs, combines the two methods into a formal, cohesive process for addressing complex systems challenges and producing scalable interventions (Banerjee 2014).We define a complex system as a set of elements interconnected in such a way that they produce their own internal dynamics that are difficult to understand, predict, manage, and/or change (Magee andDe Weck 2004, Meadows andWright 2008).Thus, designing an intervention for changing how system components interact requires the integration of multiple stakeholder perspectives, and a high level of comfort with developing and testing solutions in the face of uncertainty and continually shifting system dynamics.

Deepwater Horizon as a complex system challenge
The Deepwater Horizon oil spill disaster (DWH, 2010) was the largest marine oil spill, and the largest mobilization of resources to address an environmental emergency, in U.S. history (Lubchenco et al. 2012).DWH was an unprecedented disaster: oil spilled for 87 days in an extreme ocean environment, spreading rapidly throughout the water column and to the atmosphere.The constantly changing crisis presented novel policy challenges and required unprecedented engagement and collaboration among scientists from multiple disciplines across government, academia, and industry (Lubchenco et al. 2012, McNutt et al. 2012).
Large oil spills, like many environmental challenges, are often classified as "wicked problems" (Rittel andWebber 1973, Buchanan 1992).Wicked problems are extremely difficult to solve and are characterized by a lack of information, overlapping and difficult-to-map drivers, and conflicting value systems among actors (Rittel andWebber 1973, Buchanan 1992).More than six factors contributed to DWH being a wicked problem.First, DWH was the largest marine oil spill in U.S. history and its geographic breadth, the far-reaching impacts, and the urgency of the response were unprecedented (Mabus 2010).Second, there were many unknown unknowns, e.g., the flow rate of oil egressing from the damaged well, affecting multiple dimensions of the response.Third, complex dynamics within and between stakeholders affected the response, where traditional stakeholders, e.g., multiple federal and state authorities and a designated responsible party, engaged with new stakeholders, including academic scientists.The convergence of different missions, cultures, and perceptions posed coordination challenges.These differences were sometimes magnified by 24/7 media coverage, which introduced additional pressure to the stakeholders (National Commission 2011).Fourth, human cognitive bias known as "discounting," i.e., prioritizing the acquisition of present and near-term rewards rather than longer term, potentially greater rewards, negatively affected the amount of organizational, financial, and human resources invested in sustainable long-term solutions.Fifth, the diversity of actors who became involved in the response reported to different authorities, some outside the National Contingency Plan, contributing to confusion and communication challenges.Sixth, governmental, industry, academic, and public interests had overlapping priorities, yet differing levels of ability to solve the problems that emerged during the disaster (Lubchenco et al. 2012).Finally, extrinsic factors further influenced and complicated the implementation of solutions, including funding constraints, legal restrictions, and bureaucracy.
In examining the diverse scientific, social, and institutional complexities that amplified the challenge of responding to DWH, crisis decision makers consistently identify cross-sectoral collaboration as one of the most critical challenges that, if addressed, could greatly improve the social and ecological outcomes of future environmental crises (Lubchenco et al. 2012, McNutt et al. 2012).DWH revealed the wealth of science and technology resources available within the broader scientific community, their strong desire to help, and how crises can spur the rapid advancement of valuable new scientific knowledge.Yet it also exposed weaknesses in the system of information dissemination and exchange among scientists from those three sectors (Machlis andMcNutt 2011, Lubchenco et al. 2012).Traditional communication mechanisms, affected by policy, previous experience, and culture, constrained the sharing of information between responders and the broader scientific community.Lack of communication across sectors and legal constraints complicated collaboration and the ability of academic scientists to rapidly mobilize, assist with spill response, and evaluate the potential social-ecological impacts.In the words of one scientist interviewed as part of this project and who was involved in DWH: "The many stakeholders involved did not share a common language, timeframe, set of values, or pre-existing relationships."The lessons learned from prior spills such as Exxon Valdez (1989) were helpful, but ultimately did not create an effective infrastructure to support rapid collaboration among federal, industry, and academic scientists during a future spill.In addition, the unique nature of DWH being the first spill to occur at significant depth (~1500m below the sea surface) introduced new response challenges.

Systems thinking to tackle complex environmental challenges
Wicked problem characteristics are common across complex environmental problems, including climate change, extractive resource use, and biodiversity loss (Levin et al. 2012).The Deep Change Method is uniquely suited to address these problems because it considers (1) system complexity, (2) behavioral psychology of human stakeholders, and (3) adaptability for unknown unknowns (Banerjee 2015).When designers use a systems thinking lens, they can decipher interventions that will impact complex, multistakeholder challenges.Both humancentered design and systems thinking can be applied to any human experience to solve wicked problems within and across subjects, putting human behaviors that drive the problem, and are necessary for sustainably solving the problem, at the center of the process (Buchanan 1992).Contrary to some science disciplines that use linear processes to distill deterministic components of a problem and a solution, a design process assumes nonlinearity and unpredictable system dynamics.It is structured to yield previously unknown solutions, solutions that have not yet been previously generated or applied to the problem.The Deep Change Method, like other design methods, explicitly avoids taking any single disciplinary approach, and instead employs tools that integrate multiple points of view to address seemingly intractable challenges.
Using the Deep Change Method in the context of environmental disasters presents an opportunity: How might we design a solution to the relationship failures that emerge in high-stress, high-stakes disasters?We pinpoint why and how design research and the Deep Change Method can be leveraged to address diverse and complex environmental problems.Specifically, we present a series of methods applied to the challenge of scientific collaboration before, during, and after environmental crises, specifically focusing on large oils spills such as DWH.https://www.ecologyandsociety.org/vol22/iss2/art18/METHODOLOGY Our process consisted of six phases: challenge framing, ethnography, synthesis, concept generation and idea selection, prototyping, and testing and solution refinement (Fig. 1), using oil spills as our test case.These six phases map to the phases included in the human-centered design process and the Deep Change Method (Brown 2009, Banerjee 2015).However, we adapt them here based on the time frame and scope of our project challenge.Throughout the process, we hosted in-person workshops and conference calls with a group of advisors with broad experience with DWH and other environmental crises across government agencies and academia to guide our design.We had four target outcomes for the project focused on environmental response and remediation (instead of the engineering challenges posed by deep-water drilling, such as source containment and kill operations): (1) minimum oil exists in the marine and coastal environments because responders prevent or stop the flow of oil and/or released oil is contained; (2) the environmental and human impacts of the oil spill are mitigated through preparation, response, and/or restoration; (3) response efforts in future large oil spills are effective and efficient in achieving harm reduction goals; and (4) increased scientific understanding of environmental and human health impacts of an oil spill improve long-term ecosystem management.During concept generation and solution refinement, we vetted our material against these target outcomes, which provided the basis for excluding or including insights and ideas intended to address the challenge.For example, we rated how shifting a set of incentives or behaviors would likely impact each target outcome and then prioritize ideas that scored highly across all four.

Preliminary challenge framing
At the beginning of a design project, the challenge framing should be broad enough to allow the project team to discover areas of unexpected value (Seelig 2012).Our initial framing centered on the question of how we might enable "effective, scientific communication between academic scientists and federal responders during large oil spill crises."During preliminary challenge framing, we mapped the federal and state government and academic actors and resources for disaster preparedness and response, including the flow of funding, information, and partnerships.These maps helped us identify where new solutions were most critical.The exercise also expanded our understanding of the complex and overlapping drivers that influenced human relationships during DWH.As a result of the insights gathered during this preliminary stage, the final challenge framing was then revised and refined after the design ethnography stage.

Ethnography
During ethnography, a process to observe and understand the users for whom researchers are designing, two of the authors and members of the internal project team (Mease and Gibbs-Plessl) identified stakeholder interviewees.These stakeholder groups were identified during pilot interviews and through a review of the literature on preparedness and response structures.This project identified several types of stakeholders involved in the response and recovery from Deepwater Horizon: academic scientists, government scientists, industry scientists, responders, and government administrators.Key stakeholders included those (1) that directly influenced the relationship failures that occurred between government administrators and responders, government scientists, and academic scientists; (2) who had decision-making authority within the government or academic structures that our project sought to assess, e.g., elected or appointed officials, university deans; and/or (3) whose work was hampered by insufficient scientific support during the disaster response cycle, e.g., local nongovernmental organizations.The internal project team conducted semistructured phone interviews with 72 stakeholders including academic scientists, government agency staff, elected officials, and industry representatives, (Fig. 2).The interviews included narrative-style questions similar to vignettes in sociological research (Bloor and Wood 2006) to elicit feelings and underlying interests within disaster scenarios.The majority of the questions were structured to identify the goals, motivations, and perceptions of the respondents to contribute to our understanding of the needs and mindsets driving their behavior (Table A1.1).One researcher https://www.ecologyandsociety.org/vol22/iss2/art18/conducted the interview while a second researcher took detailed notes.Finally, the team used existing literature on scientific crisis preparation and response from other disaster types to guide our ethnography process, such as research on the science community's engagement during influenza pandemics and the Fukushima nuclear meltdown, as well as the broader literature on crisis communication.

Synthesis and final challenge framing
The internal project team used a postinterview capture form to interpret and synthesize the insights from the interviews.The form captured a range of information: compelling insights, challenge framing, primary responsibilities of the interviewee, resources controlled, self-perceived agency of decision making in a response situation, relationship with other system stakeholders, key underlying emotions communicated, role in the system, familiarity with existing solutions, and potential new solutions envisioned for the project challenge.From this information, we created persona profiles for four primary stakeholders: (1) academic scientist local to the disaster site, i.e., working at an academic institution in the state affected by the disaster, (2) academic scientist not local to disaster site, (3) U.S. Coast Guard administrator, and (4) National Oceanic and Atmospheric Administration (NOAA) Scientific-Support Coordinator.These profiles included the stakeholders' motivations and goals, intrinsic and extrinsic barriers to their agency, and dominant perceptions of other primary stakeholders (Fig. A1.1).Humancentered design often employs user personas to connect solutions to user needs refined from ethnography, and have been shown to improve the usability of final design products (Long 2009).In this study, persona profiles served as generalized, fictitious representations of stakeholders the project team could reference to guide evaluation of stakeholder needs, motivations, and behaviors.
From final challenge framing, shaped by interviewees, we constructed a root cause map.Root cause analysis investigates and links observable phenomena to underlying drivers.Addressing root causes can mitigate negative cascading effects and amplify positive impacts.We used the online concept mapping software Kumu (https://kumu.io/) to link the drivers of the challenge to each other and order them hierarchically, to visually depict the dominant root causes adjacent to one another in a tree format (Fig. 3).Fig. 3.An example root cause tree that illustrates how asking "why" many times yields a causal chain with a leverage point at the end node that, if shifted, will influence all of the drivers in the proceeding nodes.This tree has been truncated; there may be additional nodes bifurcating from each node.
Following root cause analysis, we identified the leverage points within the system of oil spill preparation and response, and associated nongovernmental, e.g., academic and industry, science (Table 1).Leverage points are places within a complex system where a small change in one component will produce a large change across the system (Meadows and Wright 2008).After creating a list of hypothesized leverage points (Table A1.2), we narrowed our focus points based on the following criteria: (1) impact (shifting this leverage point will create significant positive results across the four desired outcomes), ( 2) feasibility (the leverage point can be effectively shifted within 1-5 years with existing resources and minimal funding), and (3) scalability (the leverage point applies to other scales and types of disaster response situations).The team mapped the resulting strategic leverage points onto the root cause map to identify interventions that also address root causes.This process enables a design team to revisit the preliminary challenge framing and tighten it to reflect the critical root causes and stakeholder needs discovered through design ethnography.
We also referenced written material and distilled insights from interviews with stakeholders from analog challenge contexts.Analogs are systems that mirror a key component of the target system, and from which designers can derive ideas and reframe the problem.An example of an analog system is the public health disaster response for Ebola and H1N1 and their respective patterns in resource mobilization and allocation.https://www.ecologyandsociety.org/vol22/iss2/art18/Table 1.System leverage points used to identify priority root causes for intervention.
Leverage Point (in increasing order of effectiveness)

Definition Examples
The size of buffers Buffers maintain system stability; the larger the buffer the more a system can withstand a shock.
The magnitude of academic scientific interest in spill research between crises (e.g., bolstering research interest between spills, rather than letting it oscillate drastically between large spill events) provides a buffer around the relative scientific demands of any one crisis.The lengths of response delays, relative to the rate of system change Key time delays before or during the response that were barriers to the desired outcomes.
The communication of research needs from responders to academic scientists during oil spills; the length of the peer-review process slows rapid application of research findings to decision making.

Negative feedback loops
Negative feedback loops before and during the response that hindered the desired outcomes.
Intense media coverage during large disasters creates a pressure on politicians, responders, and scientists to have firm and unwavering, "sound-byte" positions.This degrades efforts at collaboration between stakeholder groups.

Positive feedback loops
Positive loops before and during the response that helped the desired outcomes.
Some scientists who became involved in the response efforts were able to build long-term relationships with responders, leading to sustained collaboration before and during subsequent incidents.

Mindsets and perceptions
Mindsets held by key stakeholders that inform their behavior.
Discounting of future disasters by agencies, academic scientists, industry, and the public.The rules of the system Governing rules of the system across geographies and time scales.
The prescribed structure of area, regional, and national contingency plans.
The goals of the system Overarching system goals (often not directly stated) that define institutional and individual stakeholder behavior.
The goal of extracting oil.

Concept generation and idea selection
To generate solutions to the science collaboration challenges that emerge during large environmental disasters, the team identified concept generation prompts based on the strategic synthesis of the ethnography findings.Where root causes overlapped with system leverage points, the team constructed "How Might We" statements.How Might We statements frame a challenge as surmountable and spur the design team to generate a wide range of possible solutions through brainstorming.For example, the root cause of "academic scientists and government staff do not codevelop priority research questions between crisis events" is an example of a "length of delays, relative to the rate of system change" leverage point (the delay in the communication of research needs from responders to academic scientists during oil spills).The corresponding How Might We statement was "How might we enhance academic participation in preparedness and contingency planning?"For the root cause of "a lack of consistent scientific interest in applied spill response research," the leverage point is the "size of buffers" (increased buffer of time around spill response research between spills), and the How Might We is "How might we create long term funding cycles for oil response research?" The internal project team and a group of 15 expert project advisors (including coauthors Reddy, Ludwig, and Lubchenco) conducted concept generation during a two-day in-person workshop, generating 15 solution ideas.The internal project team then brainstormed over 50 additional ideas, based on the system leverage points.The 15 project advisors scored the solution ideas across four criteria: impact, feasibility, novelty, and applicability to other disaster types (on a scale of 1-5).Based on this scoring, three ideas were selected to prototype.Novelty was evaluated to gauge the innovation potential of the idea, as an indicator of shifting a system from its status quo.

Prototyping
The internal project team built three prototypes to test in-person with a broader group of users at the 2015 Gulf Oil Spill and Ecosystem Science Conference in Houston Texas, attended by individuals across the target stakeholder groups.A prototype is an early model of a concept, process, or product developed to explore and test its efficacy in achieving the desired design outcomes (Brown 2009).For example, at the conference, the primary prototyping tool that was used was a "user journey scenario," presented in the form of an interactive printed booklet.
The prototypes were scenario-based and walked users through an experience flow of their imagined role in the new crisis decisionmaking system and the change in their behavior if the solution's value was realized.The booklets solicited interaction through periodic reflection questions, both written and through verbal discussion with surrounding participants.Prototyping is useful for testing specific hypotheses about the value that a proposed solution is designed to create.Prototypes allow designers to incorporate user feedback quickly, iteratively, and inexpensively.
Prototyping also helps designers identify how real-world human or system constraints compromise an idea's integrity.Importantly, prototypes often start conversations with potential users that uncover additional insights and help inform improved solutions.
During the group workshop at the conference, 36 participants were arranged into multistakeholder tables (academic scientists, responders, and oil industry reps) to facilitate cross-sector dialogue.The participants were guided through the interactive booklets, which posed the storyline, hypothetical decisions, and potential rewards associated with each idea.We collected the participant feedback through audio recording and transcription of the participant conversations and captured written feedback each participant provided in the interactive booklets.Additional prototyping feedback was received from individual 30-to 75-https://www.ecologyandsociety.org/vol22/iss2/art18/minute conversations with academic and government stakeholders at the conference.We synthesized this feedback to refine the solution structure of all three prototypes.

Testing and solution refinement
The last phase of the design process was to select and refine a single proposed solution based on prototyping and testing the solution with key system stakeholders.This phase also included implementation planning and identified target outcomes for the proposed solution.Importantly, during the testing phase designers must remain curious and open to user feedback as they continue to refine the quality and power of the proposed solution.
Because a greater diversity and number of users are engaged, designers continually enhance the robustness of the idea, tweaking the solution to serve the needs and create additional value for stakeholders.
The internal project team tested the final solution idea through one-to two-hour long phone interviews with 45 system stakeholders and seven webinar presentations.The proposed solution was described in detail, highlighting the value created for a variety of users and the structural components that underpin the solution's feasibility, impact, and scalability.We asked scenario questions to identify how the proposed solution may or may not achieve its target value with the interviewees.We then refined the proposed solution based on the feedback received over the course of three months.For example, based on user feedback from government stakeholders, the project team shifted the solution's focus from integrating nongovernmental scientific involvement in actual disaster response activities toward disaster preparedness activities, e.g., table-top exercises.In addition to phone interviews, one in-person, two-day workshop was held with nine project advisors to refine the features of the solution, build an implementation plan, and identify key benchmarks for its implementation.
The key stakeholders of the focus challenge were continuously involved in the framing, brainstorming, and testing of our project solutions.This method of "codesign" with users and stakeholders is distinct from traditional processes of product design, where interaction between the designer and user is discrete and limited.We build on the definition of codesign given by Sanders and Stappers (2009) to refer to the creative act of designers and people not trained in design intentionally working together at multiple points of the design development process.Codesign is a valuable and important aspect of systems thinking, because it harnesses the insights of users directly involved in the challenge that is being addressed.In a systems solution, there is almost never a single "end user," instead, there are multiple stakeholders who will interact with the solution.Designers must create distinct solution features to effectively engage and generate value for these different stakeholders.The complexity of this design space necessitates a participatory approach where the users become coproducers of viable solutions, and the design team takes the role of process facilitators.This methodology builds upon the scholarship of participatory design, which involves giving prototypes to nondesigners and adaptively incorporating their feedback (Björgvinsson et al. 2010).In the case of experience design, this can be done through iteratively proposing new decision points and revising the prototype to reflect differing stakeholder experiences.The close involvement of the project advisors in our design challenge-who are ultimately the users of the design solution itself-is also an example of using participatory design to ensure the outcome is grounded in the human needs and realistic constraints of the challenge system.

RESULTS
Our targeted ethnography uncovered key insights about how scientific collaboration is incentivized and motivated among governmental decision makers and academic scientists.Through interviews with key stakeholders, idea generation with project advisors, and identification of desired project outcomes, we identified a set of key system root causes and leverage points from which we could address the problem.

Final challenge framing
We synthesized the challenge framings articulated by our informants to narrow and refine the scope of the challenge.
Determining the final scope of the challenge framing is critical because it determines the scope of the solution.Many interviewees drew parallels between the challenges of collaboration in marine oil spills and other environmental disasters, thus we broadened our challenge framing to include a broader set of environmental crises (chemical spills, rail-based oil spills, hurricanes, tsunamis, and severe winter storms).Further, we heard that many of the barriers to collaboration during response were rooted in the obstacles to relationship-building and communication that exist between crises, i.e., before and after disasters occur.Thus, we extended our final challenge framing to include the collaboration challenges that emerge before, during, and after crises occur.

Design specifications
Our synthesis of ethnographic insights revealed a number of key system characteristics that, if achieved, may enable greater scientific collaboration between our target users: academic scientists and government agencies tasked with disaster preparation and response activities.These characteristics served as our design specifications as we generated potential solutions to our design challenge.We included a qualitative analysis of these design specifications when evaluating solutions for their impact, feasibility, and scalability.Specifically, we analyzed the extent to which the solution (1) builds trust between academia and the government response community before a crisis occurs; (2) maps to the existing motivations of target users by creating genuine and tangible value on relevant time scales; (3) remains active between crises to foster the relationships necessary to rapidly identify, investigate, and communicate about scientific unknown unknowns; and (4) decreases the effort (time and money) responders must expend on engaging with academic scientists during a response operation.

Root causes
The root cause map revealed 78 system attributes that drive undesirable individual, institutional, or system behaviors relative to our target outcomes.Of the 78 root causes mapped, 27 were related to the leverage point of shifting mindsets, 23 were related to material or information flows and their buffers (elements that maintain system stability), 13 were related to feedback loops, 8 were related to changing system goals, and 7 were related to changing the rules of the system.We found that the stakeholder perceptions, particularly ones that were critical or unfavorable Table 2.The 20 leverage points most powerful for achieving our four target outcomes.

Leverage Point Description
The size of buffers Increased buffers around spill response research between spills, i.e., bolstering research interest between spills, rather than punctuated interest only during spill events Increased buffers around spill response research between spills, i.e., bolstering research interest between spills, rather than punctuated interest only during spill events The delay in the communication of research needs from responders to academic scientists during oil spills The delay in mobilizing scientific resources, e.g., physical or intellectual, among key institutions to the spill site during oil spills The delay in finalizing scientific products due to processes assuring they meet scientific standards of quality (Quality Assurance Quality Control) during spills The lengths of delays, relative to the rate of system change The delay in adaptive learning among agencies and academic scientists after response drills The strength of negative feedback loops

Media pressures on politicians, responders, and academics
The gain around driving positive feedback loops Journals create publication addendums to allow data sharing during crisis, e.g., Science and Nature allow the release of nonpublished data, academics are incentivized to collaborate on data analysis Rapid response grants for academic research were disseminated to scientists, leading to greater scientific understanding and increased capacity to secure funding Some scientists who became involved in the response efforts were able to build long-term relationships with responders, leading to sustained collaboration Mindsets and perceptions Stakeholders and the public discount disasters, underestimating the likelihood of their occurrence in the future Stakeholders often have an assumption of no "unknown unknowns" before or during a response Agency perception that staff know what to do during a response and there is not a role for academics Academic perception that their data will be used by decision makers if it is produced, and it is not their responsibility to translate it In human-caused disasters, stakeholders often have a need for a scapegoat, e.g., blame and distrust of responders because of relationship with the responsible party; there is not a sense of collective responsibility for an oil spill Responders often have multiple objectives, e.g., mitigate oil spill or meet public expectations, whereas academics often have a single objective, e.g., scientific discovery and publication Agency mindset that disaster planning is not a collective responsibility across agencies The rules of the system The Incident Command System and the Oil Spill Pollution Act, e.g., designation and role of the responsible party Area, Regional, and National Contingency Plans The National Resource Damage Assessment regarding the values of other stakeholder groups, were impeding collaboration and relationship building.For instance, government administrators tend to operate with a mindset that they know how to manage spills, without additional input from academic scientists, and academic scientists tend to have a mindset that their research will be used, regardless of their communication and engagement of that research.We also found that extrinsic forces, such as limited research funding for crisis science, legal and contracting hurdles, and time constraints, exacerbate stakeholder cultural conflicts.

Leverage points
Solution ideas were generated from How Might We statements based on the 20 leverage points that were identified as uniquely powerful in shifting crisis collaboration toward our 4 desired outcomes (Table 2).The final project solution explicitly incorporates features to use 16 leverage points to create tangible and durable value for all system stakeholders.A full list of the 44 system leverage points identified for enabling system change can be found in Appendix A1.3.

Prototyping
The multistakeholder prototyping workshop at the 2015 Gulf Oil Spill and Ecosystem Science Conference was valuable for provoking generative, productive exchanges between stakeholders and identifying opportunities to improve the solution ideas by creating additional mutual value.One-on-one user testing interviews were also important for receiving targeted feedback on specific solution components.We observed stakeholders adopting a focused, creative mindset when given a tangible solution prototype to interact with, as opposed to discussing solutions in the abstract, a frequent pitfall of solution processes for complex systemic challenges.The workshop and other small group prototyping sessions also provided a space to initiate and strengthen cross-stakeholder relationships important for solution implementation.

Testing
The final solution selected by the internal planning team was the Science Action Network, a community of academic and professional scientists who are linked to regional government planning and response bodies to coordinate and streamline scientific input for decision making before and during disasters.The Science Action Network would enable increased crossdisaster preparedness and would support response decision making through novel academic-agency partnerships, resource sharing, and coordination of scientific input (see Table 3 for descriptions of how the Science Action Network meets the project design specifications).
The project team selected the Science Action Network as the final project solution based on prototyping feedback and our filtering criteria (Table 4).Based on what we heard from key stakeholders while testing the network's potential value, we adapted the network to be more heavily focused on precrisis relationship Table 3. Design specifications distilled from ethnography and how the final proposed solution, the Science Action Network, met those specifications.

Design Specifications Solution Design
Builds trust between academic scientists and the response community before a crisis occurs.
The primary purpose of the Science Action Network is to increase the levels of interaction, "connectedness," and trust between academic scientists and government administrators before crises occur.The recommended mechanisms to achieve this include (a) regular scenario-based collaborative problem-solving roundtables and drills between responders and academic scientists; (b) public recognition of academics' service to the network through their institutions and professional societies; (c) increased access to research funding opportunities for network members, through relationship building and potential streamlined funding during crises; and (d) the establishment of an annual Science Action Network national conference.Maps to the existing motivations of target users by creating genuine and tangible value on relevant time scales.
The network produces compelling value for diverse stakeholders.Differences in the goals and incentives of academics, responders, and industry are a major hindrance to effective collaboration.The Science Action Network strives to create mutual value between these key stakeholders through collaborative resource sharing and joint access to information.New or enhanced incentives for academic involvement in agency decision making through the network will lead to a broader diversity of disciplinary and geographic expertise in planning and response.Trainings offered to network members around preparedness and response research needs and disaster response protocols will increase the usability and applicability of network research outputs, and enhance the cultural competency of both agency staff and academics to communicate and collaborate between and during crisis situations.Remains active between crises to foster the relationships necessary to rapidly identify, investigate, and communicate about scientific unknown unknowns.
The network sustains participation through novel cross-hazard learning and exchange.Because large disasters such as Deepwater Horizon happen relatively rarely, academic interest in disaster-relevant science wanes between incidents, weakening academics' ties with the response community.The network is flexed to prepare and respond to multiple types of disasters: thus members will be engaged more frequently and across more disciplines.
Decreases the effort (time and money) responders must expend engaging with academic scientists during a response operation.
The network leverages and amplifies existing resources and competencies.Many existing solutions are informal and customized to one institution, geography, or hazard type.The Science Action Network fills a critical gap not addressed by existing strategies: it catalyzes interdisciplinary disaster-relevant research before incidents occur, ensures that the research is integrated into response preparation, and streamlines decision makers' access to necessary scientific expertise during incidents.Academic scientists who apply to join the network are required to complete an orientation training that will familiarize them with the (1) basic legal and organizational structure of federal and state response, and ( 2) the critical cultural differences between academia and government agencies (and bridging strategies to use).Finally, the network reduces the friction of searching for necessary expertise by utilizing existing disciplinary relationships within academia to source the best science for scientific unknowns during crises.The network's regional chapter structure is designed to tap into the place-based knowledge and connections of academics at local institutions to inform decision making and enhance agency efforts to build trust and clear communication with local communities impacted by a disaster.
building through collaborative research and preparedness efforts.Further, we refined the network structure to include both a geographic organization (via 10 hubs) and a disciplinary organization, e.g., academics in the field of crisis communication connected across regions.A disciplinary structure enables liaisons to streamline identification of disciplinary expertise necessary for a particular incident, which may not exist locally.It also ensures cross-pollination of key research priorities, opportunities, and accomplishments across the network by piggybacking on existing disciplinary social networks and professional societies.

DISCUSSION
Our methodology was well suited to address the complexity and difficulty of our project challenge.The human-centered design and systems thinking techniques we used uncovered the goals and perceptions of our target users and informed a final solution that is grounded in the real-world needs and behaviors of those that have the potential to improve crisis decision making through increased cross-sectoral scientific collaboration.We found that stakeholders in the system of scientific decision making for disasters have differing motivations to engage and, thus, partly because of extrinsic forces such as funding and intrinsic forces of culture, lack the trust necessary for collaboration or communication in high-stakes contexts.These insights illuminated both why there were collaboration successes and why there were collaboration failures during Deepwater Horizon, and provided a springboard for reimagining how we might create more consistent, mutual value among stakeholders.Finally, our codesign approach enabled rapid iteration on our key insights, the generation of ideas grounded in the perceptions of the target users, and greater buy-in to the final solution.
Our ethnography uncovered goals, motivations, perceptions, and structural barriers impeding consistent collaboration among our target stakeholder groups before and during large, environmental crises.First, a profound perception gap exists between academic scientists and government administrators and responders, leading to a missed opportunity of collaboration before, during, and after a disaster event.Of the 82 root causes identified, the highest proportion related to the entrenched mindsets and perceptions of stakeholders regarding the role of science during response and the value of sharing information across sectors (n = 27).This gap in perceptions about the role of science in decision making, expectations for collaboration, and the potential mutual value that could be created among stakeholders hinders trust building.The gap is exacerbated because the two groups have dramatically different reward systems and priorities, which deepen the cultural Table 4.The criteria used to evaluate the project ideas and how the final proposed solution met those criteria.

Criteria Solution Design
Impact (creates significant positive results across the four desired outcomes).
The Science Action Network achieves impact by bridging cultural gaps between response agencies, industry, and academic scientists to create new norms for scientific collaboration; driving disaster-relevant and interdisciplinary scientific research through novel academic-agency partnerships and funding opportunities; and catalyzing cross-disaster and cross-institutional scientific exchange.Feasibility (effectively implemented within 1-5 years with existing resources and minimal funding).
The network will leverage existing institutional structures, e.g., regional preparedness and response interagency bodies, while building new working relationships between academic scientists, government administrators, and government scientists through full-time Network liaisons.The liaisons identify critical and efficient opportunities to collate relevant, local research to inform timely crisis decision making.Scalability (accounts for and scales across hazard types).
The network will explore previously unseen shared challenges and science needs across disaster types.The network's proposed Lead Advisory Committee comprises representatives from each of the federal disaster preparation and response agencies, multidisciplinary scientific professional societies, e.g., the American Association for the Advancement of Science, and private industry sectors, e.g., extractive energy companies.This diverse composition from the onset helps set up the network's membership and focus activities to strategically grow across hazard types, rather than remaining limited to only oil spills or hurricanes.
divide between them and weaken their willingness to engage productively.We also found that extrinsic forces, such as limited funding for rapid, opportunistic science during crises, legal constraints around governmental data transparency or academic contracting, and time constraints inherent to any crisis response, all amplify cultural conflicts and impede the ability of stakeholders to collaborate.By mapping how these forces affected collaborative relationship building, we were able to pinpoint the root causes of our challenge and craft a solution that addressed both the structural barriers to collaboration, e.g., lack of communication channels, and also the cultural barriers erected by polarized mindsets, e.g., the mindset of some government staff that there is no role for academic scientists in response decision making.

Key insights gained from applying the Deep Change Method
Using the Deep Change Method, which combines humancentered design and systems thinking techniques, to address our complex project challenge revealed how these tools may have broad relevancy to other socio-environmental challenges.Complementing the traditional, assessment-based research process with design methods and principles may be a powerful approach for integrating the real-world human behaviors and motivations of those affected by and/or driving system interactions.Methodologically, we took away several key lessons.

1.
Combining both traditional human-centered design and systems-oriented tools enabled more impactful and scalable solutions.Human-centered design tools such as stakeholder persona profiles increased stakeholders' understanding of each other's motivations, constraints, and worldview, and grounded solutions in real human needs.System thinking tools such as leverage point analysis enabled efficient and strategic convergence around key system behaviors that, if shifted, would achieve our target outcomes.In a complex system, both sets of tools are useful for characterizing system dynamics influencing the problem, while also empathizing with the stakeholders most salient to the design challenge.

2.
Codesign is imperative when crafting systems-oriented solutions.Moving beyond "engagement" with stakeholders and toward participatory codesign was essential for sculpting an impactful and feasible solution.Thus, partnership between the neutral lead design team and an active advisory team may improve solution viability.Critically, the internal project team had creative control over the idea generation and refinement process, which helped ensure a "whole system," focused solution, rather than one tailored towards one or two powerful stakeholders.This was particularly important given the easily politicized nature of the challenge and the collective biasing of the project outcomes if any one of the target users was able to disproportionately influence the design process.

3.
Prototyping system solutions requires human-and systemslevel testing.Experience prototypes must be tailored toward individuals to test solution assumptions around stakeholder perceptions and motivations.Prototypes must also test how the solution influences system behavior.Story-based multistakeholder prototyping (similar to role-playing) facilitated the identification of key interaction points between stakeholders and alternative pathways for increasing value in stakeholder relationships.One-on-one testing sessions with individual users were still valuable, but group prototyping sessions were critical for illuminating opportunities for generating mutual value and meeting multiple stakeholder needs.Finally, the solution must be rigorously tested with cross-sectoral groups to ensure it will be implementable within the existing system of formal political, economic, and legal structures, as well as compatible with the implicit cultural norms of the various stakeholder institutions.For example, testing the final solution across interagency working groups was important to restructure the Science Action Network to adhere to existing legal boundaries around governmental collaboration with nongovernmental bodies.https://www.ecologyandsociety.org/vol22/iss2/art18/impactful interventions to influence system behavior.The normative nature of the design process can be difficult for collaborators accustomed to assessment-based projects, e.g., most scientists, and we found that repeatedly regrounding our decision in the insights we discovered in the ethnography phase was critical for maintaining stakeholders' trust in the process and the final solutions.

Limitations
Design, by nature, is a flexible method that may look dramatically different in theory and practice across problems.When seeking solutions for complex system challenges, design processes methodologically rely on the ability to prototype in real-world conditions.If the design team does not have access to end users or real challenge scenarios to prototype ideas, the effectiveness of the final solution is limited.In this case study, we were unable to test the final solution in a real-world disaster context and our prototyping was limited to likely scenarios and stakeholder memories from past incidents.Future work would benefit from testing our final solution during an actual event or interagency table-top exercise.Finally, the field of human-centered design and systems thinking can use language and tools that make it impenetrable for stakeholders unfamiliar with the process.Thus, it is critically important to involve stakeholders early, provide clear definitions, and cocreate process goals to ensure buy-in and meaningful participation from participants and codesigners alike.

Future research
The fields of human-centered design, systems thinking, and the Deep Change Method are still nascent.Further research is necessary for the enhancement of the credibility, usability, and efficacy of the tools and processes used in this study.We outline five dimensions of a research agenda that we believe are uniquely valuable in advancing the theory and practice of complex systems thinking.
First, research is needed to understand how engaging stakeholders from analogous challenge types, e.g., involving representatives from medical crisis response, is useful in each phase of the design process.Seeking inspiration from analogous challenges is a basic innovation tool for designers because it seeds the process with new ideas and helps ensure solutions scale to other contexts.In our project, we conducted literature reviews and several conversations in the fields of nuclear disaster management, public health epidemics, and cybersecurity threats.However, we did not invite stakeholders from any of those analogous sectors to participate as core project advisors or as participants in project workshops.It would be valuable to understand how such analog stakeholders or experts influence the design process and the novelty of produced solutions.
Second, research is needed to test and identify additional techniques for multistakeholder system prototyping.Creating a tangible product or experience to represent a system intervention is difficult and complex.More tools are needed for prototyping systemic solutions, and to better understand their efficacy in testing solution assumptions, both on the human and system scales.
Third, codesign is becoming a more widely applied practice across multiple fields, e.g., policy creation and urban planning, and research is needed to test and compare codesign practice and principles.Additionally, little is known about the long-term impact of the participatory design process on stakeholders who participate in it, both on their perceptions of the challenge and the system, and of themselves and their own motivations.
Fourth, there are few metrics for evaluating the diffuse and often indirect impacts of system-oriented design solutions, as well as holding the process and designers accountable for those impacts.
A designer often sees his or her work as done once the solution proposal is finished, and is minimally involved with the process of solution implementation.In complex systems, it is difficult to assess the distributed effects of interventions.Designers need methods for tracking impact of implemented solutions to both assess the efficacy of their methods and to increase their connectivity with stakeholders who are involved in or have been affected by the implementation of their proposed solutions.
Finally, because systems thinking tools and methods proliferate across disciplines and problem contexts, research on the efficacy of mixed qualitative and quantitative methods is needed.Design is both an art and a science.A better understanding of how various methodologies influence design outcomes would be valuable, particularly as design metrics and standards for accountability become formalized.

CONCLUSION
In this study we provide a framework for using design methods in novel and diverse applications across the environmental and social sector.Using oil spills as our focus case study, we demonstrated the value of the Deep Change Method, which combines human-centered design, behavioral psychology, and systems thinking, for creating a systemic solution for scientific collaboration during environmental crises.Codesign methods created active buy-in from relevant stakeholder groups in the final solution.Human relationships, organizational culture, and trust were identified as critical barriers to collaboration by project interviewees.Thus, human motivations and organizational culture were centered in the final solution concepts.Most problems are, at their root, caused or can be remedied by humans, thus integrating human behavior and incentives in designing solutions is critical for addressing them.And, finally, the final solution is efficient and innovative by leveraging existing resources and stakeholder motivations to generate new value for multiple stakeholder groups.These outcomes demonstrate the potential for the Deep Change Method to be applied across many complex social and environmental problems.It is hoped that some of the tools and insights discussed here serve as a foundation for considering the system complexity that drives human challenges.
Responses to this article can be read online at: http://www.ecologyandsociety.org/issues/responses.php/9246

Fig. 1 .
Fig. 1.The convergent (synthesizing insights) and divergent (exploring new pathways for framing the problem and generating solutions) phases of the Deep Change Method.The recursive arrows emphasize the iterative, nonlinear nature of the method.Challenge framing: Develop problem frames, a theory of change, and associated target outcomes.Ethnography: Conduct in-depth interviews and observation of key system stakeholders to understand their roles, power, motivations, and behaviors.Synthesis: Articulate system and stakeholder behavior, resources flows, barriers, opportunities, and leverage points.Concept generation: Generate a short list of intervention concepts that are leveraged and have the potential to create large-scale impact.Prototyping: Flesh out concepts through experience design, iterative prototyping, and field-testing.Select 1-2 strategies to test.Testing & Solution refinement: Refine the strategic and tactical elements of the intervention experience, including scaling, diffusion, evaluation, and measurement strategies.Finalize a implementation plan and test with key stakeholders.

Fig. 2 .
Fig. 2. The number of interviewees across each target stakeholder group: government; nongovernmental environmental organizations, including independent research institutions, advocacy NGOs, and consultants; local academics, working at an academic institution in the Gulf of Mexico; nonlocal academics; and industry.The second annulus graph represents a breakdown of the 35 interviewees working in federal government agencies.