The Virtual River Game: Gaming using models to collaboratively explore river management complexity

Serious games are increasingly used as tools to facilitate stakeholder participation and stimulate social learning in environmental management. We present the Virtual River Game that aims to support stakeholders in collaboratively exploring the complexity of a changed river management paradigm in the Netherlands. The game uses a novel, hybrid interface design that features a bidirectional coupling of a physical game board to computer models. We ran five game sessions involving both domain experts and non-experts to assess the game ’ s value as a participatory tool. The results show that the game was effective in enabling participants to collaboratively experiment with various river interventions and in stimulating social learning. As a participatory tool, the game appears to be valuable to introduce non-expert stakeholders to Dutch river management. We further discuss how the hybrid interface combines qualities usually found in board and computer games that are beneficial in engaging stakeholders and stimulating learning.


Introduction
Stakeholder involvement and participatory approaches are increasingly important in environmental decision-making (Pahl-Wostl et al., 2008;Reed, 2008;Voinov et al., 2016). A recent shift from sectoral towards more integrated natural resources management has made stakeholder participation essential to the pursuit of cross-disciplinary objectives (Berkes, 2009;Pahl-Wostl, 2007;Reed, 2008). At the same time, stakeholder participation is recognized as an effective way to improve the quality of and acceptance in decision-making (Cundill and Rodela, 2012;Reed, 2008;Voinov et al., 2016). One method that is receiving increasing attention to facilitate stakeholder participation in environmental management is serious gaming (Aubert et al., 2018;Rusca et al., 2012;Voinov et al., 2016).
Serious games are generally referred to as games that have a primary purpose other than mere entertainment (Michael and Chen, 2005;Susi et al., 2007). In the context of environmental management and participation, serious games are defined by Mayer (2009) as "experi(m)ent(i) al, rule-based, interactive environments, where players learn by taking actions and by experiencing their effects through feedback mechanisms that are deliberately built into and around the game". To provide feedback on actions, such serious games include a simplified representation of reality in terms of both the environmental system and its stakeholders (Harteveld, 2011;Redpath et al., 2018;Rodela et al., 2019). In this way, serious games enable stakeholders to explore environmental challenges, interventions, and the effects of such interventions in an environment in which it is safe to experiment. Furthermore, serious games enable stakeholders to experience the strategic interactions between stakeholders by explicitly including interaction rules and assuming stakeholder roles in the game. Therefore, serious games facilitate stakeholders' learning about both the physical-technical and the inherent socio-political complexities (Bekebrede, 2010;De Caluwé et al., 2012;Geurts et al., 2007;Mayer, 2009). The lessons learned while playing serious games can be both relevant and transferable to real-world decision-making (Geurts et al., 2007;Mayer, 2009).
One promising opportunity for serious games in the context of stakeholder participation in environmental management is their use as what Rodela et al. (2019) categorize as learning-based interventions. Such games are developed to engage stakeholders in dialogue and activity in order to contribute to what is commonly referred to as social learning: changes in understanding through interaction in collaborative and participatory processes that go beyond the individual (see e.g. Cundill and Rodela, 2012;Muro and Jeffrey, 2008;Reed et al., 2010;Rodela, 2011). As learning-based interventions, games are therefore developed under the assumptions that these: (1) provide stakeholders with participatory environments that facilitate the stakeholder interactions and collaborative experimentation that are essential to establish social learning; and (2) contribute to individuals or groups experiencing a change in understanding as a result of the game's collaborative activity (Ampatzidou et al., 2018;Flood et al., 2018;Medema et al., 2016). In the literature, social learning is usually operationalized as: (1) cognitive learning: acquiring new or restructuring existing knowledge; (2) normative learning: changing viewpoints, values, or paradigms; and (3) relational learning: increasing the understanding of the mind-set and perspectives of other stakeholders as well as fostering the ability to cooperate among stakeholders (Baird et al., 2014;Ensor and Harvey, 2015).
In the context of environmental management, there are various examples of serious games as learning-based interventions (see e.g. reviews by Aubert et al., 2018;den Haan and van der Voort, 2018;Flood et al., 2018 for games on water management, sustainability, and climate change respectively). As one of a few recent examples (from a far longer list), Craven et al. (2017) developed SimBasin to bring stakeholders together with the aim of developing a shared understanding and sense of urgency around the management of the Magdalena-Cauca river basin in Colombia. They showed that the game was successful in creating an open discussion space to bring stakeholders and scientists together. The Sustainable Delta Game (Valkering et al., 2013;Van der Wal et al., 2016) challenges stakeholders to develop collective strategies to manage a fictional stretch of a Dutch river and aims to help them to learn about the complex interactions between river management, climate change, and changes in society. Results from twelve sessions showed that playing the game led to the convergence of the players' perspectives ( Van der Wal et al., 2016). Becu et al. (2017) developed LittoSim to enable social learning among local authority managers on Oléron Island in France about prevention measures to reduce the risk of coastal flooding. They showed that LittoSim facilitated stakeholder experimentation with and learning about risk prevention measures in relation to possible flood events. van Hardeveld et al. (2019) developed the RE:PEAT game to explore collaborative management strategies to help reduce soil subsidence in the Netherlands. Results from ten sessions showed that RE: PEAT improved cooperation among peatland stakeholders, increased their understanding of the problems, and led them to possible strategies for reducing soil subsidence.
Recent changes in Dutch river management provide a valuable case study for designing and evaluating a serious game as a participatory tool. Traditionally, river management has been dominated by dike strengthening and was therefore within the field of hydraulic engineering. A new management paradigm, involving spatial measures and multifunctional design, brings in many other stakeholders that are traditionally not usually involved with river management. The challenge of collaboration between these diverse groups of stakeholders, in what is still a rather technical field, demands a shared understanding of the physical system. In this paper, we present the Virtual River Game, a serious game to collaboratively explore river management complexity in the Netherlands. The game was played in five sessions involving both domain experts and non-experts to assess its value as a participatory tool. It was guided by the research questions: (1) to what extent does the game facilitate stakeholders collaboratively exploring and experimenting with river interventions; and (2) to what extent does playing the game lead to social learning outcomes?
The paper is structured as follows. The next section discusses the game and particularly the fact that it facilitates stakeholders to apply river interventions on a game board that has a bidirectional link to computer models. Section 3 presents the assessment approach of the game, including an overview of the sessions and the methods to collect and analyze the data in order to address the research questions. Section 4 reports on the factual game output of the sessions, observations of the in-game discussions, and the participants' self-reported learning outcomes. The paper ends with a discussion of the value of the game as a participatory tool as well as on how the game's hybrid interface combines those qualities found in board and computer games that benefit and support stakeholder participation and social learning processes.

Background and aim
To protect the deltaic floodplains from flooding, the traditional approach in the Netherlands has been to build and reinforce dikes. However, near-flood events in the 1990s shifted the approach of Dutch river management towards applying so-called spatial measures that aim to create more space for rivers to safely discharge water (Rijke et al., 2012;Warner et al., 2012); for example, by digging side channels, lowering floodplains, and moving back dikes (see e.g. Berends et al., 2019;Straatsma et al., 2019;Van Stokkom et al., 2005). In addition to lowering peak water levels, such spatial measures also aimed to restore the local ecology (Fliervoet et al., 2013;Klijn et al., 2013;Straatsma et al., 2017). This paradigm shift, while still retaining flood safety as its primary focus, led to an increasingly more integrated river management approach. As a result, it attracted new stakeholders to river management (Verbrugge et al., 2019) and emphasized the importance of stakeholder participation in decision-making (Edelenbos et al., 2017;Fliervoet et al., 2013;Zevenbergen et al., 2015).
In the context of the new Dutch river management paradigm, we set out to develop the Virtual River Game as a tool to increase and support stakeholder participation. The game can beneficially be played at an early stage of a project, as an icebreaker activity but that is disconnected from the project's actual decision-making. The game enables stakeholders to collaboratively experiment with river interventions in order to increase their understanding of Dutch river managementincluding both the physical system and the effects and trade-offs of specific interventionsand the perspectives and interests of other stakeholders in relation to such interventions. During the game's design process, we found that some stakeholdersparticularly those introduced to river management as a result of the paradigm changeview the hydrodynamic models central to Dutch river management decision-making as mysterious black boxes (den Haan et al., 2018). Therefore, to support stakeholder participation, we set out to develop the Virtual River Game to enable stakeholdersregardless of background and expertiseto work with a hydrodynamic model that is widely used in practice. To that end, we developed a hybrid interface based on tangible interaction; linking physical forms to digital information (Hornecker and Buur, 2006;Ishii, 2008). The interface features a bidirectional coupling of a physical game board to a hydrodynamic, ecological, and cost model. An impression of the Virtual River Game and its interface is shown in Fig. 1. The Virtual River Game uses hexagonal tiles to represent a typical stretch of a Dutch river that includes a main channel and floodplains and dikes on both sides of the channel. The changing of the tiles on the game board provides input to the models, while the models' output is visualized both on the game board through projection and in a game engine shown on a touchscreen monitor. An overview of the game's software and hardware components is shown in Fig. 2. In the following sections, we first introduce the models and their integration in the Virtual River Game, followed by a description of the game itself.

Delft3D Flexible Mesh hydrodynamic model
To model water flow and water levels in the game area, we incorporated the Delft3D Flexible Mesh (FM) hydrodynamic model to compute the hydrodynamic response to system change (Berends et al., 2019;Kernkamp et al., 2011). We use a rectangular numerical grid of cell size 20 m by 20 m. Initial bed levels and Chézy friction coefficients are determined at the start of a game and are updated as the game progresses. The boundary conditions are given by an upstream constant discharge and a downstream constant water level. We use default parameter settings (SI , Table 1). Water levels and flow velocities are the model outputs of interest.

BIOSAFE biodiversity model
To model the potential biodiversity of the game area, we integrated the BIOSAFE model as developed by Lenders et al. (2001), De Nooij et al. (2004, and Straatsma et al. (2017). The model calculates biodiversity scores based on the potential occurrence of protected and endangered species in each ecotope, the laws and regulations protecting the species, and the surface area distribution of the main channel and floodplain ecotopes between the dikes. Ecotopes are defined as "spatial landscape units that are homogeneous as to vegetation structure, succession stage, and the main abiotic factors that are relevant to plant growth" (Klijn and de Haes, 1994). As input, the model needs the ecotopes and their surface areas. As output, the model provides potential biodiversity scores for seven taxonomic groups: mammals, birds, herpetofauna, fish, butterflies, dragonflies and damselflies, and higher plants.

VRCost cost model
To model the costs of interventions, we created a model in Python by translating unit prices for costs for interventions in Dutch river management  to interventions in the game. Unit prices relate to costs per volume, area, or length (hexagon cross section). For example, a volume of soil may have to be excavated to construct a side channel in the river's floodplain. The model distinguishes four cost categories: excavations, construction of hydraulic structures, land use changes, and land acquisition. As input, the model needs the elevation and land use change. As output, the model calculates the total costs for changes on the board as well as the costs per type.

Interface design & model integration
For the Virtual River Game's hybrid interface, we designed and built a physical table (Fig. 1) that includes the game board plus an off-theshelf webcam, touchscreen monitor, and projector. The game board consists of 143 hexagonal tiles representing a stretch of river. The tabletop has an open aluminum mesh into which the tiles slot, leaving the bottom side of each tile visible to the webcam. Each tile contains information on terrain height and land use, which can be independently varied. Table 1 shows the five elevation levels and twelve land use types and their potential combinations. We chose the five elevation levels as a representation of the varying elevations found in Dutch rivers. For the land use types, we took inspiration from the classification of vegetation types used by the Dutch Public Works Authority (Rijkswaterstaat, 2012). Markers on the bottom of the tiles enable the conversion of the physical board to a digital board, defining each tile's elevation and land use in the game's software, based on a picture taken by the webcam. Throughout the game, the system can be updated, triggering the software to process the latest board state. In the following subsection, we explain the additional processing steps in the software needed to link the board to the models.

Model interface
The game software converts the digital board to a digital elevation model (DEM) and a roughness distribution as input to Delft3D FM. The DEM is created by an inverse distance interpolation (power of 2) of the terrain height at the center of the three nearest tiles to a regular grid, indexed to the computational grid used in Delft3D FM. The roughness classes are based on a lookup table from land use to roughness class (Table A1). Subsequently, the software calculates the hydraulic roughness by retrieving the water levels of locations from Delft3D FM and applying the vegetation friction model of Klopstra et al. (1996). The software sets the elevation and roughness coefficients and also retrieves water levels and flow velocity from Delft3D FM through the Basic Model Interface (BMI) (Peckham et al., 2013), using the Python BMI (Baart, 2017).
For BIOSAFE, the software converts the digital board to an ecotope distribution through a lookup table that links terrain height and land use to ecotopes (Table A1). A subset of 15 ecotopes out of the 82 fluvial  (Straatsma et al., 2017) is integrated in the game's software to enable sending of the ecotopes and retrieving the potential biodiversity score for the taxonomic groups.
For VRCost, the software compares the previous and new board states to detect changes in elevation and land use. Changes on the board are sent directly to the model and the costs of changes, including their breakdown into the four categories, are retrieved.

Model feedback
The Virtual River Game offers model output in two locations: on the game board itself and on the touchscreen. On the physical game board, the software visualizes the DEM, water flow patterns from Delft3D FM, and hydraulic roughness coefficients by converting these into colormaps, which are subsequently projected on the board. The flow pattern is visualized as diffusive paint blobs that follow the flow lines. The hydrodynamic effects of changing tiles on the board are therefore visualized based on the model's output and projected on the same location as which changes take place. Through these choices, we aimed at making the hydrodynamic model more accessible and transparent by providing a tangible, easy-to-use interface and by enabling players to link their actions to the model's output.
On the touchscreen, the game shows the Tygron Geodesign Platform, a 3D spatial planning modeling tool that is also used as a virtual game engine for those serious games that have a spatial development component (Bekebrede et al., 2015;van Hardeveld et al., 2019;Warmerdam et al., 2006). The game interfaces with the Tygron game engine through its API (Tygron, 2018). The inclusion of the engine serves two purposes. First, the DEM and land use types of the digital board are converted to a virtual game world that matches the game board. For example, the engine shows trees for tiles on the game board that represent the forest land use. Second, interactive panels in the engine provide players with the output from and information about all three models.
We deliberately developed the game in such a way that players are able to fully control the interface without needing the game's facilitator. Players change tiles on the board, switch visualizations through the Graphical User Interface (GUI) displayed on the touchscreen, and inspect the virtual world as well as open panels in the game engine. Updating the board statewhich involves processing changes on the board, running the models, and updating both the visualizations and information in the Tygron engineonly takes between 15 and 30 s, depending on the time needed for water levels in the hydrodynamic model to stabilize. The update times are based on a dedicated, portable computer utilizing an AMD Ryzen X3700 desktop processor to run the game and models locally.

Virtual River Game
In the Virtual River Game, players are challenged to manage a 3500 m long deltaic stretch of river incorporating a navigable main channel that has floodplains and dikes on both sides. The game scenario reflects a high river discharge as result of which the floodplains are inundated. Three teams, each consisting of one or two players, play the roles of flood manager, nature manager, and financial manager. Collectively, the players are given a budget and are tasked to improve the flood safety status and ecological value of the area. Each team is given additional objectives as well as special abilities to block implementations of interventions based on real-world stakeholders, legislation, and European Union development targets. The flood manager mirrors the Dutch Public Works Authority, which is responsible for ensuring and maintaining adequate flood safety levels. The flood manager can block interventions if these decrease flood safety levels or if land use is changed to a type leading to high hydraulic roughness (reed and brushwood, shrubs, forest, and mixtype), which reflects the legislative power of the Public Works Authority. The nature manager represents larger nature management organizations such as the Dutch State Forestry Agency, which own and manage a large percentage of the Dutch floodplains and aim to develop their ecological value. The nature manager can block interventions if those decrease the area's ecological value. To reflect the common view that nature organizations hold less power than for example the Dutch Public Works Authority in real world decisionmaking, this ability can only be used once during the game. The financial manager represents a combination of the Dutch national government, which allocates budget for river projects, and regional governments, which are the commissioners of river projects and responsible for managing the floodplains as Natura 2000 areas under the EU Birds and Habitats Directives. The financial manager can block interventions if those would take more than half of the initial budget or if these require expensive land acquisition (buildings and agricultural land), which reflects the interest of governments to pursue costeffective, win-win solutions. If players are part of any of these or similar organizations, they are assigned a role that is different from their regular role to let them experience river management from another point of view.
Each game consists of a maximum of four rounds. During each round, players apply one of six interventions: side channel construction, floodplain lowering (grading), floodplain smoothing (changing the floodplain vegetation to lower roughness), replacement of groins (alternatively termed 'wing dikes' or 'spur dikes') with longitudinal training dams, dike relocation, or dike reinforcement. Players first discuss and agree which intervention they wish to apply. They subsequently implement the chosen intervention by changing tiles on the game board while following the intervention's rules. During this implementation phase, players can continually change tiles and collectively evaluate the intervention's effects before agreeing on a final implementation.

In-game scoring
In the Virtual River Game, we included performance indicators on flood safety, biodiversity, and budget. Therefore, interventions are evaluated on hydrodynamics, ecology, and costs, as in the Delft3D FM, BIOSAFE, and VRCost models. Each indicator has its own progress bar across a 0-100% score range. Minimum (50%), good (65%), and excellent (80%) scores are provided for both the flood safety and biodiversity indicators. We determined these scores as a balance between the game's representation of reality and its playability (see Harteveld, 2011) that: (1) reflects Dutch river management practice in terms of performance; and (2) corresponds to the difficulty of attaining good scores in the game. Players can click each indicator's progress bar on the touchscreen to see graphics on the effects of interventions on that indicator. In addition, separate information panels are available to players where they can see simplified scores after each tile change to anticipate the effects of interventions.
The flood safety score is based on the water levels along the river axis in comparison to the crest height of the dike at each tile location on the board. The water levels can be at or above, just below, or well below the dike's crest height, corresponding to that location being considered unsafe, moderately safe, and substantially safe, respectively. Unsafe, moderately safe, and substantially safe locations each contribute to the flood safety score as zero, half, and full score, respectively. Consequently, the overall flood safety score is 0% when all dike locations are considered unsafe and 100% when all dike locations are considered substantially safe. Flood safety is visualized in the indicator panel as a longitudinal profile of the stretch of river that includes initial and Table 1 The five elevation levels and twelve land use types used in the Virtual River Game, including the feasible elevation level and land use combinations. current water levels and as a top view with each dike tile colored red, yellow, or green representing unsafe, moderately safe, and substantially safe, respectively. The biodiversity score is based on the potential biodiversity, the sum of the scores on the seven taxonomic groups, over the whole river reach. This sum is converted to a percentage in the 0-100% range for simplicity. We determined the 0% and 100% scores for biodiversity by running a Monte Carlo simulation to find the board layouts that result in the lowest and highest potential biodiversity scores respectively that can be achieved in the game. In the indicator panel, the potential biodiversity of the taxonomic groups, the sum, and the corresponding score are provided. Furthermore, a bar graph shows the scores on the seven taxonomic groups for both the initial and the current game board. A second bar graph shows the change for each taxonomic group expressed as a percentage.
The budget score reflects the remaining budget that players have available as a percentage of the initial budget that they received at the start of a game. The initial budget is therefore equivalent to a 100% score. Spending the whole budget results in a 0% score and spending more than the budget results in a negative score. A graph shows the budget spent per round as well as the remaining budget in the budget indicator panel, expressed both in Euros and a percentage of the initial budget. In a second graph, the breakdown of the costs incurred per round is shown as stacked bars.

In-game objectives
Teams are scored on each indicator separately throughout the game. As a collective objective, the players have to achieve the minimum score (50%) for both flood safety and biodiversity in the four rounds while (preferably) staying within the budget. Collectively, the teams may decide not to play all rounds if they reach the collective objective before the fourth round. Additionally, each team is given one main and two secondary objectives. The teams receive the instruction that, in order to win the game, they have to reach both the collective objective and their role-specific main objective. The role-specific secondary objectives are presented as bonus points that they can earn. The flood manager is given the main objective to achieve a good flood safety score (65%). The secondary objectives are to achieve an excellent flood safety score (80%) and ending the game without any unsafe dike locations. The nature manager has the main objective of achieving a the good biodiversity score (65%) and the secondary objectives of achieving an excellent biodiversity score (80%) and to end the game with five or more forest locations within the floodplains. The budget manager is tasked to limit spending to the collective budget. The secondary objectives are to achieve good scores for flood safety and biodiversity (65%). The collective objective is known to all players. Each team is given its main and secondary objectives at the start of the game by blindly drawing one of four role-specific objective cards. Players do not know that these four cards all list the same main and secondary objectives for their team. Players are not told whether or not to share their team's objectives with other players.

Virtual River Game evaluation
We developed the Virtual River Game to support stakeholder participation in the new Dutch river management paradigm. In this study, we set out to assess the potential of the game as a participatory tool guided by the research questions: (1) to what extent does the game facilitate stakeholders collaboratively exploring and experimenting with river interventions; and (2) to what extent does playing the game lead to social learning outcomes? The research questions address what Aubert et al. (2019) refer to as the process-oriented (how are these outcomes achieved) and variance-oriented (what outcomes are achieved) assessment of serious games.
The first research question focused on the game itself to evaluate its ability to engage stakeholders in dialogue and activitya prerequisite to stimulating social learning. Following the scope of the game and its interface design, we were particularly interested in evaluating the extent to which the game facilitates both domain experts and non-experts to engage in collaborative experimentation with river interventions. We considered the game to be successful as a participatory tool when participantsregardless of their background and expertise -(1) developed a shared understanding of the problem; (2) developed a collaborative strategy to address that problem; (3) engaged in discussions on how interventions affect indicators and role objectives; and (4) applied and tested various implementations of interventions.
The second research question focused on evaluating to what extent playing the game led to social learning by individual participants. Following the game's scope, we focused on learning outcomes related to cognitive and relational learning (Baird et al., 2014;Ensor and Harvey, 2015). Cognitive learning outcomes that were assessed included gaining an improved understanding of: (1) the functioning of the river system; (2) the effects of interventions and their trade-offs; (3) how hydrodynamic models work and are used in decision-making; and (4) the conflicts and opportunities for cooperation between the various stakeholder roles. Relational learning outcomes that were assessed relate to gaining an improved understanding of the mind-sets and perspectives of other participants. We considered the game to be successful when both domain experts and non-experts achieve cognitive or relational learning outcomes or both. As experts bring their knowledge and experience to the game, we expected that they would achieve fewer cognitive learning outcomes than non-experts.
To address the research questions, we organized five sessions playing the Virtual River Game. In the following subsection, we describe the setup of the sessions and their participants. In subsection 3.2, we describe the data collection methods and the data analysis used to address both research questions.

Sessions and participants
The description of a session playing the Virtual River Game is provided in Box 1. We invited both domain experts (professionals working in Dutch river management) and non-experts (participants without river management expertise) to the sessions. None of the sessions were linked to a real-world river project. This made it difficult to engage with nonexperts who are also real-world stakeholders in Dutch river management. Therefore, we invited design researchers and game designers to represent non-expert stakeholders. We organized two sessions that included only expert participants and three sessions of experts and nonexperts (Table 2). Each session had between four and six participants in a total of 26 participants. The length of each session, including the explanations and debriefings, was around 3 h. The actual length of gameplay ranged between 78 and 109 min. All sessions had the same experienced facilitator and a trained observer. We prepared an initial board (Fig. 3) as a scenario that we used in every session. The scenario reflected a stretch of river with a hydrodynamic bottleneck formed by narrow floodplains, higher floodplain terrain, and vegetation that causes high hydraulic roughness, resulting in unsafe water levels upstream from the bottleneck. After the second session, we changed one parameter of the scenario by lowering the initial budget by 30% to € 17.5 million because the initial budget was found to be too high. Although reducing the budget between sessions was undesirable, we aimed to stimulate more discussions that also addressed the cost-effectiveness of interventionsone of the main criteria in real world decision-making. We chose the 30% reduction as this aligned with the budget spent during the first session.

Data collection and analysis
We applied a multi-method approach to collecting and analyzing both quantitative and qualitative data. Our approach consisted of: (1) a pre-game questionnaire; (2) in-game data logging; (3) in-game observations; (4) a post-game questionnaire; and (5) a post-game debriefing. Before starting the game, the participants filled out a short questionnaire stating their age, expertise, experience related to river management, and experience with serious gaming, all so as to separate expert from non-expert participants (SI, Table 2). During the game, the participants' decisions and performance for each update were stored in a log file. In addition, we observed the participants' discussions during the game. We used an observation recording form to document a timeline of each session by linking discussions to the three indicators as well as to intermediate and final decisions taken in the game (SI, Table 3). We used this timeline to analyze each session's outcomes (e.g. interventions applied, final layout, score progression) and processes (e.g. discussions, consideration of options, contributions of participants) in order to address the first research question. Directly after playing the game, the participants completed a second questionnaire that focused on their overall impressions of the game and the insights gained by playing the game (SI, Table 4). In closed questions, the participants were asked to self-report their experiences by rating their agreement with statements on a 5-point Likert scale that ranged from 'strongly agree' to 'strongly disagree': a common approach in game studies (Bekebrede et al., 2018;Keijser et al., 2018;Mayer et al., 2013). In one open question, the participants were asked to list a maximum of three main insights obtained by playing the game. We categorized the open question answers in relation to cognitive and relational learning to determine what experts

Box 1
Description of a game session 1. The facilitator welcomes the participants and provides an overview of the session. 2. The participants fill in the pre-game questionnaire. 3. The facilitator briefs the participants on the game. The facilitator explains the scope of the game, the game indicators, and models behind these, the collaborative objective, the rules, and how the game interface works. 4. The participants engage in a trial round, in which they select one intervention and implement it by making changes on the board to familiarize themselves with the game's interface. 5. After the trial round, the participants take a short break, during which the facilitator resets the game system and rearranges the board to the game scenario. 6. Before starting the game, the facilitator assigns the participants to their roles in the game and lets them draw their objective cards. The facilitator explains the roles to all the participants, including their abilities to block the implementation of interventions. 7. The game starts, for a maximum of four game rounds. During each round, the participants first discuss and decide which intervention they want to apply using any and all information available to them*. If no consensus is reached, participants vote on the intervention to apply, with each role having one vote. 8. As a second part of the round, participants implement the chosen intervention. They discuss and make changes on the board based on the intervention's rules. For example, to relocate a dike, participants move dike tiles further away from the main channel, enlarging the floodplains, in such a way that all dike tiles on that side of the river remain connected (i.e. no gaps are allowed).
Participants are able to continuously make changes on the board and update the game system to inspect the intervention's effects on the indicators*. The participants can use their assigned role's abilities to block the intervention during this step. 9. The participants end the round by finalizing the implementation of the intervention. Preferably, the participants decide through consensus that they are satisfied with the intervention's implementation*. If no consensus is reached, participants vote on finalizing the implementation, with each role again having one vote. No implementation may be finalized under a legitimate use of an ability to block the intervention. Steps 7 to 9 are repeated until the fourth round finishes or sooner if the participants are satisfied with their results before reaching the fourth round. 10. Directly after finishing the game, participants fill in the post-game questionnaire. 11. To conclude the session, the participants collectively reflect on the game activity in the debriefing, moderated by the facilitator.
* During all stages of a game round, the participants themselves are in charge of deciding, testing and implementing interventions. Only if participants seem stuck in any phase of the round does the facilitator pose a question or push for a decision. Otherwise, the facilitator stays mostly passive apart from making sure that the game rules are followed. The facilitator is available at all times to answer participants' questions, but does not advise or tell players what to do or not do. and non-experts identify as the main insights from playing the game. We used the data from the completed questionnaires to address the second research question. One non-expert participant did not complete the post-game questionnaire and was therefore excluded from the results (N total = 25 for the questionnaire). To conclude each session, we conducted a debriefing to collectively reflect on the game activity. The debriefing was set up along the lines proposed by Kriz (2010): six phases to structurally reflect on the game activity. To confirm that the discussions and considerations in the game reflect those encountered in practice, we expanded the third phase of the debriefingwhich includes reflecting on the game's external validity ( Van den Hoogen et al., 2014) in the two expert sessions. Following informed consent, we recorded the debriefings to transcribe these for later analysis. We used the debriefing data to further interpret the analyses to help address both research questions. In addition, we compared the data to address the two research questions with each other to strive for data triangulation.

Collaborative experimentation with river interventions (RQ1)
The first research question was to what extent the game facilitates stakeholders to collaboratively exploring and experimenting with river interventions. Here, we present the chosen interventions and indicator scores as the factual game output, as well as the observations of the ingame discussions about and experimentations with those interventions.

Interventions and scores
The choice and implementations of interventions during the game sessions showed that good scores (between 65% and 79%) for both flood safety and biodiversity can be achieved in several ways (Fig. 4). For flood safety, only the participants in the fourth session (71%) did not achieve an excellent score (80% or higher). Excellent scores for biodiversity were not achieved in any session. In all sessions, the participants first focused on increasing the flood safety scores, generally spending most of the budget during the first two rounds.
Increasing the biodiversity scores became the focus during the later rounds. Participants in the third and fifth sessions used their remaining budget to achieve good biodiversity scores (between 65% and 79%), but only after achieving satisfactory flood safety scores. During all sessions, participants applied the floodplain smoothing intervention to increase their biodiversity scores while at times also optimizing their flood safety scores. During the second and third sessions, participants also chose the side channel construction and replaced groins with longitudinal training dams to increase their biodiversity scores.

Strategies and discussions
At the start of each session, participants explored the game scenario by changing the board visualizations, inspecting the information on the touchscreen, and linking the information from these two sources. Initial discussions focused on establishing a shared understanding of the flood safety bottleneck and on how flood safety could be improved. During the mixed sessions, expert participants were observed taking prominent roles in these early discussions. These experts supplemented the in-game information by explaining what they saw as problems, for example that the bottleneck caused higher water levels upstream, and suggested what could be done to improve the indicator scores. Non-experts initially followed their lead but then started to make their own suggestions as the games progressed. During the sessions that included only professionals, experts who regularly work with hydrodynamic models were similarly observed to take more prominent roles at the start of the game. The discussions during these sessions moved more quickly from identifying the bottleneck to possible interventions than in the mixed sessions. During these discussions, flood safety specialists were observed to start the games as if they were in their real-world roles, even though they had not been assigned to the flood safety manager team. Game discussions during the early rounds did include how tackling the flood safety bottleneck could simultaneously improve biodiversity without the costs of interventions rising unacceptably. However, no change in general direction of these discussions was observed as a result of lowering the budget after the second session.
After establishing a shared understanding of the problem, the discussions during all sessions focused on deciding which intervention to implement in the first round. In no session did the participants develop a collaborative strategy at the start of the game. During the sessions, the participants were observed to plan ahead by discussing how the intervention they were implementing could be made more effective by applying a different intervention in the next round. Experts continued to supplement the in-game information, for example by explaining that changing land use to a type with high hydraulic roughness to increase biodiversity (e.g. to a forest) is best done at locations where there is a low flow velocity (e.g. next to a dike) to limit its effect on flood safety. Discussions during the later stages of the game featured more bargaining between the teams, each seeking to pursue their main and secondary The reported length of the game in this session includes a correction for internet issues that resulted in a loss of 20 min to play the game. Fig. 3. Initial board layout that was used as a starting situation in all sessions.
objectives. For example, discussions during the third and fifth sessions included teams agreeing on the choice of intervention in one round only if the other teams agreed beforehand on the intervention to be made in the subsequent round. During these bargaining discussions, participants started to share their team objectives. Only in the fifth session did the participants share their team objectives at the start of the game to establish their own common objectives, guided mostly by one expert participant who advocated forgetting about the politics. No notable differences in late-game discussions were observed between the expert and mixed sessions. Non-experts were as active in proposing implementations of interventions as experts were and their suggestions were also applied on the board. In the three sessions after reducing the budget, the costs of interventions were emphasized more during the lategame discussions, in particular by the budget manager teams.
During the debriefings, participants confirmed that they did not have a collective strategy for the game, but that it felt natural to them to plan ahead while testing implementations of interventions. Additionally, participants in the second, third, and fifth sessions explicitly indicated that it seemed logical to them to start with the most drastic intervention and to subsequently optimize from there. Before asking about the connection of the game activity to reality during the third session (expert session), the following discussion took place: Expert A: "… and that [the game] more or less reflects how things work in reality. I am working together with Expert D on an area, in which we have the same type of discussions." Expert B: "I really liked that you could quickly update and inspect the results. Like, how does this work out? How far are we now? That is very useful. The risk is that you start micro-managing the environment, but yeah …" Expert C: "But that is also realistic." Expert B: "Yes, true." Expert D: "[The game] feels very realistic, because the discussions we were having, about the board where we were working with, those are the same discussions you have in practice and the advantage is that you can quickly update the situation and inspect the effects, so you really get a feel of what the interventions do." The discussion illustrates that these experts found that using the board to apply interventions felt sufficiently realistic and elicited discussions similar to those occurring in practice. Experts in other sessions provided feedback that supported these findings during the debriefings Fig. 4. Overview of the game sessions, with each row showing the result of one session. The first column shows the interventions applied during the four game rounds and the resulting progression of indicator scores, with the final indicator scores listed to the right of the graphs. The number shown below each intervention icon indicates the number of implementations that participants tested during the respective round. The budget indicator scores for the first two sessions are adjusted to match the lowered budget. The second column shows the final board layout for the bed elevation and the third column shows the final layout for the land use. and in written statements made in the post-game questionnaire. Experts in the first, third, and fifth sessions did mention that they missed a role that directly represents the interest of agriculture in the game. As a result, although facing a financial penalty for the compulsory purchase of land, they mentioned that they could convert agricultural land into other land use types without the emotional discussions associated with the compulsory purchase of land by the government that would occur in real life.

Experimentation approach
In all sessions, participants used the opportunity to experiment with interventions. In sessions 1 to 4, the participants applied and evaluated between 17 and 20 implementations of interventions while playing the game (Fig. 4). The number of applied interventions in the fifth session was notably lower, which was probably caused by internet problems during the session that resulted in a loss of time to play the game. To help propose interventions, participants used the board as a map to suggest interventions by hand gestures and used the information both from the visualizations as well as on the touchscreen to formulate arguments. Participants experimented with interventions both to test implementations at several locations, e.g. constructing a side channel on the other side of the main channel (comparison), and to improve the implementation at a chosen location (optimization). Especially the floodplain smoothing intervention led to experimentation, with participants in the second and third sessions trying eight and eleven implementations, respectively. In these experimentations, non-experts were observed to realize that lowering the hydraulic roughness from locations with higher water flow velocity is effective in increasing the flood safety score. Furthermore, participants were observed to realize, through experimentation, that changing a production meadow (intensively managed grassland) to a more natural setting (less managed) is most effective at increasing biodiversity.

Experience and social learning outcomes (RQ2)
The second research question was: to what extent does playing the game lead to social learning outcomes? Here, we describe the participants' overall impression on the game and their self-reported learning outcomes with respect to both cognitive and relational learning.
The questionnaire results indicate that the game was well received by both domain experts and non-experts (Fig. 5). Most participants indicated that the game's goal was clear and that they enjoyed playing the game. During the debriefings, participants also frequently mentioned that they regarded the game as a fun activity.
Regarding the learning outcomes, the questionnaire results indicate that participants gained insights into both cognitive and relational learning (Fig. 6). Compared to experts, an equal or higher percentage of the non-expert participants gave answers to the statements that showed strong agreement. Below, we discuss the results for experts and nonexperts separately. Fig. 6 shows that non-experts mostly agreed or strongly agreed with the twelve statements. Non-experts agreed most with statements 11 and 12 (μ = 4.40 and 4.60 respectively) related to relational learning. In relation to cognitive learning, non-experts agreed most with statements 1 and 10 (both μ = 4.20). Only on statement 4 (μ = 3.40) about the costs of interventions did two non-experts give answers that showed strong disagreement. These participants explained that they had difficulty interpreting information about costs during the game. In the open question, non-experts most frequently mentioned insights related to the functioning of the river system (10) and interventions and trade-offs (6) categories (SI , Table 5), both associated with cognitive learning. Most of these insights were broadly formulated, such as "the relation between flow velocity and flood safety" and "trade-offs between decisions".
Experts gave more mixed responses to the statements than nonexperts. Experts agreed most strongly with statement 5 (μ = 3.87) related to the trade-offs between interventions. They also rated statements 11 and 12 (μ = 3.67 and 3.80 respectively) on relational learning positively, with most experts only agreeing but a few strongly agreeing. Statements 6 and 7 (μ = 2.67 and 2.93 respectively) on hydrodynamic models were rated poorly, with more experts disagreeing than agreeing with both statements. This was not an unexpected result as experts regularly work with these models anyway. In the open question, experts most frequently mentioned insights related to interventions and tradeoffs (8) and player perspectives (7) (SI , Table 5), associated with cognitive learning and relational learning, respectively. On the interventions and trade-offs, insights were formulated more specifically than by non-experts, such as "insight into the ratio of costs of different interventions" and "more insights into the costs of interventions in relation to their effectiveness". Experts indicated in both the questionnaires and the debriefings that they gained such insights as a result of playing a role different from their day-to-day role.
During the debriefings, participants collectively reflected on what helped them to obtain their main insights and mentioned the game's feedback (all sessions), the shared exploration and experimentation with interventions (sessions 2, 3, 4 and 5), explanations from other participants (sessions 1, 2, 4 and 5), and finding it easier to interpret information from the game board than from a screen (session 5). In all three mixed sessions, non-experts explicitly pointed out that the experts had helped them to understand the problem as well as to predict the consequences of interventions. Experts in turn indicated that collaborative experimentation with interventionsin combination with the game's feedbackhelped them to explain river management principles to non-

Discussion
In this section, we first reflect on the potential value of the Virtual River Game as a participatory tool based on the study's results. Next, we discuss the design of the game and its interface. We end the section by reflecting on the study and outlining its limitations.

Game as a participatory tool
The results of the sessions indicate that the game was successful at engaging both domain experts and non-experts in collaboratively exploring and experimenting with river interventions. In all sessions, the participants developed a shared understanding of the problem. To address the problem, they applied and tested various implementations of interventions. Although the participants did not develop a collective strategy from the start, they did collaboratively plan ahead while applying interventions, discussing how future interventions could further improve the indicator scores and contribute to achieving the various objectives. Experts and non-experts were both active in the games' discussions, with experts taking more of a leading role, especially at the start of the games. Furthermore, experts noted that the discussions and considerations that they had during the game reflect well those found in real-world projects.
The results further indicate that the Virtual River Game stimulated social learning for both experts (N = 15) and non-experts (N = 10). Considering the game's scope and the operationalization of social learning, we looked specifically at cognitive learning (acquiring new or restructuring existing knowledge) and relational learning (increasing the understanding of the mind-set and perspectives of others) (Baird et al., 2014;Ensor and Harvey, 2015). With only a few exceptions, non-experts rated all statements in relation to the game's learning outcomes positively. According to their comments, the main insights that non-experts emphasized were almost exclusively associated with cognitive learning. However, experts rated the statements in a more mixed way, with some statements rated positively but some statements associated with cognitive learning neutrally to negatively. We expected that experts would report less cognitive learning given their background knowledge. The statements on relational learning were rated positively by experts, which was also reflected in their reported main insights. Taken together, we conclude that both domain experts and non-experts learned by playing the game. However, the extent to which they learned and emphasis they placed on it varied. The game appears to be particularly valuable as a participatory tool to introduce non-expert stakeholders to river management in the Netherlands. Keijser et al. (2018) reported similar results from their game on marine spatial planning, showing that their game worked well as an introductory game to engage non-experts on the topic.
On learning more generally, our results are in line with evaluations of other serious games (see e.g. reviews by Aubert et al., 2018;den Haan and van der Voort, 2018;Flood et al., 2018), which have shown that serious games are an effective way of increasing the understanding of physical systems (Becu et al., 2017;Carson et al., 2018;Keijser et al., 2018), of raising awareness (Onencan and Van de Walle, 2018;Stefanska et al., 2011;Van Pelt et al., 2015), and of increasing the understanding of alternative views and perspectives (Douven et al., 2014;Jean et al., 2018;Souchère et al., 2010). Combining the results on collaborative exploration and learning, this study offers further evidence that serious games are effective ways of improving social learning (Hofstede et al., 2010;Medema et al., 2016;Savic et al., 2016).

Game and interface design
Serious games are increasingly explored as learning-based interventions in environmental management (Aubert et al., 2019;Flood et al., 2018;Rodela et al., 2019). In terms of the design of such games, this work contributes to the growing body of literature by proposing a new, hybrid interface design concept: the bidirectional coupling of a physical game board to computer models. Whereas other serious games Fig. 6. Learning outcomes of the game as reported in the post-game questionnaire based on a 5-point Likert scale. Rating strongly agree counts as 5; strongly disagree as 1. The statements relate to the functioning of the river system (1), the effects of interventions and their trade-offs (2-5), how hydrodynamic models work and are used in decision-making (6-7), the roles of and conflicts between stakeholder roles (8-10), and the views and perspectives of other players (11-12). Graphical representation based on Heiberger and Robbins (2014). on environmental management tend to use simplified, custom computer models that are location-specific (e.g. Craven et al., 2017;Valkering et al., 2013), our focus was on creating a simplifiedyet realistic -3D representation of a typical stretch of a Dutch river in combination with models currently used in practice. Designing serious games always requires some form of simplification to strike a good balance between an accurate representation of reality, the playability of reality, and the meaning of the game (Harteveld, 2011). In our approach, the biggest simplification of the physical reality was not in the models, but in the fact that the board uses a fixed, hexagonal grid. We applied this simplification mainly to increase playability; the fixed number and shape of the tiles reduces the level of detail in relation to a real stretch of river and provides a manageable structure by limiting the number of possible arrangements in the game. The sessions showed that the game board design facilitated participants working together and, as an interface, enabled especially non-experts to work with models currently used in practice. Moreover, experts determined that the game triggered discussions that also occur in practice. Therefore, simplifying a river stretch to a hexagonal grid proved to be a suitable approach to balancing a good representation of reality with playability. Increasing the level of detail in the game could be further explored, but is not necessary to facilitate the collaborative exploration of and experimentation with river interventions.
In addition to serving its specific purpose, the hybrid interface offers qualities that we see as beneficial to facilitating stakeholder participation in environmental management and to improving social learning. Serious games have been developed as board games (Hertzog et al., 2014;Keijser et al., 2018;Speelman et al., 2014), as digital games (Ayadi et al., 2014;Carson et al., 2018;Craven et al., 2017;Hill et al., 2014), and as hybrid games combining elements of board and digital games (Cleland et al., 2012;Magnuszewski et al., 2018;Valkering et al., 2013;Van der Wal et al., 2016). Such games have a common feature that participants are provided with an experimentation environment in which they can collaboratively and safely explore both the techno-physical and socio-political complexity of an environmental system. With the hybrid interface design of the Virtual River Game, we combined three qualities that are separately found in board and computer games. These qualities are described below.
First, the hybrid interface provides participants with a physical, tangible object to engage with in shared exploration. Through the game board and touchscreen, control of the game is shared and not limited to a facilitator or one participant controlling an input device. Participants engage in discussions by moving game tiles or pointing to locations on the game board, making their views explicit to other participants (Stanton et al., 2001;Suzuki and Kato, 1995).
Second, as an interface, the game board appeared to lower the threshold for participating while allowing participants to collaboratively work with computer models by using tangible objects, removing the need to have specific expertise in using these models. Computer games have been developed as interaction layers to models based on a GUI (Chew et al., 2013;Craven et al., 2017;van Hardeveld et al., 2019), but without the use of tangible objects as a direct source of input. In turn, games have been developed that use a non-digital board in combination with computer models (Magnuszewski et al., 2018;Stefanska et al., 2011), but without either a direct connection to the models or the models' output being visualized on the board. In the Virtual River Game, the bidirectional link between the board and models provides participants with visualizations of their actions at the same location as where they made them.
Third, for a serious game in general, being able to experiment with interventions is a valuable way of triggering learning (Becu et al., 2017;Ferrero et al., 2018). Multiplayer serious games that focus on managing a spatial area generally include interventions as binary options that can be turned on or off for defined locations (Carson et al., 2018;Craven et al., 2017;Rusca et al., 2012). This can include a few options of the same interventions at the same locations, such as small, medium, and large versions (Onencan et al., 2016;Savic et al., 2016;Valkering et al., 2013). In the Virtual River Game, participants determine the location, direction, shape, and size of interventions by replacing tiles on the board while following rules defined outside the software. Therefore, the interface increases the experimentation potential by including the design of interventions rather than choosing predefined intervention options. In serious games, similar functionality is found in tile-based computer games (e.g. Becu et al., 2017;Chew et al., 2013), but not in combination with tangible tiles. This type of functionality is also found in planning support systems in which participants can draw and apply interventions on a tabletop surface (see e.g. Leskens et al., 2014;Vonk and Ligtenberg, 2010). To summarize, the hybrid interface combines the strengths of board games (accessibility, tangibility) with the power and flexibility of computer games (modeling, visualizations) which, in combination, are beneficial to supporting stakeholder participation and improving social learning.

Reflection and limitations
In all sessions, the participants focused mainly on increasing the flood safety score in the early game rounds and only focused on the biodiversity score during later rounds. This approach is consistent with current Dutch river management practice in which improving flood safety is the primary objective (see e.g. Deltaprogramma, 2019). However, some choices in the Virtual River Game's design and the choice of participants invited to the sessions may have influenced the participants' focus on improving their flood safety scores. Although no distinction between primary and secondary objectives is made in the game, the in-game interventions are based on interventions implemented in reality that do have the improvement of flood safety as the primary objective. The sessions' starting conditions may have further nudged participants to first focus on the flood safety indicator as its initial score (29%) was lower than the biodiversity score (47%). In addition, some flood safety specialists were observed to approach the game as if they were initially in their real-world roles.
Besides these possible influences on the results, the study has two other limitations. In general, a serious game aims to provide participants with a safe space to experiment, to take on another role, and to defend positions that they may not take in reality (De Caluwé et al., 2012;Geurts et al., 2007;Mayer, 2009). The results of this study suggest that the game provides this sense of safety as participants experimented with interventions to pursue the objectives of their assigned role. In particular, domain experts were observed to propose and experiment with interventions that ran counter to objectives that they would normally pursue in their real-world roles. However, the sessions were organized outside of a real-world policy-making setting. Therefore, the first limitation of the study is that we do not know if the game would also establish the same sense of safety in experimenting with interventions and defending positions of other stakeholder roles when played in the context of a real-world river project. This needs to be explored in future research. Before the game is used in a real river project, the roles in the game must be reviewed. In particular, based on the results of this study, adding an additional role that actively defends the interests of the agricultural sector must be considered. This role was initially left out to limit the game's complexity. In order not to increase the game's complexity, an alternative could be to consider the current game as an introductory level and add a second game level with additional objectives, indicators, roles, and interventions.
The second limitation lies in the limited number of sessions and participants. Conducting studies on serious games is time-consuming, both for the participants and for the researchers. As a result, the number of expert and non-expert participants was too low to be able to draw statistically significant conclusions. Consequently, we looked at these two groups separately and have not made comparisons between the learning outcomes of experts and non-experts. However, despite the limitations, the results indicate that the game may well be useful as a tool to help support stakeholder participation in Dutch river management.

Conclusion
In this paper, we presented the design of the Virtual River Game and investigated its potential value as a participatory tool in Dutch river management. Based on the results of five sessions, we showed that the game was successful in: (1) facilitating domain experts and non-experts to collaboratively experiment with river interventions that are commonly applied in the Netherlands and to elicit discussions similar to those that occur in real-world river projects; and (2) establishing social learning outcomes as both experts and non-experts indicated that they had learned by playing the game. Through self-reporting, non-experts indicated that most gained insights into the game's learning objectives and emphasized their learning about the functioning of the river system and about river interventions and their inevitable trade-offs. Experts, in turn, reported more mixed results regarding the learning objectives and emphasized learning about river interventions and their inevitable trade-offs, as well as about the views and perspectives of other participants. As a participatory tool, we conclude that the substantial value of the game lies in introducing non-expert stakeholders to Dutch river management.

Funding
This research is part of the research program RiverCare: towards selfsustaining multifunctional rivers, supported by the Netherlands Organization for Scientific Research (NWO) and partly funded by the Ministry of Economic Affairs under grant number P12-14 (Perspective Program).

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.