Effective policy instrument mixes for implementing integrated flood risk management

Central to integrated flood risk management is the integration of water management with spatial planning. Existing studies often focus on analyzing the policy instruments in the initiation and planning phases of integrated flood risk management. Little is known, however, about the policy instrument mixes that enable implementation of integrated flood risk management. Therefore, in this article we analyze the Dutch Room for the River program to identify what mixes of policy instruments enable successful integrated flood risk management in the implementation phase. We collected archival and survey data and analyzed 19 implemented projects in the Room for the River program applying Qualitative Comparative Analysis (QCA). We conclude that no single policy instrument is necessary or sufficient for successful implementation. We found three policy instrument mixes: an integrated contract mix, a project management mix, and an outside-government mix.


Introduction
Nowadays, it is widely agreed that traditional approaches to flood risk management-which focus on 'defending against the water'-are no longer sufficient and that integrated flood risk management is required instead (Hartmann & Driessen, 2017;Merz et al., 2010;Pahl-Wostl, 2007;Warner et al., 2012;Woltjer & Al, 2007).In integrated flood risk management, traditional flood defense measures (e.g., building or strengthening dikes) are combined with measures that stress the 'accommodation of the water' (e.g., dike relocation); measures that take other land use functions (e.g., housing, recreation, and nature conservation), and the extent to which these measures increase local and regional spatial qualities, into account (Johnson & Priest, 2008;Nillesen & Kok, 2015;Van Herk et al., 2015).Whereas existing studies often focus on the policy instruments involved in the initiation and planning phases of integrated flood risk management (e.g., Busscher et al., 2019;Ran & Nedovic-Budic, 2017;Thieken et al., 2016), research into policy instruments in the implementation phase remains limited (see e.g., Moss, 2008).Implementation can be seen as the process that starts with the contracting of a project and ends when construction is finished.
In the Netherlands, the €2.3 billion Room for the River program is often considered the incarnation of the Dutch approach to integrated flood risk management (Jong & Van den Brink, 2017;Meyer, 2009;Van Buuren et al., 2010).The objective of the Room for the River program is twofold: the accommodation of higher flood levels, i.e., water safety, and improving the spatial quality of the riverine areas (Ruimte voor de Rivier, 2007).While much has already been learned from current research about the program (Rijke et al., 2012;Zevenbergen et al., 2015), research providing insight into the implementation of the Room for the River projects, and in particular the policy instruments used, is still scarce.Now that the Room for the River program has come to an end (Olde Wolbers et al., 2018), this provides the opportunity to analyze what policy instruments enable successful integrated flood risk management in the implementation phase.
It is increasingly stressed that successful policy implementation depends on the simultaneous and strategic deployment of a combination of various instruments-so-called policy instrument mixes.At the same time, analyses of the interplay between policy instruments are scarce and more empirical studies are called for (Cunningham et al., 2013;Rogge & Reichardt, 2016).The research question of this article therefore is: which mixes of policy instruments were applied in implementing successful integrated flood risk management projects?To answer this question, we collected archival and survey data and analyzed it using the configurational method of Qualitative Comparative Analysis (QCA) (Rihoux & Ragin, 2009;Schneider & Wagemann, 2012).In QCA, it is assumed that a combination of factors-coined 'conditions' in QCA--explain an outcome of interest.There may exist multiple configurations explaining a particular outcome.Given the complexity of spatial planning in general and integrated flood risk management in particular, QCA is a useful method to analyze the effective configurations of policy instruments (Verweij & Trell, 2019).
This article is further structured as follows.In Section 2, the background of the Room for the River program is provided.In Section 3, policy instruments are discussed both in theory and in correspondence to the program.In Section 4, the data and method are explained.In Section 5, the results and the analysis are presented.In Section 6, we draw our conclusions.

The room for the river program
In the Room for the River program, Rijkswaterstaat, provinces, municipalities, and regional water authorities (i.e., water boards) cooperated in the implementation of 34 projects along the river Rhine and its branches the Waal, IJssel, and Nederrijn.The program had a dual objective; each of the projects was expected to increase the capacity of the rivers to cope with high water levels and simultaneously improve the spatial quality of the riverine area.The program was coordinated by the Room for the River Program Office, which was part of Rijkswaterstaat.Rijkswaterstaat is the executive agency of the Ministry of Infrastructure and Water Management.The Program Office was responsible for ensuring that both objectives would be met.Within the Program Office, the 'Cluster Spatial Quality' was responsible for the coordination of the program's second objective, focusing on directing, facilitating, and monitoring the different projects in achieving spatial quality.The program followed a decentralized approach: the individual projects were implemented by various appointed governmental bodies.The projects were advised by the Cluster Spatial Quality during their implementation (Collignon-Havinga et al., 2009;Feddes & Hinz, 2013).The Cluster was supported and informed by an independent 'Quality Team' (i.e., Q-team).The Q-team was chaired by the National Advisor for the Landscape and consisted of five other independent experts from different disciplinary backgrounds, ranging from landscape architecture to river engineering (Klijn et al., 2013).The Q-team was specifically tasked to evaluate spatial quality and to report about the achieved spatial quality to the Minister of Transport, Public Works, and Water Management.This means that the Q-team evaluated all projects that were part of the program.Their evaluations were seen as highly authoritative, which is demonstrated by the fact that the Cluster Spatial Quality typically indiscriminately ratified their evaluations.
In the Room for the River program, spatial quality was defined in reference to the 'Vitruvius Triplet' that says that structures should exhibit three qualities: they should be useful (utilitas), beautiful (venustas), and robust (firmitas) (Hooimeijer et al., 2001).This triplet was translated by the Q-team as the relationship between hydraulic effectiveness, cultural meaning and aesthetics, and ecological robustness (Klijn et al., 2013).Projects were evaluated on the achieved spatial quality by comparing the (relationship between) the three spatial quality aspects in each project to the baseline situation prior to the initiation of the program (Klijn et al., 2013).This means that, also if spatial quality was already high prior to the project, the project had the task to further improve it.
The 34 projects that were part of Room for the River were coined 'measures'.The measures varied in type and scale (see Fig. 1 for an overview of potential measures).In general, three types of measures were distinguished (see Busscher et al., 2019;Ruimte voor de Rivier, 2007, 2016a): (1) technical measures, such as strengthening dikes, that S. Verweij et al. represent the traditional flood risk management measures, (2) measures within the banks (i.e., the 'wet area' inside the banks not protected against floods) such as floodplain excavations, and (3) measures beyond the banks (i.e., the area behind the dike where housing can be found) such as dike relocations and depoldering.The most renowned and large-scale projects of the program, such as the Depoldering of the Noordwaard and the Overdiepse Polder, the Dike Relocation Lent, and the Bypass near Kampen, are all examples of this third type.The program, however, also included multiple relatively small-scale measures (see e.g., Van den Brink et al., 2019).
All projects in the program achieved the primary objective of improving the protection of the river basin against floods (Olde Wolbers et al., 2018).The spatial quality objective was therefore the objective at stake.That is, studies showed that particularly this objective was vulnerable to be abandoned or attenuated when push came to shove (Van Twist et al., 2011).

Policy instrument mixes in the room for the river program
In the Room for the River projects, a combination of instruments has been applied to improve spatial quality (Busscher et al., 2017).Instruments refer to the means or techniques by which governments seek to achieve their policy goals (Knill & Tosun, 2012), such as the improvement of spatial quality.As explained by Hood (2007), governments can employ four different types of resources to "get people to do things that they might not otherwise do; or [to] enable people to do things that they might not have done otherwise" (Schneider & Ingram, 1990, p. 513).In the implementation of the Room for the River projects, 'people' refers mostly to private contractors responsible for the execution of the project.The resources governments can use, in order to ensure that these private contractors are implementing the project in correspondence with the spatial quality objective, are nodality, authority, treasure, and organization (Hood, 1983;Hood & Margetts, 2007).These can be seen as overarching principles to structure and organize the wide variety of potential instruments.

Nodality
Nodality refers to the central position of governments with regard to the collection, use, and distribution of information.Governments are often at the center of the information network that underpins the development and implementation of policy.This central position not only implies nodality, but also means that governments are in a position in which they can strategically distribute information and detect new information, to change the beliefs and perceptions of public and private actors (Knill & Tosun, 2012).
In the context of Room for the River, a typical instrument representing nodality is the Cluster Spatial Quality (CLUS).The Cluster Spatial Quality was responsible for the coordination of the program's spatial quality objective and facilitates the projects herein.To this purpose, it used various resources, ranging from collecting information about the project progress to a helpdesk where projects can go to for questions and assistance (Collignon-Havinga et al., 2009;Hulsker et al., 2011).The midterm evaluation indicated that the Cluster's role as a facilitator, "without sitting on the designer's seat", was influential in achieving spatial quality (Hulsker et al., 2011).The Cluster thus used and distributed various forms of information.In Section 5, we explore whether the information provision and closer involvement of the Cluster in a project as facilitator of spatial quality was necessary or not for achieving spatial quality, and if so, which other instruments were important as well.

Authority
Authority refers to the legal or official powers held by governments that can be used to regulate and sanction actors.This often includes prohibitions, bans, permits, and standards (Hood & Margetts, 2007).In this sense, instruments that use authority as a government resource prescribe what is expected of other actors.
In the context of the Room for the River program, a typical instrument building on authority is the inclusion of spatial quality requirements in the project Tender Documents (TEN).These documents played an important role in anchoring spatial quality (Feddes & Hinz, 2013;Van Herk et al., 2015) as they specified project requirements that needed to be met (Mees et al., 2014).In the projects, "spatial quality was a selection criterion in tender procedures and was detailed in accompanying ambition documents" (Van Herk et al., 2015, p. 93).However, the projects differed in the extent to which this criterion was explicated in the tender documents (Feddes & Hinz, 2013).In the project Ruimte voor de Waal, for instance, the spatial quality to be obtained was specifically defined (see Table 1), which made it an explicit requirement for the project design by the private contractor (Brouwer et al., 2017).In other projects, spatial quality was merely broadly defined or not mentioned as a hard criterion at all in the tender documents, leaving the realization of spatial quality open to the interpretation of the private contractor.With the analysis in Section 5, we explore whether specifically defining spatial quality in tender documents was necessary or not.

Treasure
Treasure revolves around money.It relates to the use of economic instruments to direct actors.An important aspect of instruments drawing on treasure is that they are voluntary in nature.The actors that are addressed by the instruments are not legally obliged to adopt the measures involved (as would be the case under authority).Instead, they are incentivized to perform certain actions (Knill & Tosun, 2012).
In the Room for the River program, a typical instrument is the use of different Contract Types (CONT).In the Netherlands, contracts are especially seen as a way in which the private sector's capacities can be harnessed to achieve effective policy implementation.In the program, four contract types have been used (Feddes & Hinz, 2013).The contracts vary in terms of their inclusiveness, i.e., the extent to which the private implementation actor is responsible for multiple project phases.In more inclusive contracts, more responsibilities are allocated to the private implementation actor, for which it is financially compensated.The inclusion of more phases is expected to lead to efficiency and quality gains.
▪ Traditional design-bid-build contracts (RAW), where the project client specifies the technical design including the "underlying calculation of materials needed and construction time" (Lenferink et al., 2013, p. 617) to be implemented by the private contractor (Feddes & Hinz, 2013).▪ Engineering and Construct (E&C) contracts, where the contractor is responsible for working out the technical and logistic details of the design (Feddes & Hinz, 2013;Lenferink et al., 2013).▪ Design and Construct (D&C) contracts, where the contractor is responsible for the entire design and not just the working out of the details of the design (Feddes & Hinz, 2013;Lenferink et al., 2013).▪ Plan, Design, and Construct (PD&C) contracts, where the private contractor, in addition to the design, is also responsible for the spatial planning process (PIANOo & Unie van Waterschappen, 2016; Ruimte voor de Rivier, 2015).
The PD&C is the most inclusive contract type because it integrates the different project phases into a single agreement.The idea is that this incentivizes the private contractor to (partially) align the phases of planning, design, and construction, expectedly allowing for faster, more efficient, and, potentially, higher spatial quality (Lenferink et al., 2013; PIANOo & Unie van Waterschappen, 2016; Ruimte voor de Rivier, 2015).The D&C contract is a bit less inclusive, but still builds on this idea.In the case of E&C and the design-bid-build contracts, the influence from private parties is limited.These contracts are established to assign a pre-defined task and explicate the legal relationship between principal (the project client) and agent (private contractor).In these contracts, spatial quality is specified by the principal and the agent has to construct it in line with these specifications.Section 5 will show which contract types were effective in combination with which other instruments.

Organization
Organization describes the resources in terms of staff, building, and technology that governments have at their disposal.Candel (2020) considers organization to relate to the employment of people with a particular skillset.
A typical instrument in the context of the Room for the River program is the use of a Landscape Architect (LAND).The landscape architect played an important role in the Room for the River program (Klijn et al., 2013;Van den Brink et al., 2019).In the traditional architectural approach, "spatial design is often understood as a product, with a strong focus on the content of plans and designs" (Heeres et al., 2016, p. 412).In that capacity, landscape architects are involved in creating a specific spatial quality plan only as part of the project scope.In these instances, a landscape architect is only asked to, rather autonomously, deliver a specific product.In integrated approaches, designs are "a way to manage a wider creative process of arriving at decisions and action" (Heeres et al., 2016, p. 412).In that capacity, the landscape architects, as members of an integral design team (cf.Van Buuren et al., 2012), were not only asked to deliver a product but played an important role in the implementation process as well.In both capacities, although in different ways, the landscape architect was a boundary spanner between the domains of water management and spatial planning ( Van den Brink et al., 2019).With the analysis in this article, we explore whether and how the involvement of a landscape architect is effective.
Governments usually mobilize mixes of policy instruments throughout the different phases of the planning and implementation process in order to attain formulated goals and objectives (see e.g., Bemelmans-Videc et al., 2011;Howlett & Del Rio, 2015).In the Room for the River program, the use of different mixes of instruments is reinforced by the fact that the program was decentralized in nature: national and local governmental bodies each implemented individual projects.In each of these projects, the involved bodies applied a project-specific combination of instruments to realize the overarching spatial quality objective.We are interested in which combination of instruments (i.e., policy instrument mixes) proved sufficient to improve spatial quality.

Data and method
We analyzed the Room for the River program with QCA (see Busscher et al., 2017).We adopted the explorative, substantive interpretability approach to QCA (see Thomann & Maggetti, 2020).The analysis consisted of five steps (Rihoux & Lobe, 2009;Gerrits & Verweij, 2018).The first step is case selection.The program consisted of 34 projects.At the time of the study, 26 projects had been completed and 8 projects were still being realized (Ruimte voor de Rivier, 2016a).Out of these, we selected 19 projects applying the following selection criteria: the projects were completed; the Q-team had evaluated the achieved spatial quality and this evaluation had been ratified by the Cluster Spatial Quality; and Rijkswaterstaat had data available for these projects.Table 1 provides the project details.We note that the case identifiers refer to an original dataset and that therefore the numbering goes to 23 (instead of 19).In the second step, which partly coincided with the first step, the data were collected.The first data source was written documents from the program's archives.These were collected in the period September 2016 to April 2017.Access to the data was provided by Rijkswaterstaat.The collection of the data and the identification of the policy instruments (see Section 3) evolved iteratively (cf.Berg-Schlosser et al., 2009;Rihoux & Lobe, 2009).That is, the collection of documents and interactive workshops with the program managers from Rijkswaterstaat progressively provided insight into the instruments, and this, in turn, informed the researchers about the data that needed to be collected.The second data source was questionnaires; since the archives did not provide data on all the instruments, additional data had to be collected.A small survey was sent out to the project managers tasked with the implementation of the projects.The survey data were collected in the period April 2017 to May 2017.After all the data were collected, the data matrix was constructed (i.e., Table 1).
In the third step, we calibrated each policy instrument (see Table 2).This allows the transparent measurement and comparison of cases (Gerrits & Verweij, 2018).In our study, we applied qualitative calibration, where no precise 0.5 anchor point is required (Schneider & Wagemann, 2012, p. 38).Table 2 lists the calibration rules.Besides the Contract Type, all other policy instruments were calibrated dichotomously.In Table 2, scores of 1 represent the expectation of the instrument having a positive contribution the outcome.The contract type (CONT) was calibrated into a four-value fuzzy set.PD&C and D&C are inclusive contracts and were hence calibrated as 0.5 < x ≤ 1.0, indicating a positive contribution to the outcome.The other two contract types have scores below 0.5.In the calibration of the outcome (QUAL), we used the scale that the Q-Team used in their project reports: worsened, not improved, barely improved, improved, and strongly improved (see Table 2).This five-value scale was calibrated into a five-value fuzzy set: 0.0 (worsened), 0.3 (not improved), 0.6 (barely improved), 0.8 (improved), and 1.0 (strongly improved).We calibrated 'worsened' as 0.0 and 'not improved' as 0.3, because each project was tasked with improving the spatial quality regardless of the situation at the start; also, if spatial quality was already high, the project was still tasked to comply with the program objective to improve spatial quality of the riverine area.We calibrated the middle value of 'barely improved' as 0.6, expressing that it is still more positive than negative, but not overly positive.The application of the calibration rules from Table 2 to the raw data in Table 1 resulted in the calibrated data matrix, which is provided as Table 3.
In the fourth step, the calibrated data matrix was transformed into a so-called truth table, which is provided as Table 4, using the QCApackage in R (Dus ¸a, 2007(Dus ¸a, , 2016)).This step identified the policy instrument mixes that were applied across the different cases.A truth table sorts the cases over logically possible configurations.Each truth table rows represents a logically possible configuration, i.e., a combination of policy instruments.With four instruments, 16 (2 4 ) configurations-or policy instrument mixes-are logically possible.On the basis of the calibrated data matrix, each of the 19 cases is assigned by the software to one of the truth table rows.Then, based on the cases in the truth table row, and based on the consistency scores, the truth table row was assigned a score on the outcome, thus indicating whether or not spatial quality was improved as a consequence of the application of the particular policy instrument mix represented by the truth table row.Consistency basically expresses "the degree to which empirical evidence supports the claim that a set-theoretic relation exists" (Rihoux & Ragin, 2009, p. 182), i.e., that the configuration presented by the truth table row is indeed sufficient for achieving spatial quality.For instance, Configuration 11 was assigned a positive outcome (spatial quality was improved) because Case_14 and Case_15 that represent this policy instrument mix, according to the evaluations by the Q-team, both have strongly improved the spatial quality (see Tables 1 and 3).
In the fifth step, the truth table was minimized using the QCApackage in R (Dus ¸a, 2007(Dus ¸a, , 2016)).This involved the pairwise comparison of the truth table rows that agreed on the outcome and differed in only one of their instruments (Schneider & Wagemann, 2012).It means the comparison between policy instrument mixes that produce a similar outcome, but a are different in one of the included instruments.For instance, Configurations 1 and 2 from Table 4 could be compared because both configurations are associated with improving spatial quality and only differ in the condition CLUS.This means that whether the Cluster Spatial Quality was involved and influential or not, did not matter in these cases; spatial quality was improved irrespectively.In QCA, this means that the instrument in which two truth table rows differed-CLUS-is logically redundant and is minimized away.This fifth step resulted in those policy instrument mixes that are related to achieving of spatial quality, as shown in Table 5 in Section 5. We have not included any logical remainders in the truth table minimization and thus present the conservative solution.

Analysis
The results of the analysis were produced by the QCA-package in R and are shown in Table 5.Each of the three paths in Table 5 represents a policy instrument mix that is sufficient for improving spatial quality.
As explained in Section 3, responsible government bodies had to collaborate with private contractors in order to realize the spatial quality objective in the implementation of the projects, as private parties are commonly responsible for project management and construction.Accordingly, the results in Table 5 also show two policy mixes in which the responsibilities between the government organization and the private contractors are shared (Paths 1 and 3) and a policy mix in which the government has only little involvement in the realization of spatial quality (Path 2).
Path 1 can be seen as an integrated contract mix.This policy instrument mix combines integrated contracts ("CONT"), in which private parties are considered to have a large influence on the spatial quality that will be achieved, with tender documents that include explicit spatial quality requirements ("TEN").Through the use of such explicit requirements in the tender documents, governments specify to a large extent the spatial quality that has to be achieved.As such, in the integrated contract mix, both governments and private parties influence the spatial quality that will be achieved.This is also reflected in the cases that can be considered as strong representatives of this strategy: the    cases Nederrijn Doorwerthsche Waarden (Case_09) and Nederrijn Machinistenschool Elst (Case_12), a lowering floodplains and a removing obstacles case, respectively.In both cases, Rijkswaterstaat was the responsible government for the implementation of the projects.At the time of the projects' tendering processes, the dominant perspective at Rijkswaterstaat was to regard private parties as qualified knowledge partners (see e.g., Van den Brink, 2009;Metze, 2010), and to involve private parties wherever possible.In the projects, this was even taken one step further as both were tendered using Best Value Procurement.This procurement method is designed to reduce risk and increase project value through a transparent and interactive dialogue between the government and the private contractor before the contract is awarded (Perrenoud et al., 2017).In these interactions, government demands are matched with private parties' feasibility estimations, which means that the project is developed by both the government as principal and the private contractor as agent.This is indeed also reflected in this policy instrument mix, where governments use treasure-i.e., economic tools that stimulate private parties to work towards high spatial quality solutions-in combination with authority, i.e., explicit government demands regarding spatial quality.
In Path 3, which can be seen as a project management mix, the responsibilities for realizing spatial quality during implementation are shared between government and private parties in a different way.In contrast to the previous policy instrument mix, here governments do not use integrated contracts, but instead use traditional contracts ("con").In these types of contracts, governments specify the spatial and technical design, and private parties are responsible for its construction and, in the case of Engineering and Construct (E&C) contracts, also for working out technical and logistic details of the provided design.Since the design is likely to already include the spatial quality requirements that need to be realized, the project management mix includes the negation of explicit-i.e., implicit-spatial quality requirements in the tender document ("ten").In order to ensure that the spatial design provided at the start of the implementation will also be achieved during construction, a landscape architect is involved during implementation, but only from a distance ("land").This path may point towards the importance of organizing spatial quality in the planning phase, laying the fertile ground for the implementation phase.In these cases, spatial quality may had already been so well-developed-both in terms of content (what is to be realized) as well as process (how will this be realized) -that it did not need specific attention anymore in the implementation phase.
The case of Het Munnikenland (Case_04), a dike relocation, can be seen as a strong representative of this strategy.This project included a spatial design that, prior to implementation, was already developed in considerable detail by the responsible government Water Board Rivierenland (see also Busscher et al., 2019).The novel feature of this design was a multifunctional dike, which included a recreational gallery, trees, and a safe haven for the wild animals in case of high water.The landscape architects involved in the implementation of the project indicated that they "operated within the boundaries that were provided, in order to enhance and fine-tune the plan" (www.parklaan.nl).Examples of their work included designing and detailing the height of the crowns of the trees on the dike-a technical detail in an otherwise already developed plan, also explaining why the negation of treasure, authority, and organization still proves to be an effective mix to improve spatial quality.
Path 2 is different from the first two paths that have been discussed above.In Path 2, which can be characterized as an outside-government mix, the government has only limited involvement in achieving spatial quality: the negation of explicit spatial quality requirements in tender documents-i.e., implicit requirements-is used ("ten") in combination with some, albeit limited, involvement from the Cluster Spatial Quality ("clus"), which implies that there is only little government influence on spatial quality.Instead, it are mostly the private contractors who take up the responsibility to achieve the spatial quality objective during implementation.This is also reflected in the case Depoldering Overdiep (Case_07), which can be seen as a strong representative of this policy instrument mix.On the one hand, this case is similar to the project management mix, in the sense that it continues to build on the spatial design developed in the planning process.Remarkably, however, in this case the spatial design, the so-called 'Mounds Plan' (Terpenplan) was not developed by a government organization, but instead was a bottom-up initiative developed by local farmers (Roth & Winnubst, 2014).The province and the water board only had to facilitate the development of the plan (Edelenbos et al., 2017).This also applies to the implementation of the plan, where the water board explains that "the private contractor is very self-organizing.We only have to control whether made agreements are adhered to" (Geluk, 2012, p. 6).These agreements, to a large extent, refer to demands from the local farmers, that were incorporated by the water board in the contract on their behalf, and regard issues such as safety, accessibility, and continuity of their business operation.In other words, this mix is characterized by governments not using resources to "get people to do things that they might not otherwise do" (Schneider & Ingram, 1990, p. 513), but instead facilitate the resources employed by parties outside government.

Conclusions
This article has explored the policy instrument mixes that were deployed to successfully implement (i.e., improve the spatial quality of) the Room for the River measures.Using QCA, we found three different policy instrument mixes: an integrated contract mix, a project management mix, and an outside-government mix.Each mix includes a different combination of policy instruments, related to nodality, authority, treasure, and organization.The different policy mixes also reflect the changing position of governments in flood risk management (see also Pahl-Wostl et al., 2010;Edelenbos et al., 2017).In the project management mix, governments still have their traditional role as project principal, prescribing the work that needs to be implemented.In the two other instrument mixes, however, governments take up different roles.In the integrated contract mix, the role of government is mostly to stimulate private parties to achieve spatial quality.In the outside-government mix, the role becomes strongly facilitative; governments predominantly act to facilitate initiatives developed outside the boundaries of the government organization.What these results also indicate is that no single policy instrument is necessary or sufficient for successful implementation.In different projects, different mixes of policy instruments have contributed to the achievement of the spatial quality objective.We highlight two instruments in particular.
First, more inclusive contracts can indeed contribute to improving spatial quality (cf.Lenferink et al., 2013), but not necessarily so.This finding ties in with research into the advantage of integrated contracts over more traditional contracts, where the performance of different contract types in infrastructure development is comparatively analyzed, with findings indicating that more integrated contracts may indeed perform better (e.g., Verweij & Van Meerkerk, 2020).The question remains, however, as with the present study, in which ways exactly the more inclusive contracts stimulate higher spatial quality.That is, it could be that it is not so much the contract type per se that is important, but rather what is actually determined as the project scope in the contract, and the way contracts are managed and implemented (Verweij, 2015).Second, with respect to the involvement of the landscape architect, it is noticeable that his role in implementation has been less prominent compared to the planning phase (Klijn et al., 2013;Van den Brink et al., 2019).He was closely involved in just five of the projects.This provides an interesting avenue for further research, as we cannot find clear explanations as to why the involvement of the landscape architect has decreased towards the implementation phase.On the one hand, it may be that, as a consequence of more integrative contracts, landscape architects are losing position in project implementation, because private contractors are often less inclined to hire an independent landscape architect, and instead want to make use of the design expertise available in the own firm or consortium.The design expertise then does not necessarily have to come from a landscape architect (see also Hulsker et al., 2011).On the other hand, as we also postulated in the case of the project management mix, it may also be that in some instances spatial quality had already been so well-developed, amongst others by the landscape architect, that its role in the implementation phase became redundant.This calls for more research into the evolution of spatial plans and collaboration throughout the planning lifecycle, i.e., how the planning phase and the implementation interact and build upon each other in the course of the process.
The analysis in this article has showed how integrated flood risk management is implemented 'on the ground' in the Netherlands.The identified practical policy instrument mixes may prove valuable for integrated flood risk management practices in other countries as well.The identified mixes and instruments can serve as a starting point for further in-depth analyses and comparative work on other integrated flood risk management programs.

Fig. 1 .
Fig. 1.Different types of measures in the Room for the River program (Ruimte voor de Rivier, 2016b).

Table 1
Raw data matrix.

Table 2
Operationalization and calibration.
(continued on next page)

Table 3
Calibrated data matrix.

Table 4
Truth table.: the numbers in the first column refer to the particular logically possible configuration of conditions (only empirically present configurations are shown); "Incl."concerns the consistency of the truth table rows; and PRI concerns their Proportional Reduction in Inconsistency.Configuration #10 is not included in the truth table minimization because Case_03 has a negative outcome as also indicated by the lower consistency. Note

Table 5
Results truth table analysis (conservative solution).
S.Verweij et al.