What’s Left Before Participatory Modeling can Fully What’s Left Before Participatory Modeling can Fully Support Real-World Environmental Planning Support Real-World Environmental Planning Processes: A Case Study Review Processes: A Case Study Review

A B S T R A C T In environmental participatory modeling (PM), both computer and non-computer-based modeling techniques are used to aid participatory problem description, solution, and decision-making actions in environmental contexts. Although many PM case studies have been published, few efforts have sought to systematically describe and understand dominant PM processes or establish best practices for PM. As a first step, we have reviewed a random sample of environmental PM case study articles (n = 60) using a novel PM process evaluation instrument. We found that significant work likely remains for PM to fully support participatory and integrated planning processes. While PM reports systematically address knowledge integration and learning, they often neglect the facilitation of a multi-value perspective within a democratic process, and the integration across organizations within a governance system. If not reported, we suspect these aspects are also neglected in practice. We conclude with key research and practice issues for improving PM as an approach for real-world participatory planning and governance.

Over the past few decades, environmental governance has been shifting to include more public participation throughout the decisionmaking process, with collaborative approaches called for in newer policy and regulatory directives (Leong et al., 2011;Reed, 2008;Sterling et al., 2019). Benefits of participation include increased public trust, transformation of adversarial relationships, social learning, and higher quality and more durable decisions (see Reed, 2008 for review). A large body of literature describes what successful participatory processes involve, but it also recounts the difficulties and problems that may arise from these efforts (e.g., Ansell and Gash, 2008;Schuett et al., 2001;Susskind et al., 2012). For example, groups of stakeholders may be overor under-represented in participatory processes, leading to democratic problems. Relatedly, power imbalances among the participants can make some participant's voices heard at the expense of others'. Furthermore, the complexity of environmental issues makes it crucial to coordinate decision-making across geographical scales, organizational boundaries, and policy fields. To achieve effective coordination and cooperation between public authorities in complex decision-making contexts has proven difficult (see e.g. Hedelin, 2017). So, successfully engaging public participants in such nested decision-making processes is a major challenge (Dietz et al., 2003). Perhaps as a consequence of that, in practice, one-way activities for public input, such as public hearings and public notice and comment, have become standard, while collaborative approaches, such as co-management and community science, remain underutilized (Innes and Booher, 2004;Jacobsen et al., 2012;Leong et al., 2011).
Within the large and multifaceted research area that concerns methods and tools for planning and decision-making, 1 a field that specifically refers to the concept of participatory modeling (PM) has developed during the last couple of decades. It includes various methods and tools that can support collaboration and stakeholder involvement throughout an environmental planning process, from problem formulation, to generating knowledge about system dynamics, to developing and evaluating decision alternatives, and implementation. Although extensive and systematic reviews of environmental PM literature are lacking, PM has been widely touted as a useful approach for understanding complex socio-environmental problems by improving social learning and integrating expert and stakeholder knowledge (Davies et al., 2015;Pahl-Wostl et al., 2007;Zellner et al., 2012b). We 2 therefore describe PM as a purposeful learning process for action that engages the implicit and explicit stakeholder knowledge to create formalized and shared representations of reality . Further, PM has also been shown to support conflict resolution, trust building, and collectively identifying and agreeing on problem solutions (Becu et al., 2008;Smajgl and Ward, 2013a). The growing popularity of PM studies (Seidl, 2015) has been attributed to the shifting norms for stakeholder engagement, coupled with improvements in cyberinfrastructure that have surfaced novel ways to engage stakeholders in collaborative representations of complex socio-environmental problems Seidl, 2015;Sterling et al., 2019). Given the inherent abilities of modeling methods to structure complex problems, to explicate the relationships between system components, and to demonstrate the trade-offs between key values, PM has important potential to support the difficulties of participatory planning.
Although there are rich literatures on both participatory planning and PM with respect to environmental issues (for overviews, see Nared and Bole, 2020;Voinov et al., 2016), little work has explicitly integrated both domains. While PM has shown clear contributions and great promise to improve public participation, studies indicate a need within the PM community to better understand the complexities of participatory processes, and to investigate how best to select and implement methods and tools to achieve desired benefits Hedelin et al., 2017a;Jordan et al., 2018;Seidl, 2015;Voinov et al., 2018). Another area of development is the need to understand PM in relation to the real-world complex decision-making contexts of environmental problems, which generally span diverse legislations and policy fields at multiple geographical scales, administrative levels, and actors (Hedelin et al., 2017a;Voinov et al., 2018). Furthermore, theoretical and critical studies of PM are rare. Such studies can provide increased understanding of issues such as whether certain types of participants influence decision-making in PM more than others (issues of power), if and how knowledge generated in PM may become accessible by others than those directly involved (issues of efficiency), and how decision-making in PM can be related to the overall democratic system (issues of democracy). A planning perspective on PM can help to further develop PM with respect to these issues, and would at the same time open avenues for transferring methods and tools of PM to the participatory planning field. To support the development of best practices of PM, with a view to adopt and implement PM into real-world planning and decision-making, we will here explicitly study PM from a participatory planning perspective. We ask, 'What's left to do before PM can provide full support to real-world participatory planning?' As a first step towards answering that question, we systematically review the field of environmental PM according to ideals for participatory planning in complex governance settings. To lay the basis for further studies of the complexities of participatory processes, we focus on procedural aspects. To do this, we firstly draw a random sample (n = 60) of environmental PM case articles from a larger pool of PM articles that represents a near-census of peer reviewed PM studies (as of mid-2017). The sampled articles are reviewed with a novel evaluation instrument that structures the review and provides the participatory planning lens for the study. The instrument is based on two complementary participatory planning frameworks: one focuses on descriptive criteria (e.g. number of participants) and the other is theoretical and focuses on prescriptive process criteria (e.g. how participants are selected), (see section 2). After presenting our findings, we conclude with key research and practice issues for improving PM as an approach for real-world participatory planning and governance.

Data collection
Our review is based on 60 published case study articles (listed in Appendix C). These were randomly selected from a pool of 212 articles compiled by searches on the academic databases, Web of Science and Science Direct using the search terms "participatory modeling," "collaborative modeling" and "companion modeling." Articles were further narrowed through filtering for "environmental" applications. One of this paper's authors (to ensure consistency) was devoted to scanning each compiled abstract and removing it if it was not an environmentallyfocused PM case study. Case studies for the purposes of this review were limited to English-language, peer-reviewed scholarly journals.
Choices of methods will always limit the study. A main limitation follows from the choices of databases, search terms and application areas. For example, research of environmental applications that do not use the selected search terms, such as operations research, multi-criteria decision making and decision theory, are not included. Furthermore, the use of scientific papers as data limits our possibilities to make statements about the performed studies, as not everything about the study is reported in a scientific paper (e.g., due to length restrictions). There may also be an overrepresentation in the reports of what the authors see as successful cases, processes and activities.

Evaluation instrument
To evaluate the selected PM case papers, we developed a novel, 21question evaluation instrument (see Appendix A). The review questions are hereafter referred to as Q1, Q2, etc. This instrument is based on two complementary frameworks for participatory planning processes. The first frameworkthe Comparison of Participatory Processes (COPP; Hassenforder et al., 2015) informed review questions about the case context (e.g., the geographic scale, the problem inspiring the PM 1 This area can be structured and described in different ways depending on the entrence points choosen, such as area of application, type of tools in focus and the aim of the research. Examples of previous overviews are: Huang et al. (2011), Johnson et al. (2018, Lamé et al. (2020), Rouwette and Vennix, 2006. 2 The US National Science Foundation's National Socio-Environmental Synthesis Center (SESYNC) funded an interdisciplinary team of social scientists, biophysical scientists, software developers, and participatory modeling practitioners to discuss the processes, products, and outcomes associated with participatory modeling and its approaches. The group of authors was part of this team. For a complete list see: Gray, Voinov, Paolisso, and Jordan Participatory Modeling. https://www.sesync.org/project/enhancing-socio-environ mental-research-education/participatory-modeling.
process, the PM process initiator, the goals of the PM process, and the number and type of participants in different steps of the process, (Q2-10)). The second frameworkthe Sustainable Procedure Framework (SPF; Hedelin, 2007Hedelin, , 2015 was used as the theoretical and normative basis for evaluating procedural aspects of the cases, (e.g., how participants were selected, how power imbalances were managed, what resources the process required, and how the PM process was related to the surrounding decision-making system (Q12-21)). Section 2.3 outlines the SPF. Question 11 asked about specific tools and methods used in the case study. It was also used to complement a survey of PM practitioners . Our PM process evaluation instrument was elaborated through a stepwise and iterative procedure engaging the whole group of authors 2 . The process started with a smaller group who developed, pre-tested, and revised an initial set of questions against a subset of case articles. These pre-tests and revisions were repeated several times. To integrate collective knowledge and experiences of the PM field in the final version of the instrument, the whole group of authors iteratively discussed and refined questions and answer categories. We also focused on making this instrument useful to those beyond our author team, so we have posted it online for sharing. 3 The questions were separately loaded into a webbased questionnaire to assemble the review data and to generate basic reports including tables and diagrams. Each paper was double coded weeks apart by the same coder, ensuring consistency in evaluations across the papers.
To further support data collection and analysis, a generic process framework was used based on Evers et al. (2012) andAlkan Olsson et al. (2011). It includes six steps covering the main topics that can be handled in a PM process from a planning perspective, although not all steps were described in all case study articles (Fig. 1).

Theoretical basis: implications of participatory and integrated planning
The theoretical basis of this study is operationalized by the Sustainable Procedure Framework (SPF). The SPF prescribes what a planning process needs to include, support, or promote to be both integrative and participatory (Table 1). To that end, it synthesizes relevant theories and research from a broad review of literature as well as from interviews with senior researchers in planning, public administration, economics, political science, resilience studies, adaptive governance, deliberative democracy, integrated management, and ecological economics. For a full explanation of the SPF and how it was derived, see Hedelin (2007;. The 16 SPF criteria stem from the two concepts of Integration 4 and Participation and are structured by five themes, which together support a structured and theory-based analysis of any participatory planning process-in this review, those using PM.  There are a number of participatory frameworks and best practice guidelines for participation and PM in literature, such as Barreteau et al. (2010), Hassenforder et al. (2015); Perez et al. (2014), Smajgl and Ward (2013b). The SPF is used as basis in this study because it allows us to simultaneously perform: • a focus on procedure (compared to a focus on output; e.g., a management plan, an implemented measure, a developed model) 5 ; • a theory-based analysis; • a critical perspective (based on the deductive and normative character of the SPF); and • a governance perspective on PM (due to inclusion of issues such as organizational integration and stakeholder representation, which stems mainly from the principle of Integration).
The use of the SPF has provided deeper understanding and practicable advice related to several cases of planning and planning tools already. See Hedelin (2017) for illustration of the SPF as analysis tool for 5 p.m. cases. See Hedelin and Lindh (2008) and Hedelin (2015a) for examples of analyses of planning processes, and see  for an analysis of planning legislation.

Results
We begin by summarizing a number of contextual features of PM studies, including the number of publications per year, subject focus areas of the studies, their geographical distribution and other dimensions. After that, we present our analysis around the more processoriented SPF themes: (1) integration across disciplines, (2) integration across values, (3) integration across organizations, (4) participation contributing to the process, and (5) participation generating commitment, legitimacy or acceptance (see Table 1). Lastly, we explore characteristics around evaluation and theoretical connection of the case studies.
Appendix B includes complementary graphical summaries of the results (referred to as Figure B1, B2 etc.)

Contextual features of the case studies
The 60 reviewed papers in our sample were published between 2003 and the summer of 2017, when the literature search was finalized. More than half of the reviewed papers were published during or after 2012 ( Figure B1). Among our case studies, agriculture (n = 13; 22%) and river basin management (n = 11; 18%) were the most common focus areas. The issues of food availability and climate change were not studied in any case studies reviewed ( Figure B2; Q2). The reviewed studies were conducted in all permanently populated continents, with most studies in Europe (n = 17; 28%) and fewest in South America (n = 1; 2%) ( Figure B3; Q3). Almost all of the studies had a regional (n = 42; 71%) or local (n = 16; 27%) scale of interest, with just one case study explicitly including a range of geographic scales ( Figure B4; Q4).
Project initiators were described in just over 60% of the cases (n = 36). Of those, 30% (n = 11) identified multiple types of initiators: governmental agencies were the most commonly listed followed by scientists and NGOs. Local community members were not identified as project initiators in any of the studies. For studies that did not explicitly describe who initiated the study or project, we inferred who initiated the study. In almost all cases, our review indicated that scientists had initiated the study, suggesting they have the strongest role in initiating PM projects and case studies ( Figure B5; Q5). Scientists also had a very strong role in leading the PM processes. Most papers report the type of actor that leads the PM process (n = 40; 67%). Of those studies, 85% (n = 34) were led by scientists and none were led by a local community member ( Figure B6; Q6).
For most studies, the number of participants was either listed or could be inferred. The smallest number of participants listed explicitly was 6, and the largest was 602.30% of the studies had 25 or fewer participants, 20% had 26-49, 35% had more than 50 participants, and for 15% of the studies, the number of participants could not be determined ( Figure B7; Q7). A detailed examination, however, revealed that over 90% of the cases involved less than 65 participants. The two largest studies, involving 373 and 602 participants, respectively, both relied on online participation.
Question 8 queried the cases' process frameworks, by which we mean several explicitly described steps, including their relations, which provide an overview and a structure for the PM process. A majority of the studies (n = 33; 55%) reported some form of such a defined process framework. There is great variation and lack of consistency of how process frameworks are described in the PM papers. Only two frameworks -Companion Modeling (Gurung Tayan et al., 2006) and Group Model Building (Vennix, 1999) were mentioned in more than one case.
The PM studies were undertaken for a variety of reasons. The most common of our listed purposes, stated by 26 studies (43%), concerns the development or application of a PM process framework. No other purpose was cited in more than 12 of the cases (20%) ( Figure B8). In contrast to the diversity of purposes, there was much more consistency in the types of reported results. At least 80% of the studies reported that gathering knowledge from stakeholders (all studies), developing modeling method (n = 57), developing and applying a PM processes (n = 53), and stakeholder learning (n = 52) were important results ( Figure B9; Q9).
The case studies drew on a range of methods and tools, which were used for a variety of purposes ( Figure B10; Q11). Here, we define tools as specific software types (for example agent-based modeling, system dynamics modeling, decision-trees etc.), and we define methods as processoriented approaches (e.g., brainstorming, scenario planning, which can use a range of tools), although in environmental PM there is often some degree of fluidity between the two . We recorded both methods and tools used in three phases of PM: problem identification, data collection about the problem, and problem analysis. The cases often used multiple methods and tools both for problem identification and for data/knowledge collection. The most commonly used methods and tools were interviews (n = 43) and expert elicitation/panels (n = 44), followed by surveys, questionnaires and polls (n = 35), and scenario building (n = 30) and focus groups (n = 27). Most cases used only one kind of method or tool for problem analysis, but no single tool dominated across our case study sample. Most popular were system dynamics modeling (n = 24) and geographic information systems (GIS; n = 22).

Procedural aspects of the case studies
3.2.1. Integration across disciplines 3.2.1.1. Integration of knowledge (criterion A). We reviewed case studies for whether, and how, they included procedures to identify the knowledge necessary for the PM process, including the tools used for that purpose (Q12). Different types of knowledge were targeted, such as scientific, expert, and layman knowledges. Nearly every study (55; 92%) described a systematic procedure for addressing this issue. A systematic procedure means that comprehensive and explicit procedures are used to identify the main pieces of knowledge needed in the process, including justification of what knowledge to include (Fig. 2).
As detailed in Table 2, participants (e.g., experts, stakeholders, community members) were the main source of information used to identify knowledge needs. Interviews were performed in 39 cases (65%), expert elicitations were used in 25 cases (42%), and brainstorming occurred in 12 cases (20%). While no additional process to structure knowledge was mentioned in most studies, we found that 15 studies engaged in a form of qualitative mapping (25%). These forms of mapping differed significantly, ranging from causal maps (e.g., as a final product or as a step towards a system dynamics model) to fuzzy cognitive maps. The use of previous studies was less common, as seen in 9 cases (15%), and only 6 studies (10%) collected or used data other than qualitative transcripts of interviews and focus groups (e.g., administrative data, spatial data).
Many of the methods and tools applied in the cases ( Figure B10; Q11) may also support integration of knowledge. Examples of such tools frequently used include scenario building (n = 30; 50%), system dynamics modeling (n = 24; 40%), focused group discussions/interviews (n = 27; 45%), brainstorming (n = 23; 38%), GIS (n = 23; 38%), and cognitive/concept mapping (n = 20; 33%). (e.g., positivist, relativist; criterion B). We did not code the papers for this criterion explicitly, as our early pilot efforts to create and apply our evaluation instrument (along with the authors' collective, prior experience) determined that this issue is rarely (if ever) handled explicitly in PM processes, and even less so in PM case study applications. A more in-depth review approach is necessary to target this issue.

Handling different kinds of uncertainty (criterion C).
Of the reviewed case studies, 37 cases (62%) report a systematic approach towards uncertainties ( Fig. 2; Q12). The rest of the papers report a limited approach towards uncertainties (n = 14; 23%), or do not report any uncertainty handling (n = 9; 15%). A range of methods and tools are used; for those that report using systematic approaches, the use of scenarios is the most common approach used (n = 15). Other systematic approaches used are agent-based models (n = 6), multi-criteria analyses (n = 4), and conditional probability tables (n = 3). See Table 2.
We also coded the cases for the types of uncertainties addressed (Q13). The most common type concerns uncertainties in the system at hand, such as alternative system functions (n = 40; 74%). The least common type is related to the proper formulation and representation of a conceptual model (n = 24; 43%). The other uncertainty types addressed in the reviewed papers are distributed within that range (n = 25-32) ( Figure B11). Most (79%) of the papers that report uncertainties address more than one type of uncertainty. One fifth of the cases concurrently addressed all five types of uncertainty that we evauated.

Identification ofand rational argumentation based onrelevant values (criteria D and E).
The main questions for examining these criteria concern how studies reported handling the main values involved, and which methods and tools were used to support those efforts (Q12). Just over 40% (n = 25) of the cases reported systematic approaches to values (Fig. 2), meaning that a comprehensive and explicit approach was applied to identify the main values, and a justified decision was made about which values to include and how to adjust the process accordingly. There was a broad diversity of methods and tools used by cases that took a systematic approach to the issue ( Table 2). Out of the 25 cases using systematic approaches, 7 cases used interviews and 3 used a multi-criteria approach.
Some cases did not include the important values-focused process

Table 2
Most commonly used approaches for addressing five process issues systematically. See Appendix A Q12 for our complete explanation of a systematic approach.

Process issue Prevalence of a systematic approach
Most common systematic methods and tools used (by the cases that repot systematic approaches) Identification of the participants to involve 18% (n = 11) • Sampling (n = 6), usually by snowball (n = 5) • Application of selection criteria (n = 2) • Nomination by a local group/ committee (n = 2) • Involves all participants (n = 1) Identification of the knowledge needed to understand and address the issue 92% (n = 55) • Interviews (n = 39) • Expert elicitation (n = 25) • Mapping (e.g., qualitatively, causal loops, as part of a system dynamics of fuzzy cognitive maps) (n = 15) • Brainstorming (n = 12) • Literature review (n = 9) • Use of data (e.g., governmental data, collection of spatial data) (n = 6) Handling of the uncertainties most relevant to the problem 62% (n = 37) • Use of scenarios (n = 15) • Agent-based models and multi-agent systems (n = 6) • Multi-criteria analyses (n = 4) • Conditional probability tables (n = 3) Handling of the main values affected by the process • Multi-criteria analysis (n = 3) Fitting the process into the formal decision-making context 15% (n = 9) • Reliance on participants, community members, or stakeholders (n = 4) steps of formulating and evaluating alternatives or assessing of proposed decisions. For those that did, 64% (n = 32) made clear within the process how values were affected by the key alternatives considered, such as alternative policies, plans and measures. The rest of the relevant cases (n = 18; 36%) did not report how the involved values were affected (Q12b). The most used methods for analysis cited in the case studies were system dynamics modeling and GIS ( Figure B10; Q11). While these can be strong tools for analyzing how some values are affected by alternative decisions, we cannot tell from our analysis whether these tools were also used to analyze effects on a set of identified main values. In comparison, multi-criteria analysis, used in 9 cases (17%), is a method that provides strong support for explicitly deliberating about how alternatives affect a set of selected values. Likewise, cost-benefit analysis is another tool for communication around values, but was used in only 1 case (2%).

Integration across organization
3.2.3.1. Organizational learning (criterion F). By organizational learning we mean learning that reaches beyond the set of people directly involved in the PM process ("non-participant learning"). Examples of such approaches that we looked for included process evaluation and documentation, the establishment of shared databases, and the development of institutions (e.g., meeting routines) of the organizations represented by individuals in the process.
Just over 30% of the cases (n = 19) reported on the issue of organizational learning, and only 15% reported "explicitly" on this issue (meaning that activities that might support organizational learning and discussions of organizational learning were both included in the paper; Fig. 3; Q16). The main way to accomplish organizational learning reported was to trust those directly involved to disseminate knowledge gained during the PM process to their home organization, village, family, etc. Other ways included the distribution of maps and oral presentations. Structural or institutional approaches to organizational learning with long-term continuation established as part of the process were not reported.

Handling of formal planning context (criterion G).
A systematic way of fitting the PM process into the formal planning contextincluding mapping of the formal context and a strategy for handling the formal contextwas reported in 15% of the reviewed cases (n = 9). Nearly 60% did not report the issue at all (n = 35; Fig. 2; Q12). Only 3 cases reported that explicit communication around participants' roles and mandates occurred as part of the process (5%; Fig. 6; Q18).
No method or tool clearly emerges as common practice among the nine cases that used a systematic approach to the planning context issue.
Although approaches are different, we note that four of the studies are unified in their reliance on participants to fit the PM process into the formal decision-making context (Table 2).

Handling of incentives, including resources and efficiency (removal of thresholds; criterion H).
The issue of resources and efficiency is fundamental for the implementation of PM processes in practice. We evaluated case studies for how they described the resources required for carrying out the PM process, e.g., money, time, expert skills, and data. Only 10% (n = 6) of the cases reported something about the resources required (Q14).
Among the few cases that did report resources, we wanted to know if the trade-offs involved could be understood, i.e., What was the potential increased value that accompanied the added cost of the PM process, and what can be learned from the case about efficient PM process designs? These cases provided only sparse accounts, which did not enable crosscase comparisons of alternative process designs. The cases mainly accounted for issues of timeeither the time required to complete the PM process, as such, or that the process required more time than an alternative, non-PM process. Two cases discussed monetary costs of the process in general terms, and only one case reported on the total project budget. Four cases discussed resources in relation to process outcomes.

3.2.3.4.
Handling "human" aspects (e.g. trust, engagement, conflict management; criterion I). Over half of the cases did not report taking any actions to handle the "human" aspects of the process, such as trust, engagement, and inter-personal conflict (n = 33; Fig. 4; Q17). The most common action, applied by almost one third of the cases, was to engage a skilled process leader (n = 19). Almost one third of the cases made the PM process transparent (e.g., by documenting the reasons behind the decisions made). Only 3 (5%) of the cases took actions to understand the social relationships of the participants before the process (e.g., value positions, disputes, and conflicts among the group of participants).

Inclusion of knowledge owned by relevant actors (criterion J).
As stated above, more than 90% of the cases reported using a systematic approach to address the issue of knowledge identification (n = 55; Table 2; Q12). To include the knowledge owned by relevant actors, identification and engagement of these actors in the process is also a prerequisite. However, less than 20% of the cases reported using systematic procedures for identifying and selecting participants (n = 11; Table 2; Q12). On the positive side, only a few cases (n = 2; 3%) did not report this issue at all. Furthermore, among the cases that did describe participant identification (systematically or not; n = 58), all cases (100%) reported that their main reason for selecting and inviting participants was that they own knowledge that was needed in the PM process (Q12a).

Inclusion of the ideological orientations represented by relevant actors (criterion K).
Just over 40% of the cases reported systematic procedures for identifying the main values related to the issue at hand (n = 25). But, as we just saw, less than 20% of the cases reported using systematic procedures for identifying and selecting participants (n = 11; Table 2).

Participation in the most critical phase(s) of the process (criterion L).
We reviewed case studies with respect to each of the generic PM steps shown in Fig. 1: first, as to whether each step was explicitly described, and then, if so, which types of actors participated in the step ( Fig. 5; Q10). Almost all papers included a description of the work for achieving the baseline understanding of the case system (n = 58), and nearly all case studies included a discussion of model development and use (n = 52). The outcome of the process, such as a decision, plan, proposed measure or an agreement, was the step least described (less than 30% of the cases; n = 17). The rest of the planning stepsprocess preparation and setup, formulation of objectives, and formulation and evaluation of alternativeswere included and described in 70-80% of the cases.
Considering the stepwise involvement of actors, there is a somewhat unbalanced focus among the case studies towards the more knowledgebased process steps compared to those of more managerial, valueoriented and operational characteristics (process preparation, objectives-alternatives-evaluation and outcome, respectively).  5 shows who was involved in each phase of the reported PM processes. Scientists were the type of participant reported to be involved most frequently in PM planning steps. This is in line with our earlier finding that scientists were leading most of the PM processes. In the preparation step, scientists were involved more than twice as often as any of the other participant types. However, when it comes to the PM process outcome, local community members were involved the most. Non-governmental organizations (NGOs) and local community members were rarely involved in the process preparation step. Furthermore, NGOs had the lowest involvement in all steps (except for preparation).   Fig. 1) included and described in the case papers, and the actors participating in each of these steps (Q10).

Participation generating commitment, legitimacy and acceptance
3.2.5.1. A procedure for defining the actors that should be involved (criterion M). Which actors participate has fundamental and cross-cutting impacts on the quality of the process. It directly affects the fulfilment of criteria J and K, and indirectly affects most of the other criteria. Less than 20% (n = 11) of the cases reported systematic procedures for addressing the issue of identification of participants i.e., the participants were identified and selected by a comprehensive and justified approach ( Fig. 2; Q12). We noted four different approaches among these 11 studies with systematic procedures (Table 2), with sampling being the most common approach (6 cases). The remaining cases used some sort of pre-established selection criteria (2), following the nominations of a local group or committee (2), or inviting all possible participants (1).
On the positive side, only a few cases did not report this issue at all (n = 2). The participant identification and selection criteria that we coded (Q12a) were applied by most of the cases that reported on this issue (systematically or not): participants who may support implementation (n = 47), who own valuable knowledge and perspectives (n = 58), and those who are affected by the process (n = 52). Only 5 cases applied other participant selection criteria.

Handling power asymmetries (criterion N), and Procedures that ensure that ideological orientations are not suppressed (criterion (O).
Half of the cases did not report any activities for managing power asymmetries ( Fig. 6; Q18). For the rest, three approaches were applied most commonly: (1) separated meetings, (2) structured communication procedures to obtain input from each party (each by 30%; n = 18)), and (3) documentation to increase transparency (just over 20%; n = 13). Only 5% of cases reported the use of telecommunication (e.g., via a webpage) as a way of handling power issues, and only 5% reported that they clarified the roles and mandates of the participants involved in the process.
Because knowledge is a resource that brings power, an additional and important way to manage power is to support learning, which takes us to SPF criterion P.

Stakeholder learning (criterion P).
The review focuses on learning processes rather than on "learning outputs", i.e., whether and how much the participants learned from the PM processes (by Q15). Over 80% (n = 49) of the cases described activities that may support stakeholder learning (Fig. 7). Just over half of the cases, however, reported explicitly on how they supported learning, meaning that activities that supported learning were described, and, learning, as such, was discussed in the paper. This does not mean that an evaluation of the learning was reported, which could show if the process activities indeed have resulted in learning (see section 3.3). Among the cases that reported explicitly on learning, participant interactions and discussions of various forms, such as workshops, modeling, and field trips, were reported as activities aimed to support learning.

Evaluation and theory connection of the case studies
Although all reviewed papers were published in scientific journals, over 60% (n = 37) did not include any kind of evaluation of their project (Q20). Of the evaluated cases, 30% lacked a description or justification of the evaluation (n = 7).
Of the cases that included evaluations, most cases used data on the participants' understanding of the process using interviews or questionnaires (n = 13). The second-most commonly used approach was to use process data other than the participants' self-reported experiences, such as observations of process activities or interview with the process leader. Only one case used theory to support their evaluation. See Fig. 7 and Fig. 8.
We also reviewed cases for whether the outcomes of the planning processes were recognized in any decisions or actions outside the PM effort itself (Q21). Almost 90% of the cases lacked such a recognition. We categorized the seven cases where outcomes of the PM effort had  been recognized: recognition by laws or regulations (n = 1), by a management action/measure (n = 6), by an institutional arrangement (n = 0) or by other means (n = 1).

Discussion: learning from PM research
PM research is generally carried out in collaboration with public authorities, organizations, and persons who are engaged in managing their real-world problems. As seen above, many of the studies we evaluated aimed to support learning, whether about the socio-ecological problem at hand, about different ways of understanding it, or about alternative ways forward. However, one can also learn about the PM process, e.g., the characteristics and function of methods and tools employed, process designs, process leadership and conflict management, etc. As PM researchers, it is mainly this kind of knowledge that we seek. To contribute to a growing body of shared and collectively reflected knowledge of PM processes, our main strategy has been to report our studies in scientific papers. These papers became our data for this review. 6 They generally describe PM processes and their contexts, focusing on the specific study objective, which, as we have learned, can vary considerably ( Figure B8). To this end, Gray et al. (2018) suggest a procedure to standardize the reporting of PM studies to allow for better systematic review across studies, and thus to better understand and inform the evolution of this field. The 4Ps of PM are: Purpose, Process, Partnerships, and Products. These 4Ps should be addressed explicitly in all publications, whether peer reviewed or not, to ensure their contribution to generalizable knowledge about how to conduct PM exercises towards impactful outcomes.
In addition, while project evaluation should also be regarded as an important part of a regular research paper, we have found that most papers do not include an evaluation. Moreover, for those that do include evaluations, reporting standards are low, lacking justificationsuch as a detailed description of how the evaluation was performedand clear evaluation criteria. For the papers that include a justified evaluation (n=7), most are based on participants' perceptions of the PM process, and only one case used a specific theory to support evaluation. This decreases the usefulness of the evaluation performed because it cannot easily be understood by anyone outside the casewhat standards are the assessment being made against? Whose standards? Furthermore, whilst this review focuses on process, evaluation of both process and outcome criteria are important to include if we want to contribute to a complete comprehension of PM. Our study shows that currently, case study reports seldom include evidence of outcomes.
In this study, we have purposely chosen to be very specific about our basis of assessment because it allows us to compare PM against a current framework of knowledge about what a participatory and integrative planning process should imply (the SPF). A summary interpreting the results of our analysis is provided in Table 3.
Because the criteria are related to each other in various ways, the criteria-wise result can be summarized across several lines. We have identified three cross-cutting patterns that merit discussion: (1) knowledge integration and learning, (2) values and democracy, and (3) integration across organizations.

Knowledge identified and often integrated
The SPF states that a fundamental characteristic of a participatory planning and decision-making process is that it integrates the main pieces of knowledge related to the issue at hand (SPF criteria A and J). For river basin management in a regulated river, for example, knowledge of river ecology, flow regulation, alternative hydro-power technologies would be key, as would knowledge of policy, law, economy, and planning. Depending on other case specifics, knowledge of tourism, recreation, land preservation, heritage, flood management, irrigation technology, and more could be likewise important.
Knowledge comes in various forms, such as disciplinary knowledge (criterion A) from experts, databases, and reports, and also more local and contextual knowledge (criterion J) from local stakeholders, NGOs, and public authorities, which may not be documented as text or numbers. Examples of the latter include local farmers' harvesting and fertilizing routines, fishers' observations of fish stock dynamics, a regional authority's current plans for the infrastructure system, and local attitudes towards a specific policy or measure.
Engaging experts and local actors who can represent all the pieces of relevant knowledge may be very expensive. But missing out on a key piece of knowledge could have devastating consequences. Therefore, it is important to prioritize knowledge carefully, and to be open to reassessing knowledge needs as the PM process evolves. To rely solely upon the expert judgement of a few persons is not sufficient (Glaas et al., 2010), in great part because experts are likely to emphasize the knowledge that lies close to their own expertise; we do not know of the knowledge that we do not know (Glaas et al., 2010;Hedelin and Lindh, 2008). Instead, a more comprehensive, transparent, and justified procedure is necessary. Our finding, that over 90% of reviewed cases apply such approaches to knowledge identification (Table 2) is gratifying. Furthermore, the frequent use of methods and tools that can integrate knowledge, such as scenario building, system dynamics modeling, agent-based modeling, GIS and cognitive mapping, is also a very strong feature ( Figure B10), which implies that the PM field has an important contribution to make to environmental planning and decision-making when it comes to knowledge integration. Note, however, that the application of these methods and tools is no guarantee that the knowledges have been integrated effectively. For example, focus group Fig. 8. Out of the 16 cases that include a description of the evaluation, the following acted as the basis for the evaluation (Q20). 6 As noted in the method section, the use of academic articles has its limitations. It may for example be that systematic procedures are being used to identify participants (criteria M) in most cases but there are 'systematic gaps' in the reporting of this (and potentially other criteria) in the PM literature. In other words, more focus is given to the outputs and descriptions of modeling methods and tools rather than other process-issues. Complementary studies would have to be made to study that. discussions that are poorly led will not function well, and any approach that neglects actors with key knowledge will fall short. We have coded the cases for issues such as representation, power imbalances, and conflict management, which may indicate whether the methods and tools applied function well. See result for criteria M, N, O and I.
Difficulties with social power and tool use are interwoven with the different ways participants understand knowledge and its connection to values, i.e., their different epistemologies (Halpern and O'Rourke, 2020;O'Rourke et al., 2019). For instance, can some knowledge be considered as objective, while other knowledge needs to be treated as depending on individual experiences and preferences? Not only are epistemologies usually different, but also, depending on their disciplinary or professional training, life experiences, and other personal characteristics, participants of the PM process will be differentially equipped for managing epistemological differences. The PM process-including leadership, resources, methods and tools-will affect how these dual differences of content and capacity play out (Hedelin et al., 2017a). Process features can become obstacles for knowledge integration or they can enrich the process by supporting a better understanding of the complex planning issue at hand (SPF criterion B).
We did not code the papers for SPF criterion B explicitly, because this issue is hardly ever reported in PM papers. See Voinov et al. (2018) for guidance on how conceptual modeling methods such as fuzzy cognitive mapping (FCM)/cognitive mapping can complement quantitative methods to avoid that kind of problem. More studies are needed, however, to explore the issues of epistemology in participatory processes, and of how PM may provide support. For example, how does the epistemology of the process leader affect the design and orchestration of a participatory (PM) process when it comes to knowledge integration, learning, and management of alternatives? How aware of our epistemological positions are we, and of how they manifest in our behaviors and actions as PM process leaders, as PM researchers, or as modeling tool developers? How can methods and tools in PM help to expose and manage alternative views of knowledge so that such differences do not hamper the participatory process? Such studies could raise of awareness of this issue among PM researchers, and hopefully epistemological aspects will become explicit in PM studies that focus on knowledge integration and learning. For instance, in their analysis of 5 p.m. cases (Hedelin et al., 2017b), showed that methods and tools regularly used in PM have the potential to act as learning platforms supporting discussions among the involved researchers and experts of ways of understanding both reality and knowledge (which are fundamentally connected). However, sometimes the methods and tools used can hamper discussions of knowledge views and integration of knowledge (Hedelin et al., 2017b). For example, because of the time and resources that extensive quantitative simulation models rely on, it may be difficult to reconsider the foundations or components of the model in accordance to the understanding gained from an interdisciplinary learning process since this may not be included in the project time plan and budget (Hedelin et al., 2017b).

Understanding uncertainty
In the context of knowledge and learning, the issue of uncertaintieswhat we don't knowis critical (Bammer, 2013). The complexity inherent in the socio-environmental systems targeted by planning efforts makes it impossible to eliminate uncertainty no matter how much time and resources are spent on increasing the level of factual knowledge (Bradshaw and Jeffrey, 2000;Zellner, 2008). Thus, uncertainty will be a regular feature of the decision-making process and it may have large impact on the various decision alternatives at handthe potential outcome of the process. Logically, an effective approach towards handling uncertainties must involve understanding which uncertainties are the most important in terms of how they affect the outcomes of alternative decisions. Clear understanding of the relative importance of different uncertainties should guide where evaluation and research effort should focus. An open, comprehensive and systematic approach Table 3 Summarizing our analysis relative to the SPF. The criteria that are assessed as being handled well in almost all cases are marked with a happy face. The criteria that are assessed as being handled poorly or not at all in almost all cases are marked by a sad face. Between these grades, a slightly happy face or a troubled face mark the criteria that are assessed as being handled well or poorly respectively by a majority or close majority of cases. Relatively low involvement of local community and NGOs in steps that need value-based input and in process preparation but high involvement in knowledge-based steps. M a procedure for defining the actors that should be involved.

SPF Criteria
Less than 20% of the cases describe systematic procedures for identification of actors. N handling of power asymmetries.
50% of the cases do not report any activities to manage power asymmetries. O procedures that ensure that ideological orientations are not suppressed.
50% of the cases do not report of any activities to manage power asymmetries. P stakeholder learning. Just over half of the cases report explicitly on how the process supports learning.
towards the full range of types of uncertainties is therefore imperative. By our results, we can see that knowledge and practice concerning the management of uncertainty is relatively well established within the PM field. Over 60% of the cases have handled this issue by systematic procedures (comprehensive and justified, Fig. 2). Even though there are additional types of uncertainty (e.g. of the modeler's skills), the types that we reviewed are covered quite well by the cases, which signals that none of these broad types are being completely overlooked ( Figure B11).

Whose knowledge?
Integration of knowledge depends on more than identifying knowledge and uncertainties. Knowledge is held by actual groups and persons, so appropriately broad representation of stakeholders is also essential to knowledge integration. The SPF criterion of carefully defining the actors that need to be involved (M) is fundamental for the fulfillment of inclusion of actors' knowledge (J), and likewise important for many other key process criteria, such as inclusion of actors' ideological orientations (K) and for establishing a democratic process. To meet these criteria, the most relevant actorsthose directly or indirectly affected by the planning issue at handneed to be identified and involved in the process, directly or by representation (Dryzek, 2013;Lidskog, 2005). Unfortunately, we found that less than 20% of cases described systematic procedures for participant selection (Fig. 2).

Stakeholder learning is becoming central to PM process
The final key knowledge integration criterion concerns stakeholder learning (P). Socio-environmental problems are generally complex, both technically and socially (DeFries and Nagendra, 2017). Local actors, who are directly concerned by the problem at stake, can provide important knowledge to the process that complements expert knowledge, and their participation may also generate commitment, legitimacy, and acceptance for the resulting decision and facilitate implementation (Bryan, 2004). However, if stakeholders (including experts) are going to take part in the decision-making process, they need to understand the problem that the decision concerns (Sterling et al., 2019). Stakeholder learning is therefore a fundamental component of participatory processes, both for reasons of knowledge integration and democracy (we return to the issue of democracy in the next section).
We have shown that over half of the PM cases reported explicitly on stakeholder learning, including how they intended to support it through process methods and procedures. Another 30% reported on methods and tools that may support learning, such as workshops, modeling activities, and field trips. Nevertheless, our judgement is that although the field has come far in supporting stakeholder learning, progress can still be made, especially when it comes to the question of whose knowledge is shared and whose is represented in analytic models. Another urgent aspect to further develop PM in this respect is to increase the number and quality of standards for evaluating learning (see section 3.3 about evaluation).

Values and democracy (SPF criteria D, E, K-O)
Our values influence the way we understand a problem and how we are likely to respond to it. The issue of how to deal with values is therefore a fundamental question for establishing a good planning process, and the planning literature is much concerned with it, e.g. the separation of knowledge and values, the role of the planner/process leader, and how to deal with the issue practically (for overview and examples, see Allmendinger, 2002;Thomas, 2012;Wallace et al., 2020). Practically, the way forward is to identify the most important values connected to the problem at hand (SPF criterion D) and to use these as the basis for rational argumentation around alternative choices that needs to be made during the planning process (SPF criterion E).

Which values?
Just over 40% of the cases reviewed describe systematic approaches for identification of the most important values (Fig. 2). So, for a majority of the cases, one cannot be sure that the values considered are the ones most affected by the decision at hand. For example, if a PM process concerns a plan for a regional agricultural district, it might be that the management alternatives are analyzed for their impact on water quality, bio-diversity, and farmers' income levels, but not for climate or for securing food supply. Key values can be excluded due to organizational, administrative, or practical reasons, such as the agenda of the funding agency, the mandate of the initiating authority, or the formulation of regulations that protect a certain value. Other possible reasons are ignorance, inattention, and the influence of unbalanced power relations.

Whose values?
Generating a comprehensive understanding of a problem and possible solutions requires input from those actors who represent the array of ideological orientations connected with the issue. Hemmati et al. (2002) express this nicely by stating that participation ''aim[s] at multi-subjectivity rather than objectivity,'' which is a claim based on the view that everyone has a subjective understanding of an issue and can therefore only contribute with parts of the overall picture (Hemmati et al., 2002, pp. 44 and 300). But in our review, over 80% of the studied cases did not describe participant selection procedures that could secure inclusion of the full spectrum of ideological orientations related to the issue at stake (Fig. 2).
Seventy percent of the studied processes took place at the regional scale ( Figure B4). There are various ways for capturing the full array of ideological orientations at this scale. In addition to a comprehensive and justified participant selection approach (SPF criterion M), such approaches need to include practical means of communication. One strategy can be to open up the process for a large number of participants (direct involvement) using some form of web portal for distanced communication. Only four cases among the studies we reviewed used a web portal, however, and as the median number of participants in the studies cases was 25, it appears that this strategy has not yet been applied much within the PM field.
Another way to capture the values of many persons could be to use some form of crowd-sourcing, but here too, the PM field has not yet engaged much (only two cases use a crowd sourcing technique; Figure B10). Yet another main strategy to gather information on people's values is to seek representation of a broad set of views by carefully selecting persons and/or interest organizations representing different views. Currently, this seems to be the most common strategy (although, again, most cases do not use a systematic selection procedure). The advantage here is that it allows for a close involvement of the actors, which gets more difficult as the group of participants grows larger. For selecting persons that can represent main ideological orientations at a regional scale, one could expect NGOs to be a key type of participant. Instead of finding and justifying the selection of several unorganized persons that can represent the key values or interest groups at that regional scale it would certainly be more feasible to commit a number of NGO leaders who are already organized to represent key values at larger scales (than individual). Surprisingly, however, NGOs are the participant type with lowest involvement in most of the planning steps (Fig. 5).
Connected to values and representation, it is also important to consider which planning steps are in focus, and which types of participants are involved in the different steps (SPF criterion L). The cases reviewed generally described several of the steps in our 6-step model (Fig. 1), but there was an unbalanced focus towards the more knowledge-based process steps compared to the more managerial, value-oriented, and operational steps (process preparation, objectivesalternatives-evaluation, and outcome respectively). Furthermore, NGOs, and local community members had relatively low rates of involvement in the value-oriented steps, such as objective setting and evaluation, where one could expect these types of participants to play a key role.

Power and rationality
Power can come in different forms: knowledge, social skills, monetary resources, and formal legal mandates. It is generally accepted that power imbalances exist, and that they have a negative effect on the conduct of a democratic participatory process of rational deliberation (e. g., Flyvbjerg (1998) and Allmendinger (2002)). Power asymmetries can be detrimental to democratic outcomes, but only if we ignore them and allow them to work against transparency and rationality (SPF criterion N; Arnstein, 2019;original publ. 1969;Flyvbjerg, 2002). Deliberationwhere the various voices of the affected actors will interact and transform, rather than simply be aggregated in an unchallenged manneris actually one way to account for and handle power imbalances (Roberts, 2004). Participatory processes are often tightly related to the idea of consensusidentification of common interests on which all involved agree. Because of power imbalances, however, critics warn that social groups or ideological orientations might be systematically excluded from decision-making if it is based on such a consensus ideal (McGuirk, 2001). Therefore, consensus-based processes specifically need to include measures that ensure that this does not happen (criterion O). One way is to design the process as compromise seeking, rather than consensus building (Zellner et al., 2020).
We found that half of the reviewed cases did not report any activities to manage power imbalances, such as separated or structured meetings, documentation, anonymous involvement, or clarification of roles and mandates (Fig. 6). This is troublesome because it will favor the solutions brought forward by those with the greatest power rather than solutions that are democratically defined. Various methods and tools used in the reviewed PM cases, however, have a potential to support management of power, e.g., by providing structure, clarity and transparence to the process, by learning, by documentation of a discussion and the reasons behind a decision, and by enabling electronic voting and communication.
Furthermore, several methods and tools can also clarify how alternative measures/decisions/plans specifically affect the values included in the process. Good examples are GIS, multi-criteria analysis and costbenefit analysis (BenDor and Scheffran, 2019). Such tools can support rational argumentation based on both knowledge and values, which is a fundamental feature of a democratic participatory process (SPF criterion E). Out of the cases that include the planning step of evaluation of alternatives, 64% used tools to show explicitly how the included values were affected by the main alternatives considered in the process. This is an auspicious result, showing that knowledge and tools related to this issue are well established in the PM field. We must continue to raise awareness within the PM community that the values of concerned actors, just as much as their knowledge, need to be systematically integrated in the PM process.
To summarize, the potential within the field is strong, but the issues of representation and power need to attain a much greater focus before PM processes can be commonly regarded as democratic.

Integration across organizations: PM in the governance system (SPF criteria F-I, L)
Planning in the context of complex socio-environmental issues is generally characterized by equally complex organizational and structural settings (Dietz et al., 2003) This includes organizations of different typespublic authorities, companies, NGOs, and stakeholder networks operating at different geographic scales ranging from multi-national to highly localized. These organizations have different roles, relationships, mandates, responsibilities, and powers, and typically, none have the full capacity or authority to unilaterally manage the problem at hand (Glaas et al., 2010). This is a fundamental reason why an effective planning process needs to be integrated across the main organizations related to the issue at hand, and embedded in its planning context (Dietz et al., 2003). On this important dimension of participatory planning, we find that PM falls short of its promise.

The planning context: PM disconnected
An important part of the planning context of a PM case is formal, including legislation, authorities, binding decision-making mandates, and ongoing planning and decision-making processes that relate to the PM issue at hand in various ways. It generally includes multiple administrative scales and sectors. The need to understand and handle these complex institutional and organizational settings of cross-sectorial issues relates to the fact that different actors have different formal mandates and capacities (Glaas et al., 2010;Prager, 2010). For example, a central authority may be formally responsible for developing a plan but may not have a mandate to implement it at the local level, where implementation depends on decisions made by local governments and the general public, and on investments and 'know-how' from private organizations. A basis for the successful management of cross-sectoral issues is, therefore, to firstly generate a broad understanding of these interdependencies and of the structural prerequisites of the planning process among the involved organizations, and secondly, to manage this in the planning process (SPF criterion G).
How does the PM process relate to the representative democratic system around it? What is the formal role and mandate of the process leader? What can actually be managed by the participatory process? And, hence, how should the objective of the process be formulated to effectively adjust to that? What other processes and decisions need the process be coordinated to address, and who needs to be involved to that end? Our results on this topic are alarming, because only 15% of the cases reported satisfactory approaches to the issue of crossorganizational integration, and almost 60% did not report addressing it at all. Furthermore, a large portion of the papers did not report who the project leader was and who initiated the process; only 5% of the cases reported explicit communication around roles and mandates having taken place as part of the process (Fig. 6). This indicates a lack of understanding of the importance of clarity around formal roles and mandates of the participants-what mandates do the initiator and leader really have? What are the reasons for establishing the process? What will the output of the process really mean when it comes to implementation?
Most PM cases (70%) took place at the regional scale, a scale that commonly requires coordination with both larger and smaller scales. Only one study, however, reported an explicit strategy to connect scales in the planning process. This lack of connection is troubling. For example, a PM process at a regional scale might establish a plan that disrupts the implementation of a local decision made by democratically elected politicians. Clarity of roles and processes are critical with regard to decision implementation, to the socio-ecological problems at stake, to the local representative democracy, and to the purpose of the regional PM process.
Within political science, these kinds of problemsrelated to the fuzziness of regions as political entities and the need of many issues to be coordinated at the regional scalehave even been termed the "regional mess" (Allen and Cochrane, 2007;McCallion, 2008). We believe that PM has a great potential to help manage this so-called mess through the use of tools and methods that can clarify complex system behaviors and aid the establishment of well-founded strategies and plans. However, to make this happen, and to prevent PM from actually adding to the mess, PM processes must have a strong self-awareness, and must be understood by, and embedded within, the surrounding governance system.
Other results further underline this point; almost 90% of the cases do not present planning outcomes that are recognized in any decisions or actions outside the scope of the PM processes. Moreover, we did not find a single case that addressed climate change mitigation or food availability in our randomized case pool. We hypothesize that one reason may be that these areas clearly require connection among global to local scales and actors, and that PM practitioners are not yet comfortable or knowledgeable about how to manage those connections. Considering the inherent potential of PM to connect scales and actors, we urgently need to develop our tools and procedures to handle this difficulty.

Why participate?
As described above, a foundational reason to integrate a PM process with its planning context is to involve all the actors that together own the capacity to handle the problem at stake. Once these actors are defined, however, there are still several barriers that can disrupt their collaboration. To overcome these, the issue of incentives, including resources and efficiency, is fundamental (H; Ansell and Gash, 2008). Why do participants get involved? What thresholds do they need to pass? Do authority, mandates, legislation, budgets and schedules make participation feasible? How can the process be set up to increase its efficiency and to decrease the time and money required to participate? What alternative set-ups exist, and what are their pros and cons in respect of resources and efficiency?
Incentives and resources cuts through many focal issues of participatory planning, such as knowledge integration and learning, values and democracy, and collaboration and coordination. If there are no incentives, there will be no participation. Despite this, the issue of resources is the most neglected one in our review: 90% of the cases did not report anything related to it. The few cases that did address resources, did so in a limited manner that focused on time and did not provide any grounds for cross-case comparison of methods or process designs.
Furthermore, we know from our review that participation is spread over many steps in the processes (Fig. 5). The question is therefore, have the resources (participants' time and engagement) been spent efficiently and effectively, or will the experience of the process rather make future engagement unlikely because the participants found it too costly? Generally, participation needs to be focused to the phases of the planning process where it is the most useful given its objectives (SPF criterion L). The initial stages of a process generally have a greater influence on the process outcome than later stages, and involving the participants in the early steps, such as process set-up and formulation of process objectives, is also a good way to create transparency and a sense of ownership (e.g., SPF criteria I, N and O). Different groups of participants may need to be involved for different purposes and hence at different phases. Because this issue is fundamental for implementing PM on a broad scale, further studies are needed to investigate the state of knowledge here in more detail.
Incentives can also come in less tangible forms and be connected to so-called human aspects, such as trust, engagement, and conflict management (SPF criterion I). These pyscho-social aspects depend on deeprooted behaviors, power relationships, organizational cultures, history, and more, and their importance for establishing a successful participatory process where people share their knowledge and perspectives is often underestimated. Trust is fundamental for making collaboration happen, and "social carrots" are needed to make it work well (Zellner et al., 2012a). For example, in order for people to make room in their already stressful working schedules, it is key that participating persons need to feel warmly welcomed to meetings, that their ideas and opinions are received with respect, that initiatives are encouraged, that critical views are allowed, among others. Another key is to establish a sense of ownership, as when it is made clear that all of the involved actors are valuable in terms of the overall capacity to manage the issue at hand (Reed, 2008). Furthermore, sometimes personal conflicts or controversial issues obstruct productive and evidence-based reasoning and deliberation (Weiss and Hughes, 2005). To manage all of this, competent process leadership, including preparedness to handle conflicts, is fundamental, as are measures such as process transparence, managing power imbalances, and setting up ground rules for participant interaction, etc. (Milz, 2018;Müller-Seitz, 2012). Our results show that, compared to the more tangible incentives (SPF criterion H), human aspects (SPF criterion I) are better managed. There is still much left to hope for however, because more than half of the cases do not report any activities to address this issue (Fig. 4).

Learning only for those directly involved
As we have discussed, learning is widely seen as key in participatory planning (SPF criterion P). We have also seen that within the PM field, most of the cases apply methods and tools that aim to support stakeholder learning ( Figure B15). When it comes to supporting integration across organizations, a vital aspect is the need to establish learning structures within the participating actor groups and organizations (Glaas et al., 2010). A PM process can support this by making time to discuss it with the participants and suggesting alternative ways of establishing learning structures; for example, connecting to organization and meeting routines at the participants' home organizations, establishing ways to share data, and creating shared communication platforms. A lack of such structures can result in an unsufficiently used project report, with some participants having learned something, but without real connections being formed between the process and the collaborating organizations and actor groups. The collaboration, including its learning processes, then becomes a 'bubble' outside of the organizations and groups that they should represent, which will, at best, work as long as the concerned individuals stay with their organization (Rashman et al., 2009). Because consultants or short-term employment positions are common in many organizations, this issue remains an important obstacle. For integrated planning to be effective on a long-term basis, the institutionalization and long-term continuance of learninglearning being 'built into' the participating organizationsis key (see SPF criterion F). Unfortunately, understanding the importance of supporting organizational learning in the process seem far from universal in the PM field; none of the cases reported efforts to establish any type of structures to support learning between process participants and the respective organizations that they represent.

Conclusions and key research issues
PM has great potential for supporting planning and decision-making processes in the governance of complex socio-environmental systems. Such governance urgently needs innovative and efficient participatory processes that can be implemented in the real world.
Overall, however, our assessment suggests that significant work remains for PM to be fully effective in supporting participatory planning. While the papers we reviewed indicated that environmental PM is very effective at promoting knowledge integration and learning among participants, our case studies also handled 11 out of 15 SPF criteria poorly or very poorly (Table 3). Judging by their presentations, these studies often fell short of facilitating a multi-value perspective within a democratic process, and in integrating across organizations within a governance system. A main underlying reason may be that the studies systematically and purposely leave certain aspects of their PM interventions out of the papers describing their work. 7 To implement PM within planning and decision-making in the real world, however, including the whole range of SPF criteria is a high, but important, standard to meet. To establish truly participatory and integrative processes for implementation, future study designs and research reports need to adjust to reflect that end. New forms of funding, of building structures within and between research institutions and practice, and of publishing scientific work can support this development. Therefore, several research questions are of key importance, and need prompt investments and engagement to pursue the field's potential.
For questions of knowledge integration and learning, the PM field can already make important contributions to participatory planning and decision-making, especially when it comes to approaches to knowledge identification and tools that integrate knowledge. Approaches for managing uncertainty, and methods and tools that support learning processes among the involved stakeholders, are already well developed. 7 The most common objective among our PM case studies, however, was the development of a PM process framework (43%), which implies that many of the studies had a broad focus.
However, some vital improvements are still needed, especially related to the question of whose knowledge is represented: • Are there some types of knowledge that are generally included while other types are excluded? For example, is expert knowledge more likely to be included than lay knowledge, or is knowledge in the form of data and numbers more likely to be included than what exists in text and other non-numeric forms? How can procedures for participant identification and selection be developed and improved to ensure that all the main pieces of knowledge are included?
Questions of values and democracy have been given too little attention in the practice of PM. While there are a variety of methods and tools within the field that could make important strides in addressing these aspects of planning processes, most PM processes cannot currently be set as a standard for value-based and democratic participatory processes. Efforts to address several research questions could help close that gap, which include: • What types of values are currently included in PM processes? Are there patterns of value types that are commonly included or excluded (intentionally or otherwise)? For example, are values that are more difficult to capture by available tools simply left unacknowledged?
Are the values of participants who are less process-oriented, less analytical, or simply less well-organized left out? Are certain types of actors systematically over-or under-represented in PM efforts? • Do we need new procedures, methods, or tools for value identification, selection of participants, and management of power? What knowledge, tools and approaches for expanding these aspects of PM can be integrated from other disciplines and fields (e.g., public administration, social work, urban planning, operational research, and multi-criteria decision analysis)? Valuable insights into such methods and tools fields are provided by for example: Huang et al. Lastly, there remain important questions of organizational integration and governance; the understanding that participatory processes need to be understood by, and connected to, their surrounding governance system is not well established in the PM field. Without advancements in this area, we believe PM faces significant limits in practice. Addressing this problem requires prompt and extensive research efforts that confront broad questions, including: • What procedures need to be added to PM processes to ensure that they are sufficiently coordinated with the planning and decisionmaking context that surrounds it? • How can PM processes be developed that incentivize participation among all relevant actors? How can efficient and effective PM processes be designed? For example, at what planning stages should different types of participants most efficiently and effectively be involved? What are the participatory costs and benefits related to alternative PM process designs? • What structures and institutions could facilitate accurate spread of knowledge created in PM process beyond those actors who were directly engaged?
By this study, we hope to inspire and support PM leaders and practitioners in their planning and reflection upon past, future, and current PM processes. We also highlight important questions that we hope will guide our field into the future. To vest our support, we have adapted our review instrument to facilitate practitioner evaluation of their own PM processes on the collaborative PM website: https://participatorymod eling.org/pm-for-participatory-planning-and-decision-making-a-rev iew-tool/. The instrument may also support design and documentation of PM studies.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.