Introduction

Climate change poses increasingly complex problems for resource managers and policymakers that require balancing scientific information, local needs and values, and the role of local knowledge in decision-making. The traditional science production model, called “Mode 1” (Nowotny et al. 2003), has been critiqued for failing to inform environmental management and decision making in an effective way, because it consciously separates science, policy, and society in ways that inhibit the use of science for problem solving (Gibbons et al. 1994; Jasanoff and Wynne 1998; Latour 1998; Jasanoff 2009). The critiques have led to the acknowledgment of the connections between science and society (Jasanoff and Wynne 1998), and encouragement of deliberate links between the two (Jasanoff and Wynne 1998; McNie 2007; Lovbrand 2011).

The critical need to increase the usability of science for management and decision-making (Dilling and Lemos 2011; Lemos et al. 2012) has led to the call for and the emergence of new science-policy models (Kirchhoff et al. 2013). Interest in a co-production approach to knowledge production has sharply increased in climate sciences, climate change adaptation and broadly in environmental management and governance (Lemos and Morehouse 2005; Visbeck 2008; Ziervogel et al. 2016; Wamsler 2017). This appeal is underscored by the fact that co-production of knowledge promises to increase the relevance and usability of science for society (Lemos and Morehouse 2005; Meadow et al. 2015; Wall et al. 2017), while breaking the unidirectional and linear traditional collaboration and participation structures (Wamsler 2017).

Our research focuses on the use of co-production of knowledge in climate and environmental management. Co-production of knowledge in this context refers to the contribution of multiple knowledge sources and capacities from different stakeholders spanning the science–policy–society interface with the goal of co-creating knowledge and information to inform environmental decision-making (Lemos and Morehouse 2005). We use nine published case studies to explore and elucidate factors within collaborative research projects that either support of inhibit the co-production of knowledge.

Literature review

The growing body of research on co-production of knowledge suggests that the driving force behind the success of the approach is the direct connection and collaboration between researchers and practitioners (whether policymakers or on-the-ground resource managers). The degree of interaction between the groups has been linked, in many contexts, to the generation of more usable knowledge and tangible project outputs (Howarth and Monasterolo 2017; Kirchhoff et al. 2013; Lemos and Morehouse 2005; Lemos et al. 2012; Reed et al. 2014; Roux et al. 2010; Walter et al. 2007). However, researchers and practitioners continue to struggle with the complexities involved with collaborative, multi-party, and transdisciplinary research (Kirchhoff et al. 2013), with some recently describing the approach as vague and ambiguous in practice and lacking empirical implementation strategies (Thompson et al. 2017).

Previous research has disentangled several principles that support greater collaboration between researchers and practitioners and contribute to the co-production of knowledge. Lemos and Morehouse (2005) noted that successful co-production processes involved stakeholders in multiple stages of research, including definition of the problem, development of the research question, research design, data collection, data analysis, meaning making, testing results, and dissemination of knowledge. Hegger and Dieperink (2014) and Hegger et al. (2012) identify seven “success conditions” for co-production of knowledge, including who is involved, achieving shared understanding of project goals, and how project responsibilities are divided-up. Polk (2015) used the findings from a comparison of five transdisciplinary research projects to identify several key components in the research processes, including co-leadership of the project, stakeholders participating in analysis, democratic meeting methods, use of communication and implementation plans, and joint analysis of data.

Early work on the usability of social science research identified practices such as two-way communication and iterativity, appropriate dissemination of research to decision-makers (not just through peer-reviewed journals), and consultation and involvement of end-users throughout the project as key components in successful applied research (Glaser and Taylor 1973). Different facets of stakeholder engagement in collaborative work and knowledge co-production have been explored. Evely et al. (2010) directly linked more stakeholder engagement to use of research findings outside of the peer-reviewed literature, where they are more likely to be accessible to decision-makers. Wamsler (2017) demonstrated that the types of stakeholders engaged influence the outcomes of the process and went further to highlight that the power constellation and the broader governance context shape the stakeholder engagement process. Sherman and Ford (2014) recommended careful consideration of the inclusion and detail of stakeholder engagement strategies in climate change adaptation projects.

Another factor associated with successful production of useful and usable science is ensuring a good fit between the research questions and the information needs of the organization(s) involved (Feldman and Ingram 2009; Landry et al. 2001; Lemos et al. 2012). In addition, Ford et al. (2013) note the importance of forethought in planning collaborative projects, so that the elements noted above are integrated into the project from the beginning. Reed et al. (2014) similarly stress the importance of designing projects specifically for co-production of knowledge. Dilling and Lemos (2011) point to the need, often, to have specific actors or organizations design and facilitate co-production processes because of the complexities involved.

Even when collaborative knowledge production projects are well designed, some challenges seem to be inherent in the process. Cvitanovic et al. (2015a, 2015b) identify inadequate measures of science impact that do not account for engagement activities, a lack of organizational support for engagement activities, insufficient time to conduct engagement activities in addition to other responsibilities, and a lack of funding to support engagement activities as barriers to knowledge co-production between scientists and resource managers. Polk (2015) noted that even in a center devoted to transdisciplinary research, participants struggled with barriers related to the time required for collaborative research and mis-matches in expectations between researchers and practitioners.

Having identified a number of strategies and principles to support co-production of knowledge in the context of climate change and environmental management, the challenge becomes how to overcome the above-identified barriers in ways that make the collaborative process more effective for all participants. The lack of a standardized or precise empirical framework for co-production of knowledge means that a wide range of practices, attitudes, and expectations are often applied, which according to Thompson et al. (2017) could undermine the advancement of the approach.

Research questions and objectives

In this paper, we ask how complex collaborative climate change and environmental management research projects were carried out from beginning to end; what challenges the teams faced and how they overcame them; and how successful the projects were in producing information usable by decision-makers. To achieve that, we review empirical examples from a set of nine international case studies of collaborative work to identify how researchers and practitioners designed and implemented projects aimed at the co-production of knowledge. We explore—based on lessons we gleaned—how these research teams’ experiences can provide a more fine-grained understanding of co-production of knowledge in practice, and ultimately inform future work involving the co-production of usable climate and environmental management knowledge.

Our results identify several common themes regarding specific components and actions (we think of these as the modus operandi of co-production) across the cases. The set of case studies highlight issues such as the integration of social and professional cultures, gender and social equity, and power dynamics and provide examples of how researchers have addressed these issues. The analytical understanding of the co-production modus operandi—the empirical process of successfully co-producing knowledge that is useful and usable— provided in this paper contributes to the advancement of the applied research model of co-production of knowledge.

Methods

We used the Grounded Theory Literature Review method (GTLR) (Strauss and Corbin 1990, 1998; Glaser and Strauss 1967) to analyze a sample of papers that addressed the operational processes involved in undertaking the co-production of knowledge in the context of climate change and environmental management decisions. GTLR works on the assumption that an accurate review of cases will yield a grounded theory that carries new insights (Wolfswinkel et al. 2013). In this review, we present the first four iterative steps in GTLR—Define, Search, Select, and Analyze—in the Methodology section and the fifth step—Present—in the Results and discussion sections.

We sought an international (including North America) perspective on practices contributing to the successful co-production of knowledge within the realm of climate science and environmental management. We identified our specific domain of interest as studies exploring how climate-related science as well as agricultural and/or environmental management sciences have been implemented through collaborative research approaches.

To identify case studies in our defined domain, we used a two-stage literature search. First, we used a process analogous to “snowball sampling” in which we started with a sample of known, relevant papers broadly cited in other publications on co-production of knowledge (Wall et al. 2017; Meadow et al. 2015; Polk 2015; Fazey et al. 2014; Phillipson et al. 2012). We then used Google Scholar to identify (1) new papers within our domain of interest that cited these and (2) papers cited by our key papers that fit our domain. This step yielded 14 papers.

Our second-stage involved using Web of Science to identify additional articles using the search terms: TITLE: (co-production* knowledge) OR TITLE: (Knowledge* exchange) OR TITLE: (Interdisciplinar* transdisciplinar*) AND TOPIC: (Climate change) AND TOPIC: (Agricultural* sciences) AND TOPIC: (Environmental* sciences) AND TOPIC: (Environmental* management). This second search resulted in 690 papers. We further refined the search to focus on our domain and identified 34 papers (including some of the papers from stage one). The search was done between September 2016 and January 2017.

From the sample of 34 papers, we selected only those that provided detailed descriptions of the research teams’ collaborative research methods, such as the nature of the research team and its formation; the characteristics of the geographical and the institutional environments of the cases; the specific approaches to stakeholder engagement and communication. We also looked for descriptions of the challenges the research teams encountered during the process and reflections on skills and capacities used or needed. We also selected papers based on their descriptions of project outputs and their management and any feedback they received from their stakeholder partners regarding project successes or failures. Ultimately, we identified nine (09) papers that addressed these elements. Our selected articles provide examples from research projects in 11 different geographical settings. The research focused on a number of different management sectors including marine resources, forestry, water resources, agriculture, and pollution remediation. The selected papers provide descriptions of a range of collaborative research approaches such as participant observation, qualitative interviewing, focus groups, household surveys, archival analysis, stakeholder workshops, and participatory or joint experimentation (Table 1). More detailed descriptions of the selected of papers, including their geographic coverage, authors, themes, timeframe, and summary, is presented in the appendix.

Table 1 Description of the case studies included in our research
Full size table

We applied a two-stage content analysis to the sample papers. First, we used an open coding approach (Charmaz 2014) to identify common themes across the nine cases that either supported or inhibited collaboration or co-production of knowledge. The unrestricted coding analysis consisted of highlighting, for each paper, relevant findings, with a focus on the tangible methods and techniques, the authors applied to their collaborative efforts to co-produce knowledge. These include the institutional factors; professional cultural differences; logistics factors; core proficiency and skills; power dynamics, equity, and inclusivity aspects; core activities that made up the process; and elements of output management and dissemination. We identified these variables as representing a package of the “how” to conduct knowledge co-production work.

After identifying these broad themes, in stage two, we placed specific examples from each paper into a theme category in order to gauge the relative rate-of-use of each of them across all the cases. We used an iterative process and two coders to bin the excerpts into the previously identified broad categories. We iterated on categorization until both coders agreed. We then used cross-coding (Wolfswinkel et al. 2013) to explore the links between the different variables and excerpts from the coding exercise.

Results

We placed the identified elements, principles, and processes associated with co-production of knowledge from our nine case studies into a logic model framework (Inputs, Process, Outputs, Outcomes–Impacts, and Context (CDC 2004)) (Fig. 1) to demonstrate how and where each variable falls within a project management structure. The use of the logic model for presenting and organizing our results is to help investigators/managers of co-production work project to think through each step of their project cycle for a better planning of their activities.

Fig. 1
figure 1

Snapshot of the variables of knowledge co-production work

Full size image

The variables discussed below are not exhaustive, but they highlight some of the elements that contributed (or hindered) successful co-production in the nine cases we examined.

Context factors

By context, we mean the environment in which the project takes place and other factors often beyond the control of researchers (CDC 2004). In this analysis, we include as contextual factors: institutional attributes, professional cultural differences between the actors involved, and logistical factors involved in coordinating the project.

Institutional factors

Several institutional characteristics (of management agencies, funding agencies, and academic institutions) were common among the cases including management structures, funding mechanisms, and level of support for collaborative research activities.

In two cases, inflexible management structures within the resource management agencies hindered practitioners’ ability to collaborate with the research teams. Procedural requirements in Foley et al. (2017) limited information sharing among project participants. Rules restricting field time kept practitioners from collaborating directly with the research team in the case of Cvitanovic et al. (2016). However, more flexible structures within a sponsoring agency—particularly regarding funding—supported the work of Castellanos et al. (2013) when the research team recognized that they lacked necessary skills in collaborative research. Flexible funding allowed the research team to bring on the skill set they required through a new hire. “[W]e were fortunate to have a funding agency that emphasizes the communication of scientific findings to policymakers and therefore encouraged us to incorporate new approaches and communication experts into the research process; but we recognize that not all funded projects have this opportunity” (Castellanos et al. 2013: 25).

Institutional support can also include providing education and professional development training on knowledge co-production, as described in Cvitanovic et al. (2016: 6), “[…] we need to teach scientists how to gain access to the right people and gain credibility with those people.” Further examples of how institutions can support co-production efforts come from Cvitanovic et al. (2016) and Podesta et al. (2013), who both noted the need for research institutions to formally recognize and promote activities relevant for knowledge co-production in job descriptions and integrate incentives and rewards for high performance in staff performance appraisal.

Cultural differences

The cultural norms of an organization or institution help to determine how it prioritizes resources and time, which can affect how it perceives the process of collaborative research. Cvitanovic et al. (2016) and Campbell et al. (2016) each were challenged to reconcile different institutional cultures. Cvitanovic et al. (2016) reported that the efforts and amount of time the researchers in their case study needed to devote to academic outputs and fund raising seemed to reduce the amount of time they had available to properly engage managers and communicate their research findings. In addition, the limited planning time that managers usually face, because of their day-to-day responsibilities, prevented them from “staying abreast of the science,” and contributing to generating new knowledge. Campbell et al. (2016: 1275) point to the need for managers to have “patience to work with researchers, who often operate at slower pace than the customary management timeline.” Campbell et al. (2016) also note that, in one case, building into the project informal “hanging out” opportunities for researchers and managers translated into greater recognition and understanding between the two groups.

Logistic factors

Logistical factors flagged in two of the case studies were the issue of geographic distance between researchers and management partners, which can constrain meetings and increase project costs. Cvitanovic et al. (2016) struggled with conducting research in a remote location that lacked internet access and was difficult to reach. The research team reflected that the remoteness prevented managers from sharing resources with scientists and having access to research outputs. Facing a similar challenge, Podesta et al. (2013) utilized information and communication technologies when possible, but still recommended face-to-face meetings to increase active learning.

Co-production inputs

We define inputs as resources that go into a project, such as funds, time, and expertise (both subject matter and expertise in collaborative processes). While adequate financial support is an important input that needs to be deployed in support of co-production work (Cvitanovic et al. 2016), in this section we focus on a set of intangible inputs: core proficiencies of researchers and practitioners as well as factors that support the embedding of trust, inclusivity, and legitimacy at the start of a project.

Proficiency and expertise for knowledge co-production

Five of the case studies identified intrinsic aptitudes, individual proficiencies, and expertise that support successful design and implementation of a research program focused on co-production. A stakeholder in the Cvitanovic et al. (2016: 5) case explained: “everyone involved in the program has to have the ability to talk to people, to be friendly, to be approachable, and to be able to speak in plain English and not just science.” Cvitanovic et al. (2016), Campbell et al. (2016), Kraaijvanger et al. (2016), and Akpo et al. (2015) each provide examples of the ways in which communication, engagement, and facilitation skills among the researchers were crucial in the process of collaboration. Castellanos et al. (2013: 26) provided an example of how lack of these proficiencies hindered their work at the outset, “As typical of many researchers, we lacked experience communicating research results to the general public and policy makers.” As a remedy, the research team needed to add the capacity through hiring an external group to facilitate communication with policymakers and farmers.

Possessing a working knowledge of the decision context of one’s partners in co-production is another key proficiency. Two papers provided examples of the benefits of having resource managers who are comfortable working within a Western science framework (Cvitanovic et al. 2016; Campbell et al. 2016). For example, Campbell et al. (2016) described situations in which managers asked questions in ways that were answerable with scientific studies in one case, which they noted eased the process of producing research knowledge affecting management practices. Researchers having an understanding of local knowledge is also a crucial element; we discuss this issue in greater detail in the Process section.

Legitimacy and trust

Cash et al. (2002: 2) defined legitimacy as “how fair an information producing process is and whether it considers appropriate values, concerns, and perspectives of different actors.” While legitimacy is often considered part of the process of co-production, we include it in Inputs because its pre-cursors—trust and relationship-building—may need greater attention at the start of a project, according to five of the case studies.

An example of the role of trust and relationship-building as an Input comes from Foley et al. (2017), who describe the process of attempting co-production work in an area where past research was unsuccessful and damaging to the community, sowing mistrust between local residents and outside researchers. While trust building takes time (Podesta et al. 2013; Castellano et al. 2013), examples of ways to overcome history of mistrust and/or building trust are provided in the cases illustrated in Campbell et al. (2016), including harnessing pre-existing personal and professional relationships, spending time with communities as a participant observer, or participating in community activities. Castellano et al. (2013) established structured feedback mechanisms and created opportunities for open discussion. Akpo et al. (2015) used research site selection as an opportunity to demonstrate how transparency built trust with the local research participants. In that case, it was the local learning group, not researchers, who identified the three candidate sites and then came to consensus about the final site selection.

Inclusivity

Three cases addressed the issue of inclusivity, including gender issues, in detail. Kraaijvanger et al. (2016) observed that gender representation affects the involvement of participants and gender diversity supports ideas and knowledge sharing among farmers. In Castellanos et al. (2013), the research team recognized the need for inclusivity, but were challenged to overcome language and culture barriers, such as cultural norms that dissuaded women from speaking in the presence of men and a language barrier between researchers and local participants. Akpo et al. (2015: 374) addressed a similar situation and reported, “We paid particular attention to the language issues and made sure that enough time was taken to share different ideas and gain mutual understanding. Participants were encouraged to use the local language (Nagot) instead of French, as it is the one all participants understood. The language issue was also concerned with the way participants understand and express the objects being tested.”

Process

In Process, we include the activities conducted and the outputs produced during the research project (see Fig. 1). Our nine case studies revealed a set of four process components or stages: (1) Setting-up, (2) Development and Design, (3) Implementation (including research methods, stakeholder engagement, and communication), and (4) Outputs management and dissemination. These components are not necessarily sequential, however. For example, stakeholders may be involved with research question development before the research team is fully assembled in the setting-up component. Here, we discuss them sequentially for the sake of clarity.

Setting-up component

Setting-up is discussed in three papers as the moment when the work of building the research team begins. First, it is important to ensure that the research team possesses the appropriate expertise and experience to carry out the work (as discussed in the Inputs section). Kirono et al. (2014: 360) describe circling back to reconsider team composition after defining the research questions: “The process of assembling the research team was driven by the range of tasks and approaches envisioned during project development.” Podesta et al. (2013) recruited a new team member after the research question had been defined because a proposal reviewer noted they lacked expertise in decision-making under uncertainty.

Experience with collaborative and participatory research was also considered in team composition in several cases. Podesta et al. (2013: 40) stressed the importance of “recruiting [a disciplinarily diverse team of] investigators with an open attitude toward interdisciplinary interaction.” Castellanos et al. (2013) suggest using the Setting-up time to foster a common language and shared understanding among the research team in order to create an environment that favors the interdisciplinary collaboration that supports co-production work.

Development and design component

Five of the case studies identify a Development and Design component in which the project is designed collaboratively in order to incorporate the different values, interests, and insights of all the identified relevant stakeholders as well as the different scientists (Cvitanovic et al. 2016; Akpo et al. 2015; Kirono et al. 2014; Podesta et al. 2013; Castellanos et al. 2013).

Other aspects of this component include co-conceptualization, co-definition of methods, and co-planning. Three cases illustrated how co-developing research questions aligned the research with stakeholder priorities. Akpo et al. (2015: 372) conducted preliminary field visits in order to explore stakeholders’ perceptions about the seedlings the research team had proposed to focus on. Kirono et al. (2014: 360) needed to redefine their question “[a]s a result of the stakeholder consultation… to focus more strongly on climate change impacts on regional surface water resources and on adaptation options.” Cvitanovic et al. (2016: 4) illustrated what can happen when co-development does not occur. Some participants in that study remarked that “Some of the research that had been done really wasn’t what was needed … it was what the researchers wanted rather than the management agencies […] you have to have all of the different agencies and end-users, including traditional owners, at the table prioritizing what needs to be done.”

Four papers discuss ways to co-define research methods, instruments, and analytical frameworks (Akpo et al. 2015; Kirono et al. 2014; Castellano et al. 2013; Podesta et al. 2013). For example, Akpo et al. (2015) worked with all the practitioners involved to collectively agree to use the farmers’ criteria to measure successful plant nursery practices; reasoning that farmers are the people who ultimately will purchase the seedlings. Similarly, Podesta et al. (2013) and Kirono et al. (2014) incorporated stakeholders’ inputs in their co-modeling. Stakeholders’ regular reviews of an agent-based model of agricultural production was necessary to enhance model transparency, and to ensure “face validity” of concepts and features (Podesta et al. 2013).

Finally, two papers note the need to integrate the stakeholders into the project planning efforts such as managing financial decisions. Podesta et al. (2013) reflect that an equitable allocation of budget and resources may emerge from a process involving all project participants during the project design stage, while participants in Cvitanovic et al. (2016) attributed the perceived lack of meaningful engagement and collaborative activities to poor planning of knowledge co-production strategies during the program’s development.

Implementation component

The implementation phase is what we often consider the heart of the co-production processes because it is where the research team and stakeholders most actively collaborate to undertake the research. Three key aspects of implementation are the consistent use of engagement activities, appropriate communication strategies, and integration of local knowledge to increase usefulness. The cases also provided examples of specific research methods useful in collaborative research.

Engagement between researchers and stakeholders is the foundation of co-producing knowledge. Young et al. (2016) explain that stakeholders seek different information depending on their position (social, economic, employment, gender, among others), and this affects the manner in which they prefer to engage with researchers, which is reflected in examples from the case studies. Five of the cases provided examples of working within existing community structures to maximize outreach and engagement (Cvitanovic et al. 2016; Kraaijvanger et al. 2016; Foley et al. 2017; Akpo et al. 2015; Kirono et al. 2014). This approach included identifying the local “champions and leaders”—locally influential people (Kirono et al. 2014)—who could identify and make connections with the most relevant local participants. Although starting with local leaders can be effective, three of the cases (Kraaijvanger et al. 2016; Akpo et al. 2015; Foley et al. 2017) discuss the importance of democratizing engagement, or being aware of the ways in which local power structures can inhibit diverse participation. Akpo et al. (2015: 373–374) “made sure that all participants expressed their opinions on the ongoing activities. We [They] intervened in such way that the process was democratic and not dominated by any single stakeholder. We [They] encouraged all participants, particularly illiterate farmers, to speak out their mind…” Foley et al. (2017) encountered power asymmetries among the stakeholder groups, including uneven distribution of knowledge, resources, and decision-making power, and reflected that failure to overcome them can lessen the quality of the engagement.

Some cases directly address the role expertise bias (on the part of the researchers) may play in shutting down engagement because participants may feel that they lack that particular type of knowledge (Kraaijvanger et al. 2016; Akpo et al. 2015; Foley et al. 2017). Kraaijvanger et al. (2016) and Akpo et al. (2015) suggest that scientists should often step back, act mostly as observers when the stakeholders are active, and focus on facilitating participatory activities. One way to address expertise bias, as well as ensure the research is useful to local practitioners, is to integrate diverse types of knowledge into the research (Kraaijvanger et al. 2016; Akpo et al. 2015; Castellanos et al. 2013). For example, Akpo et al. (2015) recognized the need to focus on the actual seedling production practices of the local nursery holders, rather than research-recommended practices, because the local methods were more useful to local producers.

Communication is often only considered as a tool to disseminate research results (as discussed in the Output management and dissemination component below). However, seven of the case studies point to its role in strengthening stakeholder engagement (Cvitanovic et al. 2016; Young et al. 2016; Foley et al. 2017; Akpo et al. 2015; Kirono et al. 2014; Castellanos et al. 2013; Podesta et al. 2013). Effective communication determines the quality of facilitation, mediation, and negotiation approaches. Effective communication includes addressing language gaps between and among the stakeholders and researchers, which required the use of interpreters, knowledge brokers, or boundary organizations in four of the case studies (Cvitanovic et al. 2016; Young et al. 2016; Podesta et al. 2013; Castellanos et al. 2013). Two cases used alternate education tools, including drawing and visual representation (Young et al. 2016; Akpo et al. 2015). For example, Akpo et al. (2015) translated their experimentation protocol into drawings based on signs and symbols familiar to stakeholders, which allowed them to more easily follow the experimentation requirements in their plots. The research team also may encounter communication challenges among themselves, such as disciplinary language differences, as discussed in the setting-up component above. Castellanos et al. (2013: 23) provided an illustration from their project; “Although the social scientists knew the theoretical approach, natural scientists invited to participate in the team had to learn the terminology and theoretical framework on vulnerability and livelihoods,” and then discussed the challenges associated with finding a common language and integrating the interdisciplinary team.

Collaborative research methods were common across the cases. Here we highlight several specific examples of approaches to collaborative research. Podesta et al. (2013) used participatory modeling to develop an agent-based agricultural model. Cvitanovic et al. (2016) used citizen science approaches as a way to include decision-makers in the research and promote a sense of ownership of the research. Kraaijvanger et al. (2016) and Akpo et al. (2015) used participatory experimentation to ensure that researchers were not the sole leaders of the project. Castellanos et al. (2013) used a suite of methods, underscoring their interdisciplinary approach: (1) qualitative consultations and interviews with key informants, (2) a household-level survey, (3) remote sensing analysis, (4) community engagement and participatory workshops to consult with farmers on the findings, and (5) participatory confirmatory analysis to ensure the findings were valid. Akpo et al. (2015) used a series of restitution workshops in which feedback on the progress of the work was provided to keep participants updated on the research project.

An overarching theme among the cases was the need to be flexible about the ways in which projects are implemented. For example when stakeholders working with Podesta et al. (2013: 44) felt frustrated by the time required in meetings to keep all participants abreast of project developments, the research team “replace[d] extended plenary meetings with short, tailored updates to individual investigators or groups by the project coordinator.”

Outputs management and dissemination component

Co-production of knowledge does not end with the engagement and communication activities, according to three of our case studies (Cvitanovic et al. 2016; Young et al. 2016; Castellanos et al. 2013). Rather, the Process stage expands to include managing and communicating the outputs so that the research products are accessible to stakeholders. As Castellanos et al. (2013: 23, 26) explain, this component is often overlooked: “Most academics have little training in how to communicate research results to stakeholders, and usually they do not receive scientific recognition for such effort… Research projects rarely incorporate communication strategies from their inception.”

Outputs that are in formats accessible and available to stakeholders increase the usability and salience of the co-production research results. Cvitanovic et al. (2016) highlighted several elements that may undermine research knowledge dissemination, including outputs that are unclear and/or that fail to clearly articulate the implications of the findings, outputs that are not consolidated and easily findable, or outputs that are not accessible to those people who are in position to use them. Stakeholders in the Cvitanovic et al. (2016) study had specific requests for a searchable, regularly updated, archival database with interactive GIS maps and expressed frustration that those were not available (Cvitanovic et al. 2016).

Several cases point to the importance of making research relevant by translating the science into language commonly used by local stakeholders. Castellanos et al. (2013) used a communication specialist to translate their research outputs. Another technique, used by Young et al. (2016: 177) was story-telling, which “goes beyond substituting jargon with lay terms. Story-telling means using narrative devices such as plots, characterizations, and in-depth descriptions to connect scientific findings with the interests, values, and priorities of potential users.” Castellanos et al. (2013) used a puppet play in a similar fashion. Other examples from the case studies included providing frequent research summaries to key participants (Akpo et al. 2015), identifying and using appropriate dissemination channels (Castellanos et al. 2013), and creating multiple versions of outputs to meet multiple stakeholder needs (Castellanos et al. 2013). Castellanos et al. (2013) ultimately created reports in lay language, a puppet play, used calendars to display key findings, and created radio messages to broadcast to key communities.

Outcomes and Impacts features of co-production

While following best practices in terms of Inputs and Process can help keep a collaborative project on-track, ultimately these projects are usually judged based on Outcomes and Impacts. However, outcomes from a co-production of knowledge process often differ from those of standard research projects and need to be evaluated differently (Jordan et al. 2012). The nine case studies considered here highlight beneficial changes in practice for both researchers and stakeholders as project outcomes. While the cases focused less on measuring actual impacts of their co-production work, several addressed the relevance and salience of the information to local farmers, managers, and/or decision-makers (Cvitanovic et al. 2016; Kraaijvanger et al. 2016; Akpo et al. 2015; Castellanos et al. 2013).

Beneficial changes in practice

Changes in practice relate to both changes in research practices and management practices. Cvitanovic et al. (2016), Kirono et al. (2014), and Podesta et al. (2013) all discuss creating opportunities for researchers (including students) to develop skills relevant to collaborative processes as an important outcome of a co-production process. Cvitanovic et al. (2016) and Akpo et al. (2015) also highlight opportunities for social learning, through development of researchers-practitioners networks, as an intangible benefit to both researchers and stakeholders.

Resource managers and other end-users can also develop beneficial management practices —from intangible to tangible—following their participation in a collaborative research project. Intangible benefit is the extent to which local collaborators are empowered and gain knowledge and new skills through the co-production process. Akpo et al. (2015) noted that the joint experimentation process has increased the participating stakeholders’ use of research processes and curiosity about specific agricultural and environmental management knowledge. With their participatory experimentation, both Kraaijvanger et al. (2016) and Akpo et al. (2015) have observed changes in farmers’ agricultural management-related literacy, network building, organizational skills, monitoring capacity, and farm experimentation skills. Similarly, Castellanos et al. (2013) noted that organizing small group activities in the villages, designed to encourage farmers to educate each other as well as scientists about local knowledge, was effective.

While tangible beneficial change is the ultimate goal of knowledge co-production particularly in the sector of climate change adaptation, such changes can take time and are affected by multiple factors often unrelated to the research project. However, Kraaijvanger et al. (2016) were able to track increased crop productivity and positive financial capital change after their project. Akpo et al. (2015) reported some changes in seedling production practices by nursery holders and new management methods instituted by the participating extension agent.

Cvitanovic et al. (2016) reflect on the need to foster collateral positive impacts (both unplanned and indirect positive impacts). For instance, improvement of tourism sector development and opportunity was an unexpected impact of their co-production work.

Relevance and salience

The intangible, tangible, and collateral benefits reported above from the cases demonstrate how the authors assessed whether their research findings were salient to the end-users. Akpo et al. (2015: 383–384) attributed the observed changes in seedling production and management practices to the research model and process, “[t]he observed changes in practices on the different participants could be explained by their full involvement in the research from the problem identification to the experiment implementation, and monitoring & evaluation.” Caution should, however, be taken in both directly attributing positive changes to a specific research project or in judging a project’s success solely based on tangible changes in practice. In some cases, researchers acknowledged the success of the process and social learning as outcomes rather than an actual impact of practical changes from knowledge co-production. Castellanos et al. (2013: 26) stated that “we can conclude with confidence that our research dissemination strategy was effective for those stakeholders… ultimately our success was in communication rather than knowledge co-production.”

Discussion

The criteria we gleaned from our analysis of the nine case studies help to paint a picture of the modus operandi of knowledge co-production. The elements highlighted reflect the character of knowledge co-production as more than just collaborative research—but its own, complex research form (Kirchhoff et al. 2013). However, as we expand upon below, each of the features, while necessary, are not sufficient to result in a truly co-produced project or product, if taken independently. We discuss some of the most common elements highlighted in the case studies, including Development, Design, and Implementation Components; and Context and Inputs features. We also discuss several other features that have been less noted in knowledge co-production literature, but which have the potential to influence a project focused on the co-production of knowledge, such as Setting-up Component; Outputs management and dissemination Component; and Outcomes–Impacts features. Finally, we emphasize elements that are worth considering although they were context-dependent and thus not widely discussed in the cases, such as Professional cultural differences, inclusivity, and logistic factors. The current research findings corroborate the earlier efforts to understand and explain co-production of knowledge with initial suggestions of fostering flexible research agendas, mechanisms promoting iterativity, and institutionalizing incentives for co-production (Lemos and Morehouse 2005; Dilling and Lemos 2011).

Development, design, and implementation components

The most common elements discussed across the nine papers were those related to Process, particularly research methods, stakeholder engagement practices, and communication strategies.

Engagement and communication activities are widely recognized as the cornerstones of producing usable information (Kothari et al. 2005; Oh and Rich 1996; van de Vall and Bolas 1982) and co-production of knowledge (Dilling and Lemos 2011; Lemos and Morehouse 2005; Reed et al. 2014). Not only does deep engagement with end-users tend to increase the likelihood that the information will be used (Evely et al. 2010), there is evidence that it improves the quality of decisions resulting from the process (Shirk et al. 2012; Beierle 2002). However, much of the outcome rests on the quality of the participation (Meadow et al. 2015; Stern and Fireberg 1996), including the degree to which researchers and stakeholders interact (Dilling and Lemos 2011), which will allow for better alignment of the research to stakeholders needs (Shirk et al. 2012).

The nine studies discussed here highlighted several approaches to ensuring that stakeholders needs drive the research process, such as co-defining the research questions, co- conceptualizing, and even co-designing the entire research project. Similarly, Polk (2015) notes the importance of co-leadership of transdisciplinary projects as a key element in success. An additional consideration for project co-design and co-leadership is the influence that the process has on allowing for full integration of local knowledge into the research process. The goal of collaboratively designing the research approaches is to move away from “expert-built” analytical frameworks that may fail to capture local knowledge or management practices. An analytical framework that does not properly model the context and the local knowledge may be misleading and results in inadequate research findings (Crane et al. 2016). True integration of local knowledge can improve planning for communities facing the most serious environmental risks (Corburn 2003; Bautista et al. 2017). However, this does require researchers to take on the challenges of integration (Pohl et al. 2010).

Many of the nine cases considered here point to the need for additional resources—both time and funds—required to fully engage with stakeholders to co-produce knowledge. The amount of attention and time required to both engage effectively with stakeholders and achieve outcomes related to stakeholder learning and changes in practice, as well as produce standard research outputs (i.e. peer-reviewed articles) is significant (Cvitanovic et al. 2015a, 2015b; Reed et al. 2014). However, several of the case studies discussed the role of funders and academic organizations in reducing those challenges through specific support for collaborative research activities. Polk (2015) notes, though, that even in an organization designed for transdisciplinary, collaborative research participants still encountered time and resource constraints. How to consistently overcome these barriers remains an area in need of further attention.

Context and Inputs features

A second group of criteria centers on specific variables that, when present, can enable co-production of knowledge. These variables alone do not ensure success, but their absence—based on our analysis—can reduce the likelihood of success.

Several themes that emerged from the case studies focused on whether the organizations involved (funders, research institutions, and management agencies) had sufficiently flexible structures to accommodate the unique requirements of co-producing knowledge. For example, funding organizations that have the flexibility to fund non-research activities, like hiring a communication specialist, can allow joint teams to adjust plans as needs become apparent. Boyd and Kramer (2017) note the crucial role that sufficient and flexible funding plays in encouraging collaboration between researchers and decision-makers. Furthermore, academic or research institutions that provide both training in and incentives for collaborative research approaches can help researchers overcome barriers related to how academic research is generally rewarded (Bell et al. 2011; Brugger et al. 2016). Van Kerkhoff and Lebel (2015) and Wyborn (2015) label the institutional characteristics that support collaborative work “coproductive capacities”—the combination of scientific resources, cognitive, social, and normative capacities that can be used to connect knowledge with action. Some case studies provided negative examples of the ways in which rigid management agency bureaucracies inhibited the ability of managers to communicate with each other and with the research team. This lack of flexibility, in addition to limiting collaboration, may also inhibit use of information in general (Guston 2007; Nelson et al. 2002; Ingram et al. 2002; Rayner et al. 2005; Meinke et al. 2006).

Similarly, individual expertise as an input in collaborative and participatory research approaches contributes to the overall coproductive capacities (Schuttenberg and Guth 2015). Some individual competencies, such as facilitation, trust building, communication, stakeholder engagement, and knowledge broker’s skills can be acquired through formal or professional development trainings, as suggested by Cvitanovic et al. (2016). Others, including networking, motivation to engage with people, understanding of decision environments, and ways to influence decision-making are intrinsic to each individual and increase with experience. However, these latter may be encouraged with incentives, such as systems of rewards, recognition, and support. Van Kerkhoff and Lebel (2015) suggest embedding coproductive capacity development explicitly into rules for funding or criteria for project monitoring and evaluation in order to generate systematic approaches for knowledge co-production.

Several case studies discussed the benefits of including stakeholders with scientific capacities. While having a shared pool of knowledge between the researchers and stakeholders can certainly benefit a collaborative project, we do not believe Western scientific training should be a prerequisite for participation in research. Researchers should cultivate an understanding of the decision contexts of resource managers in order to ease the collaborative process of developing applicable knowledge that fits management goals and be used ultimately (Dilling and Lemos 2011; Skolnikoff 1999). Likewise, resource management agencies should not send only representatives that fit in the Western science framework because doing so may limit the integration of local knowledge(s) along with science-based approaches.

Imbuing a co-production process with trust and legitimacy is often considered important part of the process (Harris and Lyon 2013). However, several cases point to the need to consider trust building as a necessary pre-cursor to a collaborative project. This is particularly true in places that have been negatively impacted by previous research work, especially those that have sought to supplant local knowledge(s) (van Kerkhoff and Pilbeam 2017). Researchers should not assume that because their own intentions are noble that they are immediately entitled to local peoples’ trust—they should consider ways to demonstrate the legitimacy of their process at the earliest stages of collaboration.

Setting-up component

An often-overlooked Process component that was addressed several times in the nine cases studies is the act of setting up research teams. Perhaps the ability of research teams to work together effectively and answer scientific questions is taken for granted.

A first consideration when setting-up the research team is ensuring that the team can work together effectively. In order for the researchers to function as an interdisciplinary team (as opposed to unconnected multidisciplinary researchers) (Little 2003; NAS 2005), they should take time to learn each other’s language, disciplinary jargons, diversity, and differences (Lach 2014) and function as a highly integrated science team (NRC 2015; NAS 2005). Other elements highlighted by the cases in the setting-up component raise a number of practical questions, including:

  • How can adequate expertise be recruited if the team must wait for stakeholders’ input before defining a research question?

  • Are financial resources available to design collaboratively the research project before requesting funding for the work?

  • What approaches to follow from writing a grant ex-nilo, or applying/responding to a specific call for interest or request for proposal, or changing the research approach of an already-funded project?

These concerns are important to address and there is no one-fit-size prescriptions. However, the new strand of research called Team Assembly (NRC 2013; NRC 2015: 88-95) might provide some guidance in this area. In addition, a Research Networking Systems that helps to find adequate research expertise for potential cross-institutional collaborations (Obeid et al. 2014) might be a useful tool for team formation and expertise recruitment. Furthermore, Enengel et al. (2012) also provide insights into how roles and responsibilities can most effectively be divided-up among the different stakeholders involved in co-production research project. They illustrate how to build on the knowledge and skill domains of three set of actors—professional practice experts, strategic case actors, and local case actors—to strategically design projects that adopt co-production of knowledge approach.

Outputs management and dissemination component

The ways in which project outputs are packaged and managed seems likely to bolster the outcomes of collaborative projects. These case studies provided several examples of how making outputs available in different forms, based on a sound understanding of stakeholders needs, increased the usability of the science produced. Previous work seeking to address the challenge of integrating scientific information into management decision-making process has recommended improving knowledge transfer among scientists and end-users by means of diverse approaches. These include the use of knowledge brokers, boundary organizations, and research papers’ briefs (summaries) in order to overcome the inaccessibility of primary scientific information and knowledge that weakened its integration in decision-making (Cvitanovic et al. 2014).

Outcomes–Impacts features

Several of the case studies suggest that increasing the literacy and the understandings of the local participants with regard to the issues on which collaborative research took place is a valid and achievable outcome from collaborative research. The main challenge centers on making the whole research process into a social learning opportunity, from which stakeholders achieve some tangible gains in terms of capacity building and development (Cumming et al. 2008). Scholars have demonstrated that the combination of such types of outcomes holds the potential of affecting resource management decisions (Pahl-Wostl et al. 2008, Armitage et al. 2009). Moreover, both individual and community-level outcomes are important to foster and need to be acknowledged, besides program outcomes, during the collaborative process’s evaluation (Jordan et al. 2012).

Professional cultural differences, inclusivity, and logistic factors

These characteristics were each discussed in several papers—but were not highlighted widely, perhaps because these factors depend more heavily on the context in which the research is taking place—such as language barriers or geographic barriers between researchers and stakeholders. However, the lessons from these context-specific factors can inform collaborative research more broadly.

Professional cultural differences (Davidson et al. 2001; Evely et al. 2008; Pohl et al. 2010; Schein 2010) can play out within a research project in ways that frustrate and disincentivize both parties (Hicks and Katz 1996; Melin 2000). Scientists, who are generally rewarded for research outputs, may wish to move slowly through the research in order to focus on publishable outputs. However, resource managers may feel pressured to take action and grow frustrated waiting for results. Practitioners may also feel excluded from the scientific process, particularly if their own knowledge base has not been acknowledged or included with Western scientific knowledge. As such, Harris and Lyon (2013) reflect that in collaborative works, even in presence of incentives to cooperate, challenges pertain to and cross professional cultural boundaries and need particular attention.

An extension of the professional cultural factors is gender sensitivity and the need for inclusivity, especially when diverse groups of stakeholders are involved. Neglecting the influence of gender issues and norms is often tied to ineffective outcomes (Holdcroft 2007), particularly in the realm of climate change, where marginalized groups—especially women—are more vulnerable to negative impacts than men (United Nations 2016). As such, being attentive to the issues of gender, class, caste (if any), and cultural identity (ethnicities, religions, etc.) among the participating groups in the knowledge co-production work is important. The consideration of such factors are highly stressed in environmental health studies in order to account for health or other disparities and inequity as well as to foster equity, empowerment, and social justice where needed (Minkler and Wallerstein 2008: Chapters 1 and 5).

The impediments associated with logistics are mostly addressed with increased travels to reach managers and other local level stakeholders (farmers or communities) for face-to-face meetings. Limited financial resources to address challenging logistics were also raised as an issue in several of the cases.

Unpacking and repositioning stakeholders for adequate engagement and communication style

The observation that emerged from these cases, which is not discussed deeply in much of the co-production literature, is the heterogeneity of stakeholders or “end-users” of information and the impact their diversity has on both the utility of information produced and the methods used to produce it. In the nine cases reviewed here, we identified three types or groups of stakeholders. Each presents different information and engagement needs: (1) farmers at households or farms level, or any other communities at local level; (2) managers or practitioners involved in the day-to day operations at local and regional levels, but who make no decisions rather than applying policies; and (3) stakeholders who make larger-scale policy decisions, but lack the direct interactions from the field-based day-to day operations. Distinguishing these distinct needs of these different stakeholders—and being aware of any power differentials between them—is an important part of making research useful for a variety of decisions (Reed et al. 2009; Sherman and Ford 2014).

Conclusion

Building upon on the growing research body that epitomizes co-production research model in climate and environmental studies, the current paper fills a gap in the practical “know-how” of co-production work. This study has sought to compile examples of different aspects of the art of conducting knowledge co-production work; i.e. how different research teams have approached the challenges of complex collaborative research and co-production of knowledge. We refer to them as the modus operandi of co-production implementation. The institutional support and incentives needed as well as the individual competencies and skills are some of the avenues that need to be strengthened, and require more support.

The current study has some shortcomings. First, while we put forward an international focus, our interest in descriptive or “process” papers limited the geographic scope of cases. The relatively new aspect of the co-production research model may make it difficult to have many published papers from all around the world, although we expand the search to capture similar thematic. Second, we also acknowledge the fact that some gray literature or unpublished works, or works in languages other than English may exist, but these were not included in our search approach. Nevertheless, while recognizing these limitations, the discussion we provide above constitutes a window to inform on how to practically engage and conduct co-production in climate and environmental research projects directed to yield actionable science.

We reiterate that our ultimate goal is to help researchers navigate the different approaches to the process of conducting co-production of knowledge projects, in a variety of contexts and with a variety of research questions. Although we explored a relatively small sample of case studies, the grounded theory analysis approach we used has allowed us to identify key features of those studies that may prove useful to other researchers. The methodological descriptive guidance and details provided here will help scientists, wishing to embrace knowledge co-production work to run flexibly and successfully plan and manage their projects.