A global assessment of Indigenous community engagement in climate research

For millennia Indigenous communities worldwide have maintained diverse knowledge systems informed through careful observation of dynamics of environmental changes. Although Indigenous communities and their knowledge systems are recognized as critical resources for understanding and adapting to climate change, no comprehensive, evidence-based analysis has been conducted into how environmental studies engage Indigenous communities. Here we provide the first global systematic review of levels of Indigenous community participation and decision-making in all stages of the research process (initiation, design, implementation, analysis, dissemination) in climate field studies that access Indigenous knowledge. We develop indicators for assessing responsible community engagement in research practice and identify patterns in levels of Indigenous community engagement. We find that the vast majority of climate studies (87%) practice an extractive model in which outside researchers use Indigenous knowledge systems with minimal participation or decision-making authority from communities who hold them. Few studies report on outputs that directly serve Indigenous communities, ethical guidelines for research practice, or providing Indigenous community access to findings. Further, studies initiated with (in mutual agreement between outside researchers and Indigenous communities) and by Indigenous community members report significantly more indicators for responsible community engagement when accessing Indigenous knowledges than studies initiated by outside researchers alone. This global assessment provides an evidence base to inform our understanding of broader social impacts related to research design and concludes with a series of guiding questions and methods to support responsible research practice with Indigenous and local communities.


Abstract
For millennia Indigenous communities worldwide have maintained diverse knowledge systems informed through careful observation of dynamics of environmental changes. Although Indigenous communities and their knowledge systems are recognized as critical resources for understanding and adapting to climate change, no comprehensive, evidence-based analysis has been conducted into how environmental studies engage Indigenous communities. Here we provide the first global systematic review of levels of Indigenous community participation and decision-making in all stages of the research process (initiation, design, implementation, analysis, dissemination) in climate field studies that access Indigenous knowledge. We develop indicators for assessing responsible community engagement in research practice and identify patterns in levels of Indigenous community engagement. We find that the vast majority of climate studies (87%) practice an extractive model in which outside researchers use Indigenous knowledge systems with minimal participation or decision-making authority from communities who hold them. Few studies report on outputs that directly serve Indigenous communities, ethical guidelines for research practice, or providing Indigenous community access to findings. Further, studies initiated with (in mutual agreement between outside researchers and Indigenous communities) and by Indigenous community members report significantly more indicators for responsible community engagement when accessing Indigenous knowledges than studies initiated by outside researchers alone. This global assessment provides an evidence base to inform our understanding of broader social impacts related to research design and concludes with a series of guiding questions and methods to support responsible research practice with Indigenous and local communities.  Maynard 2014, Maldonado et al 2015. Reasons behind this call include improvements both in the effectiveness of research and in the standards of ethical research (Pearce et al 2009). Indigenous communities whose knowledge and subsistence systems remain tightly woven with ancestral lands often suffer disproportionate impacts from accelerating climaterelated biological disruptions and land-loss, as well as from political, social and ideological marginalization and persecution (United Nations 2009, Ford et al 2016a. Currently, we have neither a standard set of indicators for responsible inclusion of Indigenous knowledge and communities in environmental sciences, nor an understanding of the degree to which research projects follow responsible research practices. This study fills these two research gaps and develops an analytical framework that can be applied for assessing Indigenous community engagement across a wide range of research fields.

Answering the global call for inclusion
Within the past decade, global networks of scientists and practitioners have formally recognized the immense value of Indigenous knowledge systems for the adaptive capacity of humankind in times of extreme climate variability. Both the fourth and fifth assessment reports of the IPCC's Working Groups emphasize Indigenous knowledge systems as critical resources for effectively adapting to climate change. Regarding human security, the latter report found 'high agreement among researchers that involvement of local people and their local, traditional, or indigenous forms of knowledge in decision making is critical for ensuring their security' (Adger et al 2014, p 765). In the United Nations publication titled Weathering uncertainty: traditional knowledge for climate change assessment and adaptation, an emphasis on collaboration, co-production of knowledge, and cross-cultural methods reflect the call to shift to more inclusive methods in scientific research (Nakashima et al 2012). Similarly, in the National Aeronautics and Space Administration's (NASA) most recent Native People Native Homelands Climate Change Workshop Report several regional working groups identified community involvement and utilizing diverse ways of knowing as important action strategies and as areas needed to implement coping and adaptation strategies (Maynard 2014). Most recently, the Climate and Traditional Knowledge Workgroup published guidelines for US tribes, agencies, and organizations in an effort to inform culturally ethical, tribally-led partnerships that weave multiple knowledge sources for climate initiatives (CTKW 2014). As the number of climate research studies engaging Indigenous knowledge systems continues to increase, Indigenous communities from across the globe are simultaneously coordinating efforts to reclaim authority over their knowledge systems, languages and practices. One of the most formative efforts, the United Nations Declaration on the Rights of Indigenous Peoples (UNDRIP), directly addresses concerns regarding Indigenous peoples' authority over their knowledge systems in Article 31: Indigenous peoples have the right to maintain, control, protect and develop theirKtraditional knowledge-Kas well as the manifestations of their sciences, technologies and cultures KThey also have the right to maintain, control, protect and develop their intellectual property (UN General Assembly 2007, p 10).
A decade since this declaration was established, there remains little evidence of action in the climate science research community for addressing these concerns in practice. In this time of increasing climate variability and ongoing socio-political vulnerability, the need persists to ask how the scientific processes for engaging Indigenous knowledge systems support or neglect the rights and capacities of the communities maintaining these knowledge systems (United Nations Framework Convention on Climate Change 2013, Whyte 2017). This study documents a systematic analytical exploration of climate research studies that draw from Indigenous knowledge systems to provide the first comprehensive global assessment on how Indigenous knowledge systems and the communities who hold them are engaged in scientific studies. We first develop an analytical framework synthesizing theory of effective practices for responsible community engagement in research and then apply this framework to examine the degree to which these practices are followed in climate research with Indigenous communities.
2. Methods for assessing community engagement in climate studies 2.1. Conceptualizing indigenous communities and knowledge systems For the context of this interdisciplinary study, we recognize the need to build a shared conceptual understanding of terms such as 'Indigenous communities' and 'Indigenous knowledge systems'. When the United Nations Working Group on Indigenous Peoples developed the Draft Declaration on the Rights of Indigenous Peoples (2007), they emphasized the need to retain autonomy within each respective community for defining Indigenous communities and peoples. For the context of this study, 'Indigenous' should be understood to reference a community of peoples sharing intergenerational ancestry and cultural aspects with original (pre-colonial) occupants of ancestral lands in a specific region of the world. Within this definition, membership to an Indigenous community should be understood as a sovereign right established both through self-determination and community acceptance (Anaya 2004). Following this understanding, the concept of 'Indigenous knowledge systems' is included in this study to mean dynamic systems of knowledge collectively held by Indigenous community members that draw from intergenerational, placebased, culturally-embedded relationships and experiences. Shared terms also used in academia and policy that reflect these unique systems of knowledge include: 'Indigenous environmental knowledge', 'indigenous knowledge', 'indigenous ways of knowing', 'Native science', 'traditional ecological knowledge', and 'traditional knowledge' (Cajete 2000, International Council for Science 2002, Nakashima and Roué 2002, Berkes 2008, Burkett 2013).

Developing an analytical framework
The following section describes our development of an analytical framework for examining community engagement in research practice, including two main components: a scale of levels of community participation and a set of indicators for responsible research practice with Indigenous communities. Recent calls for improvements in inclusion and collaboration with Indigenous communities in climate science research Sharp 2012, Ford et al 2016a) guided us towards developing a relevant framework grounded in Indigenous and community-based participatory research guidelines and ethical standards. We also recognized that beyond levels of engagement, consistency and quality of engagement for community members also require consideration for effective research practice (Israel et al 1998, Reed 2008, Pearce et al 2009.

Scaling levels of community participation
Within participatory research we commonly find scales and typologies for assessing participation in environmental science related literature (Arnstein 1969, Biggs 1989, Lilja and Ashby 1999, Rodriguez-Izquierdo et al 2010, Shirk et al 2012. We adapted these scales for our analytical framework while also drawing from Indigenous, community-based and participatory action research theory. An ever-growing number of Indigenous researchers and communities continue to develop and implement research frameworks reflective of their own cultural values and systems through Indigenous epistemologies and research models (e.g. Weber-Pillwax 1999, Estrada 2005, Wilson 2008, LaFrance and Nichols 2009, McGregor et al 2010, Smith 2012. These frameworks share considerations outlined in participatory action and community-based research, such as who retains authority over research design and whose interests are served (Chilisa 2012). Indigenous research frameworks are also distinct in their explicit emphasis on self-determination and relational accountability to one's own community, including non-human communities (Weber-Pillwax 1999, Wilson 2001). Another common theme among Indigenous research frameworks is that the research process is centred on values, definitions and protocols developed within the Indigenous community engaged in the research (LaFrance and Nichols 2009). These research models recognize and account for colonial, historical and socio-cultural contexts in which research takes place, and the unique challenges and strengths inherent in Indigenous communities (Kovach 2010).
When implemented within a culturally-relevant, community-based framework, research should reflect the value-centred approach of Indigenous research methodologies (Cochran et al 2008, LaVeaux and Christopher 2009, Smith 2012. Participatory action research closely reflects this value-centred process (Kindon et al 2007). However, the interpretation and applied practice of community-based and participatory research varies considerably from study to study (Cornwall and Jewkes 1995). In an effort to identify best practices in participation, Reed's (2008) review of stakeholder participation in environmental management finds that the quality of participation is highly dependent on the quality of the process it builds from, and furthermore the philosophy upon which that process is built. Philosophical qualities identified in Reed's review include participants' ability to engage effectively in decision-making, recognizing and limiting pre-existing power inequalities, and supporting ongoing two-way learning between participants and researchers.
For our adapted scale, we assess levels of community participation ordered along a continuum ranging from contractual (employment-related) participation in which community members have at most a contracted role in the study with no decision-making authority, up to an Indigenous process in which all aspects of a study are contextualized and decided upon within the community (figure 1). Each level in the scale varies according to what degree community members engage in the process and who holds primary decision-making authority in the research.
One key feature emphasized in best practices for community participation, is continuity in community engagement throughout all stages of the research process (Fisher and Ball 2003, Johnson et al 2003, Sims and Kuhnlein 2003, Reed 2008. Likewise, the United Nations Framework Convention on Climate Change technical paper on best practices for use of Indigenous and traditional knowledge calls for 'involving indigenous and local knowledge in all assessment phases, from conception through to outputs', and that a prerequisite to including Indigenous knowledge is 'full and effective participation of (knowledge) holders' (2013, p 24). Each stage in the research process holds a unique purpose and impact for the participating community members. Following these recommendations, we apply this scale to various stages along the research process of field studies identified for this review, including: design (proposal development, defining goals and objectives, defining research questions etc), implementation (fieldwork, sampling, data collection, monitoring etc), and analysis (data interpretation, evaluation etc).

Identifying indicators for responsible community engagement
In addition to levels of engagement, we developed indicators for responsible research standards to assess quality of engagement undertaken by field studies. Withstanding centuries of challenges to their rights to maintain their own knowledge systems and practices, Indigenous peoples continually speak to a need for quality standards for research in their communities. Numerous Indigenous peoples' and collaborative science working group reports and codes of conduct For the purpose of this study, we identified best practices from literature on community-based, participatory and Indigenous research methods, and Indigenous community concerns identified in working group reports to develop indicators for responsible community engagement. Indicators reflect both foundational standards for responsible research (e.g. free prior and informed consent) and benefits for local communities (e.g. accessibility to findings). We include six indicators for responsible research practice in our assessment of reported data from climate field studies that speak to these concerns: 2.2.2.1. Indicator 1-Access: are findings accessible to Indigenous community members? Access indicates whether field studies address community access to findings (e.g. local presentations and distribution of publications regarding findings from study; data available to or stored with community members; materials from study produced in local languages). Mechanisms for community access should be addressed from the outset of the study design (CIDA 2002). This indicator reflects the 'principle of reciprocity', often lacking in existing academic research ethics protocols, for disseminating study results back to contributing community members in an accessible language and format (Smith 2012, p 16). Reciprocity in knowledge sharing is also identified as a method for 'promoting inter-cultural exchanges, knowledge and technology transfer' (Convention on Biological Diversity 2011).

2.2.2.2.
Indicator 2-Relevance: are findings reported in the context of concerns, issues or interests defined by Indigenous community members? Relevance is defined by the degree to which reported findings are explicitly relevant to concerns and interests pre-identified by members of the community. The UN DRIP (2007), Kaupapa Māori principles (Smith 2015), and the International Society of Ethnobiology (ISE) Code of Ethics (2006), all reflect this philosophy towards looking critically at how developments and research designs support communities within their existing cultural and organizational needs and concerns.
2.2.2.3. Indicator 3-Credit: how were Indigenous community members credited for their knowledge contributions and efforts (acknowledgement, coauthorship)? Credit is defined by the degree to which research credits knowledge holders for their contributions (i.e. no acknowledgement, acknowledgement only, co-authorship). This indicator reflects the Climate and Traditional Knowledges Workgroup call for ensuring that 'contributions of tribal partners are recognized in final products, publications, and efforts to publicize projects' (2014, p 8). The ISE Code of Ethics' 'Principle of Acknowledgement and Due Credit' also emphasizes the importance of crediting knowledge contributions: 'researchers will act in good faith to ensure the connections to original sources of knowledge and resources are maintained in the public record' (2006, p 8).

Indicator 4-Ethics
: did the study report ethical guidelines followed, such as Free Prior and Informed Consent? Fundamental ethical principles that have long been recognized in medical and legal practice, such as 'free, prior and informed consent' (FPIC), now form basic requirements for academic and institutional research with human communities. Ethics indicates whether studies reported ethical guidelines followed in the research process (e.g. FPIC, approval from Indigenous ethics group, reference to applied code of ethics). Ethical responsibilities researchers hold to Indigenous communities require careful consideration in order to promote benefit for community members and reduce harm (Piquemal 2001, Williams andHardison 2013). As one example of ethical guidelines, the principle of FPIC works to ensure that knowledge holders within Indigenous communities retain informed decisionmaking authority regarding their participation in the research process. Numerous ethics guidelines emphasize the importance of providing knowledge of and receiving prior approval for research that impacts 2.2.2.5. Indicator 5-Cause no harm: did the study address intellectual property rights or risks for Indigenous communities?
The principle of cause no harm or do no harm represents another fundamental ethical standard relevant to climate research practice with Indigenous communities (CTKW 2014). Cause no harm indicates whether studies address concerns regarding the Indigenous intellectual property (e.g. community review and/or ownership of data, sensitive data identified and protected). The principle of cause no harm denotes a critical step in assessing risk and potential harm, both socio-cultural (e.g. appropriation of cultural and intellectual property) and material (e.g. resource exploitation), for Indigenous communities in the research process. The Mataatua Declaration on Cultural and Intellectual Property Rights of Indigenous Peoples, calls for state, national and international agencies to 'recognise that indigenous peoples are the guardians of their customary knowledge and have the right to protect and control dissemination of that knowledge' (1993, p 3). In addition, guides for best practices in international project planning with Indigenous peoples call for safeguards that increase Indigenous peoples' decision-making authority proportional to higher levels of risk associated with sharing of Indigenous traditional knowledge (CIDA 2002). This consideration also reflects issues reported by Native Americans, Alaska Natives, and Pacific Islanders disproportionally impacted by climate change regarding the issue of intellectual property, in which they emphasized Indigenous communities as the best resources for ensuring that protocols for integrating diverse knowledge systems include cultural protections (Maynard 2014).
2.2.2.6. Indicator 6-Outputs: did the study report any outputs or outcomes for the Indigenous community? Outputs and outcomes that indicate quality practices include: whether projects lead to any actions or changes within the community (e.g. capacity-building, adaptation plans), or whether any products developed from a given study directly benefit the community (e.g. maps, curriculum materials). Much like access, this indicator also links back to the foundational principle of reciprocity centred in Indigenous research methodologies and codes of ethics. For example, the ISE Code of Ethics recognizes the right for community members to benefit from outcomes and results produced by research that accesses Indigenous knowledge systems (ISE 2006). Further, in their study on integrating local and scientific knowledge for environmental management, Raymond et al state that an indication of project success is 'the extent to which the knowledge integration outputs are used by those who input their knowledge' (2010, p 1770).
The scale and indicators described in detail above served as analytical tools for assessing the degree of Indigenous community engagement in climate studies. The following section further describes how we identified studies and specific criteria for justifying inclusion in our global review.

Protocol development and expert review panel
We adapted methods for this review from existing systematic review frameworks (Pullin and Stewart 2006, Grant and Booth 2009, Munroe et al 2012. The process included developing a review protocol in consultation with a panel of experts, a comprehensive search of peer-reviewed and grey literature, qualitative synthesis and a quantitative analysis. The supplementary materials associated with this article include full details of the search and screening results, description of attributes and codes, data analysis results, and a bibliography for all articles included in the review. Based on systematic review guidelines adapted from health services research for environmental sciences (Pullin and Stewart 2006), we invited subject experts early in the process to ensure a more robust protocol that is relevant to research practice and policy. In addition to the two authors, eight additional panel members (see credit and acknowledgements section) whose work focuses on Indigenous knowledge systems and environmental sciences provided feedback to help refine the research questions and search terms and to identify relevant field studies. Five of the panel members are Indigenous community members currently working on climate research, including the lead author David-Chavez. The five remaining members maintain expertise in working collaboratively with Indigenous communities and knowledge systems.

Screening and inclusion criteria 2.4.1. Search terms
Search terms developed and refined with the expert panel represent three unique overlaying conceptsone identifying the appropriate populations and two defining thematic content: The final list of search terms presented here represent results from several scoping searches through the Web of Science database in which we identified terms that most accurately and comprehensively located thematically relevant climate field studies. We also checked database results for field studies pre-identi-

Criteria for selection for full review
Screening criteria for titles and abstracts included three main criteria for inclusion. Firstly, we required original field studies containing climate-related environmental research. 'Climate-related environmental research' includes knowledge systems and biocultural relationships humans hold in regard to long term weather patterns, their environmental impacts, and environmental adaptation in a particular place. Secondly, we required studies to include Indigenous knowledge system(s) (e.g. traditional ecological knowledge, Native science) from specific Indigenous community(ies). Thirdly, we required studies to be published in English (due to reviewers' language fluency limitations). Publications meeting these criteria, or those we could not clearly determine based on title and abstract alone, were included for full-text review (n = 232). Publications meeting all criteria that also contained enough methodological and contextual information regarding the research process were included in the final analysis (n = 140; figure 3).

Data collection, coding and analysis
We recorded data obtained from full-review articles meeting all inclusion criteria in a spreadsheet for cross-case qualitative and quantitative content analysis (Stemler 2001). For multiple articles containing content from the same field study (n=15), we aggregated notes and coded data as a singular case under the most current publication year to allow for a balanced comparison in quantitative analyses of research practices across field studies. Attributes recorded for each field study included demographic data, disciplines of study authors, levels and methods of Indigenous community engagement in all stages in the research process (initiation, design, implementation, analysis, dissemination), and quality indicators for responsible research practice. This process for data collection allowed for an inclusive analysis of a diverse range of qualitative, quantitative and mixed methods studies. We developed categorical codes to indicate the levels of Indigenous community participation at the initiation phase of field studies: A-outside academic researchers, M-mutual agreement between outside researchers and Indigenous community members, C-community initiated, O-other, NR-not reported. For the design, implementation, and analysis stages of the research we applied the ordinal scale developed in our analytical framework to record levels of Indigenous community participation: 0-Contractual/No participation, 1-Consultative, 2-Collaborative, 3-Collegial, 4-Indigenous. Although at the time of this review no studies met criteria for Indigenous level on scale, we retained this code for future analyses. We also coded reported data for each indicator of responsible community engagement as follows: access (0-not reported, 1-accessibility is directly addressed); relevance (0-not reported, 1-relevance for community is directly addressed); credit (0-not reported, 1-acknowledgement only, 2-co-authorship); ethics (0-not reported, 1-some form of ethical guidelines/consent process reported); cause no harm (0-not reported, 1-intellectual property rights/risks addressed); outputs (0-not reported, 1-proposed, 2-actual). Upon completion, the final spreadsheet comprised a case-based matrix with coded values visually linked with qualitative notes referencing evidentiary criteria used for identifying patterns of association (Bazeley 2013). After excluding aggregated studies, a secondary reviewer unfamiliar with the study beyond the specific coding criteria analysed and coded a random sample of field studies (n=29). We included an inter-rater reliability test (kappa=0.907) using Cohen's weighted kappa to assess for consistency in coded values (Cohen 1968).
In our analyses we searched for patterns across space, patterns across disciplines, patterns across time, and patterns in responsible community engagement in climate field studies. These included geospatial and statistical analyses using R Studio software (version 3.4.2). To view patterns across time, we compared levels of community participation within the design, implementation and analysis research stages. Different research stages represent unique components in terms of the research processes and we recognize that active participation may vary across the length of a study. Further, in the ISE Code of Ethics 'Principle of Active Participation', community participation is stressed within these distinct stages (2006, p 6). We also compared levels of participation based on who initiated each study (i.e. outside researchers, mutual collaborations between researchers and community, or community initiated). Practical guidelines outlined in the ISE Code of Ethics recognize that 'objectives, conditions and mutually agreed terms should be totally revealed and agreed to by all parties prior to the initiation of research activities' (2006, p 11). To test for evidence whether proportions for each indicator of responsible community engagement reported in studies varied by who initiated the studies we used descriptive statistics and Fisher's exact test of independence with Holm's correction. Due to limitations in language fluency and time, we acknowledge some potential underestimation in values for relevant case studies published in languages other than English and for grassroots-driven climate studies occurring yet not represented in any publications.

Patterns across space
On a global scale we find that the vast majority of climate studies (87%) practice an extractive model in which researchers use Indigenous knowledge systems with minimal participation or decision-making authority from communities who hold them ( figure 4). Several geographic clusters denote where Indigenous knowledge systems have most often been accessed for climate research, with the most prominent groupings in the North American Arctic, Sub-Saharan East Africa and the Tibetan Plateau.
Average levels of participation vary considerably between these three geographic regions, with the highest levels of Indigenous community participation concentrated in northern Canada and Alaska. In their study on community-level climate vulnerability assessments McDowell et al (2016) also note a higher than average concentration in participatory approaches in the North American Arctic, including stakeholder consultation in developing research objectives and Indigenous evaluation of quality of results. Although the limited scope of our study did not explore drivers for these geographic patterns in detail, we note that these similarities invite further research into why these regional variations in Indigenous community participation occur. Many different factors may influence the variation in geographical distribution, including differences in research policies across countries regarding engagement and documentation of involvement with Indigenous communities. For example, in response to non-Indigenous research priorities and over-researched communities, Canadian First Nations standardized new ethical guidelines on data ownership, data sharing and self-determination (Schnarch 2004). Further, within the US Arctic, principles regarding ethical responsibilities towards Indigenous communities guide some federally-funded research initiatives (National Science Foundation 1990). Observing global distribution of field studies also reveals extensive geographic gaps representing areas where Indigenous knowledge systems and communities may not yet be included in climate research. These include areas, such as low-lying islands, drought and flood-prone regions, and coastal regions where changes in hydrological, marine, terrestrial and food systems attributed to climate impacts continue to raise increasing concern (Field and Barros 2014).

Patterns across disciplines
Climate research studies inherently span a broad reach of disciplines and approaches. We found that within climate research that engages Indigenous knowledge systems, patterns in authorship reflect an ongoing shift in scientific knowledge production from intradisciplinary approaches (drawing from a single field of theory and methods), towards application-oriented research achieved through interdisciplinary (integration of theory and methods from two or more disciplines), multidisciplinary (collaborations between people working within different disciplines), and transdisciplinary (reaching beyond disciplines to include stakeholders and practitioners) methods ( figure 5).
The highest proportions of climate studies engaging Indigenous knowledge systems include studies authored by interdisciplinary (18%), multidisciplinary natural and social sciences (25%), and transdisciplinary researchers (50%). We observed the highest levels of engagement (collaborative and collegial) in studies authored by teams that include Indigenous scientists and community members and/or local practitioners. Studies that included only non-Indigenous practitioners and non-Indigenous authors from multiple disciplines remain heavily weighted towards contractual participation. Analysis of distributions of disciplines within these climate studies demonstrate an ongoing shift beyond only traditional scientific disciplines, towards the inclusion of application-oriented and integrated disciplinary approaches that address usability and social relevance of knowledge (Van den Besselaar and Heimeriks 2001). We consider these findings in parallel with ongoing emphases in global environmental change research for collaborative and transdisciplinary research-harnessing strengths from natural sciences, social sciences, humanities and community knowledge alike, to address complex challenges (Mauser et al 2013, Palsson et al 2013, Belmont Forum 2016.

Patterns across time
We also sought to understand variations in levels of Indigenous community participation across different stages in the research process. Among studies reporting methods in the design, implementation, and analysis phases of research, we find that a substantial number of studies (39% in design, 48% in implementation, 56% in analysis) practice no or contractual (employment-related) levels of Indigenous community participation ( figure 6).
Participation in all stages of research varied considerably depending on who initiated the project. Research initiated with (n=21) or by (n=10) Indigenous communities had higher levels of engagement and inclusion throughout all stages of the research process ( figure 7). However, studies initiated in mutual collaboration between outside researchers and community members vary more widely in levels of Figure 5. Patterns across disciplines: variation in levels of participation by discipline(s) of study author(s). 'Interdisciplinary' signifies an author/authors from one interdisciplinary field, while 'multidisciplinary' signifies multiple-authored studies applying more than one discipline, and 'transdisciplinary' signifies inclusion of community practitioners/stakeholders. * Outliers represented two Indigenous scientists from the community working within these disciplines. participation, especially in the analysis phase of the research.
By contrast, studies initiated solely by outside researchers (n=89) tend to maintain lower levels of participation across all stages of the research process. The greatest variation in participation levels across study groups occurred in the analysis stage. These results may speak to needs and challenges identified in similar studies on community engagement in research, including local employment and training (Fisher andBall 2003, Pearce et al 2009), and adapting research for analyses that can draw on more diverse knowledge systems by allowing for knowledge co-production (Harvey et al 2017). Likewise, increasing levels of participation across various stages of research may call for additional capacity building among researchers towards understanding rights, risks, cultural protocols and methods for respectful inclusion of diverse ways of knowing in climate research (Cochran et al 2013, CTKW 2014. Overall, many studies that verbally referenced community inclusion and engagement (e.g. 'participatory research'), lacked evidence to demonstrate community engagement beyond contractual tasks. This parallels McDowell et al (2016) and Pearce et al's (2009) observations that although local knowledges are recognized for their importance in climate research, participatory design remains lacking in applied practice. This observation also reflects challenges with how the idea of participation is applied in research. While terminology such as 'communitybased' and 'participatory' are used in a variety of ways, they may not be consistently applied in practice.

Patterns in responsible community engagement
The majority of studies did not report the presence of 5 of the 6 indicators of responsible community engagement (figure 8). The lone exception was with credit for community contributions in the form of an acknowledgement. However, although most studies included a formal acknowledgement for Indigenous knowledge contributions to field studies, less than a quarter of all studies included co-authorship for Indigenous knowledge contributors.
Engaging Indigenous community members from the beginning of the research process does not necessarily guarantee sustained responsible research engagement throughout the length of the research, however we found statistically significant differences in all six indicators for responsible community engagement depending upon who conducted climate studies (p values: Initiated by versus Access 3.30549e-11; Initiated by versus Relevance 6.457925e-10; Initiated by versus Credit 2.186311e-11; Initiated by versus Ethics/FPIC 0.005697953; Initiated by versus Cause No Harm 1.194242e-07; Initiated by versus Outputs/ Outcomes 1.405891e-06).
Overall, studies that engaged Indigenous community members from the beginning of the research process (those initiated by community members or in mutual agreement between outside researchers and community members) consistently reported higher proportions of indicators for responsible community engagement (figure 9). Some of these indicators (i.e. ethics/FPIC, cause no harm) represent foundational research principals recommended in ethical research conduct with all human communities. Other quality indicators (i.e. accessibility, relevance, credit, outputs/ outcomes) also speak to best practices in communitybased and participatory research practices.
Just under 15% of studies initiated by outside researchers reported community access to research findings, compared to 80%-90% for those that included Indigenous community members from the initial stages of the research process. Similarly, 18% of studies initiated by outside researchers, 81% of studies initiated in mutual agreement between outside researchers and community members, and 90% of community-initiated studies reported findings within the context of concerns, issues or interests relevant to Indigenous community members.
Although all the studies in our analyses accessed and included Indigenous knowledge systems in the research, 22 of the 24 studies that attributed coauthorship to Indigenous knowledge holders or tribal communities were initiated in mutual agreement with or by Indigenous community members from the outset. When considering Indigenous knowledges in climate research studies we must also consider intellectual property rights and potential problematic risks to communities. Findings from this study infer that for most climate studies (n=101, 81%), researchers from outside the community will inevitably be cited in connection with Indigenous knowledge reported in the research findings. As Williams and Hardison (2013) point out, few social or political governing principles exist in practice regarding obligations towards Indigenous knowledge holders for outside researchers who share Indigenous knowledge with third parties and secondary users.
Among climate studies initiated by outside researchers alone, 27% report any indication of including ethical guidelines in the research process, including free, prior, and informed consent, and only 7% report any indication regarding intellectual property rights and risks for participating communities. However, substantially more studies that include Indigenous community members from the beginning of the research process (52% mutually-initiated studies, 80% community-initiated) reported the use of ethical protocols and exercising the principle to cause no harm (43% mutually-initiated studies, 90% communityinitiated).
Just under 16% of studies initiated by outside researchers reported actual outputs or outcomes for Indigenous communities in their studies. Alternately, 62% of mutually-initiated studies and 80% of studies initiated by community members reported outputs or outcomes relevant to the communities, including: capacity building, climate adaptation plans, educational curriculum, maps, and digital media resources for the study community.
We developed each of these indicators of responsible engagement based on research standards identified within both Indigenous communities and professional scientific forums. The fact that the majority of studies do not report evidence of adhering to these indicators of responsible engagement raises concerns. When considering broader impacts such as public understanding and support of scientific research, these patterns identify areas in need of improvement. These findings also represent an imbalanced exchange between outside academic researchers and Indigenous knowledge holders. The large proportion of studies reporting lack of community access to findings, relevance, credit, ethical considerations, or benefit to Indigenous communities indicate a deficiency in long-term accountability and reciprocity towards Indigenous knowledge holders. Most of the studies assessed here demonstrate an extractive process in their interaction with Indigenous communities and their knowledge systems.
Several pathways should be explored to encourage more responsible research engagement in the future.
For one, publication journals and funding agencies could hold an important role in improving research standards by requiring reporting on these indicators. For example, AlterNative: An International Journal of Indigenous Peoples, founded by New Zealand's Māori Centre of Research Excellence, requires that authors have approval from Indigenous knowledge holders through research agreements, approval from ethics committees, and that they 'design a study with participatory research and give the community and individual participants say in the anonymity and use of data' (Ngā Pae o te Māramatanga 2016). Additionally, the Climate and Traditional Knowledges Workgroup Guidelines for Considering Traditional Knowledges (TKs) in Climate Change Initiatives document calls to, 'develop guidelines for review of grant proposals that recognize the value of TKs, while ensuring protections for TKs, indigenous peoples, and holders of TKs' (2014, pp 21-2).
Increasing access for involvement of Indigenous communities in research should also be supported. Mauro and Hardison (2000) call for scientific institutions and societies to support Indigenous community rights and development in managing their own data. In their study on traditional knowledge and science, Fernandez-Gimenez et al found that studies documenting traditional ecological knowledge (TEK) were 'less useful for the integration and application of knowledge than direct involvement by TEK holders (in this case hunters) in research and management ' (2006, p 313). Agrawal (1995) also underscores the need to remain intent on who knowledge is useful for. Several studies included in our assessment claimed a need to extract and catalogue Indigenous knowledges due to the risk of the host cultures dying out. However, Agrawal warns against some of the risks in ex situ conservation of Indigenous knowledges, looking to in situ preservation as a new direction in research. This form of preservation, he states, requires that Indigenous communities 'possess the right to decide on how to save their knowledge, how to use it, and who shall use it ' (1995, p 432). Likewise, Tang and Gavin (2016) emphasize community-based initiatives and community engagement as a primary factor in preventing further loss and degradation of traditional ecological knowledge in Indigenous communities. Indigenous scholars such as Whyte (2018), further deepen this discussion by looking beyond the commonly referenced 'supplemental-value' (value as added input to supplement gaps in scientific data) of Indigenous knowledge systems in climate research, to further understanding and supporting 'governance-value' (value for sustainable planning initiatives and well-being in Indigenous communities), embedded within these knowledge systems. Future direction in this work will also continue to be shaped through the rapidly emerging fields of Indigenous data sovereignty and data governance, as led by Indigenous scholars and researchers around the world, which centre the inherent pre-colonial rights of Indigenous peoples to govern the collection, stewardship and dissemination of data regarding their communities (e.g. Kukutai and Taylor 2016, Rainie et al 2017, Robertson 2018. We also encourage further research assessing levels of engagement of underrepresented populations within Indigenous communities, such as marginalized genders, ethnic, and socio-economic groups. For example, not all Indigenous community members may have an interest in or access to higher levels of engagement in research studies and participating in various aspects of the research may place disproportionate burden on community members, especially in the case of marginalized populations. Areas in the Arctic heavily researched for climate impacts also note concerns with research fatigue and consultation fatigue due to tokenizing or non-reciprocal interactions with community members (Ford et al 2016b). United Nations' best practices for use of Indigenous knowledges in climate adaptation emphasizes the need for further understanding regarding 'not only how different social groups are affected but also how different groups can bring vital resources to the adaptation process' (2013, p 5).

Conclusion and recommendations
Our primary goal was to develop an analytical framework to assess how Indigenous knowledge systems are being accessed and identify how current standards in climate research practice are addressing calls for increased inclusion and engagement of diverse knowledge holders. In addition to observing temporal variation in patterns of engagement across studies, we also identified geospatial patterns, patterns in disciplines, and patterns in reported ethical practices and outcomes. By necessity our review could only examine details that researchers explicitly reported. However, researchers may have not have reported all the procedures they implemented. We encourage future research projects to recognize the importance of community engagement and to embrace transparency in all methods used across the research process. Similarly, we would suggest that academic journals require more rigorous reporting of research methods and the level of engagement with local communities. Among the handful of studies in this global review demonstrating quality practices regarding responsible Indigenous community engagement (see references highlighted in supplementary data file is available online at stacks.iop.org/ERL/13/123005/mmedia), we observed numerous opportunities that could serve as recommendations for removing barriers between researchers and Indigenous community members and increasing scientific engagement. For example, most high standard studies we observed used some form of on-site community workshops for opening pathways for science communication and discussions with community members (e.g. When viewed as a whole, our findings provide an evidence-base for identifying areas for improved standards in quality research practice. These findings reveal the variety of ways in which the scientific community is engaging Indigenous knowledge systems and communities who hold them in climate research, from extractive to action-oriented research. For the scientific community to value Indigenous knowledge systems, we understand the necessity to also respect the needs and values of the societies who hold these knowledge systems and the need for appropriate standards of responsible engagement within our research processes. Our current challenge is to develop normative standards of scientific research practice that support Indigenous communities in their ongoing efforts to maintain and practice these knowledge systems. Shifting research standards and practice will likely come with challenges such as adapting funding and timelines for cultural sensitivity and research reflexivity. However, the value that Indigenous knowledge systems hold for understanding and adapting our human communities to changes in our natural environment far outweigh the costs of meeting these challenges. Based on these findings, we call for action on the part of funding and research agencies, publication outlets, and institutional review boards to identify how they address responsible research concerns, such as those identified in the analytical framework presented here, in current standards for scientific research proposals and protocols. We offer ten guiding questions for researchers, funding agencies, journal editors, and policy makers to further reflect on how research practices address these standards for responsible research practice with Indigenous communities in every stage of the process (box 1). These guiding questions also hold application for local, non-Indigenous communities. Through the efforts of this publication and global assessment, it is our hope that this evidence-base serves as a reminder and practical guide for cultivating balanced respectful exchanges of knowledge centuries overdue within our scientific community. We also honour and recognize the value of the few shining examples highlighted in supplementary data that exist for responsible research with Indigenous communities as we strive to enhance ethical and intellectual standards for future research practice.

Credit and Acknowledgments
David-Chavez maintained primary responsibility for conceptual development, research design, data collection, data analyses and data interpretation. M C Gavin contributed with research design, development of analytical framework, and data interpretation. We express our gratitude to additional members of the expert-practitioner panel who reviewed and guided the research protocol for this review including:

ORCID iDs
Dominique M David-Chavez https:/ /orcid.org/ 0000-0001-5393-4331 Box 1. Questions for guiding responsible research practice with Indigenous communities (also may be applicable to local, non-Indigenous communities). Adapted primarily from the UN declaration on the rights of Indigenous peoples, the climate and traditional knowledges workgroup guidelines for considering traditional knowledge in climate change initiatives, and the International Society of Ethnobiology Code of Ethics. To what level do Indigenous community members have authority regarding the analysis of the research?
Are findings accessible to Indigenous community members?
Are findings reported in the context of concerns, issues or interests defined by Indigenous community members?
How were Indigenous community members credited for their knowledge contributions and efforts (i.e. acknowledgement, coauthorship)?
Did the study report ethical guidelines followed, such as Free Prior and Informed Consent?
Did the study address intellectual property rights or risks for Indigenous communities?
Did the study report any outputs or outcomes for the Indigenous community?