Skip to main content
Download PDF

Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation

Systematic Reviews volume 9, Article number: 21 (2020) Cite this article

Abstract

Background

Stakeholder engagement has become widely accepted as a necessary component of guideline development and implementation. While frameworks for developing guidelines express the need for those potentially affected by guideline recommendations to be involved in their development, there is a lack of consensus on how this should be done in practice. Further, there is a lack of guidance on how to equitably and meaningfully engage multiple stakeholders. We aim to develop guidance for the meaningful and equitable engagement of multiple stakeholders in guideline development and implementation.

Methods

This will be a multi-stage project. The first stage is to conduct a series of four systematic reviews. These will (1) describe existing guidance and methods for stakeholder engagement in guideline development and implementation, (2) characterize barriers and facilitators to stakeholder engagement in guideline development and implementation, (3) explore the impact of stakeholder engagement on guideline development and implementation, and (4) identify issues related to conflicts of interest when engaging multiple stakeholders in guideline development and implementation.

Discussion

We will collaborate with our multiple and diverse stakeholders to develop guidance for multi-stakeholder engagement in guideline development and implementation. We will use the results of the systematic reviews to develop a candidate list of draft guidance recommendations and will seek broad feedback on the draft guidance via an online survey of guideline developers and external stakeholders. An invited group of representatives from all stakeholder groups will discuss the results of the survey at a consensus meeting which will inform the development of the final guidance papers.

Our overall goal is to improve the development of guidelines through meaningful and equitable multi-stakeholder engagement, and subsequently to improve health outcomes and reduce inequities in health.

Peer Review reports

FormalPara Contribution to the literature

  • Existing guidance on stakeholder engagement largely focuses on patient, consumer, caregiver, or family involvement in guideline development but does not provide guidance on multi-stakeholder engagement.

  • The MuSE project will compile and provide evidence-based strategies for the equitable and meaningful engagement of multiple stakeholders during the guideline development and implementation process.

  • The MuSE project will develop a stakeholder engagement plan for its own project stakeholders and evaluate its development and implementation.

Background

Guidelines from various entities exist for many health conditions and health-related activities. Guidelines contain recommendations for health practice, public health, or health policy [1]. They are used by health care providers as well as policymakers, health system leaders, professional medical bodies, service organizations, funders, and regulatory authorities [2]. Recommendations in guidelines should be based on available research evidence [3]. Guideline development requires evaluating, summarizing, and making recommendations based on the available body of evidence regarding patient care, public health, and health systems. This requires weighing the benefits and risks that accompany all care and policy options before making recommendations [3]. Sonnad [4] noted that a lack of connection between guideline developers and those who use them often leads to controversy and uncertainty. Indeed, engaging guideline users during the guideline development process has been noted to improve guideline recommendation uptake [5, 6]. Schünemann and colleagues [7], for one, consider implementation in policy and practice as part of the guideline development process (steps 14 and 16). Throughout this work, we will use guideline development to encompass the guideline development and implementation processes.

In recent years, we have witnessed considerable shifts in how healthcare research is planned, delivered, shared, and evaluated. It is now increasingly expected that individuals or groups involved in or affected by health- and healthcare-related decisions, programs, or policies (termed “stakeholders”) should have a say in the planning, conduct, dissemination, uptake, and evaluation of healthcare research. In other words, stakeholders should be engaged in the entire process of guideline development. Several entities [8,9,10,11], including the World Health Organization (WHO) and the National Institute for Health and Care Excellence (NICE), recommend involving stakeholders in guideline development. As such, stakeholder engagement has become widely accepted in the production of trustworthy guidelines [12, 13]. In guideline development, stakeholder engagement is considered critical to ensuring priority guideline topics are identified and that comprehensive assessments of the evidence and other considerations are done [14,15,16].

Stakeholder involvement can help to ensure a guideline’s acceptability and feasibility to the end users. They can also ensure that equity and human rights issues are taken into consideration and support the adoption of its recommendations into policy and practice. This in turn may lead to improved adherence to any treatments and practices recommended [7, 17]. Stakeholder engagement in guideline development is part of a wider acceptance by the research community of the value of ensuring the participation of end users in the research and knowledge translation cycles [18, 19]. There is a moral imperative to engage end users in that people have a right to be involved in activities that may affect them. End user engagement may also improve the relevance, transparency, and usefulness of guidelines [20].

There are many stakeholder groups equally affected by recommendations in guidelines—e.g., patients, consumers, providers, general public, researchers, and policymakers. However, engagement with patients/public/community stakeholder groups dominates the literature, and guidance of the engagement with patient/public stakeholders is the most prominent [21,22,23]. In a review of guideline methodologies conducted by Armstrong and Bloom for example, patients/public stakeholders were consulted by 101 different guideline developers [21]. Many guideline groups that have sought to involve stakeholders have utilized limited numbers of participants or utilized slow and labor-intensive processes (e.g., time and resources needed to administer, collate, and respond to over 200 stakeholder views and comments) [12]. It is recognized that successful guideline development and implementation requires the engagement of multiple stakeholders [24] and “shared solutions” (input from patients, clinicians, and policymakers) improve health outcomes [24,25,26]. Patient/public stakeholders may potentially feel intimidated to contribute if they are only one voice among many. Keeping patient and public stakeholder voices separate from other stakeholder groups potentially shortchanges the input and influence that this group may offer. Equitable engagement of multiple stakeholder groups can help to ensure that guidelines contribute to reducing health disparities [27, 28]. However, there is a lack of consensus on how to identify and recruit relevant stakeholders, how they should be engaged, what their roles and responsibilities should be, how to evaluate the impact of their engagement in guideline development, and how to best collect and manage conflicts of interest as part of the engagement and guideline development process.

Schünemann et al. identified 18 steps in the guideline development process, based on a review of 35 guideline manuals published between 2003 and 2013 [7]. Several manuals mentioned the importance of including stakeholders, but few provided details on what stakeholder engagement should entail. In a review of 56 guidance documents for guideline development, 72% mentioned incorporating patients and their views in the process. However, the review did not provide sufficient detail on how to do this for each step of the guideline development process [29]. Armstrong et al. developed a framework for continuous patient engagement in clinical practice guideline development that outlines options for patient engagement in the steps in which they are most commonly involved [16]. While providing guidance on when to involve patients, the framework does not provide guidance on how to identify patients to participate and does not discuss other stakeholders.

Effective stakeholder engagement, ideally, facilitates the equitable contribution of relevant stakeholder groups in the guideline development and implementation process. This requires that guideline developers establish processes that prevent more financially powerful, highly vocal, or intellectually conflicted stakeholder groups from dominating the guideline development process. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) equity guidelines, for one, recommend the inclusion of underrepresented stakeholder groups in the guideline development and implementation process [30]. Stakeholders will have various levels of time, resources, and skills available to dedicate to the process, and ensuring that these differences do not result in certain stakeholder having more influence over the final guideline recommendations is important. A challenging aspect of engaging stakeholders is ensuring that interests are declared and that conflicts of interest are appropriately managed. Despite the importance of conflict of interest for guideline development, there is high variability in the process of disclosure and management of such conflicts across different organizations [27]. As a consequence, there are many inconsistencies in how stakeholder engagement is considered across guideline development groups. There should be opportunities for a variety of opinions to be heard, but it is vital that the recommendations made are objective and not unduly influenced by vested interests. Strategies such as active outreach activities, giving adequate time for comment on guideline recommendations and using processes with follow-up prompts that ensure all stakeholder comments are systematically addressed may assist in reducing potential inequities and increase guideline development transparency [12].

Study aim

The study aim is to develop guidance for guideline developers that supports the equitable and meaningful engagement of multiple stakeholders throughout the guideline development and implementation process. Guideline development will be used to encompass development, implementation, and evaluation processes. The objectives, in terms of participants, interventions, and comparators, are stakeholder/stakeholder groups engaged in guideline development, engagement in the guideline development process, and no comparator. Outcomes are outlined below for each phase of the project.

Key definitions

We define below the terms guideline, stakeholder, stakeholder engagement, levels of engagement, and under-represented groups used for this project.

Guideline

Guidelines are “systematically developed evidence-based statements which assist providers, recipients, and other stakeholders informed decisions about appropriate health interventions” [31].

Stakeholder

A stakeholder is any “individual or group who is responsible for or affected by health- and healthcare-related decisions that can be informed by research evidence” [32]. Further, we acknowledge that some stakeholders may use guidelines to inform decision-making (CIHR terms these stakeholders, knowledge users), while others may have an interest in the recommendations for other reasons [32,33,34]. We define eight stakeholder groups for this project, namely (1) persons and the public (e.g., patients, their caregivers, families, and patient and consumer advocacy organizations), (2) providers (individuals/organizations that provide care, e.g., nurses, physicians, pharmacists, mental health counselors, community-based workers), (3) payers (pays for or reimburses for health-related interventions, e.g., insurers, individuals with deductibles, others responsible for reimbursement for health-related interventions), (4) purchasers (e.g., employers, self-insured, governments, and other entities responsible for underwriting the cost of care), (5) policymakers (policymaking entities such as governments and professional associations), (6) product makers (e.g., drug/device manufacturers), (7) principal investigators (e.g., researchers), and (8) the press (e.g., publishers, news media) [32, 33].

Engagement

“Engagement” refers to the approach to gather input or contribution from stakeholders “toward the development of a guideline, completion of any stages of a guideline, or dissemination, uptake or evaluation of a guideline and its recommendations” [35]. Engagement is considered multi-directional, resulting in “informed decision-making about the selection, conduct, and use of the research” [32]. Depending on the context, engagement may also be termed collaboration, involvement, or partnership [36]. Herein, we will use the term “stakeholder engagement.”

Levels of engagement (Table 1)

The extent to which stakeholders are engaged in the guideline development process can vary. We identify four levels of engagement (see Additional file 1), adapted from previous work [37,38,39]: (1) Communication—stakeholders receive information but have no role in contributing; (2) Consultation—stakeholders provide their views, thoughts, feedback, opinions, or experiences but without a commitment to act on them; (3) Collaboration—stakeholders are engaged to influence the production of the guidelines (e.g., commenting, advising, ranking, voting, prioritizing, and reaching consensus) but without direct control over decisions; and (4) Coproduction—stakeholders are equal members of the guideline development team and participate in all steps of the guideline development process. Members benefit from each other’s knowledge, skills, and perspectives and build relationships in an open, trusting, and transparent atmosphere that encourages learning from each other. With ongoing collaboration and engagement, all members have an equal opportunity to influence each aspect of the guideline development process [25, 35, 37, 40,41,42].

Table 1 Levels of engagement [37,38,39]
Full size table

Under-represented groups

Under-represented groups refer to those individuals or groups who may experience health inequities for reasons such as a lack of inclusion in research, health policy, or guideline development; barriers to access of health services; or because of other socially stratifying factors, such as their place of residence, race/ethnicity/culture/language, occupation, gender/sex, subject matter knowledge, religion, education, socioeconomic status, social capital, age, or other individual characteristics [27, 28].

Study design and methods

We adapted our methods from the guidance for developing research reporting guidelines by Moher et al. [43] and our internal Terms of Reference for GRADE project groups document. Moher and colleagues recommend identifying the need for guidance, reviewing the literature, identifying participants, conducting a Delphi survey to gather opinions and set priorities, and holding a face-to-face consensus meeting. We have made a slight modification by adding key informant interviews to gather opinions on what should be included in the preliminary guidance. The conception and design of the study is the product of a global consortium for Multi-Stakeholder Engagement, entitled MuSE.

The MuSE consortium

The MuSE consortium was established in 2015 and includes over 80 researchers and stakeholders in various countries including: Australia, Brazil, Canada, Germany, Italy, Lebanon, the Netherlands, the Philippines, Switzerland, the UK, and the USA. The team includes researchers, policymakers, guideline developers, research funders, clinicians, patients and patient representatives, and policymakers from various organizations including the Agency for Healthcare Research and Quality (AHRQ), the Campbell Collaboration, Cochrane, GRADE Working Group, Health Canada, Patient-Centered Outcomes Research Institute (PCORI), Research and Development (RAND) Corporation, the World Health Organization (WHO), Joanna Briggs Institute (JBI), and multiple universities. All team members share an interest in developing methods and approaches for involving patients and other stakeholders in health outcomes research [44]. The MuSE consortium includes three working groups with each undertaking projects related to the development of methods for involving multiple stakeholders in health outcomes research (see Additional file 2).

Stakeholder engagement in the MuSE project

We will develop a stakeholder engagement framework for this study that outlines how we will engage and evaluate our own stakeholder engagement processes throughout the project, drawing on principles of realist evaluation to explore what works, for whom, why, and in what context [45]. We have adopted a co-production approach, drawing on the National Institute for Health Research (NIHR) guidance which pays attention to sharing power, including the perspectives and skills of all involved, respecting and valuing the knowledge of all, reciprocity, and building and maintaining relationships [25, 37, 40,41,42, 46]. We will work with our stakeholder to agree on the best ways to operationalize those key elements of co-production. We have assembled a large, international project team of co-investigators, collaborators, and stakeholders representing the various stakeholder groups as part of the MuSE consortium. All members of the consortium will be invited to provide advice and collaborate throughout the planning of each project stage, interpreting results, and coproducing the final guidance paper. To ensure accurate and transparent reporting of our stakeholder engagement throughout this project, we will follow the Guidance for Reporting Involvement of Patients and the Public (GRIPP2) checklist [47]. We will document and report on the methods used to engage our stakeholders, the results of the stakeholder engagement, the extent to which stakeholders’ input influenced the guidance development process and outcomes, and the lessons learned from the experience [47].

GIN McMaster Guideline Development Checklist

This study will use the 18 steps contained in Schünemann et al.’s Guidelines 2.0 checklist, known as the GIN McMaster Guideline Development Checklist, for guideline development as its organizing framework [7]. It provides guideline developers with a comprehensive checklist of items linked to relevant resources and tools to facilitate the guideline development process. The GIN McMaster Guideline Development Checklist has been used by guideline developers in various settings and was used to develop extensions for specific aspects of guideline development such as adaptation and rapid guideline development [48,49,50,51,52]. The 18 steps begin with the organization, budgeting, and planning of the development process, and continue with steps for priority setting, guideline group membership, question generation, all the way to developing recommendations, dissemination, implementation, and evaluation of the guidelines. Item six on the checklist relates to consumer and stakeholder involvement. As opposed to identifying stakeholder engagement as one step along the multi-step process of guideline development, the MuSE study will seek to document evidence-based guidance for stakeholder engagement for each step in the guideline development process (see Additional file 3).

Equity and the PROGRESS-Plus framework

The Canadian Institutes for Health Research (CIHR) expects that all research applicants will integrate gender and sex into their research designs when appropriate [53]. The MuSE project recognizes gender as an important social determinant of health and contributor to health inequities. Gender refers to “the socially constructed characteristics of women and men—such as norms, roles, and relationships of and between groups of women and men. It varies from society to society and can be changed” [54]. Sex is “the different biological and physiological characteristics of males and females, such as reproductive organs, chromosomes, hormones, etc.” [54]. Gender considerations will be integrated throughout the MuSE project as part of the project’s adoption of the PROGRESS-Plus framework for equity. The PROGRESS-Plus framework identifies socially stratifying factors that can contribute to health inequities [55]. We define health inequities as difference in health status that is avoidable, unfair, and unjust [56]. PROGRESS refers to Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, and Social capital. The Plus extends the original framework to include personal characteristics associated with discrimination (e.g., age, disability), features of relationships (e.g., children of smoking parents), and time-dependent relationships (e.g., release from incarceration) [55]. We will use the framework to guide the integration of health equity considerations throughout the project. Below, we describe the project stages and the integration of equity in each.

Project stages

This study includes various stages (Fig. 1). Briefly, they are (1) the conduct of four concurrent systematic reviews, (2) the development of draft guidance, (3) an online international survey of external stakeholders and experts, (4) a consensus meeting of project stakeholders, and (5) the finalization of the guidance paper. This protocol focuses on stage 1.

Fig. 1
figure 1

Project plan

Full size image

Stage 1: Systematic reviews of the literature

We will conduct four systematic reviews to identify and assess the available literature on stakeholder engagement in guideline development. This protocol will yield four distinct but related systematic reviews that will inform the overall protocol output. To improve efficiency, we will conduct an integrated literature search, screening, and data extraction process. Once included, articles will be tagged for their relevance to a particular review(s). Analyses and reports will be conducted separately for each review. Given the qualitative and multi-stage nature of this study, we believe the GRIPP2 checklist is more suitable for reporting the conduct of this project. The systematic review stage will be developed following the PRISMA-P checklist (Additional file 4)

Below, we briefly describe the objectives of each review and the methods common to all reviews.

Review 1: Existing guidance on stakeholder engagement in guideline development

The objective of this review is to synthesize existing guidance for stakeholder engagement at each of the 18 steps in the guideline development process. Eligible studies will describe the development of a framework or process for stakeholder engagement in guideline development. This includes frameworks or processes that address the implementation or evaluation of stakeholder engagement in guideline development. Outcomes to be synthesized are methods for (1) identifying stakeholders, (2) engaging stakeholders at different stages of the guideline development process, and (3) resolving differences in opinions/perspectives. In addition, frequency and level of engagement of stakeholders at each step of the guideline development process, level of engagement in each step of the guideline development process, and how stakeholders contributed at each step of the guideline development process will be synthesized.

Review 2: Barriers and facilitators to stakeholder engagement in guideline development

The expected result of this review is to summarize the barriers and facilitators to stakeholder engagement at each step in the guideline development process. Eligible studies will describe or assess the barriers and facilitators to stakeholder engagement in guideline development. A “barrier” is defined as any variable or condition that impedes stakeholder engagement in guideline development or implementation. A “facilitator” is defined as any variable or condition that promotes stakeholder engagement in guideline development or implementation.

Review 3: Conflicts of interest of multiple stakeholders in guideline development

The objective is to systematically review the literature on conflict of interest issues when engaging stakeholders in guideline development. Specifically, this review aims to answer the following questions: (1) What are the types of conflicts of interest that stakeholders engaged in the guideline development process have and how do they vary by stakeholder group? (2) Can the conflicts of interest of individuals or organizations selecting stakeholders to participate in the development of a guideline affect that selection process? (3) What are the potential effect(s) of conflicts of interest of stakeholders on the guideline development process? (4) What are the proposed and/or implemented approaches for managing the conflicts of interest of stakeholders engaged in guideline development, and what are their respective advantages and disadvantages?

Review 4: Impact of stakeholder engagement in guideline development

This review will assess the impact of stakeholder engagement in guideline development on (a) the guideline development process; (b) guideline relevance, trustworthiness, acceptability, and uptake; and (c) the stakeholders and panel members themselves. Eligible studies will identify or assess the impact of multi-stakeholder engagement through all the stages of guideline development.

Eligibility

Included studies will discuss stakeholder engagement in guideline development that assesses existing guidance and methods for stakeholder engagement at each stage of the guideline development and implementation process, barriers and facilitators to stakeholder engagement, conflicts of interest of stakeholders in the guideline development and implementation process, and/or impact of stakeholder engagement on the guideline development and implementation process.

Population

For our purposes, stakeholders in guideline development are as described above under the “Key definitions” section.

Intervention

Eligible studies will involve/engage stakeholders in some role(s) during the guideline development process. Engagement and guidelines are as described above under the “Key definitions” section.

Comparator

Studies that do not involve stakeholders in the guideline development process will not be eligible for inclusion in this project.

Study designs

All reviews will include quantitative, qualitative, and mixed-methods studies. As such, our methods will follow the Cochrane Handbook for Systematic Reviews of Interventions and the Handbook for Synthesizing Qualitative Research, as appropriate [57, 58]. We will include randomized trials, non-randomized studies (e.g., cohort studies, before and after studies, cross-sectional studies), qualitative studies, theoretical and ethical papers, process evaluation studies, policy analysis studies, case studies, and mixed-methods studies. We will exclude editorials, commentaries, proposals, and conference abstracts. Studies without a clear methods section will be excluded.

Search strategy

We will develop one comprehensive search strategy to identify relevant studies for the four reviews. We will search the following databases: MEDLINE (OVID), CINAHL (EBSCO), EMBASE (OVID), PsycInfo (OVID), AMED (OVID), and SCOPUS. We will not place limits on language, date, or study design. In addition, we will perform both backward and forward citation tracking to identify further eligible studies. A draft of the search strategy is provided (see Additional file 5).

To identify grey literature, we will search the websites of agencies who actively engage stakeholder groups such as the AHRQ, Canadian Institute of Health Research (CIHR) Strategy for Patient-Oriented Research (SPOR), INVOLVE, the National Institute for Health and Care Excellence (NICE), and the PCORI. We will also search the websites of guideline-producing agencies, such as the American Academy of Pediatrics, Australia’s National Health Medical Research Council (NHMRC), and the WHO. We will invite members of the team to suggest grey literature sources, and we plan to broaden the search by soliciting suggestions via social media, such as Twitter (https://twitter.com/CochraneEquity).

Study selection

Titles and abstracts of the studies identified by the search strategy and the full texts of those assessed as potentially relevant will be screened independently, in duplicate using Covidence software [38]. Disagreements on study selection will be resolved by discussion or with a third member of the research team when necessary. Eligible studies will be exported into an Excel spreadsheet and independently “tagged” for their relevance to each review. One study may be included in more than one review. For example, a study may describe a process for stakeholder engagement in guideline development as well as describe the barriers and facilitators to the stakeholder engagement.

Data extraction

The data extraction form will be pre-tested and will include (as applicable) factors related to the population, intervention, comparison, and outcomes. Extracted data common to all reviews include information on study characteristics (e.g., year of publication, authors, type of publication), stakeholders included, stakeholder roles, and stakeholder characteristics. The data will be extracted independently in duplicate by two reviewers and will be piloted on ten articles. Disagreements on extractions will be resolved by discussion and with a third member of the research team when necessary. Where necessary, the corresponding author of eligible studies will be contacted for additional information

We will use the 18 steps of the guideline development process as a framework for data extraction and will map the existing guidance, barriers and facilitators, conflicts of interest, and impacts discussed in our included studies to one or more of the steps of this framework [7]. Further, we will extract data that describe study populations and results or findings by PROGRESS-Plus factors.

Risk of bias

We will examine the methodological quality of the included studies as appropriate. We will use the risk of bias tools from the Cochrane Handbook for randomized trials (ROB2) [59], the Risk of Bias in Non-randomized Studies – of Interventions (ROBINS-I) tool [60], and the Critical Appraisal Skills Programme (CASP) qualitative appraisal research tool [61] for qualitative studies. Risk of bias will be assessed independently, in duplicate, by two authors, and any discrepancies will be resolved by consensus and consultation with a third author, when necessary.

Analyses

We will use a mixed-methods approach to summarize our findings. Qualitative and quantitative studies will be analyzed and synthesized separately, while the implications for practice, policy, and research that will form the discussion and conclusion sections of the reviews will draw on both the qualitative and quantitative syntheses. We will summarize findings across the 18-steps of the GIN-McMaster Checklist, the eight stakeholder groups, and the PROGRESS-Plus factors. We will report the systematic reviews following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) [62] and the Enhancing Transparency in Reporting the Synthesis of Qualitative Research (ENTREQ) [63] reporting guidelines, as appropriate. The GRADE or Confidence in the Evidence from Reviews of Qualitative Research (CERQual) methodology will be used to evaluate the quality of evidence for each review as appropriate.

Stage 2: Drafting the guidance

In consultation with the MuSE consortium, we will use the results of the four systematic reviews to develop a candidate list of recommendations to include in the final guidance paper. These draft recommendations will provide guidance mapped to the 18 steps of the guideline development process which will allow guideline developers to consider when and how to engage different stakeholders at each stage of the guideline development process. Equity considerations at each step of the guideline development process will be outlined.

Drafting and refining the guidance will involve structured engagement similar to that described by Jull and colleagues [64] with all MuSE consortium members. Because of our large and geographically diverse team, we will consult with members via email, teleconferences, and face-to-face meetings, as appropriate, to revise the guidance and ensure that all team members have equal opportunity to contribute and influence the research output. We will work with our MuSE consortium members to equitably include diverse voices in the drafting process, including visible minorities, women, members from low- and middle-income countries, and others who are traditionally omitted from guideline development processes.

Discussion

Stakeholder engagement should be multidirectional, meaningful, effective, and enable equity for both the stakeholders and guideline developers throughout all steps in the decision-making process. This project is innovative in that we are committed to an inclusive, comprehensive, and equitable approach to ensure that the guidance we develop is representative and relevant for all stakeholders, including those who are involved in creating and implementing guidelines and those affected by the recommendations developed within those guidelines. Our stakeholder engagement strategy for the project will assist us in effectively engaging our own stakeholders and allow us to monitor our engagement processes in real-time so that course corrections can be made if there is evidence of non-meaningful engagement.

We are not including language or date restrictions on our search strategy. A potential limitation of these reviews is that we are including papers regardless of their methodological quality. This will allow us to collect and synthesize qualitative data that we may miss if standard methodological criteria are applied. We will discuss limitations further in the full review.

These reviews will contribute to the literature by identifying existing guidance, barriers and facilitators, potential impacts, and possible conflict of interest issues related to engagement of stakeholders from many stakeholder groups in guideline development and implementation.

The results of the four reviews will inform the development of draft guidance. Once drafted, we will gather opinions and priorities on the guidance items from a wide range of purposefully selected stakeholders external to the MuSE consortium, including representation from low- and middle-income countries through an anonymous, online survey. We will strive for equity in the identification of survey recipients by engaging with a diversity of respondents representing different physical capabilities, genders, geographies, socio-economic statuses, and ethnicities. We will then present the results of the survey for each candidate item and use structured discussions to reach consensus on the included items for the final guidance paper at a two-day face-to-face consensus meeting as recommended by Moher et al. [43]. Finally, based on the results of the previous stages, we will develop guidance that provides recommendations for stakeholder roles and modes of engagement at different steps of the guideline development process (including implementation and evaluation) and for managing conflicts of interest. We will use an iterative process of feedback to draft, refine, and finalize the guidance to be provided in each manuscript in consultation with the co-authors of each paper and the other members of the MuSE consortium. The final product will be included in GRADE Working Group Guidance. The GRADE Working Group has developed internationally recognized guidance for the development of clinical practice and public health guidelines [65].

The expected final guidance will contribute to improving the guideline development and implementation process by identifying strategies for the meaningful and equitable engagement of all relevant stakeholder groups at all stages. Through this project, we aim to contribute to the growing body of literature on stakeholder engagement for better quality guidance, increased uptake of guidance, more relevant health programs, policies and services, and more equitable health outcomes.

We continually welcome additional expressions of interest and suggestions for relevant literature and plan to evaluate our own stakeholder engagement throughout this work to ensure meaningful engagement.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

Abbreviations

AHRQ:

Agency for Healthcare Research and Quality

CASP:

Critical Appraisal Skills Programme

CERQual:

Confidence in the Evidence from Reviews of Qualitative Research

CIHR:

Canadian Institute of Health Research

ENTREQ:

Enhancing Transparency in Reporting the Synthesis of Qualitative Research

GRADE:

Grading of Recommendations, Assessment, Development and Evaluation

GRIPP:

Guidance for Reporting Involvement of Patients and the Public

JBI:

Joanna Briggs Institute

MuSE:

Multi-stakeholder engagement

NICE:

National Institute for Health and Care Excellence

NIHR:

National Institute for Health Research

PCORI:

Patient-Centered Outcomes Research Institute

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analysis

PROGRESS:

Place of residence, Race/ethnicity/culture/language, Occupation, Gender/sex, Religion, Education, Socioeconomic status, and Social capital

ROB:

Risk of bias

ROBINS-I:

Risk of Bias in Non-randomized Studies – of Interventions

SPOR:

Strategy for Patient-Oriented Research

WHO:

World Health Organization

References

  1. WHO. WHO Handbook for Guideline Development – 2nd Edition [Internet]. World Health Organization, Geneva, Switzerland; 2014 [cited 2019 Apr 5]. Available from: http://apps.who.int/medicinedocs/en/m/abstract/Js22083en/

  2. Kredo T, Bernhardsson S, Machingaidze S, Young T, Louw Q, Ochodo E, et al. Guide to clinical practice guidelines: the current state of play. Int J Qual Health Care J Int Soc Qual Health Care. 2016 Feb;28(1):122–8.

    Article  Google Scholar 

  3. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical practice guidelines we can trust [Internet]. Graham R, Mancher M, Miller Wolman D, Greenfield S, Steinberg E, editors. Washington (DC): National Academies Press (US); 2011 [cited 2019 Apr 5]. Available from: http://www.ncbi.nlm.nih.gov/books/NBK209539/

  4. Sonnad SS. Organizational tactics for the successful assimilation of medical practice guidelines. Health Care Manage Rev. 1998;23(3):30–7.

    Article  CAS  PubMed  Google Scholar 

  5. Moulding NT, Silagy CA, Weller DP. A framework for effective management of change in clinical practice: dissemination and implementation of clinical practice guidelines. BMJ Qual Saf. 1999 Sep 1;8(3):177–83.

    Article  CAS  Google Scholar 

  6. Gagliardi AR, Brouwers MC. Integrating guideline development and implementation: analysis of guideline development manual instructions for generating implementation advice. Implement Sci IS. 2012 Jul 23;7:67.

    Article  PubMed  Google Scholar 

  7. Schünemann HJ, Wiercioch W, Etxeandia I, Falavigna M, Santesso N, Mustafa R, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ Can Med Assoc J J Assoc Medicale Can. 2014;186(3):E123–42.

    Article  Google Scholar 

  8. Jarret L, Patient Involvement Unit. A report on a study to evaluate patient/carer membership of the first NICE Guideline Development Groups [Internet]. National Institute for Clinical Excellence; 2004 [cited 2019 Apr 24]. Available from: https://www.nice.org.uk/media/default/About/NICE-Communities/Public-involvement/Public-involvement-programme/PIU-GDG-evaluation-report-2004-1.pdf

  9. Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012 Apr 3;156(7):525–31.

    Article  PubMed  Google Scholar 

  10. Schünemann HJ, Fretheim A, Oxman AD, WHO. Advisory Committee on Health Research. Improving the use of research evidence in guideline development: 1. Guidelines for guidelines. Health Res Policy Syst. 2006 Nov 21;4:13.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Guidelines I of M (US) C on S for DTCP, Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E. Committee on standards for developing trustworthy clinical practice guidelines [Internet]. National Academies Press (US); 2011 [cited 2019 Apr 24]. Available from: https://www.ncbi.nlm.nih.gov/books/NBK209544/

  12. Cluzeau F, Wedzicha JA, Kelson M, Corn J, Kunz R, Walsh J, et al. Stakeholder involvement: how to do it right: article 9 in integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report. Proc Am Thorac Soc. 2012 Dec;9(5):269–73.

    Article  PubMed  Google Scholar 

  13. NHMRC. 2016 NHMRC Standards for Guidelines | NHMRC [Internet]. 2016 [cited 2019 Apr 5]. Available from: https://www.nhmrc.gov.au/about-us/publications/2016-nhmrc-standards-guidelines

  14. Gillard S, Simons L, Turner K, Lucock M, Edwards C. Patient and public involvement in the coproduction of knowledge: reflection on the analysis of qualitative data in a mental health study. Qual Health Res. 2012 Aug;22(8):1126–37.

    Article  PubMed  Google Scholar 

  15. Oliver K, Lorenc T, Innvær S. New directions in evidence-based policy research: a critical analysis of the literature. Health Res Policy Syst. 2014;12:34.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Armstrong MJ, Rueda J-D, Gronseth GS, Mullins CD. Framework for enhancing clinical practice guidelines through continuous patient engagement. Health Expect Int J Public Particip Health Care Health Policy. 2017;20(1):3–10.

    Google Scholar 

  17. Carroll C. Qualitative evidence synthesis to improve implementation of clinical guidelines. BMJ. 2017 Jan 16;356:j80.

    Article  PubMed  Google Scholar 

  18. Jackson CL, Greenhalgh T. Co-creation: a new approach to optimising research impact? Med J Aust. 2015 Oct 5;203(7):283–4.

    Article  PubMed  Google Scholar 

  19. Ghaffar A, Langlois EV, Rasanathan K, Peterson S, Adedokun L, Tran NT. Strengthening health systems through embedded research. Bull World Health Organ. 2017 01;95(2):87.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Esmail L, Moore E, Rein A. Evaluating patient and stakeholder engagement in research: moving from theory to practice. J Comp Eff Res. 2015 Mar;4(2):133–45.

    Article  PubMed  Google Scholar 

  21. Armstrong MJ, Bloom JA. Patient involvement in guidelines is poor five years after institute of medicine standards: review of guideline methodologies. Res Involv Engagem. 2017;3(1):19.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Lavis JN, Paulsen EJ, Oxman AD, Moynihan R. Evidence-informed health policy 2 – Survey of organizations that support the use of research evidence. Implement Sci. 2008;3(1):54.

    Article  PubMed  PubMed Central  Google Scholar 

  23. van de Bovenkamp HM, Zuiderent-Jerak T. An empirical study of patient participation in guideline development: exploring the potential for articulating patient knowledge in evidence-based epistemic settings. Health Expect. 2015;18(5):942–55.

    Article  PubMed  Google Scholar 

  24. Suman A, Dikkers MF, Schaafsma FG, van Tulder MW, Anema JR. Effectiveness of multifaceted implementation strategies for the implementation of back and neck pain guidelines in health care: a systematic review. Implement Sci. 2016 Sep 20;11(1):126.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Dunston R, Lee A, Boud D, Brodie P, Chiarella M. Co-production and health system reform - from re-imagining to re-making. Aust J Public Adm. 2009;68(1):39–52.

    Article  Google Scholar 

  26. Kumarasamy MA, Sanfilippo FP. Breaking down silos: engaging students to help fix the US health care system. J Multidiscip Healthc. 2015;8:101–8.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Shi C, Tian J, Wang Q, Petkovic J, Ren D, Yang K, et al. How equity is addressed in clinical practice guidelines: a content analysis. BMJ Open. 2014;4(12):e005660.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health. 2010 Apr 1;100 Suppl 1:S40-S46.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Selva A, Sanabria AJ, Pequeño S, Zhang Y, Solà I, Pardo-Hernandez H, et al. Incorporating patients’ views in guideline development: a systematic review of guidance documents. J Clin Epidemiol. 2017;88:102–12.

    Article  PubMed  Google Scholar 

  30. Akl EA, Welch V, Pottie K, Eslava-Schmalbach J, Darzi A, Sola I, et al. GRADE equity guidelines 2: considering health equity in GRADE guideline development: equity extension of the guideline development checklist. J Clin Epidemiol. 2017;90:68–75.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Global Programme on Evidence for Health Policy. Guideline for WHO Guidelines [Internet]. World Health Organization, Geneva, Switzerland; 2003 [cited 2019 Apr 24]. Available from: http://archives.who.int/eml/expcom/expcom14/1other/guid_for_guid.pdf

  32. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise J-M, Santa J, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27(8):985–91.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Tugwell P, Robinson V, Grimshaw J, Santesso N. Systematic reviews and knowledge translation. Bull World Health Organ. 2006;84(8):643–51.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Canadian Institutes of Health Research. Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches [Internet]. Ottawa, Ontario; 2012 [cited 2019 May 29]. Available from: http://www.cihr-irsc.gc.ca/e/documents/kt_lm_ktplan-en.pdf, http://www.cihr-irsc.gc.ca/e/documents/kt_lm_ktplan-en.pdf

  35. Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, et al. Stakeholder involvement in systematic reviews: a scoping review. Syst Rev. 2018;7(1):208.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Hoddinott P, Pollock A, O’Cathain A, Boyer I, Taylor J, MacDonald C, et al. How to incorporate patient and public perspectives into the design and conduct of research. F1000Research. 2018 Jun 18;7:752.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Oliver SR, Rees RW, Clarke-Jones L, Milne R, Oakley AR, Gabbay J, et al. A multidimensional conceptual framework for analysing public involvement in health services research. Health Expect Int J Public Particip Health Care Health Policy. 2008;11(1):72–84.

    Google Scholar 

  38. Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, et al. Development of the ACTIVE framework to describe stakeholder involvement in systematic reviews. J Health Serv Res Policy. 2019;18:1355819619841647.

    Google Scholar 

  39. Crowe S. “Who inspired my thinking?- Sherry Arnstein.” Res All. 2017;1(1):143-146(4).

  40. INVOLVE-National Institute for Health Research. Guidance on co-producing a research project [Internet]. Hampshire, UK; 2018 [cited 2019 Jun 7]. Available from: https://www.invo.org.uk/posttypepublication/guidance-on-co-producing-a-research-project/

  41. Markkanen S, Burgess G. Introduction to co-production in research: summary report; 2016.

    Google Scholar 

  42. Pohl C, Rist S, Zimmermann A, Fry P, Gurung GS, Schneider F, et al. Researchers’ roles in knowledge co-production: experience from sustainability research in Kenya, Switzerland. Bolivia and Nepal. Sci Public Policy. 2010;37(4):267–81.

    Article  Google Scholar 

  43. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Concannon TW, Grant S, Welch V, Petkovic J, Selby J, Crowe S, et al. Practical guidance for involving stakeholders in health research. J Gen Intern Med. 2019;34(3):458–63.

    Article  PubMed  Google Scholar 

  45. Wilson P, Mathie E, Keenan J, McNeilly E, Goodman C, Howe A, et al. ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study [Internet]. Southampton (UK): NIHR Journals Library; 2015 [cited 2019 Jun 12]. (Health Services and Delivery Research). Available from: http://www.ncbi.nlm.nih.gov/books/NBK315999/

  46. National Institute for Health Research. National Standards for Public Involvement In Research [Internet]. 2018 [cited 2019 Jun 12]. Available from: https://www.nihr.ac.uk/news-and-events/documents/Public_Involvement_Standards_March%202018_WEB.pdf

  47. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem. 2017;3:13.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Falzon D, Schünemann HJ, Harausz E, González-Angulo L, Lienhardt C, Jaramillo E, et al. World Health Organization treatment guidelines for drug-resistant tuberculosis, 2016 update. Eur Respir J. 2017;49(3).

    Article  PubMed  Google Scholar 

  49. Alonso-Coello P, Oxman AD, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: Clinical practice guidelines. BMJ. 2016 Jun 30;353:i2089.

  50. Fiocchi A, Pawankar R, Cuello-Garcia C, Ahn K, Al-Hammadi S, Agarwal A, et al. World Allergy Organization-McMaster University guidelines for allergic disease prevention (GLAD-P): probiotics. World Allergy Organ J. 2015;8(1):4.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Laine L, Kaltenbach T, Barkun A, McQuaid KR, Subramanian V, Soetikno R, et al. SCENIC international consensus statement on surveillance and management of dysplasia in inflammatory bowel disease. Gastrointest Endosc. 2015 Mar;81(3):489-501.e26.

    Article  PubMed  Google Scholar 

  52. Farrell B, Tsang C, Raman-Wilms L, Irving H, Conklin J, Pottie K. What are priorities for deprescribing for elderly patients? Capturing the voice of practitioners: a modified delphi process. PloS One. 2015;10(4):e0122246.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  53. Canadian Institutes of Health Research. How to integrate sex and gender into research [Internet]. 2018 [cited 2019 Jun 11]. Available from: http://www.cihr-irsc.gc.ca/e/50836.html

  54. World Health Organization. Gender mainstreaming for health managers: a practical approach [Internet]. Geneva : World Health Organisation; 2011 [cited 2019 Jun 11]. Available from: https://apps.who.int/iris/handle/10665/44516

  55. O’Neill J, Tabish H, Welch V, Petticrew M, Pottie K, Clarke M, et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014 Jan 1;67(1):56–64.

    Article  PubMed  Google Scholar 

  56. Whitehead M. The concepts and principles of equity and health. Int J Health Serv. 1992;22(3):429–45.

    Article  CAS  PubMed  Google Scholar 

  57. Sandelowski M, Barroso J. Handbook for synthesizing qualitative research. New York: Springer Publishing Company; 2007. 284 p.

  58. Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions [Internet]. The Cochrane Collaboration; 2011 [cited 2019 Apr 5]. Available from: https://handbook-5-1.cochrane.org/

  59. Higgins JPT, Sterne JAC, Savović j., Page MJ, Hróbjartsson A, Boutron I, et al. Appraising the risk of bias in randomized trials using the Cochrane Risk of Bias Tool. In: Cochrane Methods 2016 J Chandler, J McKenzie, I Boutron and V Welch (editors) Cochrane Database of Systematic Reviews 10 (Suppl 1) [Internet]. Available from: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD201601/full

  60. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016;355:i4919.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Critical Appraisal Skills Programme. Casp Checklists. Casp. 2013.

  62. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12:181.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Jull J, Petticrew M, Kristjansson E, Yoganathan M, Petkovic J, Tugwell P, et al. Engaging knowledge users in development of the CONSORT-Equity 2017 reporting guideline: a qualitative study using in-depth interviews. Res Involv Engagem. 2018;4:34.

    Article  PubMed  PubMed Central  Google Scholar 

  65. Hultcrantz M, Rind D, Akl EA, Treweek S, Mustafa RA, Iorio A, et al. The GRADE Working Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4–13.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We would like to thank the entire MuSE consortium membership for their valuable contributions and advice.

Funding

This work has been funded by the Canadian Institutes of Health Research (PJT-155970; Application number: 391110). AS is supported by a National Health and Medical Research Council Postgraduate Scholarship (GNT 1132803). SVK is funded by a NRS Senior Clinical Fellowship (SCAF/15/02), the Medical Research Council (MRC_UU_12017/13 & MRC_UU_12017/15), and the Scottish Government Chief Scientist Office (SPHSU13 & SPHSU15). PC and AP are employed by the Nursing, Midwifery and Allied Health Professions Research Unit, which is funded by the Chief Scientist Office in Scotland. IDG is a recipient of a Canadian Institutes of Health Research Foundation Grant (FDN# 143237). The funders had no role in study design, preparation of manuscript, or the decision to publish.

Author information

Author notes

  1. Vivian Welch, Thomas W Concannon, and Peter Tugwell are senior authors

Authors and Affiliations

  1. Bruyère Research Institute, Bruyère Continuing Care and University of Ottawa, 85 Primrose Ave East, Ottawa, Ontario, Canada

    Jennifer Petkovic, Alison Riddle, Rosiane Simeon & Vivian Welch

  2. Department of Internal Medicine, American University of Beirut, Beirut, Lebanon

    Elie A. Akl

  3. Clinical Research Institute, American University of Beirut, Beirut, Lebanon

    Joanne Khabsa

  4. McMaster University, Hamilton, Ontario, Canada

    Lyubov Lytvyn

  5. Bruyère Research Institute, 85 Primrose Ave East, Ottawa, Ontario, Canada

    Pearl Atwere

  6. Nursing, Midwifery and Allied Health Professions (NMAHP) Research Unit, Glasgow Caledonian University, Glasgow, UK

    Pauline Campbell

  7. Faculty of Medicine, School of Public Health Imperial College, London, UK

    Kalipso Chalkidou

  8. Agency for Healthcare Research and Quality, Rockville, MD, USA

    Stephanie M. Chang

  9. Crowe Associates Ltd., Oxford, UK

    Sally Crowe

  10. Department of Clinical Epidemiology, University of the Philippines-Manila, Taft Ave, 1000, Manila, Philippines

    Leonila Dans

  11. American University of Beirut, Beirut, Lebanon

    Fadi El Jardali

  12. National Health and Medical Research Council, Canberra, Australia

    Davina Ghersi

  13. School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada

    Ian D. Graham

  14. Ottawa Hospital Research Institute, Ottawa, Ontario, Canada

    Ian D. Graham

  15. Department of Social & Behavioral Sciences, Richard M. Fairbanks School of Public Health, Indiana University, 1050 Wishard Blvd, RG 6046, Indianapolis, IN, 46202, USA

    Sean Grant

  16. LLC/S.T.A.R. Initiative, Apple Valley, CA, USA

    Regina Greer-Smith

  17. Departments of Obstetrics and Gynecology, Medical Informatics & Clinical Epidemiology, Emergency Medicine Oregon Health & Science, University School of Medicine and the OHSU-PSU School of Public Health, Portland, USA

    Jeanne-Marie Guise

  18. University of Calgary, Calgary, Canada

    Glen Hazlewood

  19. School of Rehabilitation Therapy, Faculty of Health Sciences, Queen’s University, Kingston, Ontario, Canada

    Janet Jull

  20. MRC/CSO Social & Public Health Sciences Unit, University of Glasgow, 200 Renfield Street, Glasgow, G2 3AX, UK

    S. Vittal Katikireddi

  21. Alliance for Health Policy and Systems Research, World Health Organization, Geneva, Switzerland

    Etienne V. Langlois

  22. Cochrane Musculoskeletal Group, London, Ontario, Canada

    Anne Lyddiatt

  23. University of Ottawa, Ottawa, Canada

    Lara Maxwell

  24. The Cochrane Collaboration, Cochrane Consumer Network, London, UK

    Richard Morley

  25. Department of Internal Medicine/Division of Nephrology and Hypertension, University of Kansas Medical Center, Kansas, USA

    Reem A. Mustafa

  26. Unit of Epidemiology and Statistics, IRCCS - Institute of Neurological Sciences of Bologna, Bologna, Italy

    Francesco Nonino

  27. Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Canada

    Jordi Pardo Pardo

  28. Nursing Midwifery and Allied Health Professions (NMAHP) Research Unit, Glasgow Caledonian University, Glasgow, UK

    Alex Pollock

  29. Departments of Family Medicine and Epidemiology and Public Health, University of Ottawa, Ottawa, Canada

    Kevin Pottie

  30. Primary Care Research Group and Equity Methods Group, Bruyère Research Institute, Ottawa, Canada

    Kevin Pottie

  31. Department of Family Medicine, McMaster University, Hamilton, Ontario, Canada

    John Riva

  32. Department of Health Research Methods, Evidence, and Impact, Cochrane Canada and McMaster GRADE Centre, Hamilton, Canada

    Holger Schünemann

  33. Cochrane Consumer Executive, Ottawa, Ontario, Canada

    Maureen Smith

  34. Programa de Pós-Graduação em Ciências da Saúde, Universidade Federal de Ciências da Saúde de Porto Alegre - UFCSPA, Rua Sarmento Leite 245, Porto Alegre, RS, 90050-170, Brazil

    Airton T. Stein

  35. Centre for Health Communication and Participation, School of Psychology and Public Health, La Trobe University, Melbourne, Australia

    Anneliese Synnot

  36. Cochrane Australia, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia

    Anneliese Synnot

  37. PCORI, Seattle, USA

    Janice Tufte

  38. Alfred Deakin University, Geelong, Australia

    Howard White

  39. Journal of Development Studies and Journal of Development Effectiveness, Geelong, Victoria, Australia

    Howard White

  40. The Campbell Collaboration, Oslo, Norway

    Vivian Welch

  41. The RAND Corporation, Boston, MA, USA

    Thomas W. Concannon

  42. Tufts Clinical & Translational Science Institute, Tufts University School of Medicine, Boston, MA, USA

    Thomas W. Concannon

  43. University of Ottawa, Department of Medicine, Faculty of Medicine, Ottawa, Canada

    Peter Tugwell

  44. Ottawa Hospital Research Institute, Clinical Epidemiology Program, Ottawa, Canada

    Peter Tugwell

  45. University of Ottawa, School of Epidemiology and Public Health, Faculty of Medicine, Ottawa, Canada

    Peter Tugwell

Authors
  1. Jennifer Petkovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alison Riddle
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elie A. Akl
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joanne Khabsa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lyubov Lytvyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Pearl Atwere
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pauline Campbell
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Kalipso Chalkidou
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Stephanie M. Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sally Crowe
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Leonila Dans
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Fadi El Jardali
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Davina Ghersi
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Ian D. Graham
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Sean Grant
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Regina Greer-Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Jeanne-Marie Guise
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Glen Hazlewood
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Janet Jull
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. S. Vittal Katikireddi
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Etienne V. Langlois
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Anne Lyddiatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Lara Maxwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Richard Morley
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Reem A. Mustafa
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Francesco Nonino
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Jordi Pardo Pardo
    View author publications

    You can also search for this author in PubMed Google Scholar

  28. Alex Pollock
    View author publications

    You can also search for this author in PubMed Google Scholar

  29. Kevin Pottie
    View author publications

    You can also search for this author in PubMed Google Scholar

  30. John Riva
    View author publications

    You can also search for this author in PubMed Google Scholar

  31. Holger Schünemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  32. Rosiane Simeon
    View author publications

    You can also search for this author in PubMed Google Scholar

  33. Maureen Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  34. Airton T. Stein
    View author publications

    You can also search for this author in PubMed Google Scholar

  35. Anneliese Synnot
    View author publications

    You can also search for this author in PubMed Google Scholar

  36. Janice Tufte
    View author publications

    You can also search for this author in PubMed Google Scholar

  37. Howard White
    View author publications

    You can also search for this author in PubMed Google Scholar

  38. Vivian Welch
    View author publications

    You can also search for this author in PubMed Google Scholar

  39. Thomas W. Concannon
    View author publications

    You can also search for this author in PubMed Google Scholar

  40. Peter Tugwell
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PT conceived the project, and JP wrote the first draft of this manuscript. JP, EAA, AS, PC, KC, SMC, SC, LD, DG, IDG, SG, JMG, JJ, EVL, AL, LM, RM, RAM, JPP, AP, KP, HS, VW, HW, TWC, and PT contributed to the grant proposal that initiated this project. JP, AR, TWC, LL, JK, EAA, RS, PA, and VW held regular project development meetings. ATS, MS, JR, SVK, JJ, TWC,RGS, PA, SC, VW, FEJ, FN, AP, JMG, IDG, EVL, JT, GH, SMC, PC, AR, JP, and EAA contributed to subsequent revisions to the manuscript. JP will lead the systematic review on existing guidance, AR will lead the systematic review of barriers and facilitators, LL will lead the systematic review on impact, and JK will lead the systematic review on conflicts of interest. All authors commented on each version of the draft manuscript. All authors approved the final manuscript. JP is the manuscript guarantor.

Corresponding author

Correspondence to Jennifer Petkovic.

Ethics declarations

Ethics approval and consent to participate

The international online survey stage of the study will be submitted to the Bruyère Research Institute’s Ethics Review Board for approval prior to launching the survey.

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1: Table S1.

Levels of engagement.

Additional file 2.

The MuSE Consortium.

Additional file 3.

PRISMA-P reporting checklist.

Additional file 4.

Stakeholders across the 18 steps.

Additional file 5.

Draft Search Strategy in Medline.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Petkovic, J., Riddle, A., Akl, E.A. et al. Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation. Syst Rev 9, 21 (2020). https://doi.org/10.1186/s13643-020-1272-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13643-020-1272-5

Keywords

Systematic Reviews

ISSN: 2046-4053