Article Text
PDF
Abstract
Objective Payers, providers and policymakers in the USA are interested in developing interventions that reduce preventable or modifiable healthcare use among high-need, high-cost (HNHC) patients. This study seeks to describe how and why complex interventions for HNHC patients lead to more appropriate use of healthcare services.
Design A realist review which develops programme theories from causal explanations generated and articulated through the creation of context–mechanism–outcome configurations.
Methods Electronic databases (including PubMed and Embase) and gray literature from January 2000 to March 2021 were searched. All study designs were included if the article provided data to develop our programme theories. Included studies were conducted in the USA and focused on interventions for adult, HNHC patients.
Results Data were synthesised from 48 studies. Identifying HNHC patients for inclusion in interventions requires capturing a combination of characteristics including their prior use of healthcare services, complexity of chronic disease(s) profile, clinician judgment and willingness to participate. Once enrolled, engaging HNHC patients in interventions requires intervention care providers and patients to build a trusting relationship. Tailored, individualised assistance for medical and non-medical needs, emotional support and self-management education empowers patients to increase their participation in managing their own care. Engagement of care providers in interventions to expand support of HNHC patients is facilitated by targeted outreach, adequate staffing support with shared values and regular and open communication.
Conclusions Building relationships with HNHC patients and gaining their trust is a key component for interventions to successfully change HNHC patients’ behaviors. Identifying HNHC patients for an intervention can be best achieved through a multipronged strategy that accounts for their clinical and psychosocial complexity and prior experiences with the healthcare system. Successful interventions recognise that relationships with HNHC patients require the sustained engagement of care providers. To succeed, providers need ongoing emotional, financial, logistical and practical resources.
PROSPERO registration number CRD42020161179.
- international health services
- health policy
- health economics
Data availability statement
Most of the data generated or analysed for this study are included in this published article, associated appendices or in the full report referenced in the study. Any additional data are available from EC upon request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
To the best of our knowledge, this is the first review study using a realist approach to assess US-based complex healthcare interventions.
Our focus on high-need, high-cost (HNHC) patients is important for policymakers and payers since the cost of care for HNHC patients accounts for a disproportionately large share of healthcare costs.
A realist review approach allows us to provide explanations for why interventions for HNHC patients may be effective.
While we included grey literature to more comprehensively identify interventions for HNHC patients, scant descriptions of interventions for HNHC patients and the experiences of patients and providers limited our ability to build context–mechanism–outcome configurations and programme theories.
Although we only included literature of studies conducted in the USA, many of the mechanisms identified in this review are likely to be applicable to HNHC patient interventions in other countries.
Introduction
In the USA, patients referred to as ‘high-need, high-cost’ (HNHC) have become a particular focus for payers, health systems and providers.1–4 Although a small percentage of the total patient population, HNHC patients account for a disproportionately large share of high-cost healthcare service use, particularly emergency department (ED) and hospital inpatient care.1 5 6 Poor control of chronic conditions, especially when coupled with functional limitations, mental health conditions, substance abuse and social needs can result in patients’ over-reliance on potentially preventable or modifiable ED and hospital care.1 2 7 8 Payers, notably the Centers for Medicare and Medicaid Services (CMS), are moving from fee-for-service payment arrangements to alternative payment models that require healthcare systems and providers to provide more cost-efficient care.2 9 The viability of these models can depend, in large part, on their ability to reduce the cost of care for HNHC patients.
Many interventions have been implemented that aim to reduce HNHC patients’ inappropriate healthcare use and improve their quality of care and health outcomes. Like other systematic reviews on HNHC patients,10 11 our recently completed systematic review found that the quantitative evaluations of these complex interventions primarily focused on measuring changes in healthcare costs and utilisation.12 Overall, these complex interventions demonstrated an inconsistent impact on these outcomes.11–13 Further, little evidence measuring other key outcomes, including changes in patient engagement or quality of care, limited the systematic review’s ability to explain ‘why’ the interventions may not have been successful.12
To help explain what may (or may not) be happening in these complex interventions, we conducted a companion realist review. Realist reviews use the available literature as the basis for developing theories to explain why an intervention is likely to work, how, for whom, in which circumstances and to what extent.14 Our goal was to be able to provide stakeholders with a greater understanding of the inner workings of the complex interventions intended to address the needs of HNHC patients. Specifically, we aimed to explain under what set of circumstances and why complex interventions for HNHC patients could lead to more appropriate use of healthcare services and better health-related outcomes. Our overarching research question was ‘what are the mechanisms in interventions serving HNHC patients that lead to reductions in potentially preventable or modifiable healthcare use and result in improved health outcomes and cost savings?’
Methods
Study design
We used a realist review approach to develop programme theories explaining and describing how complex interventions for HNHC patients work (or fail), for whom and in what particular contexts and settings, as supported by evidence from qualitative and quantitative studies.14 The approach facilitates unpacking and understanding the causal processes for achieving outcomes within complex interventions. Consistent with this approach, we developed context–mechanism–outcome (CMO) configurations. CMOs are the building blocks for developing programme theories that help explain the underlying (often unseen and intangible) causal mechanisms that underpin interventions.15 Realist review findings provide support for theories about why outcomes may differ because of subtle contextual conditions.
This review was part of a report that also contained a best fit framework synthesis that identified characteristics associated with patients being HNHC and a systematic review of the effectiveness of interventions for this population. The report was performed by the RTI-University of North Carolina Evidence-Based Practice Center for the Agency for Healthcare Research and Quality (AHRQ). We followed the Realist And Meta-narrative Evidence Syntheses: Evolving Standards (RAMESES) publication standards for realist reviews.16 17 Detailed methods and findings from the full review are reported elsewhere.12 A group of seventeen content and methodological experts were involved in shaping the review and providing feedback on the refined programme theories.
Programme theory development
The team, with input from our experts, first conceptualised a map of a ‘patient journey’ to illustrate how HNHC patients may be inappropriately using the healthcare system. We identified categories of interventions intended to improve the HNHC patient journey, based on a scan of the literature. After team discussions and review of existing theories, we identified that our initial programme theory would have to incorporate the following concepts: claims only strategies were not sufficient for identifying HNHC patients; HNHC patients are clinically and socially complex and interventions needed strategies to engage them meaningfully; and providers were more likely to participate in interventions that considered the differences in their experiences treating HNHC patients compared with a general patient population. Through iterative review of the literature and discussions with our experts, we developed our initial programme theories. We then used the data from included studies to refine our concepts to form our final realist programme theories.
Search strategy
We identified potentially relevant articles for the realist review through one search strategy that encompassed all components of the review and conducted additional targeted searches as needed. An experienced information specialist searched MEDLINE via PubMed, the Cochrane Clinical Trials Central Register, CINAHL, Embase, PsycINFO, Web of Science, Academic Search Premier and Scopus from 1 January 2000 to 4 March 2021. We also searched for grey literature from several relevant organisational and government websites for unpublished literature and non-peer reviewed reports and consulted with content experts to help identify relevant literature. Search strategy details can be found in online supplemental appendix 1.
Supplemental material
Study selection and data extraction
To increase the efficiency of our searching and screening process, reviewers screened titles and abstracts simultaneously for the best fit framework, systematic review and realist review. Studies were eligible if they focused on providing care for non-institutionalised US adults who were considered HNHC based on having high healthcare use or cost, measured over ≥6 months. We considered any study design eligible for inclusion. During full-text screening, we considered all articles that were included for the best fit framework and systematic review, as well as other identified articles that would be useful for developing our programme theories. Two reviewers independently screened the titles, abstracts and full texts.
In keeping with realist review principles, we evaluated articles for their relevance and robustness in answering our key questions and their contribution to theory building and testing.16 Data were abstracted into NVivo (V.12) using a coding framework grounded in our initial CMOs. Coding of all included studies proceeded as both an inductive and deductive process, that is, we continued to identify and add to the coding framework as we proceeded.
We used RAMESES project standards to guide our judgments in quality appraisal, considering relevance, trustworthiness, plausibility of the argument and methodological rigour.16–19
Data synthesis and programme theory refinement
After all included studies were coded, one investigator sorted and grouped excerpts from studies by concepts and developed initial, partial CMO configurations. A second investigator reviewed the CMOs for agreement with their own interpretation. The team then examined the CMOs alongside the developing programme theories and iteratively reworked the CMOs as more data were added and feedback was provided by our experts. Our final programme theories were developed after considering our full set of CMOs.
Patient and public involvement
Patients were not directly involved in the conduct of the review. However, patients and the public were invited to review and provide feedback on a draft of the full report during the public posting period (26 February 2021 to 26 March 2021).
Results
Figure 1 shows the flow of article selection. In total, 48 studies (51 articles) contributed to developing and refining our programme theories. By design, all studies were conducted in the USA to ensure they reflect the experiences of patients in the US healthcare system. The majority of studies were published in the peer-reviewed literature. Approximately 21% (10 studies) were found in the grey literature, including government reports evaluating large-scale complex interventions for CMS. See online supplemental appendix 2 for additional details about the included studies.
Flow of article selection for realist review.
We developed 39 CMO configurations, based on data in the included studies, and used these to iteratively refine three key programme theories to explain mechanisms of interventions for HNHC patients. Programme theory 1 describes the pathway for identifying and targeting participants for HNHC patient interventions. Once patients are enrolled, building and maintaining trusting relationships between patients and care providers can help interventions succeed. We set out key considerations for engaging HNHC patients (programme theory 2) and for engaging care providers (programme theory 3) so that each has the necessary emotional support and practical resources to develop and maintain effective, ongoing relationships. Figure 2 shows the relationship among the three programme theories. Online supplemental appendix 3 includes the data and CMO configurations that underpin the programme theories.
Framework of optimising interventions for HNHC patients
Programme theory 1: identifying and targeting HNHC patients for inclusion in interventions
A first challenge for interventions intending to reduce potentially preventable or modifiable high-cost healthcare use among HNHC patients is to accurately identify those who are most likely to inappropriately use services in the near future. Selecting patients exclusively based on prior high costs could miss individuals at the beginning of a high use period.1 Alternatively, identification based solely on chronic disease status will include individuals who are managing their conditions as best as can be expected.1 Therefore, identifying HNHC patients requires capturing their complexity based on a combination of prior use, chronic diseases, barriers to accessing care and willingness to participate.
Data for identifying HNHC patients
Administrative claims data and electronic health records specifying disease complexity and healthcare use and cost history are often included in models.20–29 While these data are necessary, no existing algorithm or model specification alone has been able to reliably predict future high-cost use.2 27 30 One barrier to developing accurate projections is ‘regression to the mean’ (the tendency for high use and cost in a baseline period to return to an average level in the follow-up period).26 28 31 32
Greater accuracy of projections may be facilitated by: limiting projections to participants with healthcare use that is the focus of an intervention (ie, care that is potentially preventable or modifiable)33; capturing the continuum of care across care settings and payers, including public health agencies and community-based organisations2 27 28 30 33–35; focusing on service use in ‘real time’,2 29 33 36 37 if possible, while a patient is still hospitalised26 32; and, when considering ED visits, focusing on visits that occur in clusters because a cluster pattern has been found to be associated with HNHC patient ED service use.38 39
Clinical data can identify complex chronic disease. Indicators include having multiple chronic conditions,31 functional limitations1 and high scores on clinical risk severity measures.29–31 36 40–43 Data on behavioural health conditions (mental health diagnoses and substance use disorders) and social needs (eg, poverty, social isolation) may improve algorithms.1 2 26 35 38 41 44 However, selecting the correct claims to use to identify these conditions can be obfuscated by the co-occurrence of multiple medical conditions.38 Fewer visits may be associated with behavioural health conditions than expected.38
Individually assessing and enrolling patients
Additional information obtained during the intake assessment can further refine predictions of patient risk of future inappropriate use of services. Self-assessment tools can capture patients’ own evaluations of their risk42 45 46 and their readiness to manage their conditions.47 Healthcare system-related barriers that have negatively affected patients’ ability to self-manage their conditions can be identified, such as prior experiences and distrust of providers and logistical and cost challenges.35 40 48–51
The intake assessment is facilitated by being conducted in-person and can determine a patient’s willingness to be monitored and comply with the intervention.32 33 40 50 52–54 Current providers can help judge appropriateness.30 52 54 Notably, socioeconomically disadvantaged or marginalised HNHC patients may appear less willing to participate50 54 and recognising subtle indicators or extending periods of outreach and trust building may be necessary.54 55 Excluding patients who are more difficult to engage may reinforce and exacerbate disparities.54 Enrolling patients before hospital discharge can increase participation because many live in difficult social circumstances and may be challenging to locate once they return to the community.26 44 56
Programme theory 2: engaging HNHC patients in interventions
HNHC patients may have a history of personal and healthcare system barriers that kept them from receiving appropriate care. Programme theory 2 posits that patients can be successfully engaged in interventions to change their health behaviours if (1) care providers can gain a patient’s trust, and (2) interventions are individualised to meet each patient’s unique complex combination of medical and behavioural health conditions and social needs.
Addressing patients’ life circumstances and prior experiences
Personal circumstances, such as behavioural health conditions, trauma stemming from early life, extreme poverty, low literacy26 35 40 41 50 56 57 and prior negative experiences with the healthcare system,35 40 57 58 can prevent patients from seeking and accepting help. System-level barriers, such as lack of insurance, inadequate cultural competence or poor communication by providers (eg, inadequate translation services),35 37 40 43 50 58 can increase patient distrust and marginalisation. The healthcare systems’ inability to accommodate patients’ personal circumstances (eg, homelessness) can further inhibit HNHC patients from receiving needed services.48 58 Disrespect and discrimination from providers can lead patients to distrust and avoid specific providers or settings and lower their likelihood of engaging with their treatment.57
Building relationships with care providers
HNHC patients need to trust the individuals providing intervention services.40 Care managers, community navigators and social workers are often tasked with building a trusting relationship with HNHC patients.36 41 55 58–60 Providing assistance with basic life needs (eg, arranging transportation) or help with care adjacent needs (eg, modifying cooking practices to improve diet) can help with establishing trust.40 42 44 50 55 57 59 61–63
Patients value care teams that support them in managing their health and welfare, especially care managers who listen and are easy to talk to, who explain things in lay terms and who act as navigators to other care providers.40 41 48 57 60 64 Supportive care teams and HNHC patient peer support groups can create an environment that motivates HNHC patients to improve their health behaviours.35 57 58 HNHC patients feel supported and motivated by interactions with their care managers when they include feedback, especially after taking small steps on their own, and frequent, longer visits.40 48 57 58 61
Providing individualised care
HNHC patients respond to interventions that are individualised, tailored, flexible and offer access to non-medical services. Allowing interventions to be flexible in addressing the diverse needs of individual patients empowers providers to work with patients and their families to develop tailored strategies that accommodate the diversity of needs across the patient population.33 34 39 40 44 48 50 65 Non-medical issues may drive patients’ ability to benefit from healthcare-focused interventions so interventions may need to first address psychosocial needs and facilitate access to services that address these needs.26 34 40 41 44 48 For example, co-occurring problems like substance abuse and housing instability may need to be addressed concurrently or even before managing healthcare.26 48
Interventions that teach patients how to obtain needed medical and non-medical services on their own enhance patients’ confidence and self-efficacy.26 37 42 44 56 58 61 62 The optimal length of time participating in an intervention differs across individual HNHC participants and an appropriate end may be based on successfully demonstrating personal intervention objectives (eg, self-management behaviour).40 50
Recognising barriers to patient change
An intervention establishing successful relationships with HNHC patients may not lead to reductions in hospitalisations or ED visits in the short term.64 Some patients may continue to prefer visiting the ED because of financial concerns (ie, a lack of copayments) or a perception of receiving better quality of care.51 58 66 Patient-centred success may need to be evaluated in relation to incremental goals, such as changes in self-care behaviours and better relationships with providers, rather than achieving more ultimate healthcare utilisation or cost goals.48 56 64 Gradual improvements in patients’ experiences with their care providers may eventually lead to long-term benefits in health behaviours and clinical outcomes.64
Programme theory 3: supporting care providers participating in interventions
Care providers affect the success of interventions through their actions and inactions. Programme theory 3 posits that obtaining provider buy-in and building processes to support their efforts helps improve intervention performance. Care providers often include physicians and nurses, with additional services provided by care managers, social workers, community health workers (CHWs), nutritionists and behavioural health specialists.31 32 41 42 44 56 58 64 67 68
Gaining support for and from care providers
Organisational support is crucial throughout an intervention. Early support from organisational leaders and physician champions can help smooth the path for implementing new processes into care settings.32 41 Garnering support from physicians and practice staff is facilitated by tailored, face-to-face outreach28 32 41 and their belief that the intervention will benefit their patients’ care.28 30 41 The additional time needed to care for HNHC patients and support for provider participation should be acknowledged in intervention designs. These may include financial incentives such as higher provider reimbursement for treating HNHC patients, capitation payments that pay providers a set amount per patient that accounts for patient complexity, physician enrolment incentives and counting HNHC patients as more than one patient on physician panels.30 32 35 41 52 63
Managing and relieving care provider workload and stress
In addition to increased workloads and responsibilities, providers participating in HNHC patient interventions may experience increased emotional strain from focusing on sicker patients.41 69 Increased stress may result in higher turnover among physicians, care managers and other care providers.60 69 Interventions can both change how care is provided and assist care providers by including resources that support confidence and skills building. Support may include trainings, dedicated time to implement new processes and an increased number of support staff.35 41 50 60 69 Providers may be organised into care teams to both deliver more coordinated care to HNHC patients and provide mutual support for each other.40 41 48 60 66 69 Within care teams, physicians can focus on providing high-quality patient clinical care, while other care providers, like care managers and social workers, lead many of the supportive tasks (eg, building trust, managing medication, referring to community resources).29 37 39 41 50 60 63 CHWs add non-clinical, peer perspectives on patient needs and challenges.29 50 Care team members can be provided with emotional and technical support through peer counselling across offices and healthcare systems.41 60 69
Maintaining communication across care providers
Strong communication and co-ordination among care team members related to meeting patients’ needs can ultimately help teams achieve long-term intervention goals.66 Regular, open communication among care team members fosters cohesive coordinated teams with shared values and commitments.48 66 Tools to ensure regular communication among care team members include regularly scheduled newsletters, email feedback and team meetings.32 41 69 Team meetings at different organisational levels are important forums for exchanging information.66 Organisation-wide meetings may be used to discuss broader performance goals and meetings at the practice level may discuss practice-specific issues, including trainings and individual physician performance.37 41 66 At the care team level, meetings that include all care team members provide opportunities to openly discuss specific patients and develop care plans.37 41 50 63 66 69 Open discussions foster a supportive environment for discussing patient care approaches41 and performance goals66 and for providing constructive feedback.66 69 Care team members need sufficient time to attend meetings and to act on patient care plans.69
Being in the same physical location facilitates care team communication by making it easier to discuss patient concerns and coordinate care.30 48 63 66 Conversely, care teams find communicating with other providers challenging where communication processes have not been established (eg, providers are located in other healthcare systems).28 30
Discussion
Summary
This realist review sought to understand and explain complex interventions implemented in the US healthcare system that intend to improve HNHC patient healthcare use and cost. Within the population of patients who are high users of healthcare resources, HNHC patients are the subgroup for whom the type and/or level of care is considered potentially preventable or modifiable.1 70 Unlike disease management programmes that focus on educating patients with the same chronic condition, HNHC interventions often target and address a HNHC patient’s unique combination of medical and psychosocial complexity.71 Like other realist reviews investigating programmes for complex populations,71 72 our review highlights the support needed by both patients and intervention providers in building effective and sustainable relationships. Our review is unique in describing considerations in identifying and selecting HNHC intervention participants. We also identify several considerations that support improvement in HNHC interventions.
Individualisation was a key mechanism, both in identifying participants and in implementing HNHC interventions. A key was identifying and addressing individual participant complexity, particularly in their specific chronic disease and healthcare use profiles and in their ability and readiness to change.45 47 52 58 Tailored interventions support patients gradually taking on greater responsibility in more appropriately managing their chronic conditions and navigating other systems to address behavioural health and social needs.33 34 39 40 44 48 50 65 For interventions to be sustainable, providers require support and assistance in meeting the additional responsibilities and challenges in improving their HNHC patients’ ability to obtain adequate and more appropriate medical treatment for their chronic conditions.
Successfully changing patients’ behaviour can require significant investments in time. Relationships require time and intensity to develop, as does successfully modifying patients’ long-held approaches to care. As a result, evaluations measuring the impact of interventions may need to assess success in relation to intermediate outcomes and extend the follow-up time in measuring final results.
Limitations
This review has several limitations. The literature that was the basis for our analysis was primarily intervention studies, also included in a companion systematic review, that contained contextual and descriptive information. Many studies of intervention outcomes included little qualitative information about participants’ and providers’ experiences in the intervention, which may have limited our ability to identify contextual considerations and mechanisms. Therefore, our theory development is reflective of the patient and provider experiences from the subset of studies that included more descriptive information. While we consulted existing theories to develop our initial programme theories, given limitations in time and resources, we were unable to further refine our final programme theories by exploring theoretical literature concerning other interventions or system dynamics that might have improved our understanding of the mechanisms of interventions for HNHC patients.
We limited the included literature to studies conducted in the USA because the review is intended to be relevant to the US healthcare system. However, while access to healthcare and social services in the USA differs from that in other countries, many of the mechanisms identified in this review are likely to be applicable to HNHC patient interventions in other countries. Finally, given the interpretive and inductive nature of the realist methodology, other researchers may arrive at different interpretations of the data, resulting CMOs and programme theories explaining interventions for HNHC patients. Further refinement, confirmation and refutation of the programme theories should and are expected to occur as additional evidence emerges in this literature.
Implications for practice and research
Table 1 summarises the provider and organisational recommendations based on the review’s findings. Careful targeting of HNHC patients for interventions, providing individualised patient services and supporting care providers with necessary emotional and practical resources may lead to gradual, improved long-term benefits in health behaviours and clinical outcomes. These recommendations are intended to be useful in thinking through the design and implementation of interventions for HNHC patients.
Summary of recommendations for providers and organisations by programme theory
Our findings point to several areas of future research. Our choice of conducting a realist review was informed by content experts who anticipated that limited evidence of the effectiveness of these interventions would be available. To some extent this proved true. Outcomes data from these interventions tended to measure gross changes in utilisation and cost and reported virtually no health or process outcomes. We abstracted the qualitative reporting in these studies to understand the role of context and the mechanisms that are triggered through these interventions, and we used this information to develop our realist programme theories. However, even though many of these interventions are quite large, costly and integral in implementing healthcare delivery reform, the ‘black box’ of what occurs in the intervention is often not reported in studies. For example, we sought to examine whether the subset of interventions that emphasise trust was more likely to have successful outcomes, but we found that the number of reported outcomes across all studies was too limited and distal to make meaningful comparisons.
Future research studies can therefore return to these same and similar interventions and obtain additional data to round out the theories that we found across studies, as well as to uncover others. It can focus on gaining a better understanding of the role of patient trust by developing tools to measure trust and testing how patient trust impacts outcomes. Research examining the experiences of individuals providing and receiving HNHC patient interventions will help elucidate the often-unobservable mechanisms or responses to the resources provided by interventions.
Conclusions
To our knowledge, this is the first realist review investigating how and why interventions for HNHC patients lead to reductions in potentially avoidable use and cost. We found that building relationships with HNHC patients and gaining their trust is a key mechanism in interventions successfully changing patients’ behaviours. Identifying HNHC patients for an intervention can be best achieved through a multipronged strategy that accounts for the complexity of prior healthcare experiences, clinician judgement and willingness to engage in an intervention. Additionally, we found that successful interventions recognise that relationships with HNHC patients require the sustained engagement of care providers. To succeed, providers need the emotional, financial, logistical and practical resources necessary to engage and maintain relationships with HNHC patients.
Data availability statement
Most of the data generated or analysed for this study are included in this published article, associated appendices or in the full report referenced in the study. Any additional data are available from EC upon request.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
The authors gratefully acknowledge the following individuals for their contributions to this project: current or prior AHRQ staff: Christine Chang, MD, and Elise Berliner, PhD, and current or prior RTI International–University of North Carolina at Chapel Hill Evidence-Based Practice Center staff: Sharon Barrell, MA, Christopher Beadles, PhD, Linda Jiang, BA, Loraine Monroe, BS, Deborah Porterfield, MD, Caroline Rains, MPH, Julie Seibert, PhD, Meera Viswanathan, PhD, Lynn Whitener, DrPH, Roberta C. Wines, MPH, and Carol Woodell, BSPH.
A draft version of this evidence report underwent external peer review from nine content and methodological experts (Elizabeth Bayliss, MD, MSPH (Kaiser Permanente Colorado, University of Colorado School of Medicine), Steven B. Cohen, PhD, MS (RTI International), Tim Engelhardt, MHS (Centers for Medicare & Medicaid Services), Allison Hamblin, MSPH (Center for Health Care Strategies), Samantha Iovan, MPH (University of Michigan), Ellen Meara, PhD (Harvard T.H. Chan School of Public Health, The Dartmouth Institute for Health Policy & Clinical Practice), Jane Noyes, PhD (Bangor University), Lucy Savitz, PhD, MBA (Kaiser Permanente Northwest), Geoffrey Wong, MD, MA, MBBS (University of Oxford)) and public posting (26 February 2021 to 26 March 2021). Comments from reviewers were considered in preparing the final evidence review.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Presented at Findings from this manuscript were previously published in a report developed for the Agency for Healthcare Research and Quality (AHRQ). The full evidence report is available at https://doi.org/10.23970/AHRQEPCCER246.
Contributors EC and NB were involved in developing the protocol, all stages of data collection and management, analysis and writing of the manuscript and are the guarantors. RA was involved in literature searching, screening and writing the manuscript. NB acquired funding. All authors read and approved the final manuscript.
Funding This work was supported by Agency for Healthcare Research and Quality under contract (HHSA-290-2015-00011-I) and RTI International.
Disclaimer Investigators worked with AHRQ staff to develop the scope, analytic framework and key questions for this review. AHRQ had no role in study selection, quality assessment or synthesis. AHRQ staff provided project oversight, reviewed the report to ensure that the analysis met methodological standards and distributed the draft for peer review. Otherwise, AHRQ had no role in the conduct of the study; collection, management, analysis and interpretation of the data; and preparation, review or approval of the manuscript findings.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.