Enhancing ovarian cancer care: a systematic review of guideline adherence and clinical variation

Abstract

Background

Clinical variation in ovarian cancer care has been reported internationally. Using Wennberg’s classification of clinical variation as effective care we can conceptualise variation through deviation from clinical guidelines. The aim of this review was to address knowledge gaps in the effectiveness of attempts to reduce unwarranted clinical variation through addressing the following questions: What is the evidence of guideline adherence in ovarian cancer and its deviation?; what are the key factors associated with variation in guideline adherence in ovarian cancer care?; and what quality improvement approaches have been used and what is the evidence of their effectiveness in enhancing guideline adherence in ovarian cancer care?.

Methods

Keywords and synonyms for the major concepts of ovarian cancer, guideline adherence and safety were developed and combined to form the search strategy. Systematic searches of four electronic databases were undertaken of publications from January 2007 to November 2018. Retrieved articles were assessed against the eligibility criteria to determine those for inclusion.

Results

Thirty-two papers were included in the review with three broad groupings identified: adherence to and deviation from guidelines (either local, national or international guidelines); factors impacting guidelines adherence; and quality improvement approaches.

Conclusions

Unwarranted clinical variation may be used as a marker for the effectiveness of a health system, based on the outcome of this systematic review. This review found that the implementation of quality indicators through a formal quality improvement program lead to improvements in guideline adherent care. Further research on outcomes of implementing quality improvement programs in ovarian cancer care will improve the ability to implement centralised care and further identify factors that to improve outcomes in ovarian cancer care.

Introduction

Clinical variation in healthcare describes differences in healthcare practice, processes or outcomes for reasons such as the complexity of a patient’s illness, the burden of illness in different populations, or administrative differences, such as different coding of the same issue [1]. Such variations are evident throughout healthcare systems and services internationally and reflect natural differences between the individuals and population groups receiving care.

Clinical variation in ovarian cancer care has been reported internationally [2,3,4]. Variation from 5 to 55% has been reported in the receipt of neoadjuvant chemotherapy for women with stage IIIC and IV epithelial ovarian cancer in hospitals that have high volume of both surgery and chemotherapy for ovarian cancer [2]. A population based study from Australia found variation in the provision of standard chemotherapy treatment for women older than 70 years with ovarian cancer, with 68% of these women not receiving standard chemotherapy [5]. Variation in surgical staging, including adherence to guidelines, was found in a Canadian study, with only 44% of gynaecologic surgeons in Ontario adhering to guidelines for surgical staging [3].

Whilst clinical variations are to be expected and are not always problematic, they have attracted increasing interest in health systems internationally as a mechanism for understanding the quality and appropriateness of care provided to patients, highlighting quality features such as efficient, effective and timely care [6]. The dominant conceptual framework and related theory is that set out by Wennberg and colleagues. Wennberg’s classification of variation identifies three categories that can be used to identify when clinical care variation is unwarranted: effective care, preference sensitive care and supply sensitive care [7, 8]. The category of effective care identifies those services and procedures that have been proven effective in the research literature for all patients. Clinical variation related to effective care is often conceptualised through deviation from clinical guidelines.

In ovarian cancer care there are a number of international guidelines that guide effective treatment internationally. The National Comprehensive Cancer Network (NCCN) in the United States initially developed a clinical practice guideline for ovarian cancer in 1996, since then providing regular evidence and consensus based updates, with the most recent guideline published in 2017 [9]. The National Institute for Health and Care Excellence, initially set up in the United Kingdom (UK) in 1999 to reduce variation in the availability and quality of treatment and care in the National Health Service, developed a clinical guideline for clinical practice in ovarian cancer in 2011 [10]. The European Society for Medical Oncology has also published clinical practice guidelines on the management of newly diagnosed and relapsed ovarian cancer [11]. There have been recent developments in peri and intra operative surgical care with the development and implementation of Enhanced Recovery After Surgery Guidelines across a number of surgical areas, including gynaecologic oncology procedures [12, 13].

To date, there has been no synthesis of evidence regarding the degree of unwarranted clinical variation (deviation from effective care) in ovarian cancer care. Data regarding the effectiveness of attempts to reduce unwarranted clinical variation in ovarian cancer care are also lacking. This review aims to address these knowledge gaps by providing a synthesis of evidence in relation to the following questions: (1) What is the evidence of guideline adherence in ovarian cancer and its deviation?; (2) What are the key factors associated with variation in guideline adherence in ovarian cancer care?; (3) What quality improvement approaches been used and what is the evidence of their effectiveness in enhancing guideline adherence in ovarian cancer care?

Methods

This review was conducted and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines [14]. In this study we have defined safe care using Wennberg’s effective care category as rationale for focusing on guideline adherent care and the deviation from these [8].

Eligibility criteria

Published journal articles reporting studies of any design, including literature reviews, were eligible for the study if they were available in English and published in the last 11 years (2007 to November 2018). This timeframe was chosen to ensure the inclusion of journal articles relevant to contemporary healthcare. To be eligible, studies had to focus on female adults (over the age of 18 years) who had received a diagnosis of primary ovarian, fallopian tube or peritoneal cancer. To be included, the reported outcomes had to focus on guideline adherence and its deviation, the key factors associated with variation in guideline adherence and what quality improvement approaches have been used and evidence of their effectiveness. Conference abstracts, editorials, commentaries and grey literature were excluded. Studies focussed on the centralisation of ovarian cancer care have been defined for the purpose of this review as effective care, using Wennberg’s classification of variation [7, 8]. These studies have been reported on in a previous Cochrane review and have been excluded as out of scope for this study [15].

Data sources

Keywords and synonyms for the major concepts of ovarian cancer, clinical variation and guideline adherence were developed and combined to form the search strategy. Systematic searches of MEDLINE, EMBASE, SCOPUS and web of science were completed for publications between January 2007 to November 2018. Results were merged using reference management software (endnote; Thomson Reuters), with duplicates removed. The search strategy has been included in Additional file 1. The search terms used in the Medline database search were: Ovarian cancer, ovarian neoplasms, patterns of care, guideline or guideline adherence, variation of care, clinical variation, referral and consultation, referral pathway, optimal care, and framework. The terms framework and guideline were removed for search in web of science and SCOPUS due to the volume of material returned. Reference lists were also scanned to identify additional relevant studies

Study selection and data extraction

One author (KW) screened the titles and abstracts of the extracted papers, identifying papers that met the inclusion criteria. A random sample of the papers (10%) were screened by a second author (RH) who independently applied the inclusion criteria. The same process was followed for the full text review. Disagreements were resolved through consensus. The following data were extracted from the studies that met the inclusion criteria: author(s), date, method, data source, country, setting, sample/participants, objective, main findings and limitations.

Data analysis

Data were synthesised using a narrative synthesis approach [16]. This approach was chosen to allow for a text based analysis across the diverse range of research and research approaches identified in this study. The elements of narrative synthesis include developing a theory in light of the findings from the studies, developing a preliminary synthesis, exploring relationships within and between studies, and assessing the robustness of the synthesis [16].

Results

A total of 2012 papers were identified during the search, after removing duplicates 1683 studies were left. Screening of titles and abstracts found 41 studies that required full text review, with 32 studies meeting the inclusion criteria for inclusion in the final review. A flow diagram outlining the results of the search strategy is provided in Fig. 1.

Fig. 1

Flow diagram of selection process. Flow diagram outlining the selection process of studies found in the initial search and how studies were excluded, leading to the final 32 studies included in the systematic review

Excluded studies

Of the 1651 studies excluded, 185 were excluded due to wrong study design, 28 were excluded due to focussing on cancer screening and 427 were excluded as they focused on quality improvement broadly and did not specifically focus on guideline adherence and clinical variation.

Characteristics of the studies included

Of the 32 studies included in this analysis, 19 reported retrospective population registry or local registry analysis, seven reported retrospective medical record audits, four were literature reviews, one reported a survey with clinicians and the final study included both a literature review and medical record audit. There were no qualitative studies that met the inclusion criteria. The studies were primarily from the United States (17 studies) and Europe (12 studies), with two from Canada, and one from South Korea.

Using a narrative synthesis approach to analyse the 32 papers, three broad groupings of publications were identified: 1) adherence to and deviation from guidelines (11 studies which were local, national or international) [3, 17,18,19,20,21,22,23,24,25,26]; 2) factors impacting guideline adherence (12 studies) [27,28,29,30,31,32,33,34,35,36,37,38] and 3) quality improvement approaches (9 studies) [39,40,41,42,43,44,45,46,47].

Guideline adherence and deviation

Of the 11 studies grouped under guideline adherence and deviation, 8 utilised the NCCN ovarian cancer guidelines as the comparator guideline, with 3 using their own local or national guidelines [3, 17,18,19,20,21,22,23,24,25,26]. These studies collectively indicated variation in adherence to guidelines for both surgical staging and receipt of chemotherapy. Bristow et al. reported that across all ovarian cancer cases in the Californian Cancer Registry, NCCN adherence was 37.2% over a 7 year period [17]. This compares to Phippen et al. who investigated NCCN guideline adherence in a low volume institute and reported that 85.4% of patients received NCCN guideline adherent surgery over an 8 year period [23]. One of the key factors distinguishing these studies was that in the Phippen et al. study all surgery was completed by gynaecological oncologists, with surgeon type not reported in the Bristow et al. study [17, 23]. Erickson et al. found a higher rate of guideline adherent care, at 78.5%, in their single institution only study [19]. This is in contrast to the population study by Hodeib et al. using data from the Californian Cancer Registry, finding guideline adherence of 24% [21]. See Table 1 for characteristics of the studies included under guideline adherence and deviation.

Table 1 Characteristics of included studies (n = 32)

Factors impacting guideline adherence

Individual hospital and/or surgeon volume and their link to improved outcomes has been identified for surgery that is technically complex [48]. Hospital and/or surgeon volume emerged as the primary factor associated with the degree of guideline adherence identified in 11 of the included studies in this review that in turn was linked to ovarian cancer outcomes. Eight of the 11 studies identified higher volumes with improved outcomes at a national population level, with three studies focussing on higher volume leading to improved outcomes at a state or specific geographical level [27,28,29,30,31,32, 34,35,36, 38]. Survival at both 5-years and 4-years was found to be better for women treated in higher volume hospitals [29, 30, 32, 38]. The Marth et al. and Bristow et al. studies found hospital volume was an independent predictor of survival [29, 36]. Hospital volume was also found to be a statistically significant and independent predictor of optimal cytoreductive surgery, i.e. guideline adherent care [28, 29]. Shakeel et al. and Wright et al. reported that higher volume hospitals had lower in-hospital mortality and morbidity [33, 34]. The study by Uppal et al. investigated the use of 30-day readmission following cytoreductive surgery as a measure of quality [49]. This study found that although higher volume hospitals had higher rates of 30-day readmission, they also had higher rates of guideline adherent care and an improved 5-year survival when compared to lower volume hospitals with lower 30-day readmission rates [37]. This supports their proposal that 30-day readmission following ovarian cancer surgery is not a robust measure of quality care. Overall the studies found that improvement in guideline adherence was associated with improved outcomes in hospitals where there was high volume ovarian cancer care at a hospital and/or surgeon level. See Table 1 for characteristics of the studies included under factors impacting guideline adherence.

Quality improvement approaches

There were nine studies identified investigating quality improvement approaches that attempted to detect, respond to or address clinical variations. Five of the studies focussed on the development of quality indicators for guideline adherent care as a way to identify variation in clinical practice and improve outcomes [40, 44,45,46,47]. Three studies reported on the impact of quality improvement programs on improving outcomes in ovarian cancer through increasing adherence to clinical guidelines, resulting in a reduction of unwarranted clinical variation [39, 41, 42]. The final study reported on one hospital’s compliance with Society of Gynecologic Oncology quality indicators [43]. Five studies reporting on the development of quality indicators all had similarities in the indicators developed, with a focus on rates of complete surgical resection [40, 44,45,46,47]. Three of these studies focussed further on the type of surgeon and their surgical volume [40, 45, 47]. The study by Phillips et al. focussed on the importance of identifying the denominator in advanced ovarian cancer studies and quality indicators, to ensure outcomes are able to be appropriately interpreted and the ‘denominator effect’ is understood [44]. Phillips et al. defined the denominator in their study as the total number of advanced ovarian cancer cases presenting or referred to the cancer centre [44]. This then allowed them to understand the proportion of patients who received treatment and the cancer centres overall survival rates for women with advanced ovarian cancer [44]. The study by Aletti et al. found an increase in complete cytoreductive surgery from 31 to 43% following the implementation of a quality improvement program [39]. The quality improvement program implemented included comparison of hospital results with the latest evidence, confidential surgeon benchmarking, educational sessions for trainees on new and complex surgical techniques and improving the availability of experienced staff members to assist with complex surgical cases [39]. The study by Kommoss et al. found women with stage 1A-IIIA ovarian cancer had an improvement of 27 to 88.5% in complete cytoreductive surgery following the implementation of a quality improvement program [42]. Harter et al. further reported on the same quality improvement program for women with later stage disease, finding complete cytoreductive surgery rates for women with stage IIB-IV ovarian cancer improved from 33 to 62%, with overall survival improving from 26 months to 45 months following the implementation of the quality improvement program [41]. The quality improvement program reported on in the Kommoss et al. and Harter et al. studies focussed on optimal surgical staging, receipt of platinum based chemotherapy, surgeon volume and experience, as well as the implementation of dedicated ovarian cancer surgical teams, benchmarking of morbidity and outcomes, and improvement of interdisciplinary assessment and management [41, 42]. There are commonalities between the three studies that reported on outcomes of the implementation of quality improvement programs, including adherence to guideline based care, improvement in ongoing surgical education and surgeon benchmarking [40,41,42]. The study by Liang et al. found high clinical variation in the indicators of complete surgical staging and the administration of intraperitoneal chemotherapy within 42 days of optimal cytoreductive surgery [43]. See Table 1 for characteristics of the studies included under quality improvement approaches.

Discussion

This review completed a synthesis of the evidence in relation to the degree of unwarranted clinical variation for women with ovarian cancer and the effectiveness of attempts to improve clinical variation. Findings from the 32 included studies were extracted in relation to three key areas: adherence to and deviation from guidelines, factors impacting guideline adherence, and quality improvement approaches to reduce variation through enhanced guideline adherence. Despite strong evidence of the optimal care process for ovarian cancer, aligning with Wennberg’s classification of effective care, variation was apparent in the extent to which guideline adherent care was practiced for women with ovarian cancer internationally [8]. Commonalities between the studies included non-adherence to guidelines in surgical staging, optimal cytoreductive surgery and receipt of adjuvant chemotherapy [3, 16,17,18,19,20,21,22,23,24]. Variation in guideline adherent care, and specifically deviation from effective care, is likely to lead to unwarranted clinical variation in outcomes for ovarian cancer patients. Quality improvement approaches such as the use of quality indicators and team-based quality improvement projects have been utilised to reduce unwarranted clinical variation, with some evidence of success. The most successful quality improvement approaches found in this review focused on the development of quality improvement programs across gynaecological oncology services. These successes facilitated clinical change through the implementation of dedicated ovarian cancer surgical teams, education on new and emerging surgical techniques and benchmarking at a hospital and surgeon level [38, 40, 41].

Surgical volume at both a hospital and surgeon level have been associated with improved outcomes in ovarian cancer for many years, with the study by Luft et al. in 1979 being one of the earliest studies to identify this association [48]. The association between improved surgical outcomes and high hospital or surgeon volume has been identified in a number of cancer types that require technically complex surgery, such as pancreatectomy [48, 50]. One of the key factors in the relationship between volume and care outcomes is thought to be the increased likelihood of those practicing high volume care to adhere to best practice guidelines. Our findings identified evidence to support the relationship between hospital volume, surgeon volume, guideline adherence and outcomes in ovarian cancer care such that higher volume hospitals and surgeons were found to be independently associated with increased survival, as well as increased levels of guideline adherent care [27,28,29,30,31,32,33,34,35,36]. Guideline adherent surgery in ovarian cancer includes complex surgical techniques that a specialist surgeon, such as a gynaecological oncologist, may only achieve competency in though completing a high volume of these surgeries, with the support of a specialist multidisciplinary team in their hospital [15, 29].

A key finding of this review is that the implementation of quality indicators through a formal quality improvement program has led to improvements in guideline adherent care, as highlighted in the improvement of complete cytoreductive surgery in the studies by Aletti et al., Kommoss et al. and Harter et al. [39, 41, 42]. While the development of quality indicators is a key part in this process, the included studies highlighted improvements in clinical variation that can be achieved when indicators are implemented into clinical practice. Three of the studies outlined the process of developing quality indicators at national and international levels [45,46,47]. However, the four studies that reported on the implementation of quality indicators through quality improvement programs were implemented in single institutions [39, 41,42,43]. These studies demonstrated that improvements in guideline adherent care could be made, with only one study reporting on an improvement in overall survival [41].

Implications

Given the relationship between hospital and surgical volume, guideline adherence and care outcomes, our findings suggest that there is value in programs to promote guideline adherent care in hospitals with high surgical volume, at both a hospital and surgeon level [30, 33]. Development and implementation of quality indicators and quality improvement programs to improve ovarian cancer outcomes have been used internationally in an attempt to reduce unwarranted clinical variation in ovarian cancer care [39,40,41,42,43, 45,46,47]. A core strategy associated with such programs is the centralisation of care to hospitals that have specialist gynaecological oncology centres and complete a high volume of ovarian cancer surgery at both the hospital and surgeon annually [51,52,53,54]. Evidence suggests that centralising ovarian cancer care into specialist hospitals improves access to gynaecological oncologists, multidisciplinary cancer care teams and higher surgical volume hospitals and surgeons may lead to improved survival [15].

The development of gynaecological oncology as a surgical sub-speciality in Australia, the UK and North America has led to increasing specialist care in the treatment of gynaecological cancer. In a study identifying practice guideline adherence in Ontario, Canada, 81% of gynaecological oncologists reported completing surgical staging adherent to guidelines, with only 41.5% of non-oncologists completing guideline adherent surgical staging [3]. A study by Chan et al. found treatment by a gynaecological oncologist led to more women in California to undergo primary staging surgery (91.9% vs. 69.1%) and chemotherapy (90% vs. 70.1%) than those treated by a non-oncologist [55].

An important focus for future research is to establish evidence of the effectiveness of the implementation of quality improvement programs, reporting on survival over time. Evidence of the relationship between quality improvement programs and survival is important for assessing the impact of implementing quality indicators through quality improvement programs as a way to decrease unwarranted clinical variation and improve outcomes for women living with ovarian cancer. Analysis of the implementation of quality indicator and quality improvement programs across regional or national jurisdictions that aim to improve outcomes in ovarian cancer beyond a single institution level would be valuable.

Limitations

The included studies and review process were subject to some limitations. Whilst hospital and surgeon volume were identified as important, a key challenge for interpreting the results of these studies was the range of case volume for hospitals, with each study defining volume in different ways. High hospital volume ranged from > 20 to > 30 cases/yr. [29,30,31,32, 36]. High surgeon volume was more consistent, being identified as ≥10 cases/yr. in the two studies by Bristow et al. that investigated surgeon level volume [27, 28]. For health systems undertaking centralisation of ovarian cancer surgery this may lead to challenges in identifying which hospitals should be completing ovarian cancer surgery. Reliance on administrative datasets created limitations such as selection bias, as most registries did not cover the entire population, with a lack of information on stage of disease, extent of cytoreductive surgery and surgeon specialist type. Two examples are the study by Erickson et al. as a single institute study finding a high rate of guideline adherent care, at 78.5%, and the study by Hodeib et al. which used population data and found a low rate of guideline adherent care at 24% [19, 21]. A limitation in study design, using single institution versus population data, may in some way explain the vast difference in guideline adherence between the two studies [19, 21]. The review process may have omitted relevant material by including only studies published in English, the exclusion of studies allocated to broader quality of care issues, as there are many factors associated with clinical variation, guideline adherence being just one, and the date range of the past 11 years.

Conclusion

This review has reported for the first time exploring guideline adherence and the factors impacting on this towards clinical variation in ovarian cancer, highlighting guideline adherence and deviation, factors impacting guideline adherence, and quality improvement approaches. There is evidence of deviation from effective care in ovarian cancer, demonstrated through deviation from best practice guidelines. Such deviation is likely to lead to unwarranted clinical variation in ovarian cancer care. Centralising care to higher volume centres and surgeons, and the growth of gynaecologic oncology as a specialty appear to be associated with enhanced guideline adherence, reduced variation and better outcomes as a result. The development, implementation and reporting of quality performance programs may also lead to reduced unwarranted variation in ovarian cancer care, but evidence is currently limited regarding the effectiveness of these programs at regional and national levels and on longer term outcomes.