Introduction

There is growing concern over the ability, even in high income countries, to deliver affordable cancer care. [1], [2] Cost-effectiveness analysis (CEA) is recognized as an important tool for assessing value for money and an important source of information for making clinical and policy decisions. CEA can play a central role in guiding appropriate resource allocation in cancer care. [1], [2].

Cost-effectiveness analysis (CEA) is the comparative assessment of two or more interventions in terms of costs and benefits. [3] CEA is frequently used by organizations responsible for financing health care to quantify the value for money associated with adopting a new therapy compared to continued use of an existing therapy. The main output of a CEA is the incremental cost-effectiveness ratio or ICER. The ICER is the ratio of increased health expenditures divided by increased health outcomes when a new therapy is compared to an existing therapy. Health outcomes can be measured in terms of life expectancy and the resulting ICER, represents the ratio of increased expenditures to increased years of life. There is no consensus on the threshold ICER value, above which a new therapy is considered too expensive although the threshold of $100 K per life year has been commonly cited. [4] In reality, decision makers in different jurisdictions employ different criteria as threshold values. [5] A preferred measure of health outcomes, the quality-adjusted life year (QALY), is derived by weighting life expectancy on a scale ranging from 0 to 1, known as a utility. A utility indicates the desirability of a health state based on morbidity and quality of life impact. [6], [7] For example, living ten years with a chronic disease that has a utility of 0.7 is the equivalent of living 7 quality adjusted life years (QALYs). CEAs in which the outcome is measured in terms of QALYs produce an Incremental Cost Utility Ratio (ICUR), which incorporates quality of life impact into the estimate of value for money. The incremental cost per life year and incremental cost-utility ratios are collectively referred to as ICERs although this simplification can result in some confusion. Health state utilities can be estimated using a variety of methods, and different methods can result in different utilities and thus different QALY estimates. [6], [7].

In trial-based CEAs, information on health care costs and health outcomes are collected during the course of a randomized controlled trial (RCT). [3] The ICER is estimated based on the cost and health outcomes measured during the trial. More commonly, CEA estimates are derived from decision models. In model-based CEA, information on health care resource utilization, costs and health outcomes are derived from a variety of potential sources including RCTs, meta-analyses, observational studies, medical record review, patient interviews and the opinions of expert panels. The information is incorporated into a mathematical model. [8] Model-based CEAs typically extrapolate short term data available from RCTs or observational studies into longer term outcomes to estimate an ICER or ICUR. Model-based CEA provides the opportunity to clearly identify key assumptions and vary model components to understand the impact of uncertainty on estimates of value for money. Sensitivity analysis is the systematic variation in a model input to assess the impact on model outputs and it is crucial for ensuring the validity of model-based CEA estimates. [9].

Cancer drug therapies are well studied in randomized controlled trails (RCTs) and RCTs are a key source of evidence incorporated into cancer CEAs. However, there are limitations to the data from RCTs. One issue is that they often use surrogate endpoints rather than overall survival (OS). Improvements in survival are a clear benefit to patients whereas the benefits of surrogate endpoints such as DFS are less clear. The studies evaluating five years of aromatase inhibitors compared to five years of tamoxifen in the treatment of post-menopausal women diagnosed with early stage hormone receptor positive breast cancer used disease free survival (DFS) rather than OS. Aromatase inhibitors reduced the risk of breast cancer recurrence when compared to tamoxifen. [10], [11] However, follow-up data reveal that the increased DFS associated with AIs in this setting does not translate into increased OS. [12], [13].

Another well recognized problem is that RCTs of cancer drugs often do not provide detailed or complete data on adverse effects. [14] Tamoxifen and aromatase inhibitors have different side effect profiles. Tamoxifen is associated with an increased risk of endometrial cancer and thromboembolism and AIs are associated with an increased risk of fractures. [15], [16] The lack of survival benefit for women treated with AIs compared to tamoxifen has raised concerns that AI-related adverse events may mitigate the benefits of reduced breast cancer recurrence. [17], [18] Recent evidence supports the idea that the relative impact of adverse events between AIs and tamoxifen should be investigated further. A systematic review and meta-analysis of RCTs demonstrated that while AIs are associated with lower risks of venous thromboembolism and endometrial cancer, they are also associated with increased risks of fractures and cardiovascular disease when compared to tamoxifen. [12] Finally, there are issues of generalizability. Individuals in the RCT may be different than individuals treated in the real-world who may be older or have more co-morbidity, factors with important impacts on risks and benefits. Particular sub-groups, such as older women, those at low risk of breast cancer recurrence, or those at high risk of fracture may be especially vulnerable. As a result, the added benefit of AI therapy when compared to tamoxifen in real-world practice has been called into question. [18].

This paper provides a systematic review of CEA studies of AI versus tamoxifen. We describe the overall quality of these studies, focusing specifically on how uncertainty is assessed and its potential impact on study conclusions and policy implications.

Materials and Methods

Results

Literature Search

We identified 1,622 non-duplicate citations, of which 40 were recognized as potentially relevant and the full text articles retrieved. (Figure 1) Of the total of 40 studies, 13 were excluded because of the comparators, seven were excluded because the study was not a cost-effectiveness analysis, and two were excluded because the study population was not early breast cancer.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. PRISMA Flow Diagram.

PRISMA indicates Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

https://doi.org/10.1371/journal.pone.0062614.g001

Study Characteristics

A total of 18 articles were included in the final study, of which 16 were published in English, one in Spanish and one in Italian.[32]–[49] The publication years ranged from 2004 to 2010. Most studies (5/18) addressed policy decision making in European zone countries, three each addressed the United States, United Kingdom, and Canada. (Table 1) Eleven of the studies acknowledged funding from industry and the funding source was not stated for three studies. Most analyses compared anastrazole to tamoxifen (n = 13) and the remainder compared letrozole to tamoxifen. All of the studies were model based – we identified no trial-based analyses. The vast majority of studies used Markov Cohort models (n = 17). In the remaining study by Lazzaro et al, the authors indicated that a model was used but the model type was unclear. Most considered both QALY and life year outcomes (n = 11). The majority of CEAs (n = 16) conducted the analyses from the perspective of the health care system as payer, accounting for the costs attributable to provider organizations such as government payers and excluding patient costs. One-half of studies used a 3% discount rate. Detailed information on study characteristics is available on-line in Table S1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of study characteristics (N = 18).

https://doi.org/10.1371/journal.pone.0062614.t001

Cost-effectiveness Analysis Outputs

When comparing anastrazole to tamoxifen, ICERs ranged from $77 to $97,202 per life year and ICURs ranged from $7,351 to $151,608 per QALY. (Table 2) For the comparison of letrozole to tamoxifen, ICERs ranged from $24,109 to $61,278 per life year and ICURs from $25,886 to $59,620 per QALY. All ICERs from the published analyses were less than $100 K (2010 USD) per life years and the majority less than $50 K (2010 USD per life year. All ICURs with the exception of the analysis by Gil et al comparing anastrazole to tamoxifen were under the threshold of $100 K (2010 USD) per QALY and the majority under $50 K per QALY. All studies estimated a survival benefit for aromatase inhibitors compared to tamoxifen. For those studies reporting undiscounted life years gained, the mean (standard deviation) estimated increase in survival was 0·39 (0·22) years for anastrazole when compared to tamoxifen and 0·38 (0·30) years for letrozole compared to tamoxifen.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Cost-effectiveness analysis outputs.

https://doi.org/10.1371/journal.pone.0062614.t002

Data Sources and Handling of Parameter Uncertainty

The majority of CEA authors took estimates of the impact of hormonal therapies on breast cancer recurrence from a single RCT, without developing a natural history model based on other data sources. (n = 14) (Table 3) Data were taken directly from either the Arimidex or Tamoxifen Alone or in Combination (ATAC) trial which compared five years of anastrazole to five years of tamoxifen or the Breast International Group (BIG 1–98) trial which compared five years of letrozole to five years of tamoxifen. A total of 10 studies (56%) reported sensitivity analysis on the risk of breast cancer recurrences, meaning a significant proportion did not report sensitivity analyses on this important factor.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Summary of data sources and handling of uncertainty (N = 18).

https://doi.org/10.1371/journal.pone.0062614.t003

Half of the studies took data on adverse events from a single RCT with the other half incorporating external information. (Table 3) For example, Hillner et al applied the increased hip fracture risk from the ATAC trial to age-specific data on hip fracture rates from Scandinavia. [37] In one study the authors cited multiple data sources for information on harms but it was unclear how the information was incorporated into the model. [40] One-third of the studies did not perform sensitivity analysis on the risk of adverse events (n = 6, 33%). Eleven studies (61% ) performed probabilistic sensitivity analyses. No studies performed a value of information analysis to quantify the value of more research to better estimate model parameters and better inform the policy decision. Detailed information on data sources and handling of parameter uncertainty is available in Table S3.

Discussion

Our review identified 18 published CEAs that compared AIs with tamoxifen for the first line treatment of early breast cancer in post-menopausal women. We found a lack of sensitivity and sub-group analyses that could limit the relevance of the study findings. The CEA studies assumed that observed benefits of AI in terms of DFS observed in individual RCTs would lead to improved OS. Subsequent meta-analysis of RCTs and longer term follow up provide no evidence of an OS benefit with five years of AI. [12], [13] Although increased rates of adverse events, particularly fractures, were found with the use of AI, many of the published CEAs did not adequately investigate the impact of these adverse events. Potential heterogeneity of the results across important sub-groups such as older women or women at risk for adverse events was often ignored in the published studies. The limitations of the existing CEA studies of AI reduce their relevance to policy making and assessment of value for money in cancer care.

The ICERs from the 18 published analyses appear to be generally consistent with other cost-effectiveness analyses of breast cancer related interventions. A systematic review identified 89 cost-effectiveness analyses for breast cancer related interventions with a median ICUR of $27 K (2008 USD $/QALY). [50] All ICERs from the published analyses were less than $100 K (2010 USD) per life years and the majority less than $50 K (2010 USD) per life year. All ICURs with the exception of the analysis by Gil et al comparing anastrazole to tamoxifen were under the threshold of $100 K (2010 USD) per QALY and the majority under $50 K per QALY. Even though ICERs and ICURs varied somewhat amongst the analyses, the publications conveyed a consistent message. The 18 studies generally indicate that adopting aromatase inhibitors as first line therapy is good value for money when compared to tamoxifen since few of the studies exceeded commonly accepted thresholds. The validity of the assertion that aromatase inhibitors are a cost-effective alternative to tamoxifen rests on the quality and thoroughness of the analyses. Our critique of these analyses in light of best practices for assessing uncertainty raises concerns about validity and signals that the ICERs and ICURs may be underestimates of the cost-effectiveness of aromatase inhibitors for women with early stage breast cancer. The assumption that aromatase inhibitors lengthen survival is a major component of the published analyses. If aromatase inhibitors do not improve survival, then the cost-effectiveness values are underestimated and incorporating more realistic assumptions may raise cost-effectiveness ratios. Our critique of these analyses implies that health policy related to aromatase inhibitors should be revisited.

Our study provides the most recent and comprehensive view of CEAs addressing AIs in the first-line setting. Compared to other systematic reviews we employ no language restrictions. [51], [52] We also systematically assess important potential sources of bias in CEA. Our analysis also compares predictions of model-based CEAs for AIs to clinical data. Following the release of preliminary results from the ATAC and BIG trials, clinicians widely expected AI reductions in breast cancer recurrence to translate into survival gains, much as tamoxifen reduced breast cancer recurrence risk and subsequently increased survival relative to no hormonal therapy. [53] Published CEAs universally predicted increased survival for AIs but reports from the ATAC and BIG trials demonstrated no significant differences in overall survival. [54]–[57] Pooled analysis of data from both trials indicated that at 5 years the overall difference in mortality between patients treated with AIs and those treated with tamoxifen was 0·8% and this decreased to 0·2% at 8 years. [13] In neither case was the difference in OS significant. Even with longer follow-up a divergence in survival curves between AI and tamoxifen treated patients is unlikely to reach the levels estimated by the CEA models. Disparities between long-term outcomes and CEA predictions relate to methods for extrapolating trial data into the future, and assumptions about the impact of adverse events on mortality. Leading oncologists correctly note that with no real survival benefit, the ICER - the incremental cost per life year or quality adjusted life year - associated with AIs in the first-line setting is much higher than published analyses suggest. [17], [18] Indeed, clinical opinion leaders have advocated for switching from tamoxifen to an aromatase inhibitor after two to three years, a strategy that clinical trials indicate may confer a survival benefit compared to five years of tamoxifen. [58], [59]The method for translating outcomes such as disease free survival into survival gains has significant implications on CEA estimates and resulting policy guidance.

Our paper adds to a growing body of literature elucidating the mechanisms through which CEAs can produce potentially misleading results. [60], [61] For example, Polyzos et al demonstrated that industry-sponsored CEAs assessing cervical cancer screening were more likely to exclude data sources presenting favourable results for existing technologies. [60], [61] We can consider our results in light of approaches for addressing parameter uncertainty, structural uncertainty, and methodological uncertainty in CEA modeling. Despite significant parameter uncertainty about the true value of the relative risk of breast cancer recurrence for AIs compared to tamoxifen, 44% of analyses we identified did not vary this critical parameter in even one-way sensitivity analysis. Uncertainty arising from the impact of adverse events can be addressed by adapting the model structure to model adverse events and resulting outcomes. Published CEAs of aromatase inhibitors did not adequately explore structural uncertainty. Adverse events were inadequately modeled, and the potential for increased mortality resulting from adverse events largely overlooked. Methodological uncertainty was also inadequately addressed. Thirty-five percent of studies did not test methodological approaches for extrapolating short-term outcomes from RCTs to a longer time horizon. Twenty-eight percent of studies did not vary the discount rates. Our findings are consistent with a recent systematic review of all published CEAs demonstrating that many authors do a poor job of accounting for uncertainty. [26].

Our analysis has some limitations. Jang et al demonstrated that industry sponsored CEAs were significantly more likely than CEAs conducted by independent academic centres to reach favourable conclusions about the cost-effectiveness of AIs. [52] It was beyond the scope of our analysis to quantify the impact of industry sponsorship on methodological choices and CEA conclusions, even though association between industry-sponsorship and favourable CEA findings is well established. [62]–[64] We focused only on AIs in the first-line setting, and thus had limited statistical power to detect differences. However, we demonstrate how favourable results from industry-sponsored clinical studies may propagate through CEAs when study results are incorporated with no adjustment for real world settings. [65] Our analysis did not include a search of the grey literature, and thus we may have missed some CEAs published by government agencies. Some authors published multiple models identified by our systematic review and thus study data are not strictly independent. However, in each case of multiple models, authors addressed different jurisdictions, thus the question of whether guidance is policy relevant is important for each. The small number of studies precludes analysis of variance in CEA estimates arising from differences in data sources, study perspective, cost categories, time horizon, discount rates, adverse events or utility measurement techniques. Furthermore, authors provided insufficient information on factors like cost categories and utility measurement techniques to allow for this kind of analysis. Finally, it was beyond the scope of our study to summarize the results of sensitivity analyses as authors report sensitivity analyses in different ways, employing different criteria and different thresholds for identifying important variables using sensitivity analysis.

Supporting Information

Acknowledgments

Carolyn Ziegler assisted with developing literature search strategies and ran the literature searches.