A systematic review and critical analysis of cost-effectiveness studies for coronary artery disease treatment

Quality of reporting

A summary of the assessment against the CHEERS Statement are shown in Table 4. Overall, the quality of reporting was high, with studies adequately reporting against 50 to 100% of relevant items on the checklist (Table 3). A more detailed table is available in Supplementary File 3.

All studies adequately reported on the CHEERS Statement items relating: to model/analysis description, background and reasons for undertaking economic evaluations, relevant patient groups and sub-groups, comparators, time horizons and choice of health outcomes included in their analyses. The most poorly reported element related to reasons for choice of model (part of item 15). Only three studies did this. The others described the analysis undertaken, but did not give reasons for the chosen strategy.

Usefulness of evaluations

While for the most part the studies reported their analyses and findings to a good standard when assessed against the CHEERS Statement, their usefulness to decision-makers is arguably of greater importance. The summary data extracted in relation to usefulness of each paper is shown in Table 4. Our assessment of the usefulness of the reporting for decision-makers is in Table 5.

Of the five papers reviewed in full, three were trial-based analyses, one used a cohort modelling approach and there was one meta-analysis (Table 4). The timeframes analysed ranged from 1 year post-intervention to a lifetime horizon. The studies came from a wide range of countries. Three of the five studies used quality of life measures in their analyses, one of which only considered QALYs as part of sensitivity analyses. The others used length of life measures to assess cost-effectiveness and one also used the clinical endpoint of myocardial infarction. All studies based their analyses on direct costs only.

In assessing the usefulness of reporting, we found that while most studies reported on various items, reporting was not always easy to interpret in the context of decision-making. In judging the reporting we were looking for a clear direction or suggestion about how the results of the analysis could be used to improve the efficiency of healthcare resource use. Only two studies made a clear statement about changing the allocation of resources or how the outcomes are relevant to policy^17,18. We rated two studies as low usefulness, two medium and only one highly useful for decision-makers.

Caruba et al.¹⁶ carried out a meta-analysis of cost-effectiveness studies. After concluding that there was no statistically significant difference between treatment strategies on clinical endpoints of myocardial infarction or death, the analysis was conducted on costs only, over 1 and 3 years¹⁶. As a result, the analysis focusses primarily on cost differences across the treatments. They estimated that substantial cost savings could be made through the management of patients with stable angina, using medical therapy¹⁶.

A more detailed examination of the outcomes reported by Caruba et al.¹⁶ revealed that while no statistically significant differences were found, there appears to be some clinically significant difference in treatment effectiveness. The confidence intervals of hazard ratios reported for both death and myocardial infarction are very wide. For example, at three years follow-up, confidence intervals related to estimates of risk of death range from a halving to a doubling of risk for all comparator treatments¹⁶. Similarly, the probabilities of being the best treatment vary widely; from 0.49 for drug-eluting stents to 0.05 for percutaneous transluminal coronary angioplasty (for risk of death at three years follow-up)¹⁶. These results suggest both that there is a high degree of uncertainty regarding estimates of effectiveness, and therefore that a clinically significant difference between the treatments is possible. This could greatly affect estimates of cost-effectiveness. In addition to uncertainty regarding estimates of effectiveness, the authors highlight important limitations and high levels of uncertainty in the overall findings.

We rated this meta-analysis as of low use to decision-makers due to the level of uncertainty described, making it difficult to interpret how the findings might be used to direct policy or practice. While a high level of uncertainty in results should not disqualify an analysis from being useful, the authors did not make any statements which assist in determining how the findings might be used. In addition, the difference in effectiveness of treatments was not fully explored, adding even more uncertainty to the findings.

An analysis by Fidan et al.¹⁷ modelled the life years gained for 36 condition–treatment scenarios for coronary artery disease. These included everything from acute myocardial infarction to primary prevention using statins¹⁷. They used the IMPACT model; a large cell-based mortality model of coronary heart disease risk and treatment^17,21. Cost-effectiveness ratios were reported, however these are not presented as incrementally, which makes it difficult to do head-to-head treatment comparisons. All treatments were examined against the baseline mortality rates, and they found that medical and surgical treatments prevented or postponed over 25,000 deaths in patients with coronary artery disease¹⁷. The approach ranked different interventions, showing a 100-fold difference in cost-effectiveness across all treatments, but it does not provide insight into incremental costs associated with new technology or interventions¹⁰. Again, the analysis only considers length of life, not quality of life. While this provides useful information about length of life it does not consider the full effect of different treatments. Included in this assessment are treatments for chronic angina. It is a somewhat counterintuitive approach to only examine length of life gains from a treatment that targets symptom relief.

While the reporting in this study is clear, we rated it as of medium usefulness for decision-makers because it does not include an assessment of the full effect of treatments on health outcomes (i.e. it only assessed length of life). The authors did, however, present a general policy suggestion, stating that investment in secondary prevention was likely to produce gains in length of life for lower costs.

Prior to the analysis of the MASS-II trial, there had been no cost-effectiveness analysis based on a trial comparing percutaneous intervention, surgery and medical therapy together²⁰. To address this gap, Griffin et al.¹⁸ conducted an economic analysis using the Appropriateness of Coronary REvascularisation (ACRE) study cohort. The ACRE study rated patients as appropriate for percutaneous coronary intervention and/or coronary artery bypass grafting but followed them according to the treatment they actually received¹⁸. Economic analysis of the ACRE study data concluded that coronary artery bypass grafting was cost-effective compared with percutaneous coronary intervention in patients classified as appropriate for bypass grafting only or for both bypass grafting and percutaneous intervention¹⁸. The analysis also found that percutaneous coronary intervention was not cost-effective when compared with medical therapy for patients classified as appropriate for percutaneous coronary intervention only¹⁸. The results of this analysis are useful because they include quality of life outcomes. However, the approach used averaged quality of life over the 6-year period using a regression model¹⁸. This gives some good information about the average quality of life of patients receiving different treatments over the time period, but does not account for events during which patients might experience reduced quality of life, such as a period of hospitalisation for a subsequent procedure.

We rated this analysis as of high usefulness to decision-makers. While the estimates of quality of life could be improved, the authors make clear statements about the changes in costs and health outcomes achieved through different treatments. They also make a clear statement of how to make changes to resource allocation based on their results, which could benefit the health service. One limitation to the usefulness of the outcomes presented is that they only cover the six-year trial period. This may not be long enough to see the full cost-effectiveness of treatment for a chronic disease and the authors foresee extending the model over a lifetime horizon in future work¹⁸.

The analysis by Hlatky et al. in 2009 examined the cost-effectiveness of revascularisation procedures in patients with type-2 diabetes, using data from the Bypass Angioplasty Revascularization Investigation 2 Diabetes trial (BARI 2D)¹⁹. The BARI 2D study randomised patients with type 2 diabetes to medical therapy alone or medical therapy with immediate revascularisation (either PCI or CABG)²². While the effectiveness of treatment for coronary artery disease has been shown to be affected by the presence of diabetes^23–26, due to the high prevalence of type 2 diabetes in this patient population and more generally, this analysis was not excluded on grounds of being relevant only to a specific group. The rates of diabetes in other included studies are 36% in MASS II²⁰, 15% in the ACRE study¹⁸, and 9 to 33% in the studies included in the meta-analysis by Caruba et al.¹⁶. The economic evaluation of the BARI 2D study outcomes concluded that medical therapy was cost-effective compared with revascularisation (PCI or CABG), in the short-term (4 years)¹⁹. When using lifetime projections of cost-effectiveness, however, medical therapy was cost-effective compared with PCI, and CABG was cost-effective compared with medical therapy¹⁵.

The BARI 2D trial used a pragmatic approach which reflects the realities of clinical practice; patients undergoing revascularisation were not randomised to a particular revascularisation strategy (i.e. PCI or CABG); this was directed by clinicians ahead of randomisation to either prompt revascularisation or medical therapy²². The effect of this is that patients were stratified into groups based on clinical markers of disease severity. The results of the study are therefore useful for choosing between medical therapy and PCI in patients with less severe disease, or between medical therapy and CABG in patients with severe disease. They are also only relevant to diabetic patients, however, as prevalence of type 2 diabetes is increasing globally, this is relevant to an increasing number of patients.

The overall results in the BARI 2D trial are based on length of life measures; quality of life measures were only used in sensitivity analyses. It was concluded that the quality of life measures did not affect the estimates of cost-effectiveness, based on life-years only. It is unclear why this choice was made, when quality of life measures provide a more comprehensive assessment of treatment effect. We rated this study as of medium use for decision-makers as it presents an analysis of real-world practice, but does not account for the full effect of treatment on patients’ health, by all but ignoring quality of life measures. Decision-makers wishing to know the full effect of different treatments on patient health outcomes need information beyond length of life.

Vieira et al. also conducted a trial-based analysis using data from the MASS II Trial (Medical Angioplasty or Surgery Study)²⁰. This was the only trial revealed in searches which randomised patients to each of the three treatment options. Its major conclusions were that medical therapy was cost-effective compared to CABG, and CABG was cost-effective compared to PCI²⁰. While this analysis did use QALYs they were not calculated using conventional health related quality of life surveys, but estimated based on the average time to event and angina free proportion of the population in each group²⁰. This is unlikely to provide good estimates of quality of life in these patients as the measurement assumes that in the period between events the patient has full quality of life and that those with angina have no quality of life. These estimates produced average QALYs of 2.07 to 2.81, over 5 years which if averaged over that time give utility weights of 0.41 to 0.56 (e.g. 2.07/5). These values are far below the estimates of 0.69 to 0.86 used in other analyses of patients with coronary artery disease^27–31. Values of less than 0.5 are generally seen only in very debilitating conditions. The quality of life estimates in the MASS II study analysis therefore substantially undervalue the quality of life of patients with coronary artery disease. While the outcomes of that analysis do examine both costs and effectiveness of the three different treatments for coronary artery disease, the outcomes reported are not useful for those making decisions about resource allocation because they do not allow comparison with other areas of healthcare or report the incremental cost per QALY gained.

We rated the analysis by Vieira et al.²⁰ as of low usefulness to decision-makers because although quality of life was included in the analysis, it was not done in a way that makes it comparable to other studies. In addition to these novel methods of QALY estimation, the authors did not conduct incremental analyses, nor did they discuss any uncertainty in their findings.

It is worth noting that further research has been undertaken using the MASS II trial data and a validated quality of life instrument³². Unfortunately, only a conference abstract was available and it was therefore not included in the analysis. The results available in that abstract show much higher average health utility weights of 0.77 to 0.81³², aligning them with the values seen in other analyses of coronary artery disease^27–31. When published, the full analysis will add greatly to the current knowledge.

Challenge for health service decision-makers

When operating under conditions of scarce resources there is responsibility to promote cost-effective care, achieving larger health gains from available resources. In an ideal world, decision-makers would have information about the long-term costs and health outcomes achievable through different configurations of health services and be able to invest accordingly. However, without good evidence of cost-effectiveness, it is impossible for decision-makers to fulfil this responsibility with any confidence.

Current evidence and value of evidence for coronary artery disease

In the case of stable coronary artery disease, we have some information about the comparative cost-effectiveness of optimal medical therapy, PCI and CABG, but it is difficult to interpret in the context of healthcare resource allocation. Overall, the results of cost-effectiveness analyses suggest that in most scenarios optimal medical therapy is cost-effective compared with alternatives and CABG is cost-effective for certain patient groups.

However, only three of the five studies included in this review used quality of life as an effectiveness measure. This is a key outcome measure for cost-effectiveness and good decision making. For chronic diseases, improvements in quality of life are equally as relevant as improvements in length of life. In the case of coronary artery disease, relief from chest-pain is a key objective of treatment. If quality of life is not measured, two treatments affording a patient equal length of life are valued equally even where one restored the patient to better health than the other. However, if given the choice, patients and health service providers would choose the option most likely to provide the best improvements to quality of life. Therefore, analyses based solely on length of life measures do not provide a full picture of the effectiveness of each treatment.

Another omission from the literature is to neglect the lifetime costs and health outcomes. Important information about the longevity of treatment effect may be overlooked. This is particularly important for chronic diseases such as coronary artery disease, where important costs and health consequences are missed when they occur beyond the timeframe of a clinical trial.

Confidence in changing services based on current evidence

The current information about cost-effectiveness of treatments for stable coronary artery disease suggests that either optimal medical therapy or CABG could be cost-effective, over a 1-year to lifetime timeframe. There is no evidence from the papers included in this review that PCI is cost-effective when compared with other competing treatment options. Therefore the current cost-effectiveness evidence does not support the increased use of PCI that has been seen in Australia and internationally, and there is increasingly reduced evidence of clinical effectiveness³³.

However, it is unlikely that healthcare decision-makers would be confident making changes to the allocation of resources based on the economic evidence outlined in this review. Our evaluation of the relevant cost-effectiveness evidence showed that overall, information is not presented in a way useful for decision-making. We found only one study to be of high usefulness in this context (Table 5).

Our assessment of the usefulness of the cost-effectiveness studies examined suggests that poor reporting may contribute to the problem. We rated two out of the five studies as ‘low usefulness’ for decision-makers. Reporting was either too complex, making interpretation challenging, or uncertainty was not reported in way that made clear the effect of acting on the evidence. It is also apparent from our analysis, that the CHEERS Statement, while encouraging comprehensive reporting, is not sufficient alone, to assess the usefulness of economic evaluations.

Others have explored barriers to the use of economic evaluation by decision-makers³⁴. A review by Merlo et al.³⁴ used an accessibility and acceptability framework developed by Williams and Bryan³⁵ to categorise barriers to use of economic evaluations. Accessibility refers to the ability of decision-makers to interpret and use economic evidence, and includes issues of complexity and timeliness of economic evaluations^34,35. Acceptability includes factors associated with scientific rigor, applicability to the institution in which decisions are to be made, and ethical considerations such as equity^34,35.

While some studies we examined included comprehensive reporting, they did not always present results in a way conducive to decision-making. For example reporting large results tables covering many different clinical scenarios, as seen in the paper by Fidan et al.¹⁷, demonstrates the complexity but does nothing to assist those wanting to make higher-level resource allocation decisions. While clinical complexity is inherently important in many contexts, for it to be of use for decision-making, it needs to be summarised in a way which makes clear the likely impact of acting on the evidence presented. In fact, presenting all clinical complexities can mean the overall message is lost in the details, making research less accessible to decision-makers and decreasing the chance that any improvement to health services will follow. This is unfortunate, considering the purpose of economic evaluation is to provide evidence for resource allocation decisions to improve service delivery.

Most of the economic evaluations we have assessed suffer from a number of accessibility problems which prevent them from being useful for decision-making; the interpretation of results and their applicability to policy are generally lacking. The information in Table 5 reveals that only one study, by Griffin et al.¹⁸, included a clear expression of the confidence in estimates of cost-effectiveness, useful in the context of decision-making. Hlatky et al.¹⁹ made a less-clear statement. Only two studies, Griffin et al.¹⁸ and Fidan et al.¹⁷, made clear statements of what direction policy should take based on results of their research (Htlaty et al.¹⁹ and Vieira et al.²⁰ expressed a direction but less-clearly). The result of this highly complex reporting and lack of clear policy direction to follow, is that some otherwise very scientifically rigorous analyses have failed to generate any policy changes or perhaps even reach their intended audience.

In addition to these complexities, while the patient populations may be similar to those in Australia, none of these studies have been carried out in the Australian context. For the purposes of resource allocation decisions in Australia, an analysis based at least on Australian costs, is required.

Conclusion

The evaluation of the studies in this review highlights a lack of information useful for making decisions about the allocation of resources for coronary artery disease. It is concerning that over $2 billion of Australia’s annual healthcare budget is being spent on coronary artery disease, with inadequate economic evidence. Since the mid to late 1990s, increased spending has been directed towards PCI with stenting, over coronary artery bypass grafting, but there is insufficient economic evidence to support this transition. Compounding this, the current cost-effectiveness evidence which would suggest a move away from PCI and stents remains too unclear and uncertain for policy-makers to be confident in making changes. In a time of increased pressure on health budgets, economic evidence should be fundamental to resource allocation decisions.

For those wishing to make resource allocation decisions to improve the efficiency in treatment of coronary artery disease, the current evidence is insufficient. A transparent, structured, lifetime analysis of all competing treatments, incorporating quality of life measures, would be valuable for decision makers. The analysis should account for fluctuations in the quality of life of patients over their lifetimes, related to symptom relief, repeat procedures and acute events.

The findings of recent trials, in particular ORBITA^33,36 and ISCHEMIA³⁷, will strengthen the evidence about the effectiveness of conservative versus invasive therapy for stable coronary artery disease. In turn these data will enable improved cost-effectiveness analyses.

To be of use to decision-makers and have a better chance of generating policy change, analyses must be accessible; economic evaluations should include a clear indication for the direction of policy or changes to practice that should follow and a statement of the probability that such changes will the generate the predicted improvements. In their systematic review of barriers and facilitators to use of evidence by policymakers, Oliver et al. named clarity, relevance and reliability as some of the top barriers to use of evidence³⁸. For decision-makers to be able to act on the economic evidence, the expected effect of making changes based on the results needs to be clear. In cases where there is too much uncertainty, a strategy to improve the analysis should be outlined. We suggest that to improve reporting of economic evaluations for decision-making, an additional item could be included in the CHEERS Statement, relating to implications for policy and practice. Ideally, this would be a statement describing the implications of acting on the evidence presented; encompassing both expected improvements to health outcomes and confidence in the effect.