Abstract
Introduction and objectives
An increasing number of published studies of economic evaluations of health technologies have been reviewed and summarized with different purposes, among them to facilitate decision-making processes. These reviews have covered different aspects of economic evaluations, using a variety of methodological approaches. The aim of this study is to analyze the methodological characteristics of the reviews of economic evaluations in health care, published during the period 1990–2010, to identify their main features and the potential missing elements. This may help to develop a common procedure for elaborating these kinds of reviews.
Methods
We performed systematic searches in electronic databases (Scopus, Medline and PubMed) of methodological reviews published in English, period 1990–2010. We selected the articles whose main purpose was to review and assess the methodology applied in the economic evaluation studies. We classified the data according to the study objectives, period of the review, number of reviewed studies, methodological and non-methodological items assessed, medical specialty, type of disease and technology, databases used for the review and their main conclusions. We performed a descriptive statistical analysis and checked how generalizability issues were considered in the reviews.
Results
We identified 76 methodological reviews, 42 published in the period 1990–2001 and 34 during 2002–2010. The items assessed most frequently (by 70 % of the reviews) were perspective, type of economic study, uncertainty and discounting. The reviews also described the type of intervention and disease, funding sources, country in which the evaluation took place, type of journal and author’s characteristics. Regarding the intertemporal comparison, higher frequencies were found in the second period for two key methodological items: the source of effectiveness data and the models used in the studies. However, the generalizability issues that apparently are creating a growing interest in the economic evaluation literature did not receive as much attention in the reviews of the second period. The remaining items showed similar frequencies in both periods.
Conclusions
Increasingly more reviews of economic evaluation studies aim to analyze the application of methodological principles, and offer summaries of papers classified by either diseases or health technologies. These reviews are useful for finding literature trends, aims of studies and possible deficiencies in the implementation of methods of specific health interventions. As no significant methodological improvement was clearly detected in the two periods analyzed, it would be convenient to pay more attention to the methodological aspects of the reviews.
Similar content being viewed by others
References
Davies, L., Coyle, D., Drummond, M.: Current status of economic appraisal of health technology in the European Community: report of the network. The EC Network on the Methodology of Economic Appraisal of Health Technology. Soc. Sci. Med. 38, 1601–1607 (1994)
Drummond, M., Brandt, A., Luce, B., Rovira, J.: Standardizing methodologies for economic evaluation in health care. Practice, problems and potential. Int. J. Technol. Asses. Health Care 9, 26–36 (1993)
Drummond, M.F., O′Brien, B.J., Stoddart, G.L., Sculpher, M.F., Torrance, G.W.: Methods for the Economic Evaluation of Health Care Programs, 2nd edn. Oxford University Press, New York (1997)
Graf von der Schulenburg, J.M., Hoffman, C.: Review of European guidelines for economic evaluation of medical technologies and pharmaceuticals. HEPAC 1, 2–8 (2000)
Australia Commonwealth Department of Health, Housing and Community Services: Guidelines for the pharmaceutical industry on preparation of submissions to the Pharmaceutical Benefit Advisory Committee, Canberra (November 1995)
Canadian Coordination Office for Health Technology Assessment: Guidelines for economic evaluation of pharmaceuticals, 2nd edn, Ottawa (1997)
Rovira, J., Antoñanzas, F.: Economic analysis of health technologies and programmes. A Spanish proposal for methodological standardization. Pharmacoeconomics 8(3), 245–252 (1995)
López-Bastida, J., Oliva, J., Antoñanzas, F., García-Altés, A., Gisbert, R., Mar, J., Puig-Junoy, J.: Spanish recommendations on economic evaluation of health technologies. Eur. J. Health Econ. 11, 513–520 (2010)
Garattini, L., Grilli, R., Scopelleti, D., Mantovani, L.: A proposal for Italian guidelines in pharmacoeconomics. Pharmacoeconomics 7(1), 1–6 (1995)
Collège des Economistes de la Santé. Recommandations de bonnes practiques des methods d′evaluation économique des strategiques therapeutiques: La Lettre du Collège, numero special: Evaluation économique des strategies therapeutiques (April 1997)
Gruppe, H.K.: Deutsche Empfehlungen zur gesundheitsökonomischen Evaluation—Revidierte Fassung des Hannoveraner Konsens. Gesundheitsökonomie & Qualitätsmanagement 4(3), A62–A65 (1999)
Gold, M.R., Siegel, J.E., Russell, L.B., Weinstein, M.G.: Cost Effectiveness in Health and Medicine, 1st edn. Oxford University Press, New York, NY (1996)
Drummond, M.F., Jefferson, T.O.: On behalf of the BMJ Economic Evaluation Working Party: Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 313, 275–283 (1996)
NHS Centre for Reviews and Dissemination: Making Cost-Effectiveness Information Accessible: the NHS Economic Evaluation Database Project—CRD guidance for reporting critical summaries of economic evaluations. NHS Centre for Reviews and Dissemination, University of York, York (1996)
Health Economic Evaluation Database (HEED): Office of Health Economics and the International Federation of Pharmaceutical Manufacturers Associations (OHE-IFPMA). John Wiley & Sons, Ltd. London. On November 1, (2006)
Pouvourville, G., Ulmann, P., Nixon, J., Boulenger, S., Glanville, J., Drummond, M.: The diffusion of health economics knowledge in Europe: the EURONHEED (European Network of Health Economics Evaluation Database) Project. Pharmacoeconomics 23(2), 113–120 (2005)
Jefferson, T., Demicheli, V., Vale, L.: Quality of systematic reviews of economic evaluations in health care. JAMA 287(21), 2809–2812 (2002)
Ofman, J.J., Sullivan, S.D., Neumann, P.J., Chiou, C.F., Henning, J.M., Wade, S.W., Hay, J.W.: Examining the value and quality of health economic analysis: implication of utilizing the QHES. J. Manag. Care Pharm. 9, 53–61 (2003)
Boulenger, S., Nixon, J., Drummond, M., Ulmann, P., Rice, S., Pouvourville, G.: Can economic evaluations be made more transferable? Eur. J. Health Econ. 6, 334–346 (2005)
Brazier, J., Deverill, M., Green, C.: A review of the use of health status measures in economic evaluation. Health Technol. Assess. 3, 1–164 (1999)
Nord, E.: Towards quality assurance in QALY calculations. Int. J. Technol. Assess. Health Care 9, 37–45 (1993)
Neumann, P.J., Zinner, D.E., Wright, J.: Are methods for estimating QALYs in CEA improving? Med. Decis. Making 17, 402–408 (1997)
Neumann, P.J., Sandberg, E.A., Bell, C.M., Stone, P.W., Chapman, R.H.: Are pharmaceuticals cost effective? A review of the evidence. Health Aff. 19(2), 92–109 (2000)
Neumann, P.J., Stone, P.W., Chapman, R.H., Sandberg, E.A., Bell, C.M.: The quality of reporting in published cost-utility analyses, 1976–1997. Ann. Intern. Med. 132(12), 964–972 (2000)
Neumann, P.J., Greenberg, D., Olchanski, N.V., Stone, P.W., Rosen, A.B.: Growth and quality of the cost utility literature, 1976–2001. Value Health 8(1), 3–9 (2005)
Neumann, P.J.: Costing and perspective in published cost-effectiveness analysis. Med. Care 47(7 suppl 1), S28–S32 (2009)
Neumann, P.J., Fang, C.-H., Cohen, J.T.: 30 years of pharmaceutical cost-utility analyses. Growth, diversity and methodological improvement. Pharmacoeconomics 27(10), 861–872 (2009)
Earle, C.C., Chapman, R.H., Baker, C.S., Bell, C.M., Stone, P.W., Sandberg, E.A., Neumann, P.J.: Systematic overview of cost utility assessment in oncology. J. Clin. Oncol. 15(18), 3302–3317 (2000)
Gerard, K.: Cost utility in practice: a policy maker’s guide to the state of the art. Health Policy 21(3), 249–279 (1992)
Gerard, K., Smoker, I., Seymour, J.: Raising the quality of CUA: lessons learnt and still to learn. Health Policy 46(3), 217–238 (1999)
Gerard, K., Seymour, J., Smoker, I.: A tool to improve quality of reporting published economic analysis. Int. J. Technol. Assess. Health Care 16, 100–110 (2000)
Holloway, R.G., Benesch, C.G., Rahilly, C.R., Courtright, C.E.: A systematic review of the research of stroke evaluation and treatment. Stroke 30, 1340–1349 (1999)
Stone, P.W., Chapman, R.H., Sandberg, E.A., Liljas, B., Neumann, P.J.: Measuring costs in CUA. Variations in the literature. Int. J. Technol. Health Care 16(1), 111–124 (2000)
Stone, P.W., Teutsch, S., Chapman, R.H., Mell, C., Goldie, S.J., Neumann, P.J.: Cost-utility analysis of clinical preventive services. Published ratios 1976-1997. Am. J. Prev. Med. 19(1), 15–23 (2000)
Deverill, M., Brazier, J., Green, C., Booth, A.: The use of QALY and non-QALY measures of health-related quality of life: assessing the state of the art. Pharmacoeconomics 13, 411–420 (1998)
Otero, H.J., Rybicki, F.J., Greenberg, D., Neumann, P.J.: Twenty years of cost effectiveness analysis in medical imaging: are we improving? Radiology 249(3), 917–925 (2008)
Brown, J., Sculpher, M.: Benefit valuation in economic evaluations of cancer therapies: a systematic review of published literature. Pharmacoeconomics 16, 17–31 (1999)
Badía, X., Rovira, J., Segú, J., Porta, M.: Economic assessment of drugs in Spain. Pharmacoeconomics 5(2), 123–129 (1994)
Iskedjian, M., Trakas, K., Bradley, C.A., Addis, A., Lanctot, K., Kruk, D., Ilersich, A.L., Einarson, T.R.: Quality assessment of economic evaluations published in Pharmacoeconomics. The first four years (1992 to 1995). Pharmacoeconomics 12(6), 685–694 (1997)
Walker, D., Fox-Rushby, J.: Economic evaluation of parasitic diseases: a critique of the internal and external validity of published studies. Int. Health Trop. Med 5, 237–249 (2000)
Walker, D., Fox-Rushby, J.: Economic evaluation of communicable disease interventions in developing countries: a critical review of the published literature. Health Econ. 9, 681–698 (2000)
Salked, G., Davey, P., Arnolda, G.: A critical review of health related economic evaluations in Australia: implications for health policy. Health Policy 31, 111–125 (1995)
Oliver, A., Pritchard, C.: Economic evaluations relating to diabetes: a descriptive review and their compliance with guidance. Value Health 3(1), 7–14 (2000)
Pritchard, C.: What do we know about the economic evaluations literature? A review of HEED. J. Drug Assess. 4, 1–70 (2001)
Nixon, J., Stoykova, B., Glanville, J., Christie, J., Drummond, M., Kleijnen, J.: The UK NHS economic evaluation database: economic issues in evaluations of health technology. Int. J. Technol. Assess. Health Care 16(3), 731–742 (2000)
Garattini, L., De Compadri, P., Clemente, R., Cornago, D.: Economic evaluations in Italy: a review of the literature. Int. J. Technol. Assess. Health Care 19(4), 685–737 (2003)
Spiegel, B.M.R., Targownik, L.E., Kanwal, F., Derosa, V., Dulai, G., Gralnek, I.M., Chiou, C.F.: The quality of published health economic analyses in digestive diseases: a systematic review and quantitative appraisal. Gastroenterology 127(2), 403–411 (2004)
Foster, W.J., Tufail, W., Issa, A.M.: The quality of pharmacoeconomic evaluations of age-related macular degeneration therapeutics: a systematic review and quantitative appraisal of the evidence. Br. J. Ophthalmol. 94(9), 1118–1126 (2010)
Petrou, S., Henderson, J., Roberts, T., Martin, M.A.: Recent economic evaluations of antenatal screening: a systematic review and critique. J. Med. Screen. 7, 59–73 (2000)
Schrappe, M., Lauterbach, K.: Systematic review on the cost effectiveness of public health interventions for HIV prevention in industrialised countries. AIDS 12(supplA), S231–S239 (1998)
Fergusson, D., van Walraven, C., Coyle, D., Laupacis, A.: Economic evaluations of technologies to minimize perioperative transfusion: a systematic review of published studies. Transfus. Med. Rev. 13, 106–117 (1999)
Brauer, C.A., Neumann, P.J., Rosen, A.B.: Trends in cost effectiveness analyses in orthopaedic surgery. Clin. Orthop. Relat. Res. 457, 42–48 (2007)
Bereza, B.G., Machado, M., Einarson, T.R.: Systematic review and quality assessment of economic evaluations and quality of life related to generalized anxiety disorder. Clin. Ther. 31(6), 1279–1308 (2009)
Udvarhelyi, S., Colditz, G.A., Rai, A., Epstein, A.M.: Cost effectiveness and cost benefit analyses in the medical literature. Are the methods being used correctly? Ann. Intern. Med. 116, 238–244 (1992)
Anell, A., Norinder, A.: Health outcome measures used in cost effectiveness studies: a review of original articles published between 1986 and 1996. Health Policy 51, 87–99 (2000)
Smith, W.J., Blackmore, C.C.: Economic analyses in obstetrics and gynecology: a methodologic evaluation of the literature. Obstet. Gynecol. 91(3), 472–478 (1998)
Blackmore, C.C., Magid, D.J.: Methodologic evaluation of the radiology cost-effectiveness literature. Radiology 203(1), 13–15 (1997)
Lee, J.T., Sanchez, L.A.: Interpretation of cost-effective and soundness of economic evaluations in the pharmacy literature. Australian Journal of Health Promotion 48, 2622–2627 (1991)
García-Altés, A.: Twenty years of health care economic analysis in Spain: are we doing well? Health Econ. 10, 715–729 (2001)
Esperato, A., García-Altés, A.: Prevención en salud: ¿inversión rentable? Eficiencia económica de las intervenciones preventivas en España. Gac. Sanit. 21(2), 150–161 (2007)
Oliva, J., Del Llano, J., Sacristán, J.A.: Análisis de las evaluaciones económicas de tecnologías sanitarias realizadas en España en la década 1990-2000. Gac. Sanit. 16(supl 2), 2–11 (2002)
Catalá-López, F., García-Altés, A.: Evaluación económica de intervenciones sanitarias en España durante el período 1983–2008. Rev. Esp. Salud Pública 84, 353–369 (2010)
Hutton, J., Iglesias, C., Jefferson, T.O.: Assessing the potential cost effectiveness of pneumococcal vaccines in the US: methodological issues and current evidence. Drugs Aging 15(Suppl 1), 31–36 (1999)
Van der Weijden, T., Knottnerus, J.A., Ament, A.J.H.A., Stoffers, H.E.J.H., Grol, R.P.T.M.: Economic evaluation of cholesterol-related interventions in general practice: an appraisal of the evidence. J. Epidemiol. Community Health 52, 586–594 (1998)
Evers, S.M.A.A., Ament, A.J.H.A., Blaauw, G.: Economic evaluation in stroke research: a systematic review. Stroke 31, 1046–1053 (2000)
Evers, S.M.A.A., Van Wijk, A.S., Ament, A.J.H.A.: Economic evaluation of mental care interventions: a review. Health Econ. 6, 161–177 (1997)
Manuel, M.R., Chen, L.-M., Caughey, A.B., Subak, L.L.: Cost effectiveness in gynecologic oncology: methodological quality and trends. Gynecol. Oncol. 93, 1–8 (2004)
Hutter, M.F., Antoñanzas, F.: Economic evaluations in the EURONHEED. A comparative analysis. Pharmacoeconomics 27(7), 561–570 (2009)
Saleh, K.J., Gafni, A., Saleh, L., Gross, A.E., Schatzker, J., Tile, M.: Economic evaluations in the hip arthroplasty literature lessons to be learned. J of Arthroplasty 14(5), 527–532 (1999)
Bozic, K.J., Saleh, K.J., Rosenberg, A.G., Rubash, H.E.: Economic evaluation in total hip arthrosplasty. Analysis and review of the literature. The Journal of Arthroplasty 19(2), 180–189 (2004)
Kruper, L., Kurichi, J., Sonnad, S.: Methodologic quality of cost effectiveness analyses of surgical procedures. Ann. Surg. 245(1), 147–151 (2007)
Severens, J., van der Wilt, G.J.: Economic evaluation of diagnostic tests. A review of published studies. Int. J. Technol. Assess. Health Care 15(3), 480–496 (1999)
Gambhir, S.S., Schwimmer, J.: Economic evaluation studies in nuclear medicine: a methodological review of the literature. Q. J. Nucl. Med. 44(2), 121–137 (2000)
Chang, W.Y., Henry, B.M.: Methodologic principles of cost analysis in the nursing, medical, and health services literature, 1990–1996. Nurs. Res. 48(2), 94–104 (1999)
Elixhauser, A., Halpern, M., Schmier, J., Luce, B.R.: Health care CBA and CEA from 1991 to 1996: an updated bibliography. Medical Care 36(5), MS 1–9, MS 18–47 (1998)
Elixhauser, A., Luce, B.R., Taylor, B.R., Reblando, J.: Health care CBA/CEA: an update on the growth and composition of the literature. Med Care 31(7suppl), JS1–11, JS18–149 (1993)
Jacobs, P., Fassbender, K.: The measurement of indirect cost in the health economics evaluation literature. Int. J. Technol. Assess. Health Care 14, 799–808 (1998)
Uegaki, K., de Bruijne, M.C., Lambeek, L., Anema, J.R., van der Beek, A.J., van Mechelen, W., van Tulder, M.W.: Economic evaluations of occupational health interventions from a corporate perspective—a systematic review of methodological quality. Scand. J. Work Environ. Health 36(4), 273–288 (2010)
Greenberg, D., Earle, C., Fang, C.-H., Eldar-Lissai, A., Neumann, P.J.: When is cancer care cost-effective? A systematic overview of cost-utility analyses in oncology. J. Natl Cancer Inst. 102(2), 82–88 (2010)
Stone, P.W., Schackman, B.R., Neukermans, C.P., Olchanski, N., Greenberg, D., Rosen, A.B., Neumann, P.J.: A synthesis of cost-utility analysis literature in infectious disease. Lancet Infectious Diseases 5(6), 383–391 (2005)
Anonychuk, A.M., Tricco, A.C., Bauch, C.T., Pham, B., Gilca, V., Duval, B., John-Baptiste, A., Woo, G., Krahn, M.: Cost-effectiveness analyses of hepatitis A vaccine: a systematic review to explore the effect of methodological quality on the economic attractiveness of vaccination strategies. Pharmacoeconomics 26(1), 17–32 (2008)
Schwappach, D.L.B, Boluarte, T.A.: HEE-GER: a systematic review of German economic evaluations of health care published 1990–2004. BMC Health Services Research 7 art. No. 7 (2007)
Starkie, H.J., Griggs, A.H., Chambers, M.G.: Pharmacoeconomics in COPD: lessons for the future. International Journal of COPD 3(1), 71–88 (2008)
Soegaard, R., Christensen, F.B.: Health economic evaluation in lumbar spinal fusion: a systematic literature review anno 2005. Eur. Spine J. 15, 1165–1173 (2006)
Van Veen, N.H.J., McNamee, P., Richardus, J.H., Smith, W.C.S.: Cost-effectiveness of intervention to prevent disability in leprosy: a systematic review. PLoS ONE 4(2), art. no. e4548 (2009)
Richardson, G., Manca, A.: Calculation of quality adjusted life years in the published literature: a review of methodology and transparency. Health Econ. 13, 1203–1210 (2004)
Griebsch, I., Coast, J., Brown, J.: Quality-adjusted life-years lack quality in pediatric care: a critical review of published cost-utility studies in child health. Pediatrics 115(5), e600–e614 (2005)
Coelho de Soárez, P., Novaes, H.M.D., Sartori, A.M.H.: Impact of methodology on the results of economic evaluations of varicella vaccination programs: is it important for decision-making? Cad. Saude Pública 25(Sup 3), S401–S414 (2009)
Garattini, L., De Compadri, P., Koleva, D., Pasina, L., Nobili, A.: A critical review of the full economic evaluations of pharmacological treatments for colorectal cancer. Journal of Medical Economics 11, 177–187 (2008)
Griffiths, U.K., Miners, A.: Economic evaluations of Haemophilus influenza type b vaccine: systematic review of the literature. Expert Rev. Pharmacoeconomics Outcomes Res. 9(4), 333–346 (2009)
Lee, A., Chan, S., Chen, P.P., Gin, T., Lau, A.S.C.: Economic evaluations of acute pain service programs. A systematic review. Clin. J. Pain 23(8), 726–733 (2007)
Merito, M., Breitscheidel, L.: Cost estimation in health economic evaluations in Germany: a systematic review. Journal of Pharmaceutical Health Services Research 1, 99–106 (2010)
Roberts, E., Cumming, J., Nelson, K.: A review of economic evaluations of community mental health care. Medical Care Research and Reviews 62(5), 503–543 (2005)
Ungar, W.J., Santos, M.T.: Quality appraisal of pediatric health economic evaluations. Int. J. Technol. Assess. Health Care 21(2), 203–210 (2005)
Willems, D.C.M., Joore, M.A., Hendriks, J.J.E., Wouters, E.F.M., Severens, J.L.: Cost-effectiveness of self-management in asthma: a systematic review of peak flow monitoring interventions. Int. J. Technol. Assess. Health Care 22(4), 436–442 (2006)
Talmor, D., Shapiro, N., Greenberg, D., Stone, P.W., Neumann, P.J.: When is critical care medicine cost-effective? A systematic review of the cost-effectiveness literature. Crit. Care Med. 34(11), 2738–2747 (2006)
Rogowski, W.: Genetic screening by DNA technology: a systematic review of health economic evidence. Int. J. Technol. Assess. Health Care 22(3), 327–337 (2006)
Chiou, C.F., Hay, J.W., Wallace, J.F., et al.: Development and validation of a grading system for the quality of cost-effectiveness studies. Med. Care 41, 32–44 (2003)
Welte, R., Feenstra, T., Jager, H., Leidl, R.: A decision chart for assessing and improving the transferability of economic evaluation results between countries. Pharmacoeconomics 22(13), 857–876 (2004)
Antoñanzas, F., Rodriguez-Ibeas, R., Juárez, C., Hutter, M.F., Lorente, R., Pinillos, M.: Transferability indices for health economic evaluations: methods and applications. Health Econ. 18, 629–643 (2009)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hutter, MF., Rodríguez-Ibeas, R. & Antonanzas, F. Methodological reviews of economic evaluations in health care: what do they target?. Eur J Health Econ 15, 829–840 (2014). https://doi.org/10.1007/s10198-013-0527-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10198-013-0527-7