Abstract
This paper describes the key principles of why an assessment of uncertainty and its consequences are critical for the types of decisions that a body such as the UK National Institute for Health and Clinical Excellence (NICE) has to make. In doing so, it poses the question of whether formal methods may be useful to NICE and its advisory committees in making such assessments. Broadly, these include the following: (i) should probabilistic sensitivity analysis continue to be recommended as a means to characterize parameter uncertainty; (ii) which methods should be used to represent other sources of uncertainty; (iii) when can computationally expensive models be justified and is computation expense a sufficient justification for failing to express uncertainty; (iv) which summary measures of uncertainty should be used to present the results to decision makers; and (v) should formal methods be recommended to inform the assessment of the need for evidence and the consequences of an uncertain decision for the UK NHS?
Similar content being viewed by others
References
Culyer AJ, McCabe C, Briggs A, et al. Searching for a threshold, not setting one: the role of the National Institute for Health and Clinical Excellence. J Health Serv Res Pol 2007; 12: 56–58
McCabe C, Claxton K, Culyer AJ. The NICE cost-effectiveness threshold: what it is and what that means. Pharmacoeconomics 2008; 26 (9): 733–744
Phelps CE, Mushlin Al. On the (near) equivalence of cost-effectiveness and cost-benefit analyses. Int J Technol Assess Health Care 1991; 7 (1): 12–21
Stinnett A, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making 1998; 18: S68–S80
Sculpher MJ. Subgroups and heterogeneity in cost-effectiveness analysis. Pharmacoeconomics 2008; 26 (9): 799–806
Claxton K. The irrelevance of inference: a decision making approach to the stochastic evaluation of health care technologies. J Health Econ 1999; 18: 342–364
Briggs A, Sculpher M. An introduction to Markov modelling for economic evaluation. Pharmacoeconomics 1998; 13: 397–409
Briggs A, Claxton K, Sculpher MJ. Decision analytic modelling for the evaluation of health technologies. Oxford: Oxford University Press, 2006
Eckermann S, Willan A. Expected value of information and decision making in HTA. Health Econ 2007; 16: 195–209
Palmer S, Smith PC. Incorporating option values into the economic evaluation of health care technologies. J Health Econ 2000; 19: 755–766
Griffin S, Claxton K, Palmer S, et al. Dangerous omissions: the consequences of ignoring decision uncertainty. Med Dec Making 2007; 27 (1): E8
Chalkidou K, Hoy A, Littlejohns P. Making a decision to wait for more evidence: when the National Institute for Health and Clinical Excellence recommends a technology only in the context of research. J R Soc Med 2007; 100 (10): 453–460
National Institute for Health and Clinical Excellence. NICE guide to the methods of health technology appraisal. London: NICE, 2004
Cooksey D. A review of UK health research funding. London: Stationery Office, 2006
Office of Fair Trading. The pharmaceutical price regulation scheme: an OFT market study. London: Office of Fair Trading, 2007
Claxton K, Briggs A, Buxton MJ, et al. Value-based pricing for NHS drugs: an opportunity not to be missed? BMJ 2008; 336: 251–254
Claxton K, Sculpher MJ, McCabe C, et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ 2005; 14: 339–347
Thompson K, Graham J. Going beyond the single number: using probabilistic risk assessment to improve risk management. Human Ecologic Risk Assess 1996; 2: 1008–1034
O’Hagan A, Luce BR. A primer on Bayesian statistics in health economics and outcomes research. Bethesda (MD): Bayesian initiative in health economic and outcomes research, MEDTAP International, 2003
O’Hagan A, Buck CE, Daneshkhah A, et al. Uncertain judgements: eliciting expert probabilities. Chichester: John Wiley and Sons, 2006
Sutton A, Ades AE, Cooper N, et al., on behalf of the NICE Decision Support Unit. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics 2008; 26 (9): 753–767
Ades AE, Sutton AJ. Multiparameter evidence synthesis in epidemiology and medical decision-making: current approaches. JRSS (A) 2005; 16 (1): 5–35
Ades AE, Claxton K, Sculpher MJ. Evidence synthesis, parameter correlation, and probabilistic sensitivity analysis. Health Econ 2006; 15: 373–381
Bojke L, Claxton K, Palmer S, et al. Defining and characterising structural uncertainty in decision analytic models [CHE research paper 9]. York: Centre for Health Economics, University of York, 2006
McCabe C, Dixon S. Testing the validity of cost-effectiveness models. Pharmacoeconomics 2000; 17 (5): 501–513
Draper D. Assessment and propagation of model uncertainty. J Royal Stat Soc 1995; 57: 45–97
Chatfield C. Model uncertainty, data mining and statistical inference (with discussion). J Royal Stat Soc 1995; 158: 419–466
Bojke L, Claxton K, Bravo Vergel Y. Using expert elicitation to resolve issues of structural uncertainty in decision analytic models. Med Dec Making 2007; 27 (1): E8
White IR, Pocock SJ, Wang D. Eliciting and using expert opinions about dropout bias in randomised controlled trials. Clin Trials 2007; 4 (2): 125–139
Leal J, Wordsworth S, Legood R, et al. Eliciting expert opinion for economic models. Value Health 2007; 10 (3): 195–203
Griffin S, Claxton K, Hawkins N, et al. Probabilistic analysis and computationally expensive models: necessary and required? Value Health 2006; 9: 244–252
Edmunds WJ, Medley GF, Nokes DJ. Evaluating the cost-effectiveness of vaccination programmes: a dynamic perspective. Stat Med 1999; 18 (23): 3263–3282
O’Hagan A, Stevenson M, Madan J. Monte Carlo probabilistic sensitivity analysis for patient level simulation models: efficient estimation of mean and variance using ANOVA. Health Econ 2007; 16 (10): 1009–1023
Stevenson M, Oakley J, Chilcott J. Gaussian process modelling in conjunction with individual patient simulation modelling: a case study describing the calculation of cost-effectiveness ratios for the treatment of established osteoporosis. Med Dec Making 2004: 24: 89–100
Ades AE, Lu G, Claxton K. Expected value of sample information in medical decision modelling. Med Dec Making 2004; 24: 207–227
Van Hout B A, Al MJ, Gordon GS, et al. Costs, effects and c/e-ratios alongside a clinical trial. Health Econ 1994; 3: 309–319
Fenwick E, Claxton K, Sculpher MJ. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Econ 2001; 10: 779–789
Fenwick E, O’Brien B, Briggs AH. Cost-effectiveness acceptability curves: facts, fallacies and frequently asked questions. Health Econ 2004; 13: 405–415
Colbourn T, Asseburg C, Bojke L, et al. Preventive strategies for group B streptococcal and other bacterial infections in early infancy: cost effectiveness and value of information analyses. BMJ 2007; 335: 655–662
Pratt J, Raiffa H, Schlaifer R. Statistical decision theory. Cambridge (MA): MIT Press, 1995
Yokota F, Thompson KM. Value of information literature analysis: a review of applications in health risk management. Med Dec Making 2004; 24: 287–298
Poynard T, Munteanu M, Ratziu V, et al. Truth survival in clinical research: an evidence-based requiem? Ann Intern Med 2002; 136: 888–895
Philips Z, Claxton K, Palmer P. The half-life of truth: what are appropriate time horizons for research decisions? Med Dec Making 2008; 28: 287–299
Brennan A, Kharroubi S, O’Hagan A, et al. Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Med Dec Making 2007; 27: 448–470
Felli JC, Hazen GB. Sensitivity analysis and the expected value of perfect information. Med Dec Making 1998; 18: 95–109
Griffin S, Claxton K. Interpreting the expected value of perfect information about parameters. Med Dec Making 2007; 27 (1): E8
Brennan A, Kharroubi S. Expected value of sample information for Weibull survival data. Health Econ 2007: 16: 1205–1225
Conti S, Claxton K. Dimensions of design space: a decision theoretic approach to optimal research design [CHE research paper 38]. York: Centre for Health Economics, University of York, 2008
Acknowledgements
This paper was initially prepared as a briefing paper for NICE as part of the process of updating the Institute’s 2004 Guide to the Methods of Technology Appraisal. The work was funded by NICE through its Decision Support Unit (DSU), which is based at the universities of Sheffield, Leicester, York, Leeds and at the London School of Hygiene and Tropical Medicine.
The author has no conflicts of interest that are directly related to the contents of this article.
The author thanks members of the DSU who commented on the briefing document that forms the basis of this paper as well as Iain Chalmers, Alex Sutton, Alan Brennan, Louise Longworth and Carole Longson, who provided helpful comments on earlier drafts of this paper. All errors and omissions are the responsibility of the author.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Claxton, K. Exploring Uncertainty in Cost-Effectiveness Analysis. Pharmacoeconomics 26, 781–798 (2008). https://doi.org/10.2165/00019053-200826090-00008
Published:
Issue Date:
DOI: https://doi.org/10.2165/00019053-200826090-00008