Parameter uncertainty, patient heterogeneity, and stochastic uncertainty of outcomes are increasingly important concepts in medical decision models. The purpose of this study is to demonstrate the various methods to analyze uncertainty and patient heterogeneity in a decision model. The authors distinguish various purposes of medical decision modeling, serving various stakeholders. Differences and analogies between the analyses are pointed out, as well as practical issues. The analyses are demonstrated with an example comparing imaging tests for patients with chest pain. For complicated analyses step-by-step algorithms are provided. The focus is on Monte Carlo simulation and value of information analysis. Increasing model complexity is a major challenge for probabilistic sensitivity analysis and value of information analysis. The authors discuss nested analyses that are required in patient-level models, and in nonlinear models for analyses of partial value of information analysis.
Dillon A., Barnett D., Longson C.Guide to the Methods of Technology Appraisal. London: National Institute for Health and Clinical Excellence (NICE); 2004 .
2.
Claxton K., Sculpher M., Drummond M.A rational framework for decision making by the National Institute for Clinical Excellence (NICE). Lancet. 2002 ;360:711-5.
3.
Draper D.Assessment and propagation of model uncertainty. J R Stat Soc B. 1995;57:45-97.
4.
Briggs A.Probabilistic analysis of cost-effectiveness models: statistical representation of parameter uncertainty. Value in Health. 2005;8:1-2.
5.
Doubilet P., Begg CB, Weinstein MC, et al. Probabilistic sensitivity analysis using Monte Carlo simulation. A practical approach. Med Decis Making. 1985 ;5:157-77.
6.
Critchfield GC, Willard KE, Connelly DPProbabilistic sensitivity analysis methods for general decision models. Comput Biomed Res. 1986;19:254-65.
7.
Briggs AH, Goeree R., Blackhouse G., et al. Probabilistic analysis of cost-effectiveness models: choosing between treatment strategies for gastroesophageal reflux disease. Med Decis Making. 2002;22:290-308.
8.
Felli JC, Hazen GBSensitivity analysis and the expected value of perfect information . Med Decis Making. 1998;18:95-109.
9.
Brennan A., Kharroubi S., O’Hagan A., et al. Calculating partial expected value of perfect information via Monte Carlo sampling algorithms. Med Decis Making. 2007;27:448-70.
10.
Sonnenberg FA, Beck JRMarkov models in medical decision making: a practical guide. Med Decis Making. 1993;13:322-38.
11.
Hunink MGM , Glasziou P.Decision Making in Health and Medicine. United KingdomCambridge University Press; 2001.
12.
Heart Disease and Stroke Statistics-2007 Update. 2007. Available from: http://www.americanheart.org/downloadable/heart/1166712318459HS_StatsInsideText.pdf. Accessed on 29 July 2009 .
13.
Chaitman BR , Bourassa MG, Davis K., et al. Angiographic prevalence of high-risk coronary artery disease in patient subsets (CASS) . Circulation . 1981;64:360-7.
14.
Kuntz KM, Fleischmann KE, Hunink MG, et al. Cost-effectiveness of diagnostic strategies for patients with chest pain. Ann Intern Med. 1999;130:709-18.
15.
Heijenbrok-Kal MHAssessment of Diagnostic Imaging Technologies for Cardiovascular Disease. Thesis. 2004. Erasmus Medical Center, Rotterdam, the Netherlands.
16.
Arias E.Center for Disease Control and Prevention (CDC). Available from: http://www.cdc.gov/nchs/data/nvsr/nvsr54/nvsr54_14.pdf . Accessed 29 July 2009.
17.
Stinnett AA , Paltiel ADEstimating CE ratios under second-order uncertainty: the mean ratio versus the ratio of means. Med Decis Making. 1997 ;17:483-9.
18.
Coggon DI, Martyn CNTime and chance: the stochastic nature of disease causation. Lancet. 2005;365:1434-7.
19.
Griffin S., Claxton K., Hawkins N., et al. Probabilistic analysis and computationally expensive models: necessary and required? Value Health. 2006 ;9:244-52.
20.
Cronin KA, Legler JM, Etzioni RDAssessing uncertainty in microsimulation modelling with application to cancer screening interventions. Stat Med. 1998;17:2509-23.
21.
Stevenson MD , Oakley J., Chilcott JBGaussian process modeling in conjunction with individual patient simulation modeling: a case study describing the calculation of cost-effectiveness ratios for the treatment of established osteoporosis . Med Decis Making. 2004;24:89-100.
22.
Briggs AH, Ades AE, Price MJProbabilistic sensitivity analysis for decision trees with multiple branches: use of the Dirichlet distribution in a Bayesian framework. Med Decis Making. 2003;23:341-50.
23.
Stinnett AA , Mullahy J.Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998;18(2 Suppl.):S68-80.
24.
Efron B. Tr.An Introduction to the Bootstrap. New York: Chap-man & Hall; 1993 .
25.
Briggs AH, Wonderling DE, Mooney CZPulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation. Health Econ. 1997;6:327-40.
26.
O’Hagan A., Stevens JWAssessing and comparing costs: how robust are the bootstrap and methods based on asymptotic normality?Health Econ. 2003;12:33-49.
27.
Greenland S.Sensitivity analysis, Monte Carlo risk analysis, and Bayesian uncertainty assessment. Risk Anal. 2001;21:579-83.
28.
Ades AE, Cliffe S.Markov chain Monte Carlo estimation of a multiparameter decision model: consistency of evidence and the accurate assessment of uncertainty. Med Decis Making. 2002;22: 359-71.
29.
Ades AE, Sculpher M., Sutton A., et al. Bayesian methods for evidence synthesis in cost-effectiveness analysis. Pharmacoeconomics. 2006;24:1-19.
30.
Spiegelhalter DJ, Thomas A., Best N., et al. WinBUGS User Manual: Version 1.4. Cambridge (UK): MRC Biostatistics; 2001.
31.
Fenwick E., Claxton K., Sculpher M.Representing uncertainty: the role of cost-effectiveness acceptability curves . Health Econ. 2001;10:779-87.
32.
van Hout BA , Al MJ, Gordon GS, et al. Costs, effects and C/Eratios alongside a clinical trial. Health Econ. 1994;3:309-19.
33.
Lothgren M. , Zethraeus N.Definition, interpretation and calculation of cost-effectiveness acceptability curves. Health Econ. 2000;9:623-30.
34.
Fenwick E., O’Brien BJ, Briggs A.Cost-effectiveness acceptability curves-facts, fallacies and frequently asked questions. Health Econ. 2004;13:405-15.
35.
Groot Koerkamp B., Hunink MG, Stijnen T., et al. Limitations of acceptability curves for presenting uncertainty in cost-effectiveness analysis . Med Decis Making. 2007;27:101-11.
36.
Barton GR, Briggs AH, Fenwick EAOptimal Cost-Effectiveness Decisions: The Role of the Cost-Effectiveness Acceptability Curve (CEAC), the Cost-Effectiveness Acceptability Frontier (CEAF), and the Expected Value of Perfection Information (EVPI). Value Health . 2008 May 15.
37.
Claxton K., Sculpher M., McCabe C., et al. Probabilistic sensitivity analysis for NICE technology assessment: not an optional extra. Health Econ. 2005;14:339-47.
38.
Halpern EF, Weinstein MC, Hunink MG, et al. Representing both first- and second-order uncertainties by Monte Carlo simulation for groups of patients. Med Decis Making . 2000;20:314-22.
39.
Grundy PM, Healy MJR, Rees DH. Economic choice of the amount of experimentation. J R Stat Soc Ser A. 1956;18:32-48. 40. Raiffa H, Schlaifer R. Probability and Statistics for Business Decisions. New York: McGraw-Hill; 1959.
Yokota F., Thompson KMValue of information literature analysis: a review of applications in health risk management. Med Decis Making. 2004;24:287-98.
42.
Yokota F., Thompson KMValue of information analysis in environmental health risk management decisions: past, present, and future. Risk Anal. 2004 ;24:635-50.
43.
Groot Koerkamp B., Myriam Hunink MG, Stijnen T., et al. Identifying key parameters in cost-effectiveness analysis using value of information: a comparison of methods. Health Econ. 2006 ;15: 383-92.
44.
Howard RAInformation value theory. IEEE Transact Systems Science and Cybernetics. 1966;SCC-2:22-6.
45.
Thompson KM , Graham JDGoing beyond the single number: using probabilistic risk assessment to improve risk management. Hum Ecol Risk Assess. 1996;2:1008-34.
46.
Ades AE, Lu G., Claxton K.Expected value of sample information calculations in medical decision modeling . Med Decis Making. 2004;24:207-27.
47.
Claxton K., Posnett J.An economic approach to clinical trial design and research priority-setting . Health Econ. 1996;5: 513-24.
48.
Claxton K., Thompson KMA dynamic programming approach to the efficient design of clinical trials . J Health Econ. 2001;20:797-822.
49.
Willan AR, Pinto EMThe value of information and optimal clinical trial design. Stat Med. 2005;24:1791-806.
50.
Campbell H. , Briggs A., Buxton M., et al. The credibility of health economic models for health policy decision-making: the case of population screening for abdominal aortic aneurysm. J Health Serv Res Policy. 2007;12:11-7.
51.
Bojke L., Claxton K., Palmer S., et al. Defining and Characterising Structural Uncertainty in Decision Analytic Models. York: Centre for Health Economics; 2006.
52.
Basu A., Meltzer D.Value of information on preference heterogeneity and individualized care. Med Decis Making. 2007;27: 112-27.
53.
Sculpher M.Subgroups and heterogeneity in cost-effectiveness analysis. Pharmacoeconomics. 2008;26:799-806.
54.
Nijhuis RL, Stijnen T., Peeters A., et al. Apparent and internal validity of a Monte Carlo-Markov model for cardiovascular disease in a cohort follow-up study. Med Decis Making. 2006;26: 134-44.
55.
Spiegelhalter DJ, Best NGBayesian approaches to multiple sources of evidence and uncertainty in complex cost-effectiveness modelling . Stat Med. 2003;22:3687-709.
56.
Pearson SD, Rawlins MDQuality, innovation, and value for money: NICE and the British National Health Service. JAMA. 2005;294:2618-22.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
