The clinical utility of medical tests is measured by whether the information they provide affects patient-relevant outcomes. To a large extent, effects of medical tests are indirect in nature. In principle, a test result affects patient outcomes mainly by influencing treatment choices. This indirectness in the link between testing and its downstream effects poses practical challenges to comparing alternate test-and-treat strategies in clinical trials. Keeping in mind the broader audience of researchers who perform comparative effectiveness reviews and technology assessments, the authors summarize the rationale for and pitfalls of decision modeling in the comparative evaluation of medical tests by virtue of specific examples. Modeling facilitates the interpretation of test performance measures by connecting the link between testing and patient outcomes, accounting for uncertainties and explicating assumptions, and allowing the systematic study of tradeoffs and uncertainty. The authors discuss challenges encountered when modeling test-and-treat strategies, including but not limited to scarcity of data on important parameters, transferring estimates of test performance across studies, choosing modeling outcomes, and obtaining summary estimates for test performance data.
Bossuyt PM, Lijmer JG, Mol BWRandomised comparisons of medical tests: sometimes invalid, not always efficient . Lancet. 2000;356:1844-7.
3.
Lord SJ, Irwig L., Simes RJWhen is measuring sensitivity and specificity sufficient to evaluate a diagnostic test, and when do we need randomized trials?Ann Intern Med . 2006;144:850-5.
4.
Lijmer JG, Leeflang M., Bossuyt PMM.Proposals for a phased evaluation of medical tests. Med Decis Making.2009;29:E13-E21.
5.
Lord SJ, Irwig L., Bossuyt PMM.Using the principles of randomized controlled trial design to guide test evaluation . Med Decis Making.2009;29:E1-E12.
6.
Siebert U.When should decision analytic modeling be used in the economic evaluation of health care? Eur J Health Econ. 2003; 4:143-50.
7.
Tatsioni A., Zarin DA, Aronson N., et al. Challenges in systematic reviews of diagnostic technologies . Ann Intern Med. 2005; 142:1048-55.
8.
Hunink M., Glasziou P., Siegel J., Pliskin J., Elstein A., Weinstein M.Decision Making in Health and Medicine: Integrating Evidence and Values . Cambridge, UK: Cambridge University Press;2001.
9.
Sox H., Blatt M., Higgins M., Marton K.Medical Decision Making. Philadelphia, PA: American College of Physicians; 2007.
10.
Mushlin AIChallenges and opportunities in economic evaluations of diagnostic tests and procedures. Acad Radiol. 1999; 6(suppl 1):S128-31.
11.
Davies MJ, Tringham JR, Troughton J., Khunti KKPrevention of type 2 diabetes mellitus: a review of the evidence and its application in a UK setting. Diabet Med. 2004;21:403-14.
12.
Waugh N., Scotland G., McNamee P., et al. Screening for type 2 diabetes: literature review and economic modelling. Health Technol Assess. 2007;11:iii-xi, 1.
13.
Gillies CL, Lambert PC, Abrams KR, et al. Different strategies for screening and prevention of type 2 diabetes in adults: cost effectiveness analysis. BMJ. 2008;336(7654):1180-5.
Cooper NJ, Sutton AJ, Abrams KR, Turner D., Wailoo A.Comprehensive decision analytical modelling in economic evaluation: a Bayesian approach. Health Econ. 2004;13:203-26.
16.
Sutton AJ, Cooper NJ, Goodacre S., Stevenson M.Integration of meta-analysis and economic decision modeling for evaluating diagnostic tests. Med Decis Making. 2008;28(5):650-7.
17.
Habbema JD, van Oortmarssen GJ, Lubbe JT, van der Maas PJThe MISCAN simulation program for the evaluation of screening for disease . Comput Methods Programs Biomed. 1985;20:79-93.
18.
Barza M., Pauker SGThe decision to biopsy, treat, or wait in suspected herpes encephalitis . Ann Intern Med. 1980;92:641-9.
19.
Braun P.The clinical management of suspected herpes virus encephalitis: a decision-analytic view. Am J Med. 1980;69:895-902.
20.
Lau J., Ioannidis JP, Balk EM, et al. Evaluation of Technologies for Identifying Acute Cardiac Ischemia in Emergency Departments. Evidence Report/Technology Assessment No26. Rockville, MD: Agency for Healthcare Research and Quality; 2001.
21.
Loeve F., Boer R., van Oortmarssen GJ, van BM, Habbema JDThe MISCAN-COLON simulation model for the evaluation of colorectal cancer screening. Comput Biomed Res.1999;32:13-33.
22.
Zauber AG, Lansdorp-Vogelaar I., Knudsen AB, Wilschut J., van BM, Kuntz KMEvaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force. Ann Intern Med . 2008;149:659-69.
23.
Screening for colorectal cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2008;149: 627-37.
24.
Duke Center for Clinical Health Policy Research and Evidence Practice Center.Positron Emission Tomography, Single Photon Emission Computed Tomography, Functional Magnetic Resonance Imaging, and Magnetic Resonance Spectroscopy for the Diagnosis and Management of Alzheimers Dementia. Evidence Report/Technology Assessment No 26. Rockville, MD: Agency for Healthcare Research and Quality ; 2001.
25.
Decision analytic modelling in the economic evaluation of health technologies. A consensus statement. Consensus Conference on Guidelines on Economic Modelling in Health Technology Assessment. Pharmacoeconomics . 2000;17:443-4.
26.
Richardson WS, Detsky AS; Evidence Based Medicine Working Group.Users’ guides to the medical literature. VII. How to use a clinical decision analysis. B. What are the results and will they help me in caring for my patients?JAMA. 1995;273:1610-3.
27.
Richardson WS, Detsky AS; Evidence Based Medicine Working Group.Users’ guides to the medical literature. VII. How to use a clinical decision analysis. A. Are the results of the study valid?JAMA. 1995;273:1292-5.
28.
Philips Z., Ginnelly L., Sculpher M., et al. Review of guidelines for good practice in decision-analytic modelling in health technology assessment. Health Technol Assess.2004;8:iii-xi, 1.
29.
Philips Z., Bojke L., Sculpher M., Claxton K., Golder S.Good practice guidelines for decision-analytic modelling in health technology assessment: a review and consolidation of quality assessment. Pharmacoeconomics. 2006;24:355-71.
30.
Sculpher M. , Fenwick E., Claxton K.Assessing quality in decision analytic cost-effectiveness models: a suggested framework and example of application. Pharmacoeconomics. 2000;17:461-77.
31.
Weinstein MC , O’Brien B., Hornberger J., et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices- Modeling Studies . Value Health . 2003;6:9-17.
32.
Meyer KB, Pauker SGScreening for HIV: can we afford the false positive rate? N Engl J Med. 1987;317:238-41.
33.
Trikalinos TA , Ip S., Raman G., et al. Home Diagnosis of Obstructive Sleep Apnea-Hypopnea Syndrome. Evidence Report/ Technology Assessment. Rockville, MD: Agency for Healthcare Research and Quality ; 2007.
34.
Trikalinos TA , Lau J.Obstructive Sleep Apnea-Hypopnea Syndrome: Modeling Different Diagnostic Strategies. Evidence Report/Technology Assessment . Rockville, MD: Agency for Healthcare Research and Quality; 2007.
35.
Bossuyt PM, Reitsma JB, Bruns DE, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med. 2003;138:W1-12.
36.
Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD Initiative. Ann Intern Med. 2003;138:40-4.
37.
Lijmer JG, Mol BW, Heisterkamp S., et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999; 282:1061-6.
38.
Rutjes AW, Reitsma JB, Di NM, Smidt N., van Rijn JC, Bossuyt PMEvidence of bias and variation in diagnostic accuracy studies. CMAJ. 2006;174:469-76.
39.
Whiting P., Rutjes AW, Reitsma JB, Glas AS, Bossuyt PM, Kleijnen J.Sources of variation and bias in studies of diagnostic accuracy: a systematic review. Ann Intern Med. 2004;140:189-202.
40.
Hawkins DM, Garrett JA, Stephenson B.Some issues in resolution of diagnostic tests using an imperfect gold standard . Stat Med. 2001;20:1987-2001.
41.
Joseph L., Gyorkos TWInferences for likelihood ratios in the absence of a ‘‘gold standard .’’ Med Decis Making. 1996;16:412-7.
42.
Walter SD, Irwig L., Glasziou PPMeta-analysis of diagnostic tests with imperfect reference standards . J Clin Epidemiol. 1999; 52:943-51.
43.
Dendukuri N. , Joseph L.Bayesian approaches to modeling the conditional dependence between multiple diagnostic tests. Biometrics. 2001;57:158-67.
44.
Irwig L., Bossuyt P., Glasziou P., Gatsonis C., Lijmer J.Designing studies to ensure that estimates of test accuracy are transferable . BMJ. 2002;324:669-71.
45.
Loong TWUnderstanding sensitivity and specificity with the right side of the brain . BMJ. 2003;327:716-9.
46.
Goehring C. , Perrier A., Morabia A.Spectrum bias: a quantitative and graphical analysis of the variability of medical diagnostic test performance. Stat Med. 2004;23:125-35.
47.
Mulherin SA , Miller WCSpectrum bias or spectrum effect? Subgroup variation in diagnostic test evaluation . Ann Intern Med . 2002;137:598-602.
48.
Leeflang MMG , Bossuyt PM, Irwig L.Diagnostic accuracy may vary with prevalence: implications for evidence-based diagnosis. J Clin Epidemiol.2009;62(1):5-12
49.
Lejeune C., Al ZK, Woronoff-Lemsi MC, et al. Use of a decision analysis model to assess the medicoeconomic implications of FDG PET imaging in diagnosing a solitary pulmonary nodule . Eur J Health Econ. 2005;6:203-14.
50.
Cantor T., Yang Z., Caraiani N., Ilamathi E.Lack of comparability of intact parathyroid hormone measurements among commercial assays for end-stage renal disease patients: implication for treatment decisions . Clin Chem. 2006;52:1771-6.
51.
Foley RN, Li S., Liu J., Gilbertson DT, Chen SC, Collins AJThe fall and rise of parathyroidectomy in U.S. hemodialysis patients, 1992 to 2002. J Am Soc Nephrol.2005;16:210-8.
52.
Dukic V., Gatsonis C.Meta-analysis of diagnostic test accuracy assessment studies with varying number of thresholds. Biometrics. 2003;59:936-46.
53.
Littenberg B., Moses LEEstimating diagnostic accuracy from multiple conflicting reports: a new meta-analytic method. Med Decis Making. 1993;13:313-21.
54.
Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AHBivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58:982-90.
55.
Rutter CM, Gatsonis CARegression methods for meta-analysis of diagnostic test data. Acad Radiol. 1995;2(suppl 1): S48-S56.
56.
Harbord RM, Deeks JJ, Egger M., Whiting P., Sterne JAA unification of models for meta-analysis of diagnostic accuracy studies . Biostatistics. 2007;8:239-51.
57.
Karnon J., Goyder E., Tappenden P., et al. A review and critique of modelling in prioritising and designing screening programmes. Health Technol Assess. 2007;11:iii-xi, 1.
58.
National Cancer Institute, Cancer Intervention and Surveillance Modeling Network. Available from: URL: http://cisnetcancergov/2008
59.
Loeve F., Brown ML, Boer R., van BM, van Oortmarssen GJ, Habbema JDEndoscopic colorectal cancer screening: a cost-saving analysis. J Natl Cancer Inst. 2000;92:557-63.
