Sage Journals: Discover world-class research

Abstract

The author describes a new methodology to solve medical decision problems involving a choice between alternatives under conditions of risk and uncertainty when imperfect empirical information from diagnostic technologies is available. The main new concept is the gener alized ROC (GROC) curve, which extends considerably the scope of analysis of the traditional ROC curve as well as the threshold approach to medical decision making. Both techniques become special cases of the new general approach. The author shows how to apply the technique and derive comprehensive clinical guidelines to solve the medical decision problem involving one patient in the most general situation. The larger problem of evaluating the performances of diagnostic technologies for a population of patients with varying prior prob abilities of disease is then addressed. A new measure of performance is proposed that goes beyond the well-accepted area under the ROC curve index. This measure is used to compare different technologies for a population of patients, and a new methodology is proposed to carry out such comparisons. Key words: technology assessment; decision theory; diagnostic information; ROC analysis. (Med Decis Making 1991;11:208-220)

Get full access to this article

View all access options for this article.

References

Phelps CE , Mushlin AI Focusing technology assessment using medical decision theory. Med Decis Making. 1988;8:279-289.

Barnett V. Comparative statistical inference. Second edition. New York: John Wiley and Sons, 1982.

Metz CE Basic principles of ROC analysis. Semin Nucl Med. 1978;8:283-98.

Pauker SG , Kassirer JP The threshold approach to clinical decision making. N Engl J Med. 1980;302:1109-17.

Hershey JC , Cebul RD , Williams SV Clinical guidelines for using two dichotomous tests. Med Decis Making. 1986;6:68-78.

Egan JP Signal detection theory and ROC analysis. New York: Academic Press, 1975.

Hanley JA The robustness of the "binormal" assumptions used in fitting ROC curves. Med Decis Making. 1988;8:197-203.

Metz CE ROC methodology in radiologic imaging. Invest Radiol. 1986;21:720-33.

Swets JA Measuring the accuracy of diagnostic systems. Science. 1988;240:1285-93.

10.

Centor RM , Hillner BE , Meier FA The effect of prior probabilities on sensitivity and specificity. Abstract presented at Annual Meeting of the Society for Research and Education in Primary Care and Internal Medicine (SREPCIM), April 30-May 1, 1986, San Diego, California.

11.

Hlatky MA , Pyron DB , Harrell FE , et al. Factors affecting sensitivity and specificity of exercise electrocardiography . Am J Med. 1984;77:64-71.

12.

Weiner DA , Ryan TJ , McCabe CH , et al. Correlations among history of angina, ST-segment response and prevalence of coronary-artery disease in the coronary artery surgery study (CASS). N Engl J Med. 1979;301:230-5.

13.

Fryback DG , Glynn PW , Sainfort F. A difficulty in the computation of expected utilities in medical decision making. Med Decis Making (in press).

14.

Swets JA Indices of discrimination or diagnostic accuracy: their ROCs and implied models . Psychol Bull. 1986;99:100-17.

15.

Simpson AJ , Fitter MJ What is the best index of detectability. Psychol Bull. 1973;80:481-8.

16.

Green DM , Swets JA Signal detection theory and psychophysics. New York: John Wiley and Sons, 1966.

17.

Hanley JA , McNeil BJ The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29-36.

18.

Bamber D. The area above the ordinal dominance graph and the area below the receiver operating graph. J Math Psych. 1975;12:387-415.

19.

Jaynes ET Prior probabilities. IEEE Trans Systems Sci Cybernetics. 1968;SSC-4:227-41.

20.

Jaynes ET Papers on probability, statistics, and statistical physics. In Rosenkrantz RD , ed. A reprint collection. Dordrecht, Holland: Reidel, 1983 .

21.

Rosenkrantz R. Inference, method, and decision: towards a Bayesian philosophy of science . Boston: Reidel, 1977.

22.

McClish DK Analyzing a portion of the ROC curve. Med Decis Making. 1989;9:190-5.

23.

Swets JA , Pickett RM Evaluation of diagnostic systems. New York: Academic Press, 1982.

24.

Ransohoff DF , Feinstein AR Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299:926-30.

25.

Begg CB , Greenes RA Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39:201-15.

26.

Gray R. , Begg CB , Greenes RA Construction of receiver operating characteristic curves when disease verification is subject to selection bias. Med Decis Making. 1984;4:151-64.

27.

Diamond GA , Rozanski A. , Forrester JS , et al. A model for assessing the sensitivity and specificity of tests subject to selection bias. J Chronic Dis. 1986;39:343-55.

28.

Hlatky MA , Mark DB , Harrell FE , et al. Rethinking sensitivity and specificity. Am J Cardiol. 1987;59:1195-8.

29.

Dolan JG , Bordley DR , Mushlin Al. An evaluation of clinicians' subjective prior probability estimates . Med Decis Making. 1986;6:216-23.

Eualuation of Medical Technologies

Abstract

Get full access to this article

References