A Bayesian Method for Evaluating Medical Test operating Characteristics When Some Patients' Conditions Fail to Be Diagnosed by the Reference Standard

Abstract

The evaluation of a diagnostic test when the reference standard fails to establish a diagnosis in some patients is a common and difficult analytical problem. Conventional operating char acteristics, derived from a 2 x 2 matrix, require that tests have only positive or negative results, and that disease status be designated definitively as present or absent. Results can be displayed in a 2 x 3 matrix, with an additional column for undiagnosed patients, when it is not possible always to ascertain the disease status definitively. The authors approach this problem using a Bayesian method for evaluating the 2 x 3 matrix in which test operating characteristics are described by a joint probability density function. They show that one can derive this joint probability density function of sensitivity and specificity empirically by applying a sampling algorithm. The three-dimensional histogram resulting from this sampling pro cedure approximates the true joint probability density function for sensitivity and specificity. Using a clinical example, the authors illustrate the method and demonstrate that the joint probability density function for sensitivity and specificity can be influenced by assumptions used to interpret test results in undiagnosed patients. This Bayesian method represents a flexible and practical solution to the problem of evaluating test sensitivity and specificity when the study group includes patients whose disease could not be diagnosed by the reference standard. Key words: Bayesian analysis; test operating characteristics; probability density functions. (Med Decis Making 1990;10:102-111)

Get full access to this article

View all access options for this article.

References

Begg CB , Greenes RA Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39:207-15.

Begg CB , Greenes RA , Iglewicz B. The influence of uninterpretability on the assessment of diagnostic tests . J Chronic Dis. 1986;39:575-84.

Begg CB Biases in the assessment of diagnostic tests. Stat Med. 1987;6:411-32.

Greenes RA , Begg CB Assessment of diagnostic technologies: methodology for unbiased estimation from samples of selectively verified patients. Invest Radiol. 1985;20:251-6.

Panzer RJ , Suchman AL , Griner P. Workup bias in prediction research. Med Decis Making. 1987;7:115-9.

Ransohoff DA , Feinstein AR Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299:926-30.

Simel DL , Feussner JR , Delong ER , Matchar DB Intermediate, indeterminate and uninterpretable diagnostic test results . Med Decis Making. 1987;7:107-14.

Zellner A. An introduction to Bayesian inference in econometrics. New York, Wiley, 1971;38-40.

Johnson NL , Kotz S. Distributions in statistics: continuous uni-variate distributions—2 . New York, Houghton Mifflin, 1970.

10.

Hull RD , Hirsh J. , Carter CJ , et al. Pulmonary angiography, ventilation lung scanning, and venography for clinically suspected pulmonary embolism with abnormal perfusion lung scan . Ann Intern Med. 1983;98:891-9.

11.

Doubilet P. , Begg CB , Weinstein MC , Braun P. , McNeil BJ Probabilistic sensitivity analysis using Monte Carlo simulation: a practical approach. Med Decis Making. 1985;5:157-77.

12.

Critchfield GC , Willard KE Probabilistic sensitivity analysis of decision trees using Monte Carlo simulation. Med Decis Making. 1986;6:85-92.