Background. Artificial neural networks (ANN) have been used in the prediction of several medical conditions but have not been previously used to predict pneumonia. The authors used ANN to predict the presence or absence of pneumonia among patients presenting to the emergency department with acute respiratory complaints and compared the results with those obtained using logistic regression modeling. Methods. Feed-forward back-propagation ANN were trained on sociodemographic, symptom, sign, comorbidity, and radiographic outcome data among 1044 patients from the University of Illinois (the training cohort) and were applied to 116 patients from the University of Nebraska (the testing cohort). ANN trained using different strategies were compared to each other and to main-effects logistic regression. Calibration accuracy was measured as mean square error and discrimination accuracy as the area under a receiver operating characteristic (ROC) curve. Results. A 1 hidden-layer ANN trained using oversampling of pneumonia cases had an ROC area in the training cohort of 0.895, which was greater than the area of 0.840 for logistic regression (P = 0.026). This ANN had an ROC area in the testing cohort of 0.872, not significantly different from its area in the training cohort (P = 0.597). Operating at a threshold of 0.25, the ANN would have detected 94% to 95% of patients with pneumonia in the 2 cohorts while correctly excluding 39% to 50% of patients with other conditions. ANN trained using other strategies discriminated equally in the 2 cohorts but no better than did logistic regression. Conclusions. Among adults presenting with acute respiratory illness, ANN accurately discriminated patients with and without pneumonia and, under some circumstances, improved on the accuracy of logistic regression.
Get full access to this article
View all access options for this article.
References
1.
1. National Center for Health Statistics. Current Estimates from the National Health Interview Survey, United States, 1987. DHHS Pub. No. (PHS) 88-1594. Hyattsville (MD): US Department of Health and Human Services ; 1981.
2.
2. Metlay JP, Kapoor WN, Fine MJ. Does this patient have community-acquired pneumonia? Diagnosing pneumonia by history and physical examination . JAMA. 1997; 278: 1440–5 .
3.
3. Singal BM, Hedges JR, Radack KL. Decision rules and clinical prediction of pneumonia . Ann Emerg Med. 1989; 18: 13–20 .
4.
4. Heckerling PS, Tape TG, Wigton RS, et al. Clinical prediction rule for pulmonary infiltrates . Ann Intern Med. 1990; 113: 664–70 .
5.
5. Dawson NV, Speroff T. Validated decision aid for outpatient pneumonia (abstract). Clin Res. 1989; 37: 774A .
6.
6. Diehr PH, Wood RW, Bushyhead JR, Krueger L, Wolcott BW, Tompkins RK. Prediction of pneumonia in outpatients with acute cough . J Chron Dis. 1984; 37: 215–25 .
7.
7. Penny W, Frost D. Neural networks in clinical medicine . Med Decis Making. 1996; 16: 386–98 .
8.
8. Lette J, Colletti BW, Cerino M, et al. Artificial intelligence versus logistic regression statistical modeling to predict cardiac complications after non-cardiac surgery . Clin Cardiol. 1994; 17: 609–14 .
9.
9. Patil S, Henry JW, Rubenfire M, Stein PD. Neural network in the clinical diagnosis of acute pulmonary embolism . Chest. 1993; 104: 1685–9 .
10.
10. Baxt WG. Use of an artificial neural network for the diagnosis of myocardial infarction . Ann Intern Med. 1991; 115: 843–8 .
12. El-Solh AA, Hsiao CB, Goodnough S, Serghani J, Grant BJ. Predicting active pulmonary tuberculosis using an artificial neural network . Chest. 1999; 116: 968–73 .
13.
13. Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. In: DE Rumelhart, JL McClelland, editors. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge (MA): MIT Press ; 1986. p. 318–64.
14.
14. McClelland JL, Rumelhart DE. Training hidden units. In: JL McClelland, DE Rumelhart, editors. Explorations in Parallel Distributed Processing. Cambridge (MA): MIT Press ; 1988. p. 121–60.
15.
15. Faussett L. Fundamentals of Neural Networks: Architectures, Algorithms, and Applications. Englewood Cliffs (NJ): Prentice Hall ; 1994.
16.
16. Baxt WG, White H. Bootstrapping confidence intervals for clinical input variable effects in a network trained to identify the presence of acute myocardial infarction . Neural Comput. 1995; 7: 624–38 .
18. Wolfram S. Mathematica: A System for Doing Mathematics by Computer. 2nd ed. Reading (MA): Addison-Wesley ; 1991.
19.
19. Heckerling PS, Gerber BS. Feed-forward, back-propagation neural network modeling using Mathematica. Comput Meth Prog Biomed; submitted for publication.
20.
20. Hinton GE. Connectionist learning machines. In: JG Carbonell, editor. Machine Learning: Paradigms and Methods. Cambridge (MA): MIT Press ; 1990. p. 185–234.
21.
21. Richard MD, Lippman RP. Neural network classifiers estimate Bayesian a posteriori probabilities . Neural Comput. 1991; 3: 461–83 .
22.
22. Dorfman DD, Alf E Jr. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals: rating-method data . J Math Psych. 1969; 6: 487–96 .
24. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases . Radiology. 1983; 148: 839–43 .
25.
25. Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. New York: John Wiley ; 1981.
26.
26. Efron B, Tibshirani RJ. An Introduction to the Bootstrap. New York: Chapman & Hall ; 1993.
27.
27. Hosmer DW, Lemeshow S. Applied Logistic Regression. New York: John Wiley ; 1989.
28.
28. Engleman L. Stepwise logistic regression. In: WJ Dixon, MB Brown, L Engleman, JW Frane, MA Hill, RI Jennrich, editors. BMDP Statistical Software. Berkeley: University of California Press ; 1985. p. 330–44.
29.
29. Hernandez CA, Espi J, Nakayama K. How to choose the training data for neural network medical diagnosis systems . Biomed Sci Instrum. 1993; 29: 283–90 .
30.
30. Astion ML, Wener MH, Thomas G, Hunder GG, Bloch DA. Application of neural networks to the classification of giant cell arteritis . Arth Rheum. 1994; 37: 760–70 .
31.
31. Gottschalk A, Hyzer C, Geer RT. A comparison of human and machine-based predictions of successful weaning from mechanical ventilation . Med Decis Making. 2000; 20: 160–9 .
33. Tu JV. Advantages and disadvantages of using artificial neural networks versus logistic regression in predicting medical outcomes . J Clin Epidemiol. 1996; 49: 1225–31 .
34.
34. Baxt WG. A neural network trained to identify the presence of myocardial infraction bases diagnostic decision on nonlinear relationships between input variables . Neural Comput Applic. 1993; 1: 176–82 .
35.
35. Cohen IL, Sudhalter V. A neural network approach to the classification of autism . J Autism Devel Disord. 1993; 23: 443–66 .
36.
36. Astion ML, Wilding P. The application of backpropagation neural networks to problems in pathology and laboratory medicine . Arch Pathol Lab Med. 1992; 116: 995–1001 .
37.
37. Somoza E, Somoza JR. A neural-network approach to predicting admission decisions in a psychiatric emergency room . Med Decis Making. 1993; 13: 273–80 .
38.
38. Baxt WG. Analysis of the clinical variables driving decision in an artificial neural network trained to identify the presence of myocardial infarction . Ann Emerg Med. 1992; 21: 1439–44 .
40. Dybowski R, Weller P, Chang R, Gant V. Prediction of outcome in critically ill patients using artificial neural network synthesised by genetic algorithm . Lancet. 1997; 347: 1146–50 .
41.
41. Little RJA, Rubin DB. Statistical Analysis with Missing Data. New York: John Wiley ; 1987.
42.
42. Brunswik E. Perception and the Representative Design of Psychological Experiments. Berkeley: University of California Press ; 1956.
43.
43. Hammond KB, Hursch CJ, Todd FJ. Analyzing the components of clinical inference . Psychol Rev. 1964; 71: 528–30 .
44.
44. Tape TG, Heckerling PS, Ornato JP, Wigton RS. Use of clinical judgment analysis to explain regional variations in physicians’ accuracies in diagnosing pneumonia . Med Decis Making. 1991; 11: 189–97 .
