Effect of Written and Computerized Decision Support Aids for the U.S. Agency for Health Care Policy and Research Depression Guidelines on the Evaluation of Hypothetical Clinical Scenarios

Abstract

Objective. The objective of this study was to compare the effects of written and computerized decision support aids (DSAs) based on U.S. Agency for Health Care Policy and Research depression guidelines. Methods. Fifty-six internal medicine residents were randomized to evaluate clinical scenarios using either a written or a computerized DSA after first assessing scenarios without a DSA. The paired difference between aided and unaided scores was determined for diagnostic accuracy, treatment selection, severity and subtype classification, antipsychotic use, and mental health consultations. Results. Diagnostic accuracy with the written DSA increased from 64% to 73%, and with the computerized DSA decreased from 67% to 64% (P = 0.0065). Residents using the computerized DSA (vs. no DSA) requested fewer consultations (65% vs. 52%, P = 0.028). In post hoc analysis, the written DSA increased sensitivity (66% to 89%, P < 0.001) and the computerized DSA improved specificity (66% to 86%, P = 0.0020) but reduced sensitivity (67% to 49%, P = 0.011). Conclusions. A written DSA improved diagnostic accuracy, whereas a computerized DSA did not. However, the computerized DSA improved specificity and reduced mental health consultations.

Keywords

depression practice guidelines U.S. Agency for Health Care Policy and Research guideline implementations computer decision support aids

Get full access to this article

View all access options for this article.

References

Field MJ , Lohr KN , editors. Guidelines for clinical practice: from development to use. Washington, DC: National Academy Press; 1992.

Grimshaw JM , Russell IT . Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342:1317-1322.

Lomas J , Anderson GM , Domnick-Pierre K , Vayda E , Enkin MW , Hanna WJ . Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med. 1989;321:1306-1311.

Lomas J , Enkin MW , Anderson GM , Hanna WJ , Vayda E , Singer J. Opinion leaders vs audit and feedback to implement practice guidelines: delivery after previous cesarean section. JAMA. 1991;265:2202-2207.

Pestotnik SL , Classen DC , Evans S , Burke JP . Implementing antibiotic practice guidelines through computer-assisted decision support: clinical and financial outcomes. Ann Intern Med. 1996;124:884-890.

Rind D , Safran C , Phillips R , et al. Effect of computer-based alerts on the treatment and outcomes of hospitalized patients. Arch Intern Med. 1994;154:1511-1517.

Weingarten SR , Riedinger MS , Conner L , et al. Practice guidelines and reminders to reduce duration of hospital stay for patients with chest pain. Ann Intern Med. 1994;120:257-263.

Cabana MD , Rand CS , Powe NR , et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458-1465.

Halm EA , Atlas SJ , Borowsky LH , et al. Understanding physician adherence with a pneumonia practice guideline: effects of patient, system, and physician factors. Arch Intern Med. 2000;160(1):98-104.

10.

Soumerai SB , Avorn J. Principles of educational outreach (“academic detailing”) to improve clinical decision making. JAMA. 1990; 263:549-556.

11.

Soumerai SB , Salem-Schatz S , Avorn J , Casteris CS , Ross-Degnan D , Popovsky MA . A controlled trial of educational outreach to improve blood transfusion practice [see comments]. JAMA. 1993; 270:961-966.

12.

Soumerai S , McLaughlin T , Gurwitz J , et al. Effect of local medical opinion leaders on quality of care for acute myocardial infarction: a randomized controlled trial. JAMA. 1998;279:1358-1363.

13.

Hunt DL , Haynes RB , Hanna SE , Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998;280: 1339-1346.

14.

McDonald CJ . Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976;295: 1351-1355.

15.

Rosser WW , Hutchison BG , McDowell I , Newell C. Use of reminders to increase compliance with tetanus booster vaccination. Can Med Assoc J. 1992;146:911-917.

16.

McPhee SJ , Bird JA , Jenkins CN , Fordham D. Promoting cancer screening: a randomized, controlled trial of three interventions. Arch Intern Med. 1989;149:1866-1872.

17.

Tierney WM , McDonald CJ , Martin DK , Rogers MP . Computerized display of past test results: effect on outpatient testing. Ann Intern Med. 1987;107:569-574.

18.

Tierney WM , McDonald CJ , Hui SL , Martin DK . Computer predictions of abnormal test results: effects on outpatient testing. JAMA. 1988;259:1194-1198.

19.

Evans RS , Classen DC , Pestonik SL , Lundsgaarde HP , Burke JP . Improving empiric antibiotic selection using computer decision support. Arch Intern Med. 1994;154:878-884.

20.

Depression Guideline Panel . Depression in primary care, volume 1: detection and diagnosis. Rockville, MD: US Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research (AHCPR); 1993. AHCPR Publication 93-0550.

21.

Depression Guideline Panel . Depression in primary care, volume 2: treatment of major depression. Rockville, MD: US Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research (AHCPR); 1993. AHCPR Publication 93-0550.

22.

Depression Guideline Panel . Depression in primary care: detection, diagnosis and treatment. In: Quick reference guide for clinicians. Rockville, MD: US Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research (AHCPR); 1993. Publication 93-0552.

23.

American Psychiatric Association . American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington, DC: American Psychiatric Association; 1994.

24.

American Psychiatric Association . American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders. 3rd ed, rev. Washington, DC: American Psychiatric Association; 1987.

25.

Peabody JW , Luck J , Glassman P , Dresselhaus TR , Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283:1715-1722.

26.

Zimmerman M , Coryell W. The Inventory to Diagnose Depression (IDD): a self-report scale to diagnose major depressive disorder. J Consult Clin Psychol. 1987;55(1):55-59.

27.

Medow MA , Borowsky SJ , Dysken S , Hillson SD , Woods S , Wilt TJ . Internal medical residents’ ability to diagnose and characterize major depression. West J Med. 1999;170(1):35-40.

28.

Rosner B. Fundamental of Biostatistics. 4th ed. Belmont (CA): Wadsworth; 1995.

29.

Tierney WM , Overhage JM , Takesue BY , et al. Computerizing guidelines to improve care and patient outcomes: the example of heart failure. J Am Med Informatics Assoc. 1995;2:316-322.

30.

Zielstorff RD , Barnett GO , Fitzmaurice JB , et al. A decision support system for prevention and treatment of pressure ulcers based on AHCPR guidelines. In: Cimino JJ , editor. Proceedings of the American Medical Informatics Association Annual Fall Symposium (formerly SCAMC); 1996 Oct; Washington, DC. Philadelphia (PA): Hanley & Belfus;1996:562-566.

31.

Bowers J , Jorm AF , Henderson S , Harris P. General practitioners’ reported knowledge about depression and dementia in elderly patients. Aust N Z J Psychiatry. 1992;26:168-174.

32.

Rapp SR , Davis KM . Geriatric depression: physicians’ knowledge, perceptions, and diagnostic practices. The Gerontologist. 1989;29:252-257.

33.

Barrett JE , Thomas E , Barrett JAO , Gerber PD . The prevalence of psychiatric disorders in a primary care practice. Arch General Psychiatry. 1988;45:1100-1106.

34.

Coulehan JL , Schulberg HC , Block MR , Janosky JE , Areña VC . Medical comorbidity of major depressive disorder in primary medical practice. Arch Intern Med. 1990;150:2363-2367.

35.

Coyne JC , Fechner-Bates S , Schwenk TL . Prevalence, nature, and comorbidity of depressive disorders in primary care. General Hosp Psychiatry. 1994;16:267-276.

36.

Hoeper EW , Nycz GR , Cleary PD , Regier DA , Goldberg ID . Estimated prevalence of RDC mental disorder in primary care. Int J Mental Health. 1979;8(2):6-15.

37.

Kessler LG , Cleary PD , Burke JD JR. Psychiatric disorders in primary care: results of a follow-up study. Arch General Psychiatry. 1985;42:583-587.

38.

Pérez-Stable EJ , Miranda J , Muñoz RF , Ying Y-W. Depression in medical outpatients: underrecognition and misdiagnosis. Arch Intern Med. 1990;150:1083-1088.

39.

Schulberg HC , Madonia MJ , Block MR , et al. Major depression in primary care practice: clinical characteristics and treatment implications. Psychosomatics. 1995;36:129-137.

40.

Von Korff M , Shapiro S , Burke JD , et al. Anxiety and depression in a primary care clinic: comparison of diagnostic interview schedule, general health questionnaire, and practitioner assessments. Arch General Psychiatry. 1987;44:152-156.

41.

Zung WWK , Broadhead E , Roth E. Prevalence of depressive symptoms in primary care. J Fam Pract. 1993;37:377-444.