Sage Journals: Discover world-class research

Abstract

Background: Clinical decision support (CDS) for primary care has been shown to improve delivery of preventive services. However, there is little evidence for efficiency of physicians due to CDS assistance. In this article, we report a pilot study for measuring the impact of CDS on the time spent by physicians for deciding on preventive services and chronic disease management. Methods: We randomly selected 30 patients from a primary care practice, and assigned them to 10 physicians. The physicians were requested to perform chart review to decide on preventive services and chronic disease management for the assigned patients. The patients assignment was done in a randomized crossover design, such that each patient received 2 sets of recommendations—one from a physician with CDS assistance and the other from a different physician without CDS assistance. We compared the physician recommendations made using CDS assistance, with the recommendations made without CDS assistance. Results: The physicians required an average of 1 minute 44 seconds, when they were they had access to the decision support system and 5 minutes when they were unassisted. Hence the CDS assistance resulted in an estimated saving of 3 minutes 16 seconds (65%) of the physicians’ time, which was statistically significant (P < .0001). There was no statistically significant difference in the number of recommendations. Conclusion: Our findings suggest that CDS assistance significantly reduced the time spent by physicians for deciding on preventive services and chronic disease management. The result needs to be confirmed by performing similar studies at other institutions.

Keywords

preventive health services clinical decision support systems task performance and analysis time motion study chronic diseases

Introduction

The quality of preventive care delivered in the United States is suboptimal.^1,2 Recent studies have shown that many patients do not get the recommended best preventive services³—mammography, pap, and colonoscopy screening rates are 72%, 83%, and 59%, respectively.⁴ Adult vaccination rates are also low—pneumococcal vaccination coverage among high-risk adults is 20% and tetanus, diphtheria, and pertussis (Tdap) coverage of adults is 13%.⁵ Several studies have identified provider,^6,7 patient,⁸ practice,^9,10 and environmental factors^11-14 that affect care quality. The need to improve the care quality has motivated federal investments in health information technology.¹⁵

Although there is strong evidence that clinical decision support (CDS) assistance improves preventive care delivery, there is little evidence of CDS impact on workload and efficiency of care providers.^16-22 Besides increasing service delivery, CDS interventions may reduce the time required for service delivery. The time savings in turn can enable the physician to focus on the patient’s presenting problem, and consequently improve care quality. But there is no previous report of the time savings from the CDS assistance. In this article, we report a pilot study aimed at measuring the impact of CDS assistance on the time spent by physicians for deciding on preventive services and chronic disease management.

Methods

We randomly selected 30 patients from a primary care practice, and invited 10 physicians across multiple sites of the practice to participate in this study. Each patient was randomly assigned in a cross over design to 2 of the physicians, such that one physician had CDS assistance and the other physician did not have CDS assistance (Figure 1). The physicians were requested to perform chart review to decide on preventive services and chronic disease management (Table 1) for the assigned patients. To ensure equal representation of the physicians, each physician had CDS assistance for only half the assigned patients. Consequently, each patient had two sets of recommendations—one from a physician using CDS assistance and the other from a different physician without CDS assistance. We compared the physician recommendations made using CDS assistance, with the recommendations made without CDS assistance. The methodological details are described further in this section. This research was approved by the institutional review board at Mayo Clinic, Rochester.

Figure 1.

Crossover design of the study. Circles indicate patients, rectangles indicate physicians and connecting lines indicate the assignments. Each patient received 2 sets of recommendations—one from a physician with clinical decision support (CDS) assistance and the other from a different physician without CDS assistance. Each physician provided recommendations for half the assigned patients using CDS assistance, and made the recommendations without CDS assistance for the other half of the patients.

Table 1.

Preventive Care and Chronic Disease Management Decisions Investigated in This Study.^a

Lifestyle factors
Tobacco cessation advice	Patient is using tobacco
Alcohol counseling	Patient reports problems with alcohol
Lifestyle counseling	BMI is >30 kg/m²
Blood tests
Fasting glucose	Hyperglycemic with no glucose test in 1 year
Glycated hemoglobin (HbA1c)	Diabetic with no HbA1C report
Lipid panel	Nondiabetic with no cholesterol, HDL, or triglycerides in the past 5 years or diabetic with no cholesterol, HDL, or triglycerides in past 1 year
LDL	High risk for cardiovascular disease and has a high recent LDL
Creatinine	No creatinine report in the last 1 year, and patient is diabetic or receives diuretics, ARB, or ACE inhibitors
Microalbumin	Diabetic patient with no microablumin result in past year
TSH	Patient on thyroid meds, with no TSH in past year
Issue examinations/screenings
Eye examination	Diabetic with no eye examinations in the past year
Urinalysis	On cyclophosphamide, no urinalysis reported in the past 6 months
Echocardiogram	Heart failure with no reported ejection fraction
Papanicolaou (Pap)	If abnormal Pap or high-risk condition was not followed by Pap test in past year, or no Pap report for 21- to 30-year-old females in past 2 years, or 30- to 65-year-old female with no Pap report in past 3 years
Mammogram	40- to 75-year-old female with no mammogram in past year
Colonoscopy	(i) Polyp not followed up in past 3 years, or colon cancer diagnosis not followed up in past year. (ii) Family history of colon cancer with no colonoscopy in past 5 years. (iii) 50- to 80-year-old with no colonoscopy in the past 10 years, occult blood in the past 12 months, flexible sigmoidoscopy, colon x-ray or colonography in the past 5 years
Abdominal aortic aneurysm	65- to 75-year-old male with past tobacco habit and no imaging of abdominal aorta
BMD	>65-year-old female with no BMD since age 50 years
PSA	50- to 70-year-old male with no PSA in past year
STI screening	16- to 25-year-old with no report of chlamydia/gonorrhea screening
Depression screening (PHQ9)	Last PHQ9 with high score more than 6 months ago or PHQ9 not reported in past year for patients diagnosed with depression
AICD consult	CHF with ejection fraction less than 35%, who have no AICD
Immunizations due
Influenza	No immunization in flu season
HPV	9- to 25-year-old female with (i) no previous HPV vaccination, or (ii) first vaccination more than 2 months ago, or (iii) second vaccination more than 4 months ago
Pneumococcal	20- to 65-year-old with high risk and no pneumococcal immunizations in the past 5 years. Greater than 65 years with no vaccine in the past 5 years
Herpes zoster	More than 60 years with no zoster vaccine in the past
Tdap	20- to 64-year-old with no Tdap ever
Tetanus	>60 years with no Tdap or tetanus dose in the past 10 years
Hepatitis B	Diabetic, aged 19 to 59 years and (i) no past immunization, (ii) 1 month after first immunization, (ii) 2 months after second immunization and 4 months after first immunization
Start medication
Beta blocker	Not asthmatic or depressed, is not currently on beta blockers, and has either (i) heart rate is greater than 60 beats per minute, with CHF and ejection fraction is less than 0.4 or (ii) diagnosis of CAD
Warfarin	Has atrial fibrillation, who is not on warfarin, has no bleeding disorder and has high CHADS₂ score
Aspirin	Not on aspirin or warfarin, and has no bleeding disorder but has either of CAD, DM, cerebrovascular accident, or peripheral vascular disease
ACE inhibitors/ARBs	On ACE or ARB, has DM, hypertension, high microalbumin, CAD or CHF, and has no history of renal failure
Medication-related blood work
Digoxin level	On digoxin with no digoxin report in past year
Potassium level	On diuretics, ARBs or ACE inhibitors with no potassium report in past year
Sodium level	On diuretics, with no sodium report in past year
Vitamin B₁₂	No vitamin B₁₂ in past year with anemia, bypass surgery, Crohns, achlorhydria, alcohol dependency, or bacterial overgrowth
Dilantin level	On anticonvulsants and dilantin, with no dilantin report in past year
Carbamazepine level	On carbamazepine, and no carbamazepine report in past year
ALT	On cyclosporin and no ALT report in past 3 months
AST	On azathioprine, leflunomide, methotrexate or tocilizumab, and no AST report in past 8 weeks
CBC	On cyclophosphamide, no CBC in last 6 weeks. On azathioprine or leflunomide or methotrexate or tocilizumab, and no CBC in last 8 weeks. On cyclosporin, sulfasalazine, anti-IL1β or anti-TNF, or rituximab and no CBC result in the past 3 months
INR	On warfarin and no INR in past month

Abbreviations: ACE inhibitor, angiotensin-converting enzyme inhibitor; AICD, automatic implantable cardioverter defibrillator; ALT, alanine aminotransferase; ARB, angiotensin II receptor blocker; AST, aspartate transaminase; BMD, bone mineral density; CAD, coronary artery disease; CBC, complete blood count; CHADS₂, congestive heart failure, hypertension, age, diabetes mellitus, stroke or transient ischemic attack; CHF, chronic heart failure; DM, diabetes mellitus; HbA1c, glycated hemoglobin; HDL, high-density lipoprotein; HPV, human papillomavirus; IL1β, interleukin-1 beta; INR, international normalized ratio; LDL, low-density lipoprotein; Pap, Papanicolaou test; PHQ9, Patient Health Questionnaire 9; PSA, prostate-specific antigen; STI, sexually transmitted infection; Tdap, tetanus, diphtheria, pertussis; TNF, tumor necrosis factor; TSH, thyroid-stimulating hormone.

The left column gives the recommendation, and the right column summarizes the applicable trigger conditions.

Study Population

We randomly selected 30 patients from the population of approximately 103 000 patients older than 18 years who receive primary care at the Department of Employee and Community Health at Mayo Clinic, Rochester. Ten physicians were invited to participate from different locations in the practice.

Data Collection

We prepared a paper checklist of preventive services including screening and chronic condition management that are covered by the institutional CDS system. The services were grouped as lifestyle factors, blood tests, screenings, immunizations, medications, and medication-related blood work, and listed with checkboxes on a single page. Participating physicians were invited to complete the checklist for indicating which preventive services are due, by performing chart review.

Each physician was requested to complete checklists for 6 randomly allotted patients (Figure 1). For 3 of the allotted 6 patients, the physicians were requested to consider the CDS reminders that were provided as a printout, for deciding on the screening recommendations. The CDS decision logic for the preventive services is summarized in Table 1.²³ For the other 3 patients, the physicians were requested to skip the CDS assistance. The physicians recorded the start and end times on each of the checklists. The 30 cases were randomly distributed among the 10 physicians, such that 2 checklists were completed for each patient—for one of the checklists the physician was assisted by the CDS system, and the other checklist was completed by a different physician who did not use assistance from the CDS system.

Analysis

We compared the checklists across the unassisted and assisted groups, for time required to complete the checklist and number of recommendations per checklist using paired t tests. The time required to complete the checklists was computed as the difference in the start and end times recorded by the clinicians.

Results

The physicians required an average of 1 minute 44 seconds, when they were they had access to the decision support system and 5 minutes when they were unassisted. Hence the CDS system resulted in an estimated saving of 3 minutes 16 seconds (65%) of the physicians’ time, which was statistically significant (P < .0001).

Assisted and unassisted physicians made an average of 3.4 and 3.9 recommendations per patient, respectively (Table 2 and Figure 2). This difference was not statistically significant (P = .36). The results of the paired t test for difference in time and number of recommendations between the groups were robust to nonparametric assumption. The average statistics correspond to a respective total of 102 assisted and 116 unassisted recommendations, of the total possible 1290 (=43 preventive services × 30 patients) recommendations in each group.

Table 2.

Results Summary.

	Mean (SD)	No. of Observations	P Value
Time (minutes) for completing checklist			<.0001
Unassisted group	5.00 (2.73)	30
Assisted group	1.73 (1.57)	30
Number of recommendations in			.36
Unassisted group	3.87 (3.20)	30
Assisted group	3.40 (2.76)	30

Figure 2.

Time required to complete the checklist and number of recommendations per patient, for the assisted versus unassisted physicians.

The patient population in this study had more females (57%), and tended to be middle aged, with an average age of 56.53 years (standard deviation of 19.7 years). The age ranged from 25 to 94 years. The number of problems listed in the electronic health record (EHR) ranged from 1 to 95, with an average of 22.9 (20.9), and number of laboratory results ranged from 0 to 2603 with average of 324.1 (597.1).

Discussion

We compared the time for decision making between 2 groups of providers—one group had CDS assistance and the other did not. CDS assistance was found to significantly reduce the time required by primary care providers for deciding on the preventive services and chronic disease management.

Our finding is in agreement with a report by Del Fiol et al²⁴ that providing context-specific information in the EHR saves 17% of the physician’s time for searching information needed to make clinical decisions. Several other studies have reported mixed effect of health information technology on time efficiency of physicians, but none of them have specifically reported on the time savings for deciding on preventive services and chronic disease management due to CDS.^25-27

Yarnall et al²⁸ have estimated the time for delivering preventive services, but they assume zero time for deciding on the recommendations. However, our study shows that physicians spend considerable time for deciding on the recommendations.

During patient consultation, the physician’s time is divided into addressing the current patient complaints and carrying out services for preventive care and for management of chronic conditions. Although physician can order the services and delegate the administration of the services to assistants, the lack of time to perform chart review for deciding on the services, can itself pose as an obstacle for delivery of the services.⁹ By reducing the time requirements for deciding the services, CDS may augment the delivery of the applicable service and improve the quality of care.

Time utilization of the clinician²⁹ for different tasks has been previously reported using methodologies like work sampling,³⁰ surveys,³¹ and continuous observation.^31,32 The methodology of continuous observation is considered a gold standard. Although observational studies can provide a detailed perspective of how the clinicians distribute their time, they are resource intensive. Our aim was to measure the time component specific for decision making, which is difficult to determine by independent observation. Hence, we resorted to the approach of self-timing.

The time savings from decision support may seem counter intuitive, since the CDS reminders represent additional information that the physicians need to consider for deciding on the recommendations. The physicians are indeed expected to confirm the validity of decision support before acting on it, which entails chart review. As a special case, CDS interventions that generate explanations or summarization can reduce the chart review effort, by identifying the reports that are relevant for the decision.³³ However, such explanations or summarizations were not supplied in this study. Hence, the likely explanation for the time savings is that the physicians acted on the CDS reminders without performing the verifications, possibly because of the trust developed in the CDS system.^6,34

Although there was a statistically significant difference in the time required for deciding on the recommendations, there was no significant difference in number of recommendations between the 2 groups. This suggests that CDS assistance may have greater benefits in terms of time savings as compared to improvements in service delivery.

The need of services for preventive care and chronic disease management is expected to vary substantially across patients, and the physician characteristics are known to influence the delivery of preventive care.^14,36 Hence, we used a crossover design, to ensure equal representation of the physician characteristics and patients in the study groups.

Limitations

As the physicians in the study were not previously familiar with the assigned patients, the study may not reflect the real-world setting of a longitudinal physician–patient relationship, wherein previous knowledge about the patient reduces the need for chart review. Also, the chart review was not synchronous with the patient visit, and thereby excludes patient participation in the decision making. For instance, some patient data resulting from care obtained outside the institution may not be recorded in the EHR system, but may be provided by the patient. In addition, patients may refuse particular services. These factors could cause an over estimation of the efficiency resulting from the use of CDS. Moreover, our study sample is modest in size and is from a single institution. Hence, the time savings measured in this study need to be confirmed by similar studies at other institutions.

Conclusion and Future Work

Clinical decision support interventions for primary care have been shown to improve screening rates, and projected to reduce costs. However, the evidence for efficiency of physicians due to decision support is sparse. In our pilot study, the CDS assistance was found to significantly reduce the time spent by physicians for deciding on preventive services and chronic disease management. Further research is required to confirm the time savings at other institutions.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Mayo Clinic foundation. In addition, Dr. Wagholikar’s effort on this research was partly supported by National Library of Medicine (NLM) of the National Institutes of Health under award number 1K99LM011575. Mr. Cha was supported by CTSA Grant Number UL1 TR000135 from the National Center for Advancing Translational Sciences (NCATS).

Author Biographies

Kavishwar B. Wagholikar, MBBS, PhD, is a Research Associate at Mayo Clinic. His research interest is in the optimization of Clinical Decision Making, through computer base decision support, guideline refinement, natural language processing, and secondary use of electronic health records.

Ronald A. Hankey is a senior analyst/programmer at Mayo Clinic with an interest in clinical decision support systems. Mr. Hankey holds BS, MS and MBA degrees from various institutions.

Lindsay K. Decker is a lead analyst/programmer at Mayo Clinic and has worked on clinical decision support systems since 2007. She holds a B.S. degree in management of information systems and a M.S. degree in Project Management.

Stephen S. Cha graduated from University of Pittsburgh with a MS degree in applied statistics 1983 and with another MS degree in biostatistics in 1985. He has been working in cancer center statistics at Mayo Clinic from 1985-2000, and in biomedical statistics and informatics from 2000-2014.

Robert A. Greenes is a Professor of Biomedical Informatics at Arizona State University and Mayo Clinic. He has a long history of work in computer-based clinical decision support, knowledge representation, and standards-based approaches to interoperability. He is the author/editor of a major textbook in clinical decision support, now in its second edition.

Hongfang Liu, PhD, is an Associate Professor and Director of the Natural Language Processing (NLP) program at Mayo Clinic Rochester. She has over 16 years’ experience in biomedical informatics, with an extensive background in statistical and expert-based NLP. Her mission is to make NLP accessible, portable, and interoperable.

Rajeev Chaudhry, MD, MPH, is a consultant and Assistant Professor in the Department of Internal Medicine at Mayo Clinic Rochester, and Co-director of Knowledge Delivery Center of the Office of Knowledge and Information Management at Mayo Clinic. He has championed the implementation of clinical decision support (CDS) systems for primary care at Mayo Clinic, and has lead the development and deployment of the CDS system investigated in this study.

References

Agency for Healthcare Research and Quality. 2011 National Healthcare Quality and Disparities Reports. 2012. http://www.ahrq.gov/research/findings/nhqrdr/nhqrdr11/qrdr11.html. Accesed July 24, 2014.

Pham

Schrag

Hargraves

Bach

PB.

Delivery of preventive services to older adults by primary care physicians. JAMA. 2005;294:473-481.

Goodwin

Singh

Reddy

Riall

Kuo

YF.

Overuse of screening colonoscopy in the Medicare population. Arch Intern Med. 2011;171:1335-1343.

Centers for Disease Control and Prevention. Cancer screening—United States, 2010. MMWR Morb Mortal Wkly Rep. 2012;61(3):41-45.

Centers for Disease Control and Prevention. Noninfluenza vaccination coverage among adults—United States, 2011. MMWR Morb Mortal Wkly Rep. 2013;62(4):66-72.

Wagholikar

MacLaughlin

Henry

. Formative evaluation of the accuracy of a decision support system for cervical cancer screening. J Am Med Inform Assoc. 2013;20:749-757.

Flocke

Gilchrist

Physician and patient gender concordance and the delivery of comprehensive clinical preventive services. Med Care. 2005;43:486-492.

Hawley

McQueen

Bartholomew

. Preferences for colorectal cancer screening tests and screening test use in a large multispecialty primary care practice. Cancer. 2012;118:2726-2734.

Eaton

Reed

Angstman

. Effect of visit length and a clinical decision support tool on abdominal aortic aneurysm screening rates in a primary care practice. J Eval Clin Pract. 2012;18:593-598.

10.

Dahrouge

Hogg

Russell

. Impact of remuneration and organizational factors on completing preventive manoeuvres in primary care practices. CMAJ. 2012;184:E135-E143.

11.

Shires

Stange

Divine

. Prioritization of evidence-based preventive health services during periodic health examinations. Am J Prev Med. 2012;42:164-173.

12.

Cabana

Rand

Powe

. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282:1458-1465.

13.

Provost

Pineault

Levesque

. Does receiving clinical preventive services vary across different types of primary healthcare organizations? evidence from a population-based survey. Healthc Policy. 2010;6:67-84.

14.

Flocke

Litaker

Physician practice patterns and variation in the delivery of preventive services. J Gen Intern Med. 2007;22:191-196.

15.

Blumenthal

Tavenner

The “meaningful use” regulation for electronic health records. N Engl J Med. 2010;363:501-504.

16.

Bright

Wong

Dhurjati

. Effect of clinical decision-support systems: a systematic review. Ann Intern Med. 2012;157:29-43.

17.

Shaw

Samuels

Larusso

Bernstein

HH.

Impact of an encounter-based prompting system on resident vaccine administration performance and immunization knowledge. Pediatrics. 2000;105(4 pt 2):978-983.

18.

Bouaud

Seroussi

Antoine

Zelek

Spielmann

A before-after study using OncoDoc, a guideline-based decision support-system on breast cancer management: impact upon physician prescribing behaviour. Stud Health Technol Inform. 2001;84(pt 1):420-424.

19.

Schroy

3rd Emmons

Peters

. The impact of a novel computer-based decision aid on shared decision making for colorectal cancer screening: a randomized trial. Med Decis Making. 2011;31:93-107.

20.

Hahn

Ferrante

Crosson

Hudson

Crabtree

BF.

Diabetes flow sheet use associated with guideline adherence. Ann Fam Med. 2008;6:235-238.

21.

Steele

Eisert

Davidson

. Using computerized clinical decision support for latent tuberculosis infection screening. Am J Prev Med. 2005;28:281-284.

22.

Cleveringa

Gorter

van den Donk

Rutten

GE.

Combined task delegation, computerized decision support, and feedback improve cardiovascular risk for type 2 diabetic patients: a cluster randomized trial in primary care. Diabetes Care. 2008;31:2273-2275.

23.

Chaudhry

Wagholikar

Decker

. The innovations in the delivery of primary care services using a software solution: the Mayo Clinic’s Generic Disease Management System. Int J Person Centered Med. 2012;2:361-367.

24.

Del Fiol

Haug

Cimino

Narus

Norlin

Mitchell

JA.

Effectiveness of topic-specific infobuttons: a randomized controlled trial. J Am Med Inform Assoc. 2008;15:752-759.

25.

Poissant

Pereira

Tamblyn

Kawasumi

The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc. 2005;12:505-516.

26.

Chaudhry

Wang

. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742-752.

27.

Devine

Hollingworth

Hansen

. Electronic prescribing at the point of care: a time-motion study in the primary care setting. Health Serv Res. 2010;45:152-171.

28.

Yarnall

Pollak

Ostbye

Krause

Michener

JL.

Primary care: is there enough time for prevention?

Am J Public Health. 2003;93:635-641.

29.

Zheng

Guo

Hanauer

DA.

Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. J Am Med Inform Assoc. 2011;18:704-710.

30.

Hakes

Whittington

Assessing the impact of an electronic medical record on nurse documentation time. Comput Inform Nurs. 2008;26:234-241.

31.

Pizziferri

Kittler

Volk

. Primary care physician time utilization before and after implementation of an electronic health record: a time-motion study. J Biomed Inform. 2005;38:176-188.

32.

Overhage

Perkins

Tierney

McDonald

CJ.

Controlled trial of direct physician order entry: effects on physicians’ time utilization in ambulatory primary care internal medicine practices. J Am Med Inform Assoc. 2001;8:361-371.

33.

Elson

Connelly

. The impact of anticipatory patient data displays on physician decision making: a pilot study. Proc AMIA Annu Fall Symp. 1997:233-237.

34.

Friedman

Elstein

Wolf

. Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: a multisite study of 2 systems. JAMA. 1999;282:1851-1856.

35.

Hung

Rundall