Abstract
Electronic health record-caused safety risks are an unintended consequence of the implementation of clinical systems. To identify activities essential to assuring that the electronic health record is managed and used safely, we used the Rapid Assessment Process, a collection of qualitative methods. A multidisciplinary team conducted visits to five healthcare sites to learn about best practices. Although titles and roles were very different across sites, certain tasks considered necessary by our subjects were remarkably similar. We identified 10 groups of activities/tasks in three major areas. Area A, decision-making activities, included overseeing, planning, and reviewing to assure electronic health record safety. Area B, organizational learning activities, involved monitoring, testing, analyzing, and reporting. Finally, Area C, user-related activities, included training, communication, and building clinical decision support. To minimize electronic health record-related patient safety risks, leaders in healthcare organizations should ensure that these essential activities are performed.
Introduction
The United States has spent billions of dollars providing incentives for health care providers and organizations to use electronic health records (EHRs). However, implementation of EHR-related initiatives has encountered greater than expected challenges.1–4 Reports have warned of unintended consequences of health information technology (HIT) adoption, including new patient safety problems and reduced provider efficiency resulting from the implementation and use of EHRs.5–16 We define EHR safety as the design, implementation, and use of EHRs in such a way that patients are not put at risk. EHR safety occurs in a sociotechnical environment, which includes the social environment in which it is implemented as well as the hardware and software required to implement the system. 17 Therefore, risks for EHR-related errors may be related to features of the technology itself, to user behavior, and/or to organizational policies about how the EHR is routinely used, maintained, and monitored. With ever-increasing use of EHRs, organizations face mounting patient safety risks. However, little research about EHR safety has been conducted, 18 though some research has been done on risks related to computerized provider order entry (CPOE).2,5 There have thus far been no studies about activities and tasks needed for reducing patient safety risks related to the broader area of EHR systems.
To help organizations to implement and use EHRs safely, the US Office of the National Coordinator for Health Information Technology (ONC) sponsored the development of the Safety Assurance Factors for EHR Resilience (SAFER) Guides. 19 The overall purpose of the SAFER Guides project was to develop and validate proactive self-assessment tools to ensure that EHR-enabled clinical work systems are safe and effective. Eight high-risk areas identified by an expert panel became the foci of the guides: Organizational Responsibilities, System Configuration, System Interfaces, Contingency Planning, Patient Identification, CPOE with Clinical Decision Support (CDS), Test Results Reporting and Follow-up, and Clinician Communication. Their development involved a literature search, input from a panel of experts, and stakeholder engagement, but because the evidence base about EHR safety was not large, we also opted to perform a qualitative study to gather further evidence of best practices related to each guide.
Primary objective
The purpose of this study, aimed specifically at development of the Organizational Responsibilities guide, but also designed to inform development of the organizational aspects of topics covered in the other guides, was to identify the essential activities performed by people within healthcare organizations to assure that the EHR is managed and used safely. We decided to focus on activities and tasks because titles and roles differ so much across organizations.
Methods
Linstone’s original Multiple Perspectives model20–22 guided the research design, data collection, and analysis of the qualitative study. Following this framework, we gathered personal, organizational, and technical perspectives on EHR safety from a wide variety of subjects. A multidisciplinary team of informaticians, social scientists, clinicians, and usability experts conducted visits to five healthcare sites (eight different care organizations) in 2012 to assess the context within which the tools would be used and to learn about best practices for EHR safety. 23 A detailed description of the initial study protocol, written prior to conducting the study, has been published elsewhere. 24 The qualitative study methods, only briefly described in the protocol, are described in greater detail below. The qualitative study was approved by the human research committees at Oregon Health & Science University, the University of Texas Health Science Center at Houston, Baylor College of Medicine, and the relevant committee at each study site.
Selection of sites
Selection of the five sites was based on several factors, including geography, organizational type (e.g. academic medical center, privately owned outpatient clinics, and federally qualified health centers), care setting (inpatient and ambulatory), experience with clinical information systems, and comprehensiveness of approach and/or resources to address EHR safety. The expert panel members assisted with selection of sites they believed were excellent in EHR safety and met these criteria. The following organizations within the United States were selected for the site visits:
Geisinger Health System based in Danville, PA, a large integrated health delivery system with sophisticated use of Epic, having implemented it in outpatient clinics in 1996 and the first hospital in 2006;
Family Physicians Group in Orlando, FL, a group of private primary care clinics in the process of developing expertise with GE Centricity;
Alliance of Chicago, a group of safety net clinics in Chicago, IL using GE Centricity since 2006;
Four small ambulatory clinics with between 1 and 10 providers in the Denver and Colorado Springs, CO areas using five different EHR systems for between 2 and 10 years;
Partners Healthcare in Boston, MA, a large delivery system with a history of locally developed EHR systems, dating back 3 to 4 decades, that was in the process of being replaced with a commercial system.
Selection of subjects
We followed the Multiple Perspectives Model for subject selection, assuring that those we interviewed and observed could offer informed perspectives on HIT use, content, technology, and governance. A contact person at each site helped us select appropriate individuals to invite for interviews and observation and provided introductions. We asked to speak with a broad spectrum of people who perform activities related to EHR safety: individuals in leadership, informatics, information technology (IT), quality and safety, health information management (HIM), and other departments. During our observations, we shadowed users of EHRs in both inpatient and outpatient settings.
Data collection methods
We used the Rapid Assessment Process (RAP) methodology for this study.25,26 RAP is a collection of qualitative methods and it is especially useful for conducting time-efficient studies because it depends on intensive work over shorter periods of time than needed for classical ethnography due to its use by a multidisciplinary team. Our efforts involved gathering information about relevant EHR safety practices from on-site informaticians prior to the visit, a demonstration of the EHR system, semi-structured interviews (see Table 1 for a list of questions from which we selected the most relevant for each interviewee), interviews to gather input about items in draft versions of the guides, and observation of clinic activities and shadowing of clinicians. All interviews were recorded and transcribed and researchers took detailed field notes during observations.
Interview guide questions.
EHR: electronic health record; HCI: human computer interaction.
Analysis
All data from the interviews and observations were entered into NVivo 10 (QSR International, Doncaster, Victoria, Australia). We analyzed the data twice, once in a deductive manner to inform development of the guides and once inductively to generate patterns and themes for the present study. Data from the semi-structured interviews and fieldnotes were first analyzed using the template (more deductive) method of analysis identifying topics within the documents that we needed to explore for development of the SAFER guides. Two researchers read and coded each document, agreeing on which quotes were relevant to each guide. The entire team then met virtually to review each quote and perform subcoding for each guide. From the quotes clumped into each subcode, the team derived many of the recommendations outlined in the SAFER guides. Data used to develop the guide about Organizational Responsibilities for EHR safety were used for this sub-study, for which we conducted an additional more inductive analytical process. We coded all fieldnotes and transcripts again, using a grounded hermeneutic approach to discover patterns and themes specifically related to activities performed by individuals across sites. 25 Again, two researchers did initial coding of the data, reached consensus on codes, proceeded to code all the data, and the entire research team met to agree on patterns and themes.
Results
Table 2 provides information about the sites and data collected at each site. In total, we conducted 92 semi-structured interviews and spent 69 h observing.
Sites and subjects.
EHR: electronic health record.
Table 3 shows a list of the areas and activity groups, along with representative quotes about each. We identified 10 groups of activities that people or teams involved in EHR safety perform. These activities fall into three large areas.
Areas, activities, and representative quotes.
EHR: electronic health record; CDS: clinical decision support; HIM: health information management.
The first category, Area A, is decision-making related to safety. This was usually done at higher levels in the organization by individuals in leadership positions working on committees.
Area A: decision-making
Activity 1: overseeing EHR safety
Although the organizations we visited did not employ people expressly to oversee EHR safety, the larger ones had individuals who played the role of patient safety officer and, through committees, EHR safety issues were addressed. Subjects told us that these organizations were primarily reactive rather than proactive in that policies were in place for analyzing and rectifying problems, but a proactive EHR safety plan was not yet available.
Activity 2: planning for EHR safety
As noted, plans were usually made in response to issues, but these were usually general patient safety risk assessment plans. While they were not EHR-specific, they did often include some EHR-related wording. We were told at all the sites that when the SAFER guides became available, they would become desirable planning tools.
Activity 3: reviewing for EHR safety
This included activities done by committees, often CDS-related. These committees did not often address safety directly, but they kept EHR safety in mind as they made decisions about what CDS to develop or buy and what to eliminate. Individuals at all of the sites talked about wanting to avoid over-alerting providers because safety risks might increase if providers ignore important alerts.
Area B: organizational learning
The second area of activities is organizational learning. These are activities performed primarily by informatics and IT staff members.
Activity 1: monitoring for EHR safety
Staff members in these organizations tried to continuously monitor both the functionality and usage of the EHR so that unintended consequences could be avoided. They made sure that hardware, servers, backup tapes, software, and content were configured correctly and working appropriately. For example, someone would be looking at reports of response time, so if the EHR started responding more slowly than normal, that person could investigate the problem immediately. Interviewees told us they would like to have the manpower to do more monitoring, however.
Activity 2: testing
These organizations performed several types of testing before any changes were made to the EHR, most notably before an upgrade. IT staff members involved clinical users for usability testing in addition to testing in different environments (a test environment and operational environment, for example) to assure safety.
Activity 3: analyzing
This usually took the form of performing root cause analyses when problems were reported. Not only did staff members investigate the multiple causes of errors that were reported, but they also generally did follow-up for purposes of continuous quality improvement and avoidance of the problem occurring again. In most patient safety events, more often than not, the EHR was only one of multiple causes.
Activity 4: reporting
Reporting took many forms. Users were encouraged to report problems with safety implications to informatics/IT staff. Staff members then reported certain software problems to the vendor. In addition to these reactive reporting efforts, reports were sometimes monitored on an ongoing basis, often by the HIM professionals, so that issues could be addressed proactively as well.
Area C: user-related activities
While decision-making was done by leaders and committees and organizational learning was accomplished by informatics and IT-related staff members, other critical activities involved users more directly.
Activity 1: training
We were told that the education and training of all staff members who used the EHR was needed on an ongoing basis. Not only did users need to learn how to use the system and be constantly aware of changes which could pose risks, especially when upgrades occurred, but they also needed to learn the importance of accuracy to avoid input errors.
Activity 2: communicating
Two-way communication is vital for EHR safety, we were told. We observed that users needed to know about changes in the system, but they also needed to feel free to communicate back to staff members and report problems. Even patients needed to be urged to communicate through patient portals to assure the accuracy of information.
Activity 3: building and managing safe CDS
While leadership committees oversaw CDS, staff members built, modified, and kept track of it. Subjects noted that over-alerting can generate errors when users disregard appropriate alerts. EHR users played a role in that clinicians were usually involved in developing CDS as well. While CDS was implemented for quality or efficiency purposes, it sometimes also provoked dangerous unintended consequences. Therefore, subjects stressed that CDS needs testing and monitoring by staff and problems need to be reported by users.
Discussion
We have identified three areas and nine groups of activities that are perceived to be critical for EHR safety within healthcare organizations of any size. These activities must become the responsibility of identified individuals within organizations because otherwise they might be neglected. This is the first large qualitative study with EHR safety as its focus, and it has been useful in informing development of not just the Organizational Responsibilities SAFER guide, but the other SAFER guides as well. One limitation of this study is that although we attempted to select representative, excellent organizations as study sites, the results may not be generalizable to all healthcare organizations. In addition, the data were collected 6 years ago. However, the SAFER guides were updated in 2016, with a thorough review of the literature conducted for that purpose, and the results presented here remain relevant and timely.
Prior research about EHR safety 18 has indicated that organizational and behavioral aspects are important, but those aspects have not been the main focus of any studies. For example, Zhan et al. 27 found that of 7029 CPOE-related medication errors reported in 2003, 0.1 percent resulted in adverse events and the greatest contributing factor to these particular errors was the presence of human distraction in 8 out of 10 cases. Santell et al. 28 likewise found that harm is low but that most errors came from inpatient pharmacy input of data, again a behavioral issue.
The results of this study have implications for a variety of stakeholders within healthcare organizations. Decision makers at the highest levels within these organizations should become aware that the EHR can increase patient safety risk and therefore liability, and they can take action to decrease that risk. They should hire people to perform the activities outlined in this article. This study highlights the need for continuous careful monitoring of systems so that organizational learning can instigate change each time something goes wrong. IT and informatics leaders must assure that users are adequately trained and also deeply involved in higher-risk EHR-related activities like developing or modifying CDS.
Future research about the organizational aspects of EHR safety should include exploration of the extent of generalizability of these qualitative findings through survey methods, research about the impact and use of the organizational effectiveness SAFER guide recommendations which are based on this study, and identification of best practices for reviewing, prioritizing, and implementing the myriad recommendations that make up the SAFER guides.
The most important implication of our results is that people within organizations must be assigned to cover all of the activities needed to ensure EHR safety. These essential activities are conducted by a broad range of individuals within organizations who come together to work as a team, most often on multidisciplinary committees. Such teams should include organizational leaders, clinicians, IT and informatics professionals, other administrative staff, and even patients. Therefore, all of these types of people must be made aware of their shared responsibility for keeping the EHR safe.
Conclusion
EHR safety can be improved by the performance of activities related to group decision-making, organizational learning, and user-related tasks outlined in our results. While roles and job titles differ across organizations, the responsibility for accomplishing these essential activities must be shared by explicitly identified individuals and teams.
Footnotes
Acknowledgements
We would like to thank Colleen Tercek for her assistance with analysis.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported through a subcontract from Westat (HHSP-23320095655WC0095655, Anticipating the Unintended Consequences of Health IT) funded by the Office of the National Coordinator for Health Information Technology (ONC) (HHSP23337003 T). Dr. Singh is supported in part by the Houston VA Health Services Research and Development Center for Innovations in Quality, Effectiveness and Safety (CIN13-413).