Computerized provider order entry–related medication errors among hospitalized patients: An integrative review

Abstract

The Institute of Medicine estimates that 7,000 lives are lost yearly as a result of medication errors. Computerized physician and/or provider order entry was one of the proposed solutions to overcome this tragic issue. Despite some promising data about its effectiveness, it has been found that computerized provider order entry may facilitate medication errors.

The purpose of this review is to summarize current evidence of computerized provider order entry -related medication errors and address the sociotechnical factors impacting the safe use of computerized provider order entry. By using PubMed and Google Scholar databases, a systematic search was conducted for articles published in English between 2007 and 2019 regarding the unintended consequences of computerized provider order entry and its related medication errors. A total of 288 articles were screened and categorized based on their use within the review. One hundred six articles met our pre-defined inclusion criteria and were read in full, in addition to another 27 articles obtained from references. All included articles were classified into the following categories: rates and statistics on computerized provider order entry -related medication errors, types of computerized provider order entry -related unintended consequences, factors contributing to computerized provider order entry failure, and recommendations based on addressing sociotechnical factors. Identifying major types of computerized provider order entry -related unintended consequences and addressing their causes can help in developing appropriate strategies for safe and effective computerized provider order entry. The interplay between social and technical factors can largely affect its safe implementation and use. This review discusses several factors associated with the unintended consequences of this technology in healthcare settings and presents recommendations for enhancing its effectiveness and safety within the context of sociotechnical factors.

Keywords

computerized physician order entry health information technology medication errors sociotechnical factors unintended consequences

Introduction

The cornerstone report released by the Institute of Medicine (IOM) in 1999 approximates that between 44,000 and 98,000 persons die per year as a result of preventable adverse events in hospitals.¹ Shockingly, medication errors (MEs) were estimated to account for about 7000 deaths per year; they account for 1 of 854 inpatient deaths and 1 of 131 outpatient deaths.¹ Another study considering ME-related deaths in US hospitals between 1983 and 1993 has shown that the mortality rate increased due to MEs from 2876 to 7391 deaths.² Accordingly, healthcare leaders have called for actions to increase patient safety within healthcare institutions. One of the important measures recommended by the IOM was to apply the capabilities of health information systems. Despite efforts to minimize adverse events, they continue to occur.^3–5 Unfortunately, several reports have linked the implementation of computerized provider order entry (CPOE) to the occurrence of some of these adverse events.^6–8

CPOE is a complex function of health information technology (HIT), commonly understood as a set of capabilities within the order entry application that permits clinicians to place medication and other therapy orders electronically with the assistance of pre-determined rules, alerts, and knowledge databases.^9,10 This capability plays a key role in the patient management process, where it can impact a variety of systems, such as pharmacy, laboratory, radiology, and resource management. CPOE also involves different users and key players, such as physicians, pharmacists, nurses, and system developers. Implementing CPOE was supported with promises for improving the quality of patient care, decreasing costs, and reducing the risk of medical errors.¹¹

Numerous earlier studies have demonstrated the effectiveness of CPOE in reducing the unintended consequences (UCs) of healthcare therapies and treatments. Nuckols et al.¹² concluded that implementing CPOE is associated with an almost 50-percent decrease in preventable adverse drug events. Radley et al.¹³ also found that using CPOE to prescribe drugs decreases the likelihood of error by 48 percent. Although the use of CPOE is linked to notable improvements in reducing and/or limiting MEs, others argue that the use of CPOE might facilitate MEs, or CPOE itself could become a source of errors.^14–17 As an example, a recent study¹⁸ found that nearly 6 percent (63,040) of the MEs reported to MEDMARX—an anonymous, confidential, Internet-accessible, ME-reporting program that is designed and developed by the US Pharmacopeia for hospitals and health systems to systematically collect, analyze, track, and report MEs¹⁹—between 2003 and 2010 were related to CPOE. Moreover, Amato et al.²⁰ conducted a study to evaluate different CPOE systems at six sites to review potential CPOE-related MEs occurring in the medication ordering phase. They reviewed 2522 ME reports and found that more than 50 percent of the MEs were related to CPOE. These findings suggest the need for a deeper understanding of the UCs to CPOE implementation is needed.

Following the release of several reports about CPOE-related MEs, numerous researchers have aimed to systematically investigate the faults behind such errors. It is crucial to explore the possible factors that hamper the safety and effectiveness of CPOE to achieve its fundamental purpose (i.e. avoiding medical errors and improving patient safety). Experts postulate that CPOE is not just about implementation of information technology; rather, it is a major organizational process change that requires a fundamental shift in the everyday practice of virtually all clinical providers. CPOE implementation is about redesigning complex clinical processes and changing administrative and clinical policies.^21,22 Implementing a new intervention such as CPOE can significantly alter workflow, create more (or different) work, and change communication patterns.²³ Thus, CPOE developers and implementers must address several factors, including organizational readiness for change (i.e. articulating policies, building proper infrastructure) and individual willingness and preparedness to adopt the new intervention (i.e. change resistance, time constraints, and training requirements), before implementing such a system.²⁴

The literature contains a variety of recommendations for developing a CPOE system and maintaining it to be as error free as possible. However, each set of recommendations focuses on a specific element, such as addressing technical flaws by designing/redesigning and developing CPOE system, conducting usability tests to evaluate the ease of use of the system, and addressing human–computer interactions. The aim of this work is to review the recent evidence of CPOE-related MEs among hospitalized patients and to address sociotechnical factors associated with CPOE implementation and use.

Methodology

Study design

Pre-searching strategy

The determination of the type of literature review is largely dependent on many factors related to the primary/original data to be reviewed. The inconsistency of the outputs of the primary data requires the use of a specific approach for successfully analyzing them.

Initially, the term “CPOE-related MEs” has been defined as an error that resulted from or caused by the direct implementation and use of CPOE, and would not happen, or unlikely to occur, if the medication had been prescribed with the traditional way (i.e. handwritten orders). Then, content analysis was applied to deal with the heterogeneity of the available data on the CPOE-related MEs and sociotechnical systems–related factors. The content analysis was carried out in the following steps: (1) identifying the key concepts or theory (i.e. CPOE-related MEs that resulted from sociotechnical systems–related factors); (2) defining pre-determined coding categories that can help in managing and organizing the data (i.e. year of publication, language, healthcare settings, evidence on CPOE-related MEs, sociotechnical-related factors, etc.); (3) designing the sampling plan (i.e. defining inclusion/exclusion criteria); (4) collecting materials/literature to be reviewed and units of analysis (i.e. title, author and year, citation, purpose of the article, etc.); (5) determining operational definitions for each category (i.e. keywords within texts that indicate either the rate of CPOE-related MEs or specific recommendations related to a successful implemented CPOE); (6) reviewing all transcripts comprehensively and re-reading them to identify and highlight texts that describe the research question and its related answers; (7) coding the highlighted texts with the pre-defined coding categories or providing new codes if needed (i.e. when the text cannot be coded under the pre-defined codes); (8) examining the data to determine the necessity of constructing subcategories to provide a wider systematic approach for collecting and analyzing the context; (9) comparing the extent to which the data support the proposed theory; and (10) drawing summaries/conclusions from the reviewed article.

Search strategy

An integrative literature review was conducted using PubMed and Google Scholar databases for articles published in the English language between 2007 and 2019 and included the following keywords: computerized physician order entry, computerized provider order entry, medication errors, unintended consequences, health information technology, sociotechnical factors, and other recommendations for their corresponding MeSH (medical subject heading) term synonyms. The process of searching and data extraction was conducted by a single author and reviewed by another.

Inclusion and exclusion criteria

Articles that report CPOE-related MEs were critically appraised and summarized in greater detail within the text. However, studies that involved outpatients, studies that evaluated issues other than clinical or technical issues (e.g. economic burden), secondary studies and opinion reports, and studies that lacked a clear objective or methods were excluded. On the other hand, articles that discussed recommendations for CPOE system improvement, including secondary reports, were categorized based on the sociotechnical systems–related factors that expected to impact the CPOE implementation.

Data extraction

All the studies extracted using the pre-defined search strategy were scanned by title and/or abstract. Studies that matched the inclusion/exclusion criteria were read in full, critically appraised, and summarized within the context of the scope of this work (i.e. CPOE-related MEs).

Results

A total of 2228 papers were identified using PubMed and Google Scholar databases. Initially, they were screened by title and/or abstract, and after removing duplicates and irrelevant studies, this number was reduced to 288 relevant papers. We then classified them based on their use within the review into the following categories: (1) studies reporting rates and statistics about CPOE-related MEs, (2) studies reporting types of CPOE-related UCs, (3) studies reporting factors that contribute to CPOE failure, and (4) studies reporting recommendations based on addressing sociotechnical factors. Following screening and categorization, a full-text assessment was performed on the remaining articles for eligibility based on our inclusion criteria; this resulted in 106 papers that were read in full and included in this review together with 27 additional papers obtained from references (see Figure 1).

Figure 1.

PRISMA flowchart of the search plan and exclusion criteria.

CPOE-related MEs: rates and statistics

Several studies suggest that CPOE is related to an increase in the rate of MEs (see Table 1). For instance, a group of researchers³⁶ prospectively investigated the incident of CPOE-generated MEs in a surgical intensive care unit of a large tertiary hospital, considering any MEs during prescribing, administration, and documentation phases. Over 4 months, 53.6 percent (286) of the total prescriptions contained at least one error related to CPOE; the majority (82.7%) of errors occurred within the documentation phase. Another retrospective study³ compared the rates of MEs pre- and post-implementation of the CPOE system in a large tertiary pediatric hospital. MEs were identified and categorized based on their occurrence in the medication use process and their severity (see Table 2).³³ CPOE implementation was associated with a significant increase in the total MEs (2226 vs. 1741, p < 0.01), and after conducting further analysis of the error severity, there was a significant increase in the category (A and E).³ Villamañán et al.⁴ conducted a relatively short observational study to evaluate the rate of CPOE-related MEs in a large tertiary care hospital. In a 1-month period, 714 MEs were detected, where 77.7 percent were related to CPOE use, and 22.3 percent were unrelated to system use (i.e. occurred while using paper-based prescriptions). The main type of CPOE-related MEs was selection error (20.9%). Improper data placement (20.3%) was another source of errors (11.5% were due to data entry into a wrong location and 8.8% were due to inappropriate drug allergy registration).

Table 1.

Summary of recent studies of CPOE-related medication errors.

Author	Sample	Main outcomes	Comments
Shah et al.²⁶	• Pharmacist survey responses	• 227 error types eliminated: • Legibility (78%) • Time delay (6%) • Knowledge deficit (6%) • Wrong patient (5%) • Handoff (4%) • Inaccuracy in drug regimen (1%) • 199 new error types observed: • Inaccuracy in drug regimen (67%) • Computer system (10%) • Wrong patient (7%) • Input error (3%) • Time delay (6%) • Knowledge deficit (6%) • Duplicate orders (2%)	• Observational descriptive analysis • Pre- and post-implementation of e-prescription • 6 months post-implementation
Lichtner et al.²⁷	• 827 reported incidents • Pediatric oncology hospital	• Medication-related incidents: 651 (79%) • Prescribing: 35% • Administration: 25% • Dispensing: 5% • Monitoring: 5% • Others: 27%	• Retrospective analysis of voluntary-reported incidents related to eMMS • Post-implementation • 18-month period
Wang et al.²⁸	45,624 FDA manufacturer and user facility device experience reports	• 152 HIT-induced MEs: • Unsafe condition: 52 (34.21%) • Near misses: 23 (15.13%) • Incidents: 77 (50.66%) • Medical devices involved in HIT-induced MEs: • Software, transmission and storage, patient data: 58 • Medical computers and software: 39 • Calculator/data processing module, for clinical use: 14 • Medical device data system: 10 • Others: 30	• Retrospective data analysis • 9-year time period
Pontefract et al.²⁹	• 28,526 prescriptions • 2422 patients	• Total opportunities for error: 21,138 • Error rate: • Pre-implementation: 5% • Post-implementation 4% (p < 0.001)	• Prospective descriptive analysis • Pre- and post-implementation • Multicenter • 78 Pre-defined criteria for prescribing errors • 18-month period
Van de Vreede et al.³⁰	5826 medication-related incidents	• Total system-related incident: 93 • Human factors and unfamiliarity or training: 65 • Cross-encounter or hybrid system error: 20 • System build: 4 • Site build error: 2 • Hardware malfunction: 2	• Retrospective analysis of MEs related to eMMS • Multicenter • Pilot study • Predetermined error classification system • 3-month period
Kadmon et al.³¹	• 109 children • CPOE in 2007 (after implementation) • CPOE in 2015 (some changes in system)	• Rate of CPOE-related MEs in 2007: 1.4% • Rate of CPOE-related MEs in 2015: 3.2%, p = 0.03 versus 2007 • Rate of CPOE-related MEs in 2016: 1%, p < 0.0001 versus 2015 • Incorrect prescription that could cause harm to the patient if in 2007: 0.7% • Incorrect prescription that could cause harm to the patient in 2015: 2%, p < 0.01 versus 2007 • Incorrect prescription that could cause harm to the patient in 2016: 0.7%, p < 0.01 versus 2015	• Retrospective observational • PICU setting • Results of 2007 obtained from previous study
Vélez-Díaz-Pallarés et al.³²	• 117 patients • 2930 E-Rx	• Prevalence of technology-induced prescription errors: 3.65% (117) • Classification of error severity: • Did not reach the patient: 73% • Reached the patient: 19% • Reached the patient and required monitoring: 5% • Reached and harmed the patient: 4%	• Prospective study • 6-month period • No control group • Involved older patients only (>65 years old)
Amato et al.²⁰	2522 ME reports	• Total CPOE-related MEs: 1308 (51.9%) • Errors facilitated by CPOE: 171 (13.1%) • Errors failed to be prevented by CPOE: 1137 (86.9%)	• Prospective observational • Reports obtained from six different sites • Inpatient and outpatient setting
Schwartzberg et al.³	Medication orders	• CPOE implementation was accounted for increasing the total MEs (2226 vs. 1741, p < 0.01) • A significant increase in the severity category A and E post-CPOE implementation (p < 0.01)	• Retrospective study • Comparison of the rate of MEs pre- and post-implementation of CPOE • MEs identified and categorized based on the stage of medication use process • MEs were categorized by their severity (A–F) that defined by the (NCC-MERP) Taxonomy of Medication Errors³³
Sethuraman et al.³⁴	• 7268 Rx before CPOE • 7292 E-Rx after CPOE with EMAS	• Rate of MEs before: 10.4% • Rate of MEs after: 7.3%, p < 0.01 versus before • Total alerts generated by EMAS: 959 • Sensitivity of EMAS alerts in identifying errors: 45.1% • Specificity of EMAS alerts in identifying errors: 57% • Alerts corrected by physicians to avoid errors: 187 • Alerts overridden by physicians and caused errors: 88	• Prospective observational • Pediatrics emergency department • ME rate reduced but remained high • EMAS system maybe a source of MEs
Hincapie et al.³⁵	484 e-prescribing incidents	• Unsafe conditions regarding E-Rxs: 239 (49%) • Near misses: 215 (44%) • Incidents reached to the patients: 30 (6%)	• Retrospective data analysis • 18-month period
Cho et al.³⁶	• 503 patients • 534 E-Rx	• 53.6% of E-Rx contained at least one error • 82.7% of errors occurred in the documentation phase • 93.6% of errors categorized as omitted information	• Prospective observational • 4-month period • Two ICU units
Villamañán et al.⁴	85.857 drug prescriptions	• 714 MEs were detected • 77.7% of MEs were related to CPOE	• Longitudinal, observational, quantitative study • One-month study of a 3-year implemented CPOE • High rate of failures associated with CPOE during the ordering process
Leung et al.³⁷	A total of 2000 charts were reviewed	• The rate of preventable ADEs decreased by 34.0% (10.6/100 vs. 7.0/100 admissions; p = 0.007) • The rate of the potential ADEs increased by 29.5% (44.4/100 vs. 57.5/100 admissions; p < 0.001).	• Prospective before and after CPOE implementation • The study conducted from January 2005 to September 2010 at five community hospitals. • Adult patients
Wetterneck et al.³⁸	Data were collected for 4147 patient-days pre-implementation and 4013 patient-days post-implementation.	• Duplicate medication ordering errors increased after CPOE implementation (Post: 167 errors (8.1%) vs. Pre: 48 errors (2.6%); p < 0.0001).	• Most post-implementation duplicate orders were either for the identical order or for the same medication.
Singh et al.³⁹	55,992 E-Rx	• 532 E-Rx contained MEs • 44.9% of errors related to dosing • 24.4% of errors related to duration • 20.5% of errors related to administration • 68.3% of errors occurred in inpatient settings, p < 0.001 versus outpatient • Pharmacist interventions required the use of the free-text comment in 88.9% of the total prescriptions to finalize them	• Prospective study • 4-month period • Inpatient and outpatient setting
Estellat et al.⁴⁰	• 399 prescriptions through CPOE • 222 patients	• Total potential prescribing error: 95 • CPOE-related errors: 47 (49%) • Treatment adaptation: 1 • Incomplete order: 26 • Wrong route or unit: 14 • Dose: 4 • Drug omission/duplication: 2	• Prospective observational analysis • 5-day period
Weant et al.⁴¹	Reports of MEs	• The number of MEs reported increased during the initial transition period post-CPOE implementation • Most of them were originated from physicians • Majority of MEs did not result in harm to the patient.	• Observational study • Over a 2-year period • CPOE implementation in the neurosurgical ICU • Pharmacy departments and pharmacy residents play an important role for the ease and safety of CPOE implementation

CPOE: computerized physician/provider order entry; PICU: pediatric intensive care unit; NCC-MERP: National Coordinating Council for Medication Error Reporting and Prevention; EMAS: electronic medication alert system; E-Rx: electronic prescription; MEs: medication errors; ICU: intensive care unit; ADEs: adverse drug events; HIT: health information technology; eMMS: electronic medication management systems; FDA: Food and Drug Administration.

Table 2.

NCC-MERP index for categorizing medication errors.

Category	Definition
A	A circumstance or event that has the capacity to cause error with no harm to the patient
B	An error that occurred, but did not reach the patient
C	An error occurred, reached the patient, but did not cause patient harm
D	An error occurred, reached the patient, and required monitoring to confirm that it did not result in patient harm and/or the patient did not require intervention to preclude harm
E	An error occurred, reached the patient, and may have contributed to or resulted in temporary harm and required intervention
F	An error occurred, reached the patient, and may have contributed to or resulted in temporary harm and required initial or prolonged hospitalization
G	An error occurred that may have contributed to or resulted in permanent patient harm
H	An error occurred that required intervention necessary to sustain life
I	An error occurred that may have contributed to or resulted in the patient’s death

Source: Adapted from the NCC-MERP.³³

NCC-MERP: National Coordinating Council for Medication Error Reporting and Prevention.

Also, inappropriate use of the free-text field (caused by duplication or discrepancies between the structured template and the free-text comments) was another common type of CPOE-related errors and was accounted for 14.4 percent. In a study by Kadmon et al.,³¹ the authors estimated that 42–58 percent of all MEs were facilitated by CPOE implementation (1.8% and 0.4% of all prescriptions in 2015 and 2016, respectively). In another study that was conducted to quantify medication incidents related to electronic medication management systems (eMMS), a total of 93 of 5826 medication-related incidents were reported over a 3-month period (May–July 2014). Factors that facilitated eMMS-generated errors were suggested to be related to human factors (e.g. unfamiliarity or inadequate training), the application of cross-encounter or hybrid system, hardware malfunction, system build, and site build factors.³⁰

Types of UCs related to CPOE

Improving the quality of healthcare requires a full understanding of the UCs of CPOE, most particularly that they are unpredictable and they might jeopardize the safety of patient care when they occur, resulting in increased healthcare costs and greater patient and/or family suffering. For instance, CPOE might influence clinical workflow, change clinician behavior, and might generate new classes of errors.^9,23,26 Identifying major types of errors resulting from CPOE implementation and addressing their causes and prevalence can help in developing implementation and precaution strategies for safe and effective CPOE.

Ash et al.¹⁵ categorized and defined types of CPOE-related UCs for a better understanding of their nature to monitor and manage them. Accordingly, consequences related to CPOE were categorized into intended and unintended, desirable and undesirable, direct and indirect, and “two-sided” consequences (either desirable or undesirable, depending on how they are perceived). Campbell et al.⁴² identify several types of unintended adverse consequences related to CPOE and classify them into nine major categories: (1) more/new work for clinicians, (2) unfavorable workflow issues, (3) never-ending system demands, (4) problems related to paper persistence, (5) untoward changes in communication patterns and practices, (6) negative emotions, (7) generation of new kinds of errors, (8) unexpected changes in the power structure, and (9) overdependence on the technology (see Table 3). Undesirable consequences related to CPOE could be direct consequences (e.g. error and security concerns) or indirect consequences (e.g. issues related to alerts, workflow, and ergonomics). Once the UCs are highlighted and fully understood, strategies and recommendations can be proposed and applied to prevent their occurrences.^23,43,44

Table 3.

The nine categories for unintended consequences.

Type of unintended consequences	Description/example
More/new work for clinicians	CPOE can increase the workload and can add more work for clinicians due to several factors: • Requiring clinicians to do more steps compared to the old practice (e.g. entering new information; responding to excessive alerts; spending extra time in completing non-routine; complex orders) • System poor design, poor integration with other HIT, and slowness • Users training (i.e. insufficient training hinders the proper use) • Some forcing functionalities (i.e. forcing users to complete all steps)
Workflow issues	Not reflecting actual clinical practices when developing CPOE may lead to inconsistencies between the intended CPOE results and the actual workflow processes (policies and procedures, human–computer interaction issues, and situation awareness issues) and may lead to ineffective or dysfunctional workflows
Never-ending system demands	CPOE requires hardware technically advanced enough to support clinical software. Also, it requires an ongoing maintenance, training, and support efforts
Paper persistence	Using handwritten paper to store data for later entry into the computer and using paper as a portable, disposable, computer output display medium for quick reference use during the workday are error-prone practice
Changes in communication patterns and practices	CPOE has been noticed to change traditional communication patterns (interpersonal conversations and clarifying orders) and create “illusion of communication” (the belief that entering data/order into the system ensures that the right person will see it and act upon it). The time interval between discussing and entering an order may lead to omission or delayed entry of some relevant orders.
Negative emotions	Shifting from traditional paper-based to computer-based orders can evoke strong emotional responses as users struggle to adapt to the new technology
Generation of new kinds of errors	CPOE implementation may generate new kinds of errors, as incorrect text entries, confusing orders representing in option presentation and selection methods
Changes in the power structure	CPOE system may change the configuration control and may reduce the power or autonomy of physicians, while the power of the nursing staff, information technology specialists, and administration is increased
Overdependence on the technology	System failures and downtime create wreak havoc when backup systems are not readily available.

CPOE: computerized physician/provider order entry; HIT: health information technology.

This table is designed based on Campbell et al.⁴²

Factors contributing to CPOE-related MEs

Researchers⁴⁵ have postulated that the “generating new kinds of errors” category is the most serious type of CPOE-related UCs. These errors have been identified as e-iatrogenesis—patient harm caused by the use of HIT. Some examples of errors that can lead to patient harm include entering orders on a wrong patient’s file, ineffective communication among nurses and physicians (i.e. not knowing if an order has been generated), loss of information during care transitions, and overlapping medication orders.²³ Different risk factors, such as flaws in system design, unrecognized discontinued orders, unintended loss of orders, drop-down menu selection errors, and issues related to alerts (lack of drug dosing alerts, generation of inappropriate dosing alerts, and inappropriate duplication alerts), can be associated with the use of CPOE.^46–48 Lichtner et al.²⁷ classified eMMS-related incidents into four categories: (1) technical issues (e.g. bugs), (2) issues related to user experience, (3) unanticipated workflow issues (e.g. duplicated or multiple orders, system incompatibilities with existing workflows), and (4) missing safety features (e.g. absence of specific decision support). Table 4 summarizes several risk factors associated with CPOE use.

Table 4.

Risk factors associated with CPOE use.

Flaws in system design
Poor CDSS design	• UCs result from factors related to the CDSS actual content or presentation of the information
Order duplication	• Confusing alert content • High false-positive alert rate • Poor medication database design prevents duplicate order alerts from being triggered • Data display (i.e. difficulty in reviewing existing orders)
Alert fatigue	• Excessive alerting might result from duplicate alerting, wrong timing of alerts, and false-positive due to poor sensitivity and specificity • Alerts that are not patient tailored have little clinical significance and are too long or difficult to interpret • Alerts with low-priority information can lead to user frustration and medication delay • Deficiency in indication-specific dose range alerts • Generation of inappropriate dosing alerts • Dysfunction of drug duplication alerts
Poor system interface design	• Failure in the process of entering and retrieving information • Absence of acknowledging and generating messages of canceled orders • Mismatch between interface and use context could lead to juxtaposition errors.
Poor CPOE design causes errors in selecting orders, editing orders, or performing new tasks	• Juxtaposition errors, when selecting wrong order from drop-down menu • Entering orders on a wrong patient’s file • Editing pre-defined order sentence incorrectly • Constructing orders and enabling free-text orders; this may facilitate orders with misspellings and inconsistencies between narrative text and structured order information
Limitations in CPOE functions and screen display features	• Poor display of drug names on the screen (e.g. fonts, truncation, capitalization). • Displaying inconsistent names (e.g. brand vs. generic names, names of combination products) • Confusing when reviewing, selecting, and ordering medications
Sociotechnical factors
CPOE affects the communications among healthcare providers	• Placing orders remotely can lessen the physical and visual interaction and may lead to communication gap • Difficulties in integrating medication orders may lead to workarounds of informal interactions
Lack of organizational readiness for change and poor system usability	• Sudden change in workflow, short implementation time, inadequate training, inconsistency between the system and the actual workflow, and lack of proper infrastructure

CDSS: clinical decision support system; CPOE: computerized physician/provider order entry; UCs: unintended consequences.

One of the primary factors contributing to CPOE UCs is system functionality and design.^46,49,50 For instance, after analyzing the different types of CPOE-related UCs, researchers⁵¹ found that many of them were related to the design of clinical decision support system (CDSS). CDSS is defined as “a computer-based system that offers passive and active referential information as well as reminders, alerts, and guidelines” (p. 524).⁵² CPOE shows a high level of effectiveness when combined with CDSS.⁵³ The main goal of integrating the two systems is preventing adverse drug events and optimizing medication safety.⁵⁴ When both systems are designed and implemented carefully, they can improve the quality of clinical decisions by providing reminders or recommendations at the point of care.^10,55,56 However, poor system design can lead to many errors, where the UCs related to CDSS can be generated by the actual content of the decision support module (e.g. elimination or shifting of human roles, ongoing necessity for updating the algorithm-based rules, wrong or misleading CDSS content) or by the presentation of the information (i.e. the way alerts and other modes of decision support are presented to the users).⁵¹

Order duplication, defined as “two or more active orders for the identical medication regardless of dose” (p. 379),⁵⁷ is one of the CPOE-related UCs that might be generated by poorly designed systems. Wetterneck et al.³⁸ evaluated the duplicate medication orders before and after CPOE implementation; they found that duplication in medication orders increased significantly from 48 to 167 errors (p < 0.0001) after CPOE implementation. They identified several contributing factors, including ordering practice and computer availability (i.e. orders can be placed from different locations through computers by different providers on rounds within minutes); entering duplicate orders during care transitions and changing shifts (i.e. poor communication and hand-offs); poor system and medication database design (i.e. confusing alert content, high false-positive alert rate, and CDSS algorithms missing true duplicates); data display (i.e. difficulty in reviewing existing orders); and issues related to local CDSS design (i.e. medications in order sets defaulted as ordered).³⁸ Moreover, Magid et al.⁵⁷ studied duplicate orders by analyzing different aspects of the process such as specific drugs, role of prescribers, the source of the duplicate orders, and the workflow. They found that duplicate orders were generated mostly when the same medications were ordered by different prescribers than by the same prescribers, and from order sets than from single orders.

Another CPOE problem resulting from poor system design is alert fatigue—a phenomenon by which users tend to ignore many of these alerts, including the ones that notify them of potentially serious errors.¹⁰ This mainly occurs when CPOE is integrated with a CDSS that generates an excessive number of warning alerts, which can lead to a high alert override rate and decision support overload.^{6,47,48,58–61} Excessive alerting can result from duplicate alerts, wrong timing of alerts, and false-positive alerts (non-relevant alerts due to poor sensitivity and specificity).^62,63 Van der Sijs et al.⁶⁴ found that medication safety alerts were overridden by clinicians in 49–96 percent of cases (e.g. overdose, contraindications, allergies, and life-threatening drug–drug interactions). Another study evaluating the CDSS by monitoring CPOE order override rates shows that 87 and 81 percent of high-severity drug-drug interaction and drug allergy alerts, respectively, were overridden.⁶⁵ A study by Strom et al.⁶⁶ highlights the hard-stop alert feature designed to reduce undesired prescribing orders; this is defined as “an alert appeared as a pop-up window that notified the clinician that the order could not be processed because of a significant potential drug-drug interaction” (p. 1579).⁶⁶ Their study revealed that despite the effectiveness of hard-stop alerts in reducing prescribing errors, they caused a delay in medication orders, which led to an early termination of the study for ethical reasons (i.e. potential for harm in the intervention). In fact, alerts that are not patient tailored have little clinical significance and are too long or difficult to interpret; also, alerts with low-priority information cause user frustration and slow down the medication ordering task.⁶² Moreover, several studies highlight other issues related to alert design, such as a lack of a dosing alert (deficiency in indication-specific dose range alerts), generation of inappropriate dosing alerts, and dysfunction of drug duplication alerts (order duplications do not trigger alerts due to the design of the medication database and the CDSS algorithms that miss true duplicates).^38,67–71

Another CPOE problem resulting from system design flaws is poor system interface. The design of the CPOE system interface is required to be consistent with the physicians’ task behavior and decision-making processes; otherwise, it may obscure the appropriate order entry strategy and may lead to inefficient workflow and user frustration.⁶² Moreover, researchers⁷² found that some UCs were associated with the absence of adequate system interface functionality, such as the absence of acknowledging and generating messages of canceled orders. Poor CPOE interface design can result in usability problems, workflow interruptions, and subsequent MEs.^62,73

Another risk resulting from CPOE use is entering orders on a wrong patient’s file⁷⁴ or selecting wrong orders. Such a problem is known as “juxtaposition errors” and can be considered an additional flaw in CPOE system design.⁴² Errors resulting from drop-down selection menu are considerably common and may contribute to serious MEs.^75–77 Researchers found that such system-related errors occur when prescribers need to select items from drop-down menus, construct orders, edit information within the system, and/or perform new tasks not previously required.⁷⁸ Selection errors were the most frequent type of error, accounting for 43 percent of total system-related errors, while editing orders accounted for 21 percent. In addition, failure to complete new tasks found to be accounted for 32 percent of all system-related errors.⁷⁸ It is important to note that such a failure is generally related to the limited functionality of the system, which leads to developing workaround processes and adopting hybrid systems (i.e. using computer-based orders along with some paper-based format).⁷⁹ Constructing or editing orders may also create risks and jeopardize patient care and safety. Enabling prescribers to use free-text instructions to enter orders may lead to inconsistencies between narrative text and structured order information.^78,80,81 For instance, Zhou et al.⁸² found that around 17 percent of free-text medication order entries contained misspellings and led to additional errors.

An additional CPOE-related factor that might contribute to MEs is the way drug names are displayed.⁸³ A number of MEs were found to be related to screen displays and workflow issues.⁸⁴ Displaying inconsistent names, especially the display of brand versus generic drug names, is a source of confusion when reviewing, selecting, and ordering medications.⁸³

Ineffective communications among nurses and physicians is another problem associated with CPOE use. The ability to enter orders through the system remotely can lessen the physical and visual interactions and may contribute to communication gaps.⁸⁵ A mixed method qualitative study was conducted by Pirnejad et al.⁸⁶ to evaluate the impact of CPOE on nurse–physician communication; the results showed that the communication was impacted due to difficulties in integrating medication orders among nurses and physicians, which eventually led to developing workarounds of informal interactions that represent risks for MEs.

The aforementioned factors may lead to MEs resulting from a lack of organizational readiness for change and poor system usability.⁵⁰ For instance, unanticipated and sudden changes in workflow, short implementation time, insufficient users training, inconsistency between the new system and the actual clinical workflow, and lack of proper infrastructure for incubating new interventions (e.g. physical space layout, policies and procedures, software and hardware capabilities) may contribute to generating new errors and affecting the workflow; this can lead to cognitive overload, user resistance, and eventual patient safety problems.^15,62,87,88 Thus, addressing the way users interact with the system, how they perceive it, and to what level they accept it and understand it are critical elements for developing and implementing CPOE effectively and safely.

Published recommendations based on addressing sociotechnical factors

Adopting new technologies in a complex and dynamic setting such as healthcare is challenging. Implementing CPOE as a major intervention for reducing the rate of MEs was found to be a source of errors in many studies. Some may consider looking at the failure of CPOE as an opportunity for improvement by addressing potential factors that might lead to ineffective CPOE design and subsequent poor implementation. Like other health information systems, successful implementation of CPOE requires careful planning, consistent monitoring, and ongoing commitment. Identifying process flaws enables developers and implementers to successfully build a system capable of achieving its intended fundamental purpose, that is, reducing medication and order errors and maximizing patient safety.

The literature contains extensive valuable recommendations that may help in developing and implementing a successful and safe CPOE environment, such as system improvement, applying human factors, and considering sociotechnical factors. The majority of the recommended remedies were proposed by researchers based on different models of analyses of HIT interventions, such as the diffusion of innovations theory,^89,90 human factors engineering and usability testing,⁹¹ human factors and ergonomics models,⁹² the acceptance and use of technology theory,⁹³ and interactive sociotechnical analysis (ISTA).⁹⁴ It is worth noting that there are several frameworks that address errors induced by HIT. Borycki et al.⁹⁵ have elegantly summarized a number of frameworks, models, and theories that would significantly impact safe HIT development and implementation. They organized the frameworks and models into three main groups based on works emerged on other literatures: (1) human factors models and frameworks (e.g. the theoretical and conceptual cognitive taxonomy of medical errors),⁹⁶ (2) organizational behavior and sociotechnical models and frameworks (e.g. the thematic hierarchical network model),¹⁵ and (3) software engineering models and frameworks (e.g. continuum of methods for diagnosing technology-induced errors).⁹⁷ Parts of their work are summarized in Table 5.

Table 5.

Models and frameworks for considering technology-induced errors.

Type of literature	Framework/model	Potential causes of errors
The human factors literature	The theoretical and conceptual cognitive taxonomy of medical errors.⁹⁶	Errors related to execution, evaluation, action specification, action execution, perception, and interpretation
	Framework for considering the origins of technology-induced errors.⁹⁸	Errors related to government policy, model organizations, vendors, healthcare organizations, and individuals
	Multifaceted cognitive methodology for characterizing cognitive demands of systems.⁹⁹	Heavy cognitive demands on users
	A methodological framework for describing how to use simulations to identify and predict technology-induced errors.¹⁰⁰	Interface design and incorrect content display
	Framework for integrating clinical and computer-based simulations to predict technology-induced errors.¹⁰¹	Interface design and incorrect content display
The organizational behavior/sociotechnical literature	Thematic hierarchical network model.¹⁵	Interface design
	Eight-dimensional model of sociotechnical challenges involved in design, development, implementation, use, and evaluation of HIT in healthcare.¹⁰²	Poor system design, development, or configuration
	The interactive sociotechnical analysis (ISTA) framework.⁹⁴	Elimination of informal interactions and redundant checks, limited interface design, poor representation of workflow, poor communication, and loss of feedback
	Input–output model of unintended consequences.¹⁰³	Interface design, implementation, poor training/support, poor fit with workflow and decision-making, interoperability, and legislative and regulatory changes
The software engineering literature	Continuum of methods for diagnosing technology-induced errors.⁹⁷	Interface design, poor system–organization fit, poor workflow
The software engineering literature	Framework for selecting health information systems to prevent error.¹⁰⁴	Poor system-organization fit, procurement processes

HIT: health information technology.

This table is designed based on Borycki et al.⁹⁵

System improvement

Improving the system design by assessing the existing ordering process and applying modifications and/or customizations of different functionalities based on the organizational needs is one of the recommended solutions.¹⁰⁵ CPOE effectiveness can be improved by enhancing the integrated CDSS.⁵⁵ For instance, selection errors can be minimized by shortening the drop-down menu and/or enhancing some of the system functionalities, such as adding new mouse trigger designs, safeguards, colors, bolded fonts, and tall-man lettering.^46,106,107 In another study, a group of researchers¹⁰⁸ postulated that patient selection errors could be reduced by manipulating CPOE interface design, such as highlighting the selected item after the selection is made, and providing visual feedback of related information. In addition, building a more sophisticated CDSS that is capable of checking free text and providing dosing support features and indication-based alerts can help minimize CPOE-related UCs.^46,109,110 Filtering the sensitivity and improving specificity of CDSS to provide prioritized alerts according to clinician preferences is another function that can be added to minimize alert fatigue.^{59,61,69,111–115}

Considering human factors

Applying human factors engineering—a field of study that focuses on equipment/system/process design and analysis while addressing human characteristics, capabilities, limitations, and interactions with system’s components, processes, and work environment—in which addressing cognitive complexity, enhancing system usability (i.e. user-friendliness and usable user interface), and enhancing user interface design could help in facilitating the safe and effective implementation of any new HIT.^{46,99,116–120}

Sociotechnical factors

Considering sociotechnical aspects is highly recommended in literature for addressing medical errors related to HIT implementation (i.e. understanding the relationship between human and computer/processes by addressing technical features, such as hardware, software, and techniques, as well as social factors, such as culture, acceptance, and environment).^121–123 For instance, a study by Salahuddin et al.¹²⁴ highlights five major sociotechnical factors influencing the safe use of HIT, such as knowledge, system quality, task stressor, organization resources, and teamwork. One of the most comprehensive conceptual models that has been proposed is the eight-dimensional sociotechnical model: (1) hardware and software computing infrastructure; (2) clinical content; (3) human–computer interface; (4) people; (5) workflow and communication; (6) internal organizational policies, procedures, and culture; (7) external rules, regulations, and pressures; and (8) system measurement and monitoring (see Table 6).¹⁰² It addresses a wide range of factors that should be carefully covered throughout all the CPOE adoption phases (planning, designing, developing, implementing, using, and monitoring). This important framework helps in identifying HIT-related contributing factors. It also assists developers and implementers in proactively planning and monitoring the CPOE implementation process by addressing sociotechnical factors that present challenges in the adoption of these systems.^28,102 For instance, the first dimension can guide organizations to assess their technical infrastructure (i.e. the availability and capabilities of software and hardware) when they intend to adopt new interventions, such as CPOE. The second dimension concerns the availability and quality of the clinical knowledge and content and how it should be collected, managed, and employed effectively and safely; it guides organizations to configure specific software requirements, such as controlled vocabulary and order sets. The third dimension focuses on the human–computer interface, where it guides organizations to address system usability and how clinicians interact with system components; this helps organizations with ongoing assessments of system acceptance and usability (e.g. system navigation feasibility, applications response time, and screen display and alert notification simplicity). The fourth dimension concerns people who are involved in CPOE adoption phases; this is critical for addressing individuals’ needs, their capabilities and limitations, their level of acceptance, and how they perceive new interventions. The fifth dimension focuses on addressing communication and workflow processes, particularly where these guide organizations to ensure the new system does not negatively affect patient care delivery, and how the system must fit the clinical workflow (i.e. modifying the system to match the clinical workflow, and vice versa). The sixth dimension addresses the internal organizational aspects (e.g. policies, procedures, resource allocation), and it guides organizations to assess their readiness to adopt the new intervention by addressing challenges in the organizational culture and developing policies and procedures that are consistent with the clinical workflow in the presence of CPOE. The seventh dimension concerns the enablers and identifies constraints that are enforced by external bodies and regulators regarding the design, development, implementation, use, and monitoring of CPOE in the healthcare setting. These may include privacy and confidentiality legislation and fee-for-service policies. The final dimension focuses on ensuring ongoing measurement and monitoring to help organizations identify issues, monitor system usability, and measure the system impact on the overall workflow and outcomes, including the measurement of the rates and types of MEs following CPOE implementation.^102,125

Table 6.

The eight dimensions of the sociotechnical model.

The Dimension	Examples
Hardware and software computing infrastructure	It includes analysis of the technical infrastructure: • Hardware (e.g. monitors, printers, keyboards, medical or imaging devices) • Software at all levels, operating system and application (e.g. network and data storage capability) • Other machines/devices, and software required for computing functioning (e.g. high-capacity air conditioning system, the batteries that form the uninterruptable power supply)
Clinical content	It concerns with the quality and continuum of the data and information that is stored in the system. The content includes the following: • Controlled vocabulary in the drop-list menu • The logic required to fire a specific alert • Clinical data about patient’s condition (e.g. laboratory test results) • Non-clinical data (e.g. demographic information)
Human–computer interface	Hardware and software that enable users to interact with the system. It needs to be constantly reviewed, refined, and then designed within the context of the user interaction model and the ergonomic aspects of the interface.
People	This represents human and the way they perceive the system; it includes any person who is involved in any stage of the system cycle (planning, designing, developing, implementing, using, and evaluating). This person could be system developers, trainers, clinicians, and also patients.
Workflow and communication	Designing a system that is capable to help in facilitating a collaborative patient care. This requires a careful assessment of the “clinical workflow” where the system must be modified to match the workflow, and if impossible, the workflow needs to be changed to match the system.
Internal organizational policies, procedures, and culture	Readiness for change that concerns with the organization’s internal structures, policies, and procedures (e.g. budget allocation, policies concerning with data backup and downtime, policies and procedures concerning with managing HIT processes). This dimension also highlights the necessity for ensuring that policies and procedures are consistent with the actual clinical workflow in the presence of the HIT.
External rules, regulations, and pressures	It concerns with external bodies that might facilitate or constrain the HIT implementation plan.
System measurement and monitoring	Measuring and monitoring the system regularly is a critical aspect for identifying its flaws and applying remedies. Four important issues need to be addressed: 1. Availability—the extent to which features and functions are available and ready for use 2. Usability—how the clinicians are using the different features and functions 3. System effectiveness—how the system is impacting patient care and whether the anticipated outcomes are achieved 4. System failure—measuring and monitoring to identify and document unintended consequences generated from using the system

CPOE: computerized physician/provider order entry; HIT: health information technology.

Hierarchical decomposition and focusing on individual units/elements in the system is not effective in assessing HIT.¹²⁶ Thus, the eight-dimensional model described above should be used as an interactive standard that focuses on how each dimension is depending on and interacting with each other to comprehensively evaluate the effectiveness, usability, and safety of the system.^102,125 Developing an effective implementation plan requires a comprehensive model to identify weaknesses. Identifying process flaws can work as lessons learned and permit developers and implementers to re-think about how to prevent them from recurring. This model has been found to be a successful tool for evaluating the system design, development, implementation, use, and ongoing monitoring.¹⁰² Addressing the sociotechnical dimensions at each stage of the CPOE implementation plan may help software developers, healthcare organizations (implementers), and end users to proactively identify process flaws and accordingly reduce the risk of patient harm.^123,127,128

Discussion

This integrative review has been written to highlight the burden of UCs of CPOE within the context of sociotechnical factors. It reviewed a wide range of research that focuses on illustrating the extent of CPOE-related MEs. The article summarized several studies that identified and classified CPOE-related UCs; recapped a number of research studies that investigated different factors and flaws associated with the CPOE design, implementation, and use; highlighted different factors associated with CPOE implementation; and underscored the importance of evaluating HIT by employing the eight dimensions of a sociotechnical model. Despite the large body of research that supports the effectiveness of CPOE in reducing MEs, some researchers have found that CPOE can facilitate the occurrence of MEs or become a source of them. For instance, a study by Cho et al.³⁶ found that nearly half of the medication orders entered through CPOE contained at least one error. Another group of researchers³ found a significant increase in the rate of MEs after implementing CPOE. The persistence of high rate of MEs, despite the use of CPOE, has triggered an alarm about the safety and effectiveness of a system that was proposed to effectively reduce the rate of errors and substantially contribute toward improving patient safety. The CPOE-related MEs drove many researchers and experts in the field to study this matter more comprehensively assisted by the classification into nine major categories of CPOE-related UCs developed by Campbell and associates.⁴² Also, different factors and flaws associated with the CPOE use and implementation have been studied to identify a successful CPOE design and implementation.

Identifying the problem sets up only part of the eventual solution. Categorizing the UCs of CPOE and understanding its flaws have guided many researchers to study and develop strategies for minimizing the occurrences of such consequences, including those contributing to MEs. As an example, remedies for technical and social-related issues have been proposed to assist in addressing this matter. One of the recommended models documented for analyzing sociotechnical related factors intended to evaluate HIT is the eight-dimensional framework previously described.¹⁰² The domains in this model are intended to assist healthcare organizations and system developers address challenges related to sociotechnical factors at each stage of system adoption (i.e. planning, designing, developing, implementing, using, and monitoring).

Evaluating CPOE systems needs to be started at the planning stage, as soon as CPOE has been recommended for implementation. This is initiated by conducting a comprehensive assessment of the organizational readiness, which starts by assessing the appropriate infrastructure (e.g. hardware, software, the physical layout, policies, and procedures) and addressing the perspectives and involvement of healthcare professionals and other stakeholders (e.g. patients, policymakers, and system suppliers).^24,129 Adopting a new technology such as CPOE requires a strong culture in which people understand and accept the change; this may require strong top management support for allocating sufficient resources for the implementation process (e.g. staffing, procurement, maintenance, and training). In addition, having organizational champion leaders who can drive implementation, change culture, and bridge knowledge with developers would be an important step to consider.¹³⁰ Also, it is important that the system’s design serves its intended purposes, especially in the clinical arena. For instance, clinical content stored in the system including a controlled vocabulary, specific order sets, and rules/alerts needs to be carefully designed—more specific and sensitive—to minimize possible related errors. To carefully conduct this step, the organization should clearly define knowledgeable users to be involved in each stage of the CPOE implementation plan, including clinicians, trainers, and technical experts. Moreover, providing periodic training sessions, understanding users’ needs, and addressing their perceptions before implementing the system are important for detecting and correcting problems, minimizing change resistance, and increasing user acceptance and satisfaction. These actions need to be regularly monitored and assessed for meeting user expectations and maintaining system feasibility.^131,132 Addressing user–computer interactions is another factor that must be carefully addressed. For instance, an organization should conduct usability tests, simulation scenarios, and pilot trials to observe how users perceive the system and how they interact with it. These actions may help identify possible system gaps and may help in evaluating the system’s usability.¹³³ Any new system must be carefully monitored and measured on an ongoing basis,¹³⁴ which may give implementing organizations the opportunity to identify and address system flaws and hidden safety risks.¹²⁷ Besides, implementers should not underrate the importance of monitoring MEs post-CPOE implementation. For instance, it is crucial to constantly review and measure the rates and types of MEs prevented, not prevented, and/or MEs facilitated by the use of CPOE during the following years after CPOE implementation to identify CPOE problems and find solutions. By measuring and evaluating system success or failure, implementers can consider a sufficient timeline for planning and implementing the CPOE system to allow for adequate system adaptation.¹³⁵ One more aspect organizations must address is the external laws and regulations, where forces outside the organization’s control might suppress or negatively impact the CPOE implementation process. Addressing them early may provide implementers the opportunity of not rushing the adoption of new systems without a full assessment for both the system and the organization’s preparedness to address external impacts. To drive the effective and safe use of CPOE functions, organizations should automate with caution. Full assessment of the current culture and infrastructure together with a comprehensive understanding of users’ needs are important keys for a successful and safe CPOE system.

For future research and new HIT projects, organizational leadership must recognize that focusing on technical capabilities alone to implement safe HIT is not sufficient; other factors, including organizational culture, work environment, and human–computer interactions, must be considered. Evaluating sociotechnical factors by utilizing the eight-dimensional model might contribute to safe and effective CPOE implementation. This can help implementers proactively address their preparedness to implement CPOE by assessing the internal culture and developing policies and procedures consistent with the clinical workflow in the presence of the change under consideration. Also, addressing the technical infrastructure, users’ needs, and how they interact with CPOE is crucial to success. By adopting and employing the eight-dimensional model, organizations can understand how each dimension can influence one another, and they can work to build a comprehensive and careful implementation plan based on their assessments. To this end, organizations must recognize that implementing CPOE is multifaceted and requires careful consideration of internal and external factors that might impede its effectiveness.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Manal Elshayib

References

Kohn

Corrigan

Donaldson

, et al. To err is human: Building a safer health system. Washington, DC: National Academies Press, 2000.

Phillips

Christenfeld

Glynn

LM.

Increase in US medication-error deaths between 1983 and 1993. Lancet 1998; 351(9103): 643–644.

Schwartzberg

Ivanovic

Patel

, et al. We thought we would be perfect: medication errors before and after the initiation of computerized physician order entry. J Surg Res 2015; 198(1): 108–114.

Villamañán

Larrubia

Ruano

, et al. Potential medication errors associated with computer prescriber order entry. Int J Clin Pharm 2013; 35(4): 577–583.

Wachter

RM.

Expected and unanticipated consequences of the quality and information technology revolutions. JAMA 2006; 295(23): 2780–2783.

Ash

Berg

Coiera

Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004; 11(2): 104–112.

Gray

Fernandes

Van Aarsen

, et al. The impact of computerized provider order entry on emergency department flow. CJEM 2016; 18(4): 264–269.

Johnson

Lehmann

and Council on Clinical Information Technology. Electronic prescribing in pediatrics: toward safer and more effective medication management. Pediatrics 2013; 131(4): e1350–e1356.

Cowan

Literature review and risk mitigation strategy for unintended consequences of computerized physician order entry. Nurs Econ 2013; 31(1): 27–3111.

10.

Ranji

Rennke

Wachter

RM.

Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review. BMJ Qual Saf 2014; 23(9): 773–780.

11.

Institute of Medicine (IOM). Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press, 2001.

12.

Nuckols

Smith-Spangler

Morton

, et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3: 56.

13.

Radley

Wasserman

Olsho

, et al. Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assoc 2013; 20(3): 470–476.

14.

Alhanout

Bun

Retornaz

, et al. Prescription errors related to the use of computerized provider order-entry system for pediatric patients. Int J Med Inform 2017; 103: 15–19.

15.

Ash

Sittig

Dykstra

, et al. Categorizing the unintended sociotechnical consequences of computerized provider order entry. Int J Med Inform 2007; 76(Suppl. 1): S21–S27.

16.

Al-Rowibah

Younis

Parkash

The impact of computerized physician order entry on medication errors and adverse drug events. J Health Care Finance 2013; 40(1): 93–102.

17.

Elsaid

Garguilo

Collins

CM.

Chemotherapy e-prescribing: opportunities and challenges. Integr Pharm Res Pract 2015; 4: 39–48.

18.

Schiff

Amato

Eguale

, et al. Computerised physician order entry-related medication errors: analysis of reported errors and vulnerability testing of current systems. BMJ Qual Saf 2015; 24(4): 264–271.

19.

Santell

Hicks

McMeekin

, et al. Medication errors: experience of the United States pharmacopeia (USP) MEDMARX reporting system. J Clin Pharmacol 2003; 43(7): 760–767.

20.

Amato

Salazar

Hickman

, et al. Computerized prescriber order entry–related patient safety reports: analysis of 2522 medication errors. J Am Med Inform Assoc 2017; 24(2): 316–322.

21.

Drazen

Kilbridge

Metzger

, et al. A primer on physician order entry. Oakland, CA: California HealthCare Foundation, 2000, https://www.chcf.org/wp-content/uploads/2017/12/PDF-CPOEreport.pdf (accessed April 2018).

22.

Upperman

Staley

Friend

, et al. The introduction of computerized physician order entry and change management in a tertiary pediatric hospital. Pediatrics 2005; 116(5): e634–e642.

23.

Ash

Sittig

Poon

, et al. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2007; 14(4): 415–423.

24.

Farre

Heath

Shaw

, et al. How do stakeholders experience the adoption of electronic prescribing systems in hospitals? A systematic review and thematic synthesis of qualitative studies. BMJ Qual Saf 2019; 28(12): 1021–1031.

25.

Moher

Liberati

Tetzlaff

, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009; 339: b2535.

26.

Shah

Galt

Fuji

KT.

Error types with use of medication-related technology: a mixed methods research study. Res Social Adm Pharm 2019; 15(12): 1480–1483.

27.

Lichtner

Baysari

Gates

, et al. Medication safety incidents in paediatric oncology after electronic medication management system implementation. Eur J Cancer Care 2019; 28(6): e13152.

28.

Wang

Liang

Kang

, et al. Understanding health information technology induced medication safety events by two conceptual frameworks. Appl Clin Inform 2019; 10(1): 158–167.

29.

Pontefract

Hodson Slee

Shah

, et al. Impact of a commercial order entry system on prescribing errors amenable to computerised decision support in the hospital setting: a prospective pre-post study. BMJ Qual Saf 2018; 27(9): 725–736.

30.

Van de Vreede

McGrath

de Clifford

Review of medication errors that are new or likely to occur more frequently with electronic medication management systems. Aust Health Rev 2018; 43: 276–283.

31.

Kadmon

Pinchover

Weissbach

, et al. Case not closed: prescription errors 12 years after computerized physician order entry implementation. J Pediatr 2017; 190: 236–240.e2.

32.

Vélez-Díaz-Pallarés

Díaz

Gramage Caro

, et al. Technology-induced errors associated with computerized provider order entry software for older patients. Int J Clin Pharm 2017; 39(4): 729–742.

33.

National Coordinating Council for Medication Error Reporting Prevention (NCC-MERP). Taxonomy of medication errors, 1998, http://www.nccmerp.org/sites/default/files/taxonomy2001-07-31.pdf (accessed April 2018).

34.

Sethuraman

Kannikeswaran

Murray

, et al. Prescription errors before and after introduction of electronic medication alert system in a pediatric emergency department. Acad Emerg Med 2015; 22(6): 714–719.

35.

Hincapie

Warholak

Altyar

, et al. Electronic prescribing problems reported to the Pharmacy and Provider ePrescribing Experience Reporting (PEER) portal. Res Social Adm Pharm 2014; 10(4): 647–655.

36.

Cho

Park

Choi

, et al. Understanding the nature of medication errors in an ICU with a computerized physician order entry system. PLoS ONE 2014; 9(12): e114243.

37.

Leung

Keohane

Amato

, et al. Impact of vendor computerized physician order entry in community hospitals. J Gen Intern Med 2012; 27(7): 801–807.

38.

Wetterneck

Walker

Blosky

, et al. Factors contributing to an increase in duplicate medication order errors after CPOE implementation. J Am Med Inform Assoc 2011; 18(6): 774–782.

39.

Singh

Mani

Espadas

, et al. Prescription errors and outcomes related to inconsistent information transmitted through computerized order entry: a prospective study. Arch Intern Med 2009; 169(10): 982–989.

40.

Estellat

Colombet

Vautier

, et al. Impact of pharmacy validation in a computerized physician order entry context. Int J Qual Health Care 2007; 19(5): 317–325.

41.

Weant

Cook

Armitstead

JA.

Medication-error reporting and pharmacy resident experience during implementation of computerized prescriber order entry. Am J Health Syst Pharm 2007; 64(5): 526–530.

42.

Campbell

Sittig

Ash

, et al. Types of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc 2006; 13(5): 547–556.

43.

Ash

Sittig

Dykstra

, et al. Exploring the unintended consequences of computerized physician order entry. Stud Health Technol Inform 2007; 129(Pt 1): 198–202.

44.

Ash

Sittig

Dykstra

, et al. The unintended consequences of computerized provider order entry: findings from a mixed methods exploration. Int J Med Inform 2009; 78(Suppl. 1): S69–S76.

45.

Weiner

Kfuri

Chan

, et al. “e-Iatrogenesis”: the most critical unintended consequence of CPOE and other HIT. J Am Med Inform Assoc 2007; 14(3): 387–388; discussion 389.

46.

Brown

Mulcaster

Triffitt

, et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assoc 2017; 24(2): 432–440.

47.

Jacobs

Hallstrom

Hart

, et al. Lessons from a successful implementation of a computerized provider order entry system. J Pediatr Pharmacol Ther 2007; 12(2): 102–114.

48.

Galt

Fuji

Kaufman

, et al. Health information technology use and patient safety: study of pharmacists in Nebraska. Pharmacy 2019; 7(1): 7.

49.

Khajouei

Jaspers

MW.

CPOE system design aspects and their qualitative effect on usability. Stud Health Technol Inform 2008; 136: 309–314.

50.

Puaar

Franklin

BD.

Impact of an inpatient electronic prescribing system on prescribing error causation: a qualitative evaluation in an English hospital. BMJ Qual Saf 2018; 27(7): 529–538.

51.

Ash

Sittig

Campbell

, et al. Some unintended consequences of clinical decision support systems. AMIA Annu Symp Proc 2007; 2007: 26–30.

52.

Bates

Kuperman

Wang

, et al. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2003; 10(6): 523–530.

53.

Kazemi

Ellenius

Pourasghar

, et al. The effect of computerized physician order entry and decision support system on medication errors in the neonatal ward: experiences from an Iranian teaching hospital. J Med Syst 2011; 35(1): 25–37.

54.

Vardi

Efrati

Levin

, et al. Prevention of potential errors in resuscitation medications orders by means of a computerised physician order entry in paediatric critical care. Resuscitation 2007; 73(3): 400–406.

55.

Beeler

Bates

Hug

BL.

Clinical decision support systems. Swiss Med Wkly 2014; 144: w14073, https://smw.ch/article/doi/smw.2014.14073

56.

Kuperman

Bobb

Payne

, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 2007; 14(1): 29–40.

57.

Magid

Forrer

Shaha

. Duplicate orders: an unintended consequence of computerized provider/physician order entry (CPOE) implementation. Appl Clin Inform 2012; 3(4): 377–391.

58.

Colpaert

Decruyenaere

Computerized physician order entry in critical care. Best Pract Res Clin Anaesthesiol 2009; 23(1): 27–38.

59.

Slight

Beeler

Seger

DL.

A cross-sectional observational study of high override rates of drug allergy alerts in inpatient and outpatient settings, and opportunities for improvement. BMJ Qual Saf 2017; 26(3): 217–225.

60.

Zenziper Straichman

Kurnik

Matok

, et al. Prescriber response to computerized drug alerts for electronic prescriptions among hospitalized patients. Int J Med Inform 2017; 107: 70–75.

61.

Riedmann

Jung

Hackl

, et al. Development of a context model to prioritize drug safety alerts in CPOE systems. BMC Med Inform Decis Mak 2011; 11: 35.

62.

Khajouei

Jaspers

MW.

The impact of CPOE medication systems’ design aspects on usability, workflow and medication orders. Methods Inf Med 2010; 49(1): 3–19.

63.

Van der Sijs

Kowlesar

Aarts

, et al. Unintended consequences of reducing QT-alert overload in a computerized physician order entry system. Eur J Clin Pharmacol 2009; 65(9): 919–925.

64.

Van der Sijs

Aarts

Vulto

, et al. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 2006; 13(2): 138–147.

65.

Lin

Payne

Nichol

, et al. Evaluating clinical decision support systems: monitoring CPOE order check override rates in the Department of Veterans Affairs’ Computerized Patient Record System. J Am Med Inform Assoc 2008; 15(5): 620–626.

66.

Strom

Schinnar

Aberra

, et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med 2010; 170(17): 1578–1583.

67.

Jani

Barber

Wong

IC.

Paediatric dosing errors before and after electronic prescribing. Qual Saf Health Care 2010; 19(4): 337–340.

68.

Scharnweber

Lau

Mollenkopf

, et al. Evaluation of medication dose alerts in pediatric inpatients. Int J Med Inform 2013; 82(8): 676–683.

69.

Stultz

Porter

Nahata

MC.

Sensitivity and specificity of dosing alerts for dosing errors among hospitalized pediatric patients. J Am Med Inform Assoc 2014; 21(e2): e219–e225.

70.

Moxey

Robertson

Newby

, et al. Computerized clinical decision support for prescribing: provision does not guarantee uptake. J Am Med Inform Assoc 2010; 17(1): 25–33.

71.

Bell

Garfield

Khosla

, et al. Mixed methods study of medication-related decision support alerts experienced during electronic prescribing for inpatients at an English hospital. Eur J Hosp Pharm 2019; 26(6): 318–322.

72.

Schreiber

Sittig

Ash

, et al. Orders on file but no labs drawn: investigation of machine and human errors caused by an interface idiosyncrasy. J Am Med Inform Assoc 2017; 24(5): 958–963.

73.

Khajouei

Wierenga

Hasman

, et al. Clinicians satisfaction with CPOE ease of use and effect on clinicians’ workflow, efficiency and medication safety. Int J Med Inform 2011; 80(5): 297–309.

74.

Levin

Docimo

SG.

“I meant that med for Baylee not Bailey!”: a mixed method study to identify incidence and risk factors for CPOE patient misidentification. AMIA Annu Symp Proc 2012; 2012: 1294–1301.

75.

Tolley

Forde

Coffey

, et al. Factors contributing to medication errors made when using computerized order entry in pediatrics: a systematic review. J Am Med Inform Assoc 2018; 25(5): 575–584.

76.

Caruso

Gittelman

Widecan

, et al. Pediatric emergency department discharge prescriptions requiring pharmacy clarification. Pediatr Emerg Care 2015; 31(6): 403–408.

77.

Walsh

Landrigan

Adams

, et al. Effect of computer order entry on prevention of serious medication errors in hospitalized children. Pediatrics 2008; 121(3): e421–e447.

78.

Westbrook

Baysari

, et al. The safety of electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated with two commercial systems in hospitals. J Am Med Inform Assoc 2013; 20(6): 1159–1167.

79.

Van der Sijs

Rootjes

Aarts

. The shift in workarounds upon implementation of computerized physician order entry. Stud Health Technol Inform 2011; 169: 290–294.

80.

Magrabi

Day

, et al. Errors and electronic prescribing: a controlled laboratory study to examine task complexity and interruption effects. J Am Med Inform Assoc 2010; 17(5): 575–583.

81.

Palchuk

Fang

Cygielnik

, et al. An unintended consequence of electronic prescriptions: prevalence and impact of internal discrepancies. J Am Med Inform Assoc 2010; 17(4): 472–476.

82.

Zhou

Mahoney

Shakurova

, et al. How many medication orders are entered through free-text in EHRs? A study on hypoglycemic agents. AMIA Annu Symp Proc 2012; 2012: 1079–1088.

83.

Quist

Hickman

Amato

, et al. Analysis of variations in the display of drug names in computerized prescriber-order-entry systems. Am J Health Syst Pharm 2017; 74(7): 499–509.

84.

Schiff

Hickman

Volk

, et al. Computerised prescribing for safer medication ordering: still a work in progress. BMJ Qual Saf 2016; 25(5): 315–319.

85.

Wright

Frey

Scherer

, et al. Maintaining excellence in physician nurse communication with CPOE: a nursing informatics team approach. J Healthc Inf Manag 2006; 20(2): 65–70.

86.

Pirnejad

Niazkhani

Van der Sijs

, et al. Evaluation of the impact of a CPOE System on Nurse-physician communication. Methods Inf Med 2009; 48(4): 350–360.

87.

Jeon

Taneva

Kukreti

, et al. Toward successful migration to computerized physician order entry for chemotherapy. Curr Oncol 2014; 21(2): e221–e228.

88.

Maslove

Rizk

Lowe

HJ.

Computerized physician order entry in the critical care environment: a review of current literature. J Intensive Care Med 2011; 26(3): 165–171.

89.

Rahimi

Timpka

Vimarlund

, et al. Organization-wide adoption of computerized provider order entry systems: a study based on diffusion of innovations theory. BMC Med Inform Decis Mak 2009; 9: 52.

90.

Rogers

EM.

Diffusion of preventive innovations. Addict Behav 2002; 27(6): 989–993.

91.

Beuscart-Zéphir

Aarts

Elkin

Human factors engineering for healthcare IT clinical applications. Int J Med Inform 2010; 79(4): 223–224.

92.

Patel

Kannampallil

TG.

Human factors and health information technology: current challenges and future directions. Yearb Med Inform 2014; 9: 58–66.

93.

Venkatesh

Morris

Davis

, et al. User acceptance of information technology: toward a unified view. MIS Q 2003; 27: 425–478.

94.

Harrison

Koppel

Bar-Lev

Unintended consequences of information technologies in health care—an interactive sociotechnical analysis. J Am Med Inform Assoc 2007; 14(5): 542–549.

95.

Borycki

Kushniruk

Bellwood

, et al. Technology-induced errors. The current use of frameworks and models from the biomedical and life sciences literatures. Methods Inf Med 2012; 51(2): 95–103.

96.

Zhang

Patel

Johnson

, et al. A cognitive taxonomy of medical errors. J Biomed Inform 2004; 37(3): 193–204.

97.

Borycki

Keay

Methods to assess the safety of health information systems. Healthc Q 2010; 13: 47–52.

98.

Borycki

Kushniruk

Keay

, et al. Framework for identifying where technology- induced errors come from. Stud Health Technol Inform 2009; 148: 181–187.

99.

Horsky

Kaufman

Oppenheim

, et al. Framework for analyzing the cognitive complexity of computer-assisted clinical ordering. J Biomed Inform 2003; 36(1–2): 4–22.

100.

Borycki

Kushniruk

AW.

Where do technology- induced errors come from? Towards a model for conceptualizing and diagnosing errors caused by technology. In: Kushniruk

Borycki

(eds) Human, social and organizational aspects of health information systems. Hershey, PA: IGI global, 2008, pp. 148–166.

101.

Borycki

Kushniruk

Keay

, et al. Toward an integrated simulation approach for predicting and preventing technology-induced errors in healthcare: implications for healthcare decision-makers. Healthc Q 2009; 12: 90–96.

102.

Sittig

Singh

A new technical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care 2010; 19(Suppl. 3): i68–i74.

103.

Bloomrosen

Starren

Lorenzi

, et al. Anticipating and addressing the unintended consequences of health IT and policy: a report from the AMIA 2009 health policy meeting. JAMIA 2011; 18(1): 82–90.

104.

Kushniruk

Beuscart-Zéphir

Grzes

, et al. Increasing the safety of healthcare information systems through improved procurement: toward a framework for selection of safe healthcare systems. Healthc Q 2010; 13: 53–58.

105.

Cresswell

Mozaffar

Shah

, et al. Approaches to promoting the appropriate use of antibiotics through hospital electronic prescribing systems: a scoping review. Int J Pharm Pract 2017; 25(1): 5–17.

106.

Slight

Eguale

Amato

, et al. The vulnerabilities of computerized physician order entry systems: a qualitative study. J Am Med Inform Assoc 2016; 23(2): 311–316.

107.

Wei

, et al. Alternative computer mouse trigger designs in computerized physician order entry (CPOE) system to reduce clinicians’ drop-down menu selection errors. Int J Ind Ergonom 2019; 71: 14–19.

108.

Taieb-Maimon

Plaisant

Hettinger

, et al. Increasing recognition of wrong-patient errors through improved interface design of a computerized provider order entry system. Int J Hum-Comput Int 2018; 34(5): 383–398.

109.

Galanter

Bryson

Falck

, et al. Indication alerts intercept drug name confusion errors during computerized entry of medication orders. PLoS ONE 2014; 9(7): e101977.

110.

Galanter

Falck

Burns

, et al. Indication-based prescribing prevents wrong-patient medication errors in computerized provider order entry (CPOE). J Am Med Inform Assoc 2013; 20(3): 477–481.

111.

Ammenwerth

Hackl

Riedmann

, et al. Contextualization of automatic alerts during electronic prescription: researchers’ and users’ opinions on useful context factors. Stud Health Technol Inform 2011; 169: 920–924.

112.

Horsky

Phansalkar

Desai

, et al. Design of decision support interventions for medication prescribing. Int J Med Inform 2013; 82(6): 492–503.

113.

Stürzlinger

Hiebinger

Pertl

, et al. Computerized physician order entry-effectiveness and efficiency of electronic medication ordering with decision support systems. GMS Health Technol Assess 2009; 5: Doc07.

114.

Zaal

Jansen

Duisenberg-van Essenberg

, et al. Identification of drug-related problems by a clinical pharmacist in addition to computerized alerts. Int J Clin Pharm 2013; 35(5): 753–762.

115.

Czock

Konias

Seidling

, et al. Tailoring of alerts substantially reduces the alert burden in computerized clinical decision support for drugs that should be avoided in patients with renal disease. J Am Med Inform Assoc 2015; 22(4): 881–887.

116.

Etchells

Bailey

Biason

, et al. Human factors in action: getting “pumped” at a nursing usability laboratory. Healthc Q 2006; 9: 69–74.

117.

Khajouei

de Jongh

Jaspers

MW.

Usability evaluation of a computerized physician order entry for medication ordering. Stud Health Technol Inform 2009; 150: 532–536.

118.

Middleton

Bloomrosen

Dente

, et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20(e1): e2–e8.

119.

Salvendy

. Handbook of human factors and ergonomics. 3rd ed. New York: Wiley, 2006.

120.

Siewert

Hochman

MG.

Improving safety through human factors engineering. Radiographics 2015; 35(6): 1694–1705.

121.

Aarts

Van der Sijs

CPOE, alerts and workflow: taking stock of ten years research at Erasmus MC. Stud Health Technol Inform 2009; 148: 165–169.

122.

Quan

Rossos

, et al. It’s not about pager replacement: an in-depth look at the interprofessional nature of communication in healthcare. J Hosp Med 2013; 8(3): 137–143.

123.

Castro

Buczkowski

Hafner

JM.

The contribution of sociotechnical factors to health information technology–related sentinel events. Jt Comm J Qual Patient Saf 2016; 42(2): 70–76.

124.

Salahuddin

Ismail

Hashim

, et al. Sociotechnical factors influencing unsafe use of hospital information systems: a qualitative study in Malaysian government hospitals. Health Informatics J 2019; 25(4): 1358–1372.

125.

Sittig

Ash

JS.

On the importance of using a multidimensional sociotechnical model to study health information technology. Ann Fam Med 2011; 9(5): 390–391.

126.

Rouse

WB.

Health care as a complex adaptive system: implications for design and management. Bridge 2008; 38(1): 17–25.

127.

Singh

Sittig

DF.

Measuring and improving patient safety through health information technology: the Health IT Safety Framework. BMJ Qual Saf 2016; 25(4): 226–232.

128.

Odukoya

Chui

MA.

e-Prescribing: characterisation of patient safety hazards in community pharmacies using a sociotechnical systems approach. BMJ Qual Saf 2013; 22(10): 816–825.

129.

Van Dort

Zheng

Baysari

MT.

Prescriber perceptions of medication-related computerized decision support systems in hospitals: a synthesis of qualitative research. Int J Med Inform 2019; 129: 285–295.

130.

Feldman

Buchalter

Hayes

LW.

Health information technology in healthcare quality and patient safety: literature review. JMIR Med Inform 2019; 7(1): e11320.

131.

Korb-Savoldelli

Boussadi

Durieux

, et al. Prevalence of computerized physician order entry systems–related medication prescription errors: a systematic review. Int J Med Inform 2018; 111: 112–122.

132.

Sittig

Ash

Guappone

, et al. Assessing the anticipated consequences of computer-based provider order entry at three community hospitals using an open-ended, semi-structured survey instrument. Int J Med Inform 2008; 77(7): 440–447.

133.

Altuwaijri

Bahanshal

Almehaid

Implementation of computerized physician order entry in National Guard Hospitals: assessment of critical success factors. J Family Community Med 2011; 18(3): 143–151.

134.

Bubalo

Warden

Wiegel

, et al. Does applying technology throughout the medication use process improve patient safety with antineoplastics. J Oncol Pharm Pract 2014; 20(6): 445–460.

135.

Yen

McAlearney

Sieck

, et al. Health Information Technology (HIT) adaptation: refocusing on the journey to successful HIT implementation. JMIR Med Inform 2017; 5(3): e28.