Abstract
Complex socio-technical health information systems (HIS) issues can create new error risks. Therefore, we evaluated the management of HIS-related errors using the proposed human, organization, process, and technology-fit framework to identify the lessons learned. Qualitative case study methodology through observation, interview, and document analysis was conducted at a 1000-bed Japanese specialist teaching hospital. Effective management of HIS-related errors was attributable to many socio-technical factors including continuous improvement, safety culture, strong management and leadership, effective communication, preventive and corrective mechanisms, an incident reporting system, and closed feedback loops. Enablers of medication errors include system sophistication and process factors like workarounds, variance, clinical workload, slips and mistakes, and miscommunication. The case management effectiveness in handling the HIS-related errors can guide other clinical settings. The potential of HIS to minimize errors can be achieved through continual, systematic, and structured evaluation. The case study validated the applicability of the proposed evaluation framework that can be applied flexibly according to study contexts to inform HIS stakeholders in decision-making. The comprehensive and specific measures of the proposed framework and approach can be a useful guide for evaluating complex HIS-related errors. Leaner and fitter socio-technical components of HIS can yield safer system use.
Introduction
Although health information technologies (HIT), including health information systems (HIS), can potentially improve patient safety, they pose possible error risks due to their poor design, implementation, and application. The safe application of HIS has yet to be realized due to its technical and organizational challenges. Literature calls for work on identifying and assessing the nature, scope, and prevalence of HIT-related safety issues and their prevention and mitigating mechanisms.1,2 New harmful and potential HIS errors are emerging proportionately with software advancement and increasing medical complexity because of new software risk, its contextual use risk, and emerging diseases and treatments.1,3,5 “Alarming reports of deaths, serious injuries, and near misses — thousands of them — tied to software glitches, user errors, or other system flaws have piled up for years in government and private repositories”. 4 This emergence requires further work pertinent to safe HIS use in various healthcare settings to evaluate the contextual1,2,4,6–8 impact. 4 In short, medical errors (MEs) are highly attributed to poor HIS, but they are preventable through incident analyses that may not be realized otherwise.9–11
Evaluating safe HIS implementation and application will aid in developing safe systems.12,13 However, rigorous, real-time mechanisms for regular assessment of HIS safety and hazards are missing in organizations, 2 while limited published evidence on the risk level significantly results in the knowledge gap on HIT safety. Research on HIS safety is complex and has various methodological challenges, including its definitions; models, frameworks, and evidence for error causes; monitoring; and prevention methods and learning about them. 2 Moreover, an effective process is a prerequisite for quality services, 14 as most problems are attributed to the process and not people or technology. 15 However, published reports on HIS effect on workflows remain limited and unreliable. 12 Studies on MEs related to process factors are also scarce. 16 Two critical challenges in managing a clinical process are combining its technical and organizational components and selecting improvement methods to align the process with organizational goals. 14 Evaluating and establishing the integration above are imperative to address these challenges. 17 Process change is challenging because it identifies current flaws and requires difficult change efforts to improve them.14,18 Further research on HIS and process alignment is needed in work organization problems, cooperative work problem, 12 and international coordination and knowledge exchange. 2 Therefore, we attempted to address these gaps and challenges using a process approach, that is, the business process management (BPM) and Lean methods, in theorizing an evaluation framework and identifying valuable lessons from error management practices and interventions, particularly those related to HIS and medication process, as well as the ways through which they can be applied in other clinical contexts.
We have attempted to include a comprehensive conceptualization for HIS-related error, which covers formative and summative evaluations and latent and active errors. These dimensions were carefully selected based on the relevant theories and validated in multiple case studies to justify the framework scope, importance, novelty, parsimony, and falsifiability.19,20 The proposed framework, known as the human, organization, process, and technology-fit (HOPT-fit), enhanced the existing HOT-fit framework significantly by amplifying the process factors and including safety and quality aspects in the study purpose and context.21,22 In addition to process measures, Lean and Lean IT methods are used; their principles on efficiency and quality include and complement many HIS efficiency and patient safety measures, respectively, and socio-technical fit. The main contributions of this paper are: (1) introducing a new evaluation framework for HIS-related errors, (2) evaluating HIS-related error management in clinical settings from the HOPT-fit aspects, and (3) proposing the end-to-end process analysis for medication use (MU), error control, and error risk using the Lean method. This paper contributes to the literature on the phenomenon by proposing and validating the applicability of a new framework using a case study evaluation and presenting its detailed findings from the socio-technical, error management, and quality improvement (Lean) perspectives related to HIS-related errors. To our knowledge, such study purposes and contexts are scarce. We engaged theory by adopting and integrating the three aforementioned up-front theories into our study and developed theoretical contributions by extending previous works to our proposed framework. 23
We organized the paper as follows. The next section reviews the literature on HIS-related errors, followed by a critical review of existing error evaluation dimensions, measures, and frameworks. The third section provides the theoretical framework that builds upon the review, proposing an extended framework encompassing the safety features and emphasizing the process view and Lean method. The fourth section describes the case study methods, followed by the evaluation findings from the framework application. We then discussed the findings in the light of the proposed framework and presented our conclusions in the last section.
Overview of HIS-related errors
Error may be defined as “an unintended act (either of omission or commission) or one that does not achieve its intended outcome”. 24 Errors are attributed to active failures (human approach) and latent conditions (systems approach). 25 Active failures are “unsafe acts by people in direct contact with the patient or system”. 25 They can be classified into slips and mistakes. Slips are caused by a lack of focus due to competing sensors or lethargic or emotional distress, while mistakes denote wrong choices resulting from a lack of experience or skill. Latent conditions represent a system’s unavoidable “resident pathogens” from strategic decisions. They may stay inactive for years before colliding with active failures and internal triggers to lead toward error incidents. In contrast to unpredictable active failures, latent conditions can be identified and mitigated to prevent error, leading to a proactive—instead of reactive—risk management approach. Error can be caused by nearly any interaction within the care system, at any time of care provision, and in all healthcare locations. 12 Specifically, 26 we defined HIS-related errors as “medical errors (MEs) that are related to the overall stages of system development life cycle (SDLC) and HIS interactions with its socio-technical aspects”. 23 More errors occur at the post-implementation stage,1,5,11,25,27 but most prevention measures at the pre-implementation stage indicate the importance of early prevention.5,16 Minimizing MEs has been a pervasive global effort.6,9,28
Most system errors are attributable to significant failures of its development, including design and implementation.28,29 Therefore, strategizing and evaluating the quality and safety of technology during its development can avoid the costly effect of widespread diffusion and harmful incidents.5,7,30,31 Several error models6,25,32,33 and studies 2 also included latent errors to address the socio-technical challenges and enable a robust risk management strategy. 34 Furthermore, fragmented evaluation tools for IT design and implementation impede the endeavor to provide an overview of safe IT use.34,35 Although active and latent failures contribute to errors, 25 less attention is given to the latter.36,37 Accordingly, latent failures should be avoided by clearly understanding the HIS risk at the design stage, which is influenced by the previous SDLC stages. 38
Socio-technical aspects of HIS
Human and computer errors alone do not contribute to MEs; instead, MEs are caused by poor interactions and fit among technology, process, organization, and people.7,12,39,40 The perspective of multi-stakeholders also results in various adoption attitudes and practices. 14 Nevertheless, effective healthcare brings them together. Therefore, understanding the socio-technical interactions of HIS on risk, workflow, and safety can optimize the potential of HIS on patient safety.2,12,41 ME evaluation and management have three perspective trends: (1) end-to-end workflow, (2) process management, and (3) SDLC. Error risk can be identified and mitigated along SDLC phases, process steps, and process management. As the causes of MEs are multidisciplinary and multifactorial, 42 evaluators can take a systems approach toward comprehensive management and effective program controls. The approach includes an interdisciplinary safety team from multiple levels of organizational units, from individual to group, 6 with skilled and unskilled/new staff, risk management personnel,25,42 and the entire process. 43 Furthermore, process, setting, and overall organization should ideally be integrated.25,43,44
System development
According to system theory, 31 system development unintendedly results in reverse salient or problems pertinent to the ineffectiveness or failure of system components, which when solved, can lead to future system improvement and innovation. Therefore, evaluating system development identifies the problems that create opportunities for further system enhancement, a cycle akin to continuous improvement. Integrating system development and use can generate “useful, actionable, and interesting” theory and practice on socio-technical change. 30
Plan
Unclear purpose affects system planning, which then ripples to the subsequent phases.13,45,46 Unrealistic planning,46,47 inadequate user involvement, unclear user requirements, limited and complex system modules, conflicting priorities, and detachment of local settings during system selection are disastrous and costly.48–52
Design
HIT design can prevent new errors and pose safety risks, such as poor function aid or unclear manual, poor user interface, limited data sharing, and inefficient workflow. 1 System usefulness is affected by unnecessary software complexity that does not support the process, wastes development effort, and requires extra training and support.15,46 Hence, designing and simplifying HIS may be more effective than training all clinicians to use unnecessary complex systems that yield additional cognitive tasks and misunderstanding. 12 Consequently, system design and implementation must be corrected and improved to prevent errors.24,43 Incident reports and usability testing can inform design. 6 Adoption can be accelerated by identifying usability issues that are applicable to all clinicians and locations, while safety can be improved by establishing workflow procedures for downtime. 12 Examples of safe design include automated alerts and reminders through checklists, protocols, and clinical decision support features. 24 These features can greatly assist clinicians and avoid risk; however, protective and organizational measures (education and awareness) should be performed 6 in their absence. HIT design has various challenges, including hardware and software reliability, interface usability, system interoperability, and data integrity, accessibility, and confidentiality.1,2 Meanwhile, system redesign needs to consider five barriers: (1) system complexity and lack of ownership, (2) unavailable information, (3) allowing workaround, (4) low rate of serious incidents, and (5) punishment policy that hinders reporting. 2
Implementation
Chaotic implementation strategies include insufficient training, hybrid record management, simultaneous systems conversions, delayed updates release, and continuous system modifications.46,50,53 Therefore, HIS implementation must consider specific organizational needs, hardware and software customization, and integration of new HIS into existing HIS and workflows, as well as their redesign, sufficient education and training, appropriate backup and security systems, and contingency plan.1,54 Implementation also needs to be evaluated on-site to explore the safety difference between designed and actual work.12,55 Comprehensive testing is critical in ensuring smooth HIS implementation.
Testing comprises two main methods, namely, usability inspection and usability testing, which are used to identify usability problems through systematic user interface navigation and system problems through the clinical task performance of users using HIS. 56 High system usability is associated with the system’s ability to fit with clinical tasks, as acknowledged by the task–technology fit concept. 57 Corrected errors from usability testing at the pre-implementation stage could save 78% of the cost and avoid an exponential increase in long-term costs related to future software modification and retraining users due to software changes to mitigate errors.15,58 Moreover, usability testing can be extended with clinical simulation to observe users implementing realistic, complex, and representative task scenarios in real-world settings 59 and detect possible error risks. 60 The goal is to prevent and mitigate errors before system operation, 61 allocate resources, and organize training. 62 Clinical simulations enable clinicians to evaluate how the tested systems support their tasks safely and identify cumbersome workflow and communication among system entities. 63 Error reporting and monitoring strategies can likewise be established during post-implementation for detailed analysis and safety purposes.
Use
Proper and optimum HIS use requires HIS and workflow alignment, such as examining and modifying them to ensure safety and effectiveness. 2 Possible risks of HIS use are alert fatigue (leads to ignored warnings and reminders),43,64,65 automation bias (complies with system instructions although they are against clinical practice), misuse of copy-and-paste functions, and workarounds for inconvenient or inefficient system functions.1,66 Ash et al. 28 categorized two broad error types: (1) data entry and retrieval, and (2) communication and coordination. Cheung et al. 27 classified IT-related incidents as follows: (1) the principal source of IT-related problem: machine-related error (input-output transfer) or human-machine interaction-related (input-output) error; (2) nature of the error (problem): input (data entry), output (data retrieval), and transfer (data transfer between systems); (3) IT systems; and (4) IT problems in the medication process.
Evaluation and optimization
Early, continuous, effective testing and error mitigation can reduce and prevent the range and number of possible risks with each development phase.44,63 Organizational risk management begins at HIT procurement, a critical inception period. 63 Thus, organizations must collaborate with vendors to monitor and optimize HIT for identifying, assessing, and improving care quality and safety. 2
Early approaches to evaluate technology-related errors
Selected error evaluation frameworks.
Overall, all five models feature comprehensive organization measures but less detailed measures for the three other factors, particularly technology and IS. Even the models that include technology measures provide limited measures for HIS. Human and process factors require more specific measures on IS user aspect and process management and quality, respectively. Our analysis showed that the evaluation measures overlapped and complemented one another. To complement the five models, we analyzed related models and theories, namely, socio-technical, organizational change theory, 69 BPM, 18 and Lean method, as described in the theoretical framework.
Theoretical framework
To identify structured, rigorous, and specific measures for unfolding the complexity of HIS-related errors, we extended the human, organization, and technology-fit (HOT-fit) evaluation framework13,17 by reviewing multidisciplinary literature.
Leavitt model
We adapted the Leavitt model
69
to complement the HOT-fit framework and classify HIS error measures in a structured manner. The Leavitt model features an organizational change theory that consists of four interrelated factors, namely, “human,” “task,” “technology,” and “structure”.
69
Interestingly, the four factors and the balance concept align well with the socio-technical factors and the fit concept in the HOT-fit framework. We used “process” instead of “task” to represent a wider work scope and replaced “structure” with “organization” to represent various dimensions related to healthcare institutions (Figure 1). Each factor interacts multi-dimensionally with other factors and contributes to chains of error incidents. Error sources in medical informatics (adapted from Leavitt
69
).
We extended the HOT-fit framework to the HOPT-fit framework (the new elements are represented in red in Figure 2 and asterisks (*) in Table 2)23,70 through the following: • Upgrading process measures to become a factor with dimensions (workflow, process management, and Lean) and their relationships: workflow and Lean are part of process management) due to their significant contribution to error incidents and mitigation; • Applying Lean to System Development, System Use, and Organizational Structure measures; • Adding two-way relationships and fit between process and its dimensions based on the original encapsulation of the process in organization; • Adding comprehensive error and mitigation measures in HOPT factors, some of which were validated through case study findings; • Adding dynamic “holes” in all four factors representing latent and active failures that denote errors caused by system-oriented factors that may reside over time and indirectly trigger the HIT/HIS or human who committed the error. The holes are hazard-prone when aligned together, producing an error that passes through the entire system. However, if the following layer can defeat the flaw of the previous defense layer, then hazards can be avoided because they are prevented or diverted; and • Adding closed-loop feedback that gathers lessons learned from error incidents, root cause, and mitigation strategies as an organizational learning process
71
and inputs them to a Plan-Do-Check-Act (PDCA) cycle for continuous improvement. Proposed human–organization–process–technology–fit framework. Interrelated dimensions of HIS effectiveness/safety. Notes: * = new dimension
Later subsections elaborated on the new system development measures and process factor dimensions. The HOPT-fit framework includes three life cycles: system development, process management, and Lean’s PDCA, all of which require comprehensive and efficient management based on the prescribed safety principles. 2 Resilient HIS can be realized through Lean/Lean IT and control or defense mechanisms at all four factors against inevitable latent and active failures. We defined system resiliency as the capability to identify, prevent, and mitigate errors and learn from them for further improvement. Errors would ensue if all four HOPT defense mechanisms fail to prevent the risk. The initial HOPT-fit framework was applied to guide this case study and subsequently refined based on our findings.
Process
Process is central to error failure and management because errors are commonly triggered while the process is executed. We proposed three process dimensions: workflow, process management, and process improvement using Lean methods. Workflow is a set and series of interrelated work activities, for example, physical and mental tasks humans perform to transform work inputs (resources) into outputs (items) at the different units (an individual, a team, across organizations) in a particular order or path, such as sequential or concurrent.73–75 For example, medication errors can be examined based on medication use (MU) stages and their compliance with the “five rights” of medication administration, namely, the right drug, dose, route, time, and patient. Process can be scoped, managed, and evaluated using BPM techniques, such as Suppliers, Inputs, Process, Output, Customers (SIPOC) diagram, and process modeling. 18 Process analysis synthesizes problems using various techniques, such as root cause analysis (RCA), (Ishikawa/Why/Why), issue analysis, challenges, improvements (e.g., Lean methods), and waste identification. The success of process change is attributed to change acceptance and support. 14 Similarly, user support and system adoption are likely to be influenced by improved process efficiency. 76
Process management can be assessed according to various stages, such as those of BPM, while process quality and safety can be analyzed using the Lean method. Process modeling for current process (as-is) illustrates a detailed process for error identification, while future process (to-be) shows how process can be redesigned to minimize error. For example, task design failures are related to missing human factor principles, including simplicity (to reduce cognitive workload—short-term memory, planning, and problem-solving), constraints (“forcing functions” to prohibit error-prone actions), procedural standardization, and reversible (undo) action or challenging task for irreversible action. 24 BPM emphasizes early error prevention and elimination through process improvement and automation. 77 Efficiency improvement requires outlining, implementing, and understanding the efficiency purpose, efforts, and outcome. 78 Aligning and standardizing processes with a new system, variance in work, formal and actual practices, old with new, manual, and multiple systems are challenging but imperative to avoid errors. 16 Thus, errors can also be measured from process and procedure coordination and aligning policies and new processes.12,79 Coordination is mainly related to process, terminology, and integration standard aspect. 16
Lean has been used to improve the quality of healthcare and IT (coined as Lean IT) processes. Applicable Lean principles and tools include continuous improvement, waste identification, and management. The methodologies should be used during process and system changes for optimal planning and success. 44 Ideal work can be measured against four rules 80 : (1) clear work activities; (2) simple and direct step connections; (3) simplest and most direct pathway (flow of steps); and (4) guided, direct, and timely improvements. Some of the proven successes include improved clinical outcomes, reduced error, and compliance with patient safety. 81 Lean is also designed to improve process flow and efficiency by eliminating non-value-added activities, known as wastes.82,83 Wasteful practices are associated with errors, which are classified as work variation, overload, and waste. 15 Errors are caused by and lead to other types of waste in HIS. 84 Therefore, managing all waste types is imperative to prevent errors and cause other additional waste. For example, over-processing is an unnecessary complexity that contributes to IT and business misfit and is caused by the inappropriate design of business processes and supporting IS. 15 Lean principles can be applied when identifying shortcomings that could lead to avoiding harm and maintaining patient safety and quality. 81
Lean IT is achieved “through people, process, and technology – in that order” 15 and by promoting fit among them. Socio-technical fit can be accomplished and measured with the active involvement of clinical and IT staff in both unit events. 15 Improved processes may lead to improved technology related to it or the identification of a new technology that supports the newly redesigned process. Lean IT also improves process by four rights, namely, information, time, format, and audience, 15 which overlap and complement those above five (or seven) medication administration rights.
System development quality and safety
We extended system development measures in light of the quality and safety of SDLC processes, including clarity of system purpose, feasibility study, planning, design, system selection, installation strategy, conversion style, and maintenance. Other relevant measures include project management, scheduling, momentum, user involvement, and relationships with technical people. We proposed a Lean system development adapted from established Lean software development (LSD) concepts, emphasizing “the benefits of a more flexible, iterative, lightweight development process” 85 to minimize error. Extant literature has reported various HOPT waste types in various IS uses and development.1,2,84 Development waste includes excessive IS features, relearning due to lost knowledge, partially done work, handoffs, task switching, delays, and defects. 46 Thus, focusing the LSD and system use on waste management and value can lead to efficiency, effectiveness, and safety.
The seven principles of LSD, namely, waste elimination, build-in quality, knowledge creation, commitment deferment, fast delivery, respect for people, and optimize-the-whole approach, 15 can also be adopted in healthcare organizations to enhance quality and safety. LSD is advocated for early error identification and mitigation through shorter, rapid (agile) testing iterations of a small set of system features as part of a complete PDCA cycle of problem-solving. 15 Unready parts should not be released into the flow; every needed part must be completed. 86 Proper planning (P) of each cycle means less DCA (Do, Check, Act) later 15 and fewer errors. Compared with traditional measures, Lean measures for system development are simple, visual, and immediate. 15
Study context
Organizational setting
We conducted our study at one of Japan’s largest secondary care teaching hospitals (STH), with over 1000 beds and 2000 staff members. The STH pioneered in HIS by collaborating with various Japanese vendors, directed by the head of the Medical Informatics Department (MID). The MID is small and is led by three IT-savvy clinicians managing five MI staff members. Two deputy directors also acted as the system analysts for the HIS project. The Quality Management Department (QMD) manages patient safety at the STH while the Pharmacy Department (PD) is led by one director, two vice directors, and 70 pharmacists.
Methods
Our summative evaluation used a case study approach employing qualitative methods 87 to comprehensively understand the healthcare context surrounding the management of HIS-related errors by generating detailed, insightful explanations. 88 We collected data using interviews, non-participant observations, and document/artifact analysis methods.
Study flow and participants
List of participants of the STH case study.
Methods for data acquisition and measurement
We gathered the case background information and phenomena under study and reviewed relevant theories to obtain a mental concept prior to data collection. 89 The secondary data pointed us to other literature reviews. We established good rapport between the researchers and participants. During non-participant observations and face-to-face interviews, we queried informants about past and future systems use, clinical processes, and HIS-related error management practices. The informants briefed us on HIS, medication workflow, and related HIS use. We conducted one focus group interview and nine individual, semi-structured interviews, most in English and a few in Japanese, lasting 60–90 min. We gathered the main documents related to STH, HIS, and medication error management from the hospital reports, documentation, newsletters website, internet portal, and literature. We collected data iteratively on planned occasions and spontaneously through daily clinical routines, briefings, discussions, informal encounters, social events, and email. The data were audio- and hand-recorded, transcribed, and analyzed.
Methods for data analysis
We used four data analysis techniques: coding, analytic memos (e.g., reflection notes, displays, and concept maps), and contextual and narrative analyses.90,91 Themes were analyzed deductively based on the HOPT-fit framework and the literature, while new emerging themes were analyzed inductively based on the literature. The data were systematically coded, categorized, organized, and scrutinized based on a thematic analysis. 92 We read iteratively to examine content based on our research questions, theoretical framework, methodology, and literature review. Upon familiarizing ourselves with our interview and field notes, we manually and iteratively conducted open coding by identifying themes, line by line, to group texts or concepts using theory deduction from our initial HOPT-fit framework and our theoretical background to confirm and clarify related themes. 93 We annotated the field notes by underlining, color coding, and writing comments in the margins. Next, we grouped similar themes under the main and sub-theme labels. We identified emergent themes and sub-themes and explored them further. We then conceptualized and corroborated them with literature and theory. New or emerging themes were mapped to the literature and updated on the new and refined HOPT-fit framework. We extensively reviewed theories and concepts to identify those relevant to our study context. The emergent framework was validated by ensuring that it was congruent with both empirical data and the existing framework, and further validation in the following iterations was conducted until we had sufficiently covered the essence of the study phenomenon (reached theoretical saturation) 89 . This process was ensured by illustrating the constructs and propositions of the framework. We also asserted that the case data supported our framework.
Study quality
Our study was guided by the case study and reporting protocols 94 and refined accordingly. We ensured reliability by demonstrating that the study could be replicated by recording detailed data documentation and activity logs and appropriate record keeping. Potential bias common in qualitative research was overcome via reliability testing and methodology and data triangulation. Using the four data analysis techniques, we triangulated data from multiple methods and sources. For example, a fact obtained from a clinician was cross-checked with a different clinician, organizational documentation, and observation. We conducted member checking to gather participant feedback on their evaluation reports to confirm data accuracy, reduce bias, and reveal negative evidence.95,96 We adapted the Statement on Reporting of Evaluation Studies in Health Informatics 94 to guide the comprehensive and structured reporting. To ensure the case study quality, we tested construct validity to establish the appropriate operational measures for the study concepts and minimize subjectivity by associating the data collection questions and measures with the research questions. 87 The evaluation measures of the interview questions were identified based on the research objectives and the proposed framework. The results showed that the interview questions reasonably addressed each research question.
Results
Dominant factors related to the management of HIS-related error at STH.
Organizational environment and structure
The Japanese government enforced four regulations for hospitals and clinics and provided partial financing sources for the ongoing, systematic patient safety strategies of STH
97
: • The new Patient Safety Committee: a clinical risk management committee (RMC), the QMD, and area clinical risk managers • Incident Reporting System (IRS) • Specific preventive actions and improvement • Corrective actions
Patient Safety Committee
The management team believed that error is inevitable, and so humans are not to blame. However, the organization is responsible for addressing the problem (Dr A; Nurse J). Proper and detailed planning involving multiple units ensures effective implementation (Dr D). The hospital strives to provide advanced, high-quality, and safe medical care, reflected in its holistic mechanisms (Dr F). Patients are engaged in patient safety using poems, and staff engagement is embedded in the clinical process. The Japanese culture of continuous improvement, known as Kaizen, and safety were applied right from the beginning, progressively shaping safe practices in various care aspects. “We always try to implement Kaizen” (Dr D), and MID has been continuously improving systems (Dr F). The MID director and his deputies are clinicians with strong interests in IT, obtaining informal IT education through self-study from the Internet and textbooks and constantly thinking about reducing risks through HIS functions and features.
Policy and procedure for safe MU are included in the clinical manual prepared by QMD, distributed to all staff in hard copy, via email, and STH’s website, and displayed as posters (communication). Summarized, step-by-step, and updated manuals are also available any time (Dr B). Pharmacist interventions are documented digitally as a regimen system and a graphical user manual in Pharmacy IS (PhIS), but the SOPs for some tasks are unavailable. STH has a dynamic, regularly updated patient safety policy and exerts proactive QMD efforts to disseminate information.
Incident reporting system
The intranet-based IRS provides convenient reporting access, improved report legibility, and rapid information sharing. 97 Some clinicians perceived the IRS positively regarding entering and managing incident details. However, some nurses viewed the system as cumbersome due to excessive data entry (Nurse L), which is time-consuming and inflexible. Incidents must be filled within an hour; otherwise, the entered data will be deleted and must then be re-entered. System simplification is challenging as the structured incident category in the IRS requires detailed information for error mitigation (Dr D).
Preventive actions and improvement
System details and system in use
STH became fully paperless in 2010 when it implemented full HIS. STH had hired 14 vendors to develop, purchase, customize, and integrate HIS. Nurses must use the PC at the workstations and on wheeled carts in the wards. PCs in the consultation rooms and wards are connected to Accounting Information Systems (AIS) for billing and lab tests and the Pharmacy and Radiology Departments. During consultation, real-time data are transferred to these departments. A total of 45 types of HIS support various hospital services and multiple departments, including Computerized Physician Order Entry (CPOE), Injection Order System, Order Instruction System (InstS), PhIS, Dispensing System, Radiology IS, Picture Archiving and Communication Systems (PACS), Electronic Medical Record (EMR), Laboratory IS, and IRS.
STH’s EMR is very accessible and reliable for sharing patient information. A template-based (structured data entry) EMR automatically generates natural language and Extensible Markup Language (XML) files involving 2210 templates and 120,000 items. Original images can be saved and transmitted to EMR as part of patient progress notes. A Document Archival and Communication System (DACS) was developed in-house to consolidate external documents from other hospitals, maintain readability, and view documents from multiple systems. The Intensive Care Unit (ICU) uses a purchased Clinical Care IS (CCIS) that includes a drug order function. The Medical Corporation System (MCS) facilitates information exchange between STH and other hospitals. STH also has a complex, integrated HIS for MU (Figure 3). Integrated medication use systems.
System development (human dimension)
The purpose of HIS is aligned with its requirements and design to improve patient care efficiency, service level, and safety. The MID staff, mostly clinicians, obtained user requirements by interviewing clinicians to design and improve the system themselves (Head Nurse K). The MID staff simplified user requirements when they became very complicated. Requirements for safety features are described in text and plain drawings. The MID staff interactively asked vendors to justify unfeasible requirements before they decided on the second-best option. Unfortunately, their vendors were software engineers who could not design the systems; they could only implement what the clinicians specified.
Some of the STH systems were purchased as is, so they are difficult and costly to modify. Nevertheless, they are revised every five to six years. Project management involves numerous analysts from 45 working groups representing 45 subsystems from clinical departments. The PD and MID staff evaluated new systems and system changes through clinical simulations and usability testing. Various ongoing and periodic enforced training on different physical and online platforms are offered, including HIS use, risk management, sharing sessions on incident reports, patient safety, and non-technical skills, such as situation awareness, decision making, communication, teamwork, stress and fatigue management, and leadership. Pharmacists undergo a one-month PhIS training through many platforms and practical on-the-job training (Pharmacist G). Educational resources such as books, health informatics journals, magazines, and digital materials are provided in the library and e-learning portals. A clear HIS user manual is printed and placed in the respective units for easy referral.
System quality (technology dimension)
Various system features and functions support patient safety, including alerts, reminders, critical values, order sets, automated dosing, checklists, templates, color codes, and icons. For example, “CPOE is smart in preventing errors: it identifies critical findings such as abnormal results” (Dr D), provides information and alerts for allergies and contra-indications, and prompts alert messages. CPOE can also identify a drug and its different brand names through an eight-digit coding system where different names of the same drug share the same first four digits. The template-based InstS standardizes instruction description and facilitates doctor–nurse communication. The graphical interface for medication prescription in InstS (Figure 4) is represented as a tree structure that organizes information in class-likely—from general to sub-categories. Medication prescription interface in the instruction system.
Instructions displayed in calendar form distinguish between original, revised, and current instructions to ease system use and information access. The template summarizes items that can be drilled down. Original instructions in the first column are described in the tooltips. Unchanged information is represented as a single-headed arrow, while amended instructions are highlighted in yellow. Symbols indicate changes in doctor instructions and drug intake information. Prescription timing is recorded on a specific timeline (before or after a meal). The drug administration status is indicated with a dark line, double blue line, or red circle for administered, delivered, and undelivered drugs, respectively. The template aids in clarifying changes in instruction history and prevents instruction overlap/overwrite (Figure 5). It also allows free text entry, allowing physicians to use the template partially instead of using an entirely new template set. The system allows addition, reduction, and changes in drug administration, which are indicated by different colors. Nurse interface in the instruction system.
Although the HIS is designed to minimize errors, it also creates new risks, particularly medication ordering. According to Dr D, The system is sophisticated, but it is like a black box— people depend on it to prevent errors, but it cannot do that completely. It has limitations in terms of checking the drug type and dose/overdose and the handing over of information from one attending physician to another.
For example, when a physician orders a particular medication, “the list of similar drugs… may lead us to choose the wrong drug” that looks or sounds alike (LASA). When a physician enters “Ferroberin drug,” the system checks the first three letters and displays two types of drugs, Ferroberin and Ferromia, with two different doses, thus listing four drug choices on one screen. Then, an alert pops up, “This is Ferroberin, not Ferromia?” Therefore, in the physician’s view, the CPOE has failed to provide adequate guidance for choosing the right medication. Another alert example is an upper-limit dosage alert with no additional information. Nurse L added, “CPOE is confusing in distinguishing drug dose, specifically before and after breakfast [meal], like the medication box.” Dr D further elaborated: … there is a gap between human expectations and actual system capability. Currently, the gap between the system and our brain is huge, but the [features] of care processes are quite different from [those of] HIS. For example, there are too many meaningless alerts and reminders. So, there is a need to achieve [a] balance between expectation and reality. However, the balance depends on whose perspective we [are] referring to, as a patient may be cared for by different physicians. For example, one physician orders one test, but the alert on the screen is not displayed for him or the right physician. One single alert that is meant for an individual is not sufficient for all clinicians who work in a team.
Several desirable system features were recommended for easy understanding of drug dosage descriptions, thereby providing caution and reducing adverse events: bold display and using the “00 mg” instead of the “1A” format, “since the standard is different in chemistry” (Nurse J).
Automating pharmacist work is essential, especially using “medication-dispensing machines and auditing clinical processes with sub-systems according to order information” (Pharmacist G). PhIS effectively provides automated checking for drug–drug interactions; contraindications; drugs counter-indicated in pregnancy, allergies, and serious illnesses; drug name variation; and high-risk drugs. One of the PhIS subsystems, a checking system, can detect prescription mismatch between the drug information and laboratory databases. The complex pharmacy process automation explains why Dr D perceived PhIS as complicated [and] sophisticated: “When we try to improve the system, we keep adding more and more, but we have to make it simpler to balance it with our cognitive capacity.” Each item in the drug supply system, including medication, contains radio-frequency identification for tracking purposes. Information exchange, including progress notes and images, is secured between databases, systems (e.g., from PACS to EMR), or related hospitals via the intranet. Clinicians can also check a patient’s information before arrival to prepare medications in advance.
Information quality (technology dimension)
Overall, patient and medication information are adequate, accurate, and timely. Timeliness varies according to the situation. For example, physicians acknowledge a drug or food allergy but delay the data entry (Dr D). Some errors are caused by an inconsistency between the PhIS data and the Japanese master code, which is not updated immediately in PhIS. The government documents detailed drug information in the system, including side effects and contraindications, but this discourages young nurses from reading all the items.
System use (human dimension)
The instruction system templates were highly utilized by the Internal Medicine and Surgery Departments (more than 70% usage rate). However, most nurses did not use personal digital assistance (PDA) due to its inconvenience (large, non-durable, and insufficient) (Dr D). Accordingly, “they administer the medication [without it], and [they] see the error only after administering it” (Nurse K). For the IRS, most nurses voluntarily enter all incidents, but only 10% of physicians do so for every incident, excluding near misses or non-serious incidents. An incident is entered through a template or in free text. New nurses are trained to enter error incidents as instructed by their supervisor. Such reporting then becomes habitual. Some physicians perceived CCIS negatively due to the limited screen view, which can result in information oversight and subsequent errors. The system is also time-consuming and tedious, has long turnaround times, and inefficient information retrieval that challenges information searching and limits patient-physician interactions (Dr F).
Process
Clinical processes are designed intuitively, evaluated through rigorous testing and simulation, modified, and continually monitored for early detection of problems from actual use. Both process and system designs consider human factor principles24,43 and the Lean methods from user input,
43
which include simplicity, constraints, procedural standardization, error proofing, and visual aids. QMD strives to minimize errors in the four MU stages: prescribing, dispensing, dosing, and follow-up: 1) Prescribing • Prompt alerts in CPOE when physicians prescribe medications with registered adverse reactions or allergy • Create chemotherapy regimens in PhIS • Share patients’ information with community pharmacies by printing blood test data on the prescription sheet • Ward pharmacists check drug dose, contraindication, and monitoring of side effects 2) Dispensing • Utilize an automation system, such as a tablet packaging system or dispenser of powdered or injectable medicines, and calculate the weighing liquid volume of anticancer drugs • Pharmacists in the dispensing unit use a BCMA to match prescriptions to drugs • Prompt alerts to the pharmacist if the patient is a child or participating in clinical trials 3) Dosing • Nurses in the patient wards use a BCMA to identify the right patients and drugs by matching drugs and the patients’ wristbands immediately before dosing 4) Follow-up • The PD obtains information on patients’ conditions or side effects from community pharmacies through reports, especially for patients who are undergoing outpatient chemotherapy • Some hospitals or community pharmacies can browse the STH’s EMR
Excerpt from controls, risks, and impact of inpatient medication prescription.
Almost half of the error risks are associated with human slips and mistakes, followed by a 1:1 ratio of process and technology. Only 5% to10% of the reported incidents are related to HIS. If an incident occurs, the service provider will fix the system promptly. QMD monitored the PDCA cycle for process improvement to ensure progress. Despite implementing preventive measures, recurring incidents were intervened to avoid future recurrence. For example, commercial and generic names of a drug were displayed during the prescription to alert the clinician, and patients were required to wear their ID wrist tag to prevent mistaken identity.
Examples of medication error scenarios.
BCMA = Barcode medication administration; 5R = 5 Rights (right drug, right route, right time, right dose, and right patient); Nil = unavailable.
Human error rates and patterns are lower during standardized processes than specialized or highly customized clinical processes, such as workarounds and heavy workloads. Interoperability between handheld devices (e.g., barcode scanner) and the main systems (e.g., EMR or database management), or updates for drug registries or medical coding can make the process more error-prone. However, in this case study, such errors are not driven by the HIS interfaces nor by database infrastructure capability that allows real-time hospital data update and analysis. Such incidents have never happened at STH. Instead, errors related to HIS-medical devices were caused by workarounds or unstandardized processes that resulted in data oversight, while errors related to drug registries or medical coding were related to process delay and subsequent data inconsistencies.
Corrective actions
When an incident occurs, the responsible nurse talks to the physician and the head nurse and then submits an anonymous incident report to the IRS. The head nurse investigates the case and subsequently checks and confirms the incident report. Next, she forms a meeting with the ward staff nurse to discuss the incident and plans for error handling following the QMD educational plan. A simple error incident is shared with other staff nurses, but a severe error incident requires further on-site investigation and personal staff handling. If the responsible staff has mental health issues such as depression, they can privately explain the incident case to care for their mental well-being, and other staff takes over her job. Finally, the unit analyzes and discusses the problem and the way forward.
Net benefits
Efforts to improve HIS have resulted in work efficiency and error reduction. “Templates enable us to enter instructions easily and to standardize their descriptions” (Dr B). However, the work duration increased due to additional tasks, such as data entry and database and system maintenance. Different innovative educational approaches, from lectures and live broadcasts to video lectures and e-learning, continuously increased participant attendance. Strategies for evidence-based studies on quality and safety were published in numerous newsletters, high-impact journals, prestigious proceedings, and books. The new patient safety unit achieved multiple improvements in its multidisciplinary efforts. IRS was beneficial for reducing STH medication errors and promoting internal and national patient safety. IRS also led to increased staff participation in all areas to report errors consistently and guide problem-solving and imperative improvements. The changes included replacing the inconvenient PDA use with the smartphone for BCMA and searching drugs using its first three, instead of two, letters. CPOE was improved by adding extra alerts for LASA drugs, including the highly harmful ones (e.g., cancer and mental treatment). 100 The system displays an additional warning message when a user clicks on one of these drug types for further checking and confirmation. The following also significantly reduced medication error rates: delivery of drugs in single doses, pharmacologists’ routine checking of physician orders in CPOE, talking to physicians, medication checks, and patient education. The proactive and positive attitudes and behavior of the top management and clinical risk managers also contributed to safer practices.
Discussion
We identified various factors and their socio-technical fit that contribute to resilient systems due to the effective error management related to HIS, including Kaizen, which shapes the safety culture, strong management and leadership, effective communication, preventive and corrective mechanisms, IRS, and closed feedback loop (Figure 6). Meanwhile, medication errors are attributed to system sophistication and process factors, such as workarounds, variance, clinical workload, slips and mistakes, and miscommunication. STH improved its system-oriented patient safety services through a comprehensive program and enterprise-wide infrastructure.43,97 The safety culture of STH encompasses many Lean principles, including respect for people, Kaizen, and systems thinking, which are implemented and sustained throughout its PDCA cycle.
15
Proper and detailed planning (P) resulted in fewer issues and errors in the DCA phases. Strong MID, QMD, and RMC structures that are in line with government policies, high-end infrastructure, adequate resources, and top management support yield effective management strategies and multidisciplinary participation. Although small, MID strongly bridges and supports clinical and IT areas owing to its IT-savvy clinicians, contradicting the case with a larger MIS Department
13
. Essence and impact of effective HIS-related error management at STH.
Many studies also showed that simulation-based training could minimize errors. 101 In situ simulation that “focuses on detailed breakdowns of major processes” also reveals workarounds, process variance, and unique contextual factors 102 that are subsequently modified based on task and technology or intervention mapping. Standard user interface design, system templates, automated functions, and intuitive design in HIS have reduced cognitive burden 2 and subsequent error incidents. However, these purposes have not been fully realized due to design limitations with actual work practices. Prompting a more comprehensive list of similar or related medications may create or prevent risk, depending on the therapeutic knowledge of the prescriber. 103 An intuitive medication list, such as a tooltip bar or a link to detailed drug information, can assist the user in choosing the correct drug. Further work is needed to understand, determine, and improve the alert appropriateness in supporting the clinical workflow.64,65
Communication, education, and training are organized and enforced frequently on multiple platforms for all staff to provide awareness, understanding, and feedback, and gain high participation. Strong enforcement, encouragement, and financial support from the government facilitate the implementation. The safety culture, high-end infrastructure, and substantial resources activate the acquisition of integrated, accessible, and safe design, implementation, and rigorous monitoring and evaluation of HIS. Documenting a successfully prevented error incident identified system weaknesses, reinforcing the importance of multiple medication checks. 42
The reporting system is critical in handling error incidents, and its reports provide lessons learned to prevent recurring incidents that may not be realized otherwise.9–11 While continuous assessment of all error types is neither possible nor needed, periodic focused data collection can adequately determine the error rate after the system failure and the specific error indicators. 24 On the other hand, real-time approaches provide “automated surveillance and monitoring of system performance and safety”. 2 Visualized data illustrate error characteristics, while measured practice instead of the event shows high error frequency over a short period for fast recognition of intervention impact. 24 Sittig et al. 2 recommended mandatory—instead of voluntary—reporting, 24 while the Institute of Medicine recommended both reporting types. 98 However, both studies agreed on a non-punitive policy system. Serious incidents related to an internal organization, the Food and Drug Administration, and the state should be reported. While optional reporting is applied for learning purposes. Proven benefits of analyzing incident reports from large databases, such as “national or international HIT reporting systems that gather and investigate serious patient safety issues with the help of dedicated experts”, 2 include wide-scale monitoring and trend analysis.6,44
We also analyzed the medication process from multiple perspectives: socio-technical, BPM, and Lean viewpoints. The analysis of end-to-end process steps identified MU practices in task execution, control mechanisms, error risks, their impact, and beyond. Our holistic and detailed view enabled us to illustrate how medication error management can create and mitigate medication error risks and outcomes.25,78 STH examples of medication error scenarios align with studies revealing that frequent nurse workarounds in BCMA are influenced by the day of the week, heavy workloads, nurse mental state, medication time, and patient-nurse ratio.54,104,105
Proposed approach for evaluating HIS-related errors
Essence of safety measures for HIS and their fit.
Detecting potential errors should be part of the clinical routine to improve the quality process. 42 Ongoing monitoring mechanisms, such as literature updates, reporting systems, morbidity and mortality conferences, and systematic failure analysis, must be in place to address any potential risks. 6 In addition, organizational risk assessment guidelines (e.g., Institute for Safe Medication Practices, American Society of Health System Pharmacists, and failure mode and effect analysis) can support a safety culture 97 by identifying risks in advance. Data-driven approaches, such as those of STH, must be established 6 to minimize recurring events and disseminate information for patient safety knowledge or mitigation/solutions, continuous review, standard categories, and education to optimize error reporting.
A convenient and easy-to-use reporting system for near misses and incidents that reach patients is imperative. 44 Error reports should be advocated for safety indicators instead of success measures. Retrospective evaluations are important in mitigating errors and ensuring safety processes; these include RCA, MU evaluation, quality improvement, and event detection. 44 Organizations should promptly use a template and standard method to conduct RCA and report the analysis and recommendations. 44
Safe use of HIS depends on its redesigned workflows. 12 Thus, HIS acquisition and implementation should include a commitment to change the workflow and address the necessary changes to improve workflow. However, costs and possible risks might be affected by designing new and practical workflows, training staff, and facing subsequent changes. Central validation of system functionality and measures to improve safety can be taken by (1) analyzing as-is and to-be workflow and (2) constructing mechanisms and metrics to identify, increase, and rectify patient safety issues. 107 Step 1 is featured in BPM and Lean methods, and step 2 is proposed in this study. Analyzing workflow interruptions can establish the generic and specific nature of problems by showing how a poor HIS design is a mismatch with the practical work of clinicians, given that they are too busy to document what happens during service delivery.12,55,64
The fit component in the HOT-fit framework can serve as a tool to evaluate the alignment of HIS, process, organization, and human elements. 13 Meanwhile, Lean methods can analyze and optimize workflows by focusing on detailed process components. These methods then redesign the processes by removing waste, such as errors. 84 This aim could also be achieved by various means, including creating visual controls, work standards, preventive maintenance schedules, and cross-training. 108 Optimally designed systems contend to streamline handoff processes and wastes that can lead to the inefficient communication of clinical information and subsequent error risk. 77 We recommended fit evaluation and Lean methods to address the two challenges mentioned above in process management: combining its technical and organizational components and selecting improvement methods to align the process with organizational goals. 14
Lean IT can be applied to the entire SDLC process, IT project management, and most measures in the HOPT-fit framework. Lean addresses awareness through the customer’s voice, “quality at the source,” and system thinking perspectives. 15 Quality at the source refers to “doing it right the first time, every time” to avoid passing defective work to the next process by applying standardized work, training, and error-proofing. System thinking aligns with organizational learning, which advocates viewing process as a whole system of interrelated parts. As a result, errors can be analyzed from the cause-and-effect chain. Lean’s ultimate transformation creates a Lean culture that includes safety. In brief, the closed-loop feedback, PDCA cycle, and reverse salient in system development corroborate one another. The complimentary Lean and Lean IT methods include 5S (workplace organization), system thinking, error proofing, associating waste types with errors, and avoiding wastes according to safe design. 24
Pathway documentation provides a model that illustrates the end-to-end process and its interrelated steps, error risk, and safety improvements. 109 For example, safe, effective, and efficient medication requires the understanding the pathway and the relation of the steps with human, 109 organization, and technology factors. Most patient medication has multiple sources for updated and accurate information due to a fragmented healthcare system and the lack of interoperability among HISs. 76 Therefore, optimum outcomes depend on communicating accurate, comprehensive, and complete information along the pathway. 109
Study limitations
The trade-off for complexity is parsimony, 19 but we balance the two aspects by developing a comprehensive but structured framework with flexible details and broad measures. 20 Constructs and variables must adequately address the respective domain for sufficient scope coverage 20 because a broader scope allows for a wider “explanatory power.” The examples of HOPT-fit measures are not exhaustive; further validation may lead to new measures or sub-measures, expanding the framework corpus. The framework is not a silver bullet to any problem; it is a guiding tool that can be used as a basis for discussions that stakeholders can access to understand their system health. 17 Moreover, although the study was conducted in specific settings, the generic evaluation process, error control and risk, standard and best practices, lessons learned, and recommendations in managing HIS-related errors apply to any clinical setting. The lessons learned, best practices and recommendations from this study can guide effective risk prevention and mitigation.
Conclusions
HIS-related errors are inevitable, but preventive and corrective measures can be taken to minimize them. A resilient system consisting of different HOPT barrier layers may prevent errors from occurring and recurring. Error management requires multidisciplinary and multifactorial efforts to match it with complex HIS-related errors. STH showed an effective program established as a strong safety culture that adheres to clinical standards (5 Rights), with system alerts and reminders that include checklists, protocols, and error and incident reporting. The study shows that medication error management can be incorporated into standard clinical pathways and automated systems that may evolve and be institutionalized as continuous improvement and culture. We demonstrated the applicability of the proposed framework from the process approach through the case study findings by unfolding the complex socio-technical interactions of HOPT-fit through evaluation. The framework can guide practitioners in evaluating the management, effectiveness, and safety of HIS in a structured, holistic, and continuous manner. The evaluation may steer decisions to plan, design, implement, and monitor compliance with patient safety regulations through and for HIT from all HOPT-fit aspects. Regulators and practitioners may focus on any HOPT factors using structured, detailed, and comprehensive measures that act as an evaluation checklist. This study can be further analyzed using other HOPT-fit factors. To conclude, leaner (effective) and fitter (aligned) HOPT results in safer HIS. HIS emergence calls for ongoing evaluation throughout the entire SDLC and overall process stages in addressing emerging HIS-related errors and optimizing system potentials to justify its benefits in minimizing errors.
Footnotes
Author contributions
The authors have made substantial contributions to all of the following: (1) the conception and design of the study (MMY) and the acquisition, of data (MMY, YM, TT, YS, and NM), analysis, and interpretation of data (MMY), (2) drafting the article (MMY) and revising it critically for important intellectual content (YM and TT), (3) final revision for submission.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We gratefully acknowledge the funding received from the Japan Sumitomo Foundation Grant, the Ministry of Higher Education Malaysia (FRGS/1/2018/ICT04/UKM/02/5), and Universiti Kebangsaan Malaysia (DIP-2016-033), that helped sponsor this study.
