Abstract
This study examined health information technology-related incidents to characterise system issues as a basis for improvement in Swedish clinical practice. Incident reports were collected through interviews together with retrospectively collected incidents from voluntary incident databases, which were analysed using deductive and inductive approaches. Most themes pertained to system issues, such as functionality, design, and integration. Identified system issues were dominated by technical factors (74%), while human factors accounted for 26%. Over half of the incidents (55%) impacted on staff or the organisation, and the rest on patients – patient inconvenience (25%) and patient harm (20%). The findings indicate that it is vital to choose and commission suitable systems, design out “error-prone” features, ensure contingency plans are in place, implement clinical decision-support systems, and respond to incidents on time. Such strategies would improve the health information technology systems and Swedish clinical practice.
Keywords
Introduction
Health information technology (HIT) is a crucial and integral part of modern medicine for improving the workflow and making the healthcare service more efficient and safer. 1 However, despite the favourable effects of HIT, its implementation and use in healthcare have caused unintended consequences and thus raised new safety concerns. 2 It can cause considerable disruptions, particularly in interoperability and system interfaces. 3 Despite the potential of HIT, new and often unforeseen risks keep appearing during and after the implementation of HIT systems. These challenges, aggravated by various factors (human and technical), add complexity to medical encounters in daily clinical practice, which leads to unintended consequences that can cause workflow interruptions, inconvenience, delays, and harm to patients.4,5 In this context, human factors refer to the interactions among humans, as well as the other elements of a system, which affect both human and system performance. 6 Technical factors are the characteristics of practices that can influence the organisation’s performance. 7
The Swedish Medical Product Agency (MPA) collaborates with the Swedish eHealth Agency and the Swedish Authority for Privacy Protection to deliver and contribute to accessible, equitable, and person-centred care. As stipulated in the National Board of Health and Welfare’s regulations, any incidents with medical devices considered to be part of NMI (National Medical Information) systems must be reported to the manufacturer and the MPA. Similar to medical devices, incidents related to NMI systems must be reported so as to reflect an imbalance between healthcare quality, patient safety, and throughput. Moreover, all Swedish regions have established computerised reporting systems, and any healthcare practitioner can submit incidents.8,9
Therefore, a digital reporting system functions as an information provider regarding incident management for continual safety improvement processes. Studying, examining, and analysing healthcare incident reports in the form of free-text narratives provides a basis for improving healthcare quality and safety. 10 The deductive approaches – the International Classification System (ICPS) 11 and HIT Classification System (HIT-CS) proposed by Magrabi et al., 12 and thematic analysis (inductive approach) have been used to analyse and interpret HIT incidents.
A great deal of research has been conducted on HIT-related incidents12–17; however, given the considerable increase of HIT-related studies in recent years, there is noticeably limited information regarding HIT-related system issues, particularly in the context of Swedish healthcare. A recent systematic review of problems with HIT indicated that 29 out of 34 included articles originated from the USA, the UK, and Australia, 18 but none from Sweden. Therefore, there is a need to explore the issues of the HIT system that occur in the routine clinical practice of Swedish healthcare.
This study explores Swedish healthcare quality and safety by examining HIT-related incidents collected through various incident report sources (craft-based, hospital-based, and staff-based). The study focuses on HIT incidents involving system issues through the lens of both deductive and inductive approaches. This study explores the following research questions: 1. What healthcare quality and safety issues affected HIT systems’ planning, management, and implementation? 2. How did human and technical factors contribute to the HIT system issues? 3. What were the outcomes of those system issues, and what were the actions taken to improve healthcare quality and safety? 4. What are the potential preventive and corrective strategies to mitigate the issues of HIT systems?
Method
This is an exploratory qualitative study. Data was collected as free-text narratives from multiple sources and analysed using deductive and inductive approaches.
Data collection
Incidents related to Swedish healthcare were collected via semi-structured interviews and three sets of voluntary regional incident reports. A range of sources of information were also collected in the form of free-text narratives.
Two sampling strategies were used – purposive and snowball samplings. The inclusion criterion was that healthcare professionals need to be familiar with HIT systems or incidents related to HIT systems. Since not all healthcare professionals are familiar with the HIT system, these sampling strategies allowed the most relevant participants to be reached. Types of participants were healthcare professionals, healthcare quality managers, medical engineers, physicians, and nurses across 21 regions in Sweden. The participants were approached via email in both English and Swedish with four open-ended interview questions and other supplementary materials, such as the study background and participant information sheet.
The “participant information sheet” informed participants that their response to the email would be considered their consent to participate. After obtaining oral consent from the participants, the phone interview was audiotaped. Responses collected from participants were recorded or transcribed from audiotape in the file titled with participant number, occupation and region, respectively. Data was then placed in two Microsoft Excel sheets (one for deductive analysis and the other for inductive analysis).
Participant and incident characteristics. 48
Note. Bold indicates the total values for the convenience of the reader.
Data analysis
The total sample of 95 incidents was aggregated for analysis using both deductive and inductive approaches. The deductive method involved applying classification systems, namely HIT-CS and ICPS, and the inductive method included thematic analysis following the guidance by Braun and Clarke. 19 The HIT-CS was used to identify incidents in the total sample that contained system (software and hardware)-related issues. The identified system-related problems were then subjected to thematic analysis to extract detailed information about the types of system issues and their association with human and technical factors. The ICPS was used to determine the outcomes of incidents and the actions taken to reduce the risks.
Conducting a thematic analysis involved selecting keywords or expressions that suggested relevant ideas from the whole sample. These concepts were then further classified into discrete sub-themes, and those that possessed comparable characteristics were further combined into themes. Some individual themes, like “system issues,” were still independent and looked at in addition to the clustered themes. This approach explores the qualitative data in a methodical and thorough manner.
Two investigators were involved in coding the incident reports for the purpose of coding verification and reliability. The main coder carried out the thematic analysis, which the second coder verified. In case of any difference in opinion, the incident was re-investigated to reach a consensus through dialogue. Each investigator conducted the classification of the incidents using the HIT-CS separately. Interrater reliability using the weighted kappa score was calculated, and the coders came to a harmonious agreement between themselves if there was any disagreement.
Results
Types of HIT systems identified using thematic analysis.
Note. Bold indicates the total values for the convenience of the reader.
Interrater reliability for the outcomes was
System issues
Of 56 incidents from the total sample (n = 95), 80 system issues were identified using thematic analysis. These issues were used to understand what happened and extract detailed information relevant to the specific problem in question (see Figure 1). The following sections illustrate each type of system issue and its effect with relevant examples of that specific issue (see Tables 3–10). Types of system issues and their frequency. NB: The term “error” is used to indicate system malfunction or failures as it is in the reports. The categories listed here have remained as reported. If additional details had been obtainable, some of these problems could have been allocated to other categories, and some could have been unified into similar groups. For example, no difference was specified between system interface or integration issues, thereby, the issues were grouped in the same category. Types of system errors with relevant examples. Types of functionality issues with relevant examples. Types of system design issues with relevant examples. Types of system interface/integration issues with relevant examples. Types of issues related to system/server down with relevant examples. Types of issues related to system update/upgrade with relevant examples. Types of issues related to launching a new system/application with relevant examples. Types of software issues with their relevant examples.
System error
System error accounted for 22.5% (n = 18) of the system issues (Figure 1). Three types of system errors were identified: data generation/input error (n = 7), display error (n = 7), and storage error (n = 4) (see Figure 1 and relevant examples in Table 3). These errors led to incorrect or missing drug information in the pharmaceutical modules, thus posing the risk of incorrect drug dosages on the prescription. The possibility of serious injuries was evident since the healthcare providers had to make decisions without the necessary patient records. Patients also experienced delays in care delivery and the risk of wrong diagnosis or treatment.
Functionality issues
Over one-sixth (18.75%) of system issues belonged in the category of functionality issues (n = 15). Two types of functionality issues were identified: communication function issues (n = 11) and alarm/warning function issues (n = 4) (see Figure 1 and relevant examples in Table 4). These problems hindered the exchange of information and documentation between different care units. Some of these problems posed the risk of serious deterioration of patients’ health due to delayed delivery of the patient data. Staff or organisational outcomes involved additional manual work and extra documentation for healthcare staff.
System design issues
Twelve (15%) system design issues were detected. There were three types of system design issues: system defect (n = 6), inconsistency with legislation/restriction (n = 4), and structure/configuration problems (n = 2), often leading to additional resources being needed for investigations for organisations (see Figure 1 and relevant examples in Table 5). System design issues that led to missing/wrong patient content posed risks of repeated/wrong treatment to patients, such as receiving too many vaccinations or incorrect dosing of drugs. For healthcare staff, system design issues caused delays in using the system and additional back-and-forth contacts with the manufacturer.
System integration issues
Three types of system integration issues were identified. The first is the integration issues relating to the Electronic Medical Record (EMR) and Patient Administration System (PAS) (n = 5). The second involved Digital Imaging and Communication in Medicine (DICOM) channelling between medical imaging systems, such as Radiology Information Systems (RIS) and image storage systems (n = 4). The third included the integration problem between EMR and the e-prescribing system (n = 3) (see Figure 1 and relevant examples in Table 6). Problems involving DICOM mainly affected the patients requiring emergency treatment, resulting in delayed treatment and increased workload and frustration for medical imaging professionals, such as radiographers and radiologists. Integration issues often led to the risk of confusion since two different patients ended up in the same context. There was a significant risk to patient safety, malpractice, over-treatment, under-treatment, and reduced trust in the system.
System/server down
Among seven issues related to system/server down (8.75%), six issues had been identified as system down, while one involved server down (see Figure 1 and relevant examples in Table 7). In some cases, it was also reported that staff manually entered information into specific modalities during the downtime of the system and server. In these cases, it was evident that staff and the organisation needed to use extra resources, such as time and additional devices, to ensure the ongoing healthcare process during the downtimes. However, no patient harm was reported in this category.
System update/upgrade
Six disparate cases of updates/upgrades (7.5%) were evident in which unforeseen issues arose due to complex HIT systems (see Figure 1 and relevant examples in Table 8). Problematic updates/upgrades caused extra workload and frustration for the healthcare professionals. An issue caused by a software update of the decision support system for cancer treatment led to medical decisions for patients being made on erroneous grounds. For example, invalid information leads to the risk of serious drug interactions for a patient.
Launching of new (version of) system/application
Among five issues (6.25%), one involved the launch of a new system, and four were due to a new version of the system or application. (see Figure 1 and relevant examples in Table 9). These issues often resulted in missing or incorrect information in the new systems. All of these reports indicated that no patients were harmed. However, incidents that included wrong or missing information resulted in the patient’s health status not being monitored optimally and them receiving repeated vaccinations. Another case involved the delayed launch of a new Android version application for chat-based psychological treatment, which further led to the application not being available to patients for 3 days, and healthcare staff lost money as their compensation was performance-based.
Software issues
Thematic analysis identified five software-related issues (6.25%), of which two included faulty software, two involved inaccessible software, and one indicated unavailable software (see Figure 1 and relevant examples in Table 10). Two cases reported inaccessible software that handled worklist and sampling, which led to delays in sending information and resulted in incorrect information being sent to the clinical chemistry or stored in the server. There were no reports of patient harm in any of the cases in this category; however, delayed or incorrect information resulting from software problems caused patient inconvenience as treatment was delayed or involuntarily interrupted.
Contributing (human and technical) factors
Among contributing factors, 73.75% (n = 59) of system issues were contributed by technical factors, of which over half of the factors (n = 30) were related to software, code, or configuration. Human factors accounted for 26.25% of contributing factors to system issues, and 19 out of 21 human factors were related to users, staff, organisations, and system developers (see Figure 2, Table 11, and association with system issues in Table 12). Types of contributing factors. Types of human and technical factors. Note. Bold indicates the total values for the convenience of the reader. Types of system issues and their association with human and technical factors. HF, Human factor; TF, Technical factor. Note. Bold indicates the total values for the convenience of the reader.
Outcomes
Types of outcomes using the ICPS
Note. Bold indicates the total values for the convenience of the reader.
Actions taken to reduce the risks
Over half of the 56 identified incidents (n = 30; 53.57%) contained details about actions taken to reduce the risks that were introduced by system issues. Less than one-third (n = 9) accounted for HIT system-related actions, and over a quarter (n = 8; 26.67%) comprised staff-related actions (see Figure 3). Other actions, that is, “making arrangements for access to a service” (n = 5), involved the manual entry of patient data or manual handling of organisational work. Types of actions taken to reduce the risks using the ICPS.
Discussion
Swedish patients, clinicians, managers, and others nowadays use a growing range of digital services and technologies to meet their different needs. 20 While HIT adaptation is undoubtedly crucial for improving the workflow and making the healthcare service more effective, there is still evidence suggesting that HIT resulted in unintended consequences and consequently brought about new safety concerns. 8 This study provided insight into system issues occurring in day-to-day Swedish healthcare and examined the contributing factors to identified system issues. The study also shed light on the outcomes of identified system issues, indicating the consequences of inefficiencies for staff and organisations and inconvenience and harm to patients.
System issues of functionality, design, and integration
The findings of this study showed that system errors, design issues, and integration issues had dominated the eHealth system issues in Swedish healthcare. Similar system issues have been reported in the studies conducted in countries such as Norway, 21 the Netherlands, 22 and Finland. 23 System issues are difficult to locate until they appear, which is even more difficult in healthcare as heterogeneous stakeholders are involved. 24 Healthcare involves multidisciplinary collaborations, and different stakeholders of the eHealth system, such as healthcare quality managers, medical engineers, physicians, and nurses, have various priorities to fulfil in the clinical daily practice. As a result, additional burdens were placed on system designers to meet the requirements of users with different priorities at the planning stages.25,26
The integration issues among various eHealth systems were highlighted in this study due to their high-risk consequences for patient safety. Administration systems, pharmaceutical systems, and health records systems witnessed the most integration issues, which resulted in patient safety and privacy concerns. The question of why interoperability is hard in healthcare was discussed at the technology, data, human, and institutional layers, 27 and in this study, the interoperability between different eHealth systems was highlighted. In addition, The decentralised nature of healthcare administration in Sweden allows regions to have autonomy in deciding the adaptation of new eHealth systems, which adds another layer of complexity to system integration nationwide at the implementation stage.25,28
Human versus technical factors
Our findings indicated that 74% of the system issues caused by technical factors were spread widely amongst different components of the forms and functions of the HIT systems. Due to the lack of reporters’ understanding of how systems work, it was difficult to devise locally appropriate preventive and corrective strategies for the majority of these technical failures. Rather than strategies aimed at early identification and minimisation, multiple components of the system issues were considered “black boxes”. 29 Most of the evidence in this study suggested that much time and energy was spent on follow-ups and making alternative arrangements. Such alternatives involved starting the computer or healthcare professionals proceeding with the treatment by making manual entries to avoid delay in care delivery in the event of any system failure.
Human factors contributed to the problems related to HIT systems, which were identified as root causes of varied system issues and played a critical role in exacerbating the negative consequences. Some human factors, especially those involving patients, contribute to system issues with already existing system deficiencies. The incident reports in this study indicated that developers’ lack of knowledge and not following related legislation contributed to system issues at the system design stage. Thus, besides continuous improvement needed by systems, training for end-users is also crucial in minimising the system issues. Healthcare professionals are seldom provided with enough training or education to properly use and operate HIT systems, resulting in a lack of proficiency in handling them in healthcare settings. 32 Similar suggestions were also made by 25 et al., recommending that clinical staff receive initial and ongoing training for incorporating upgrades. 21
Involvement of multiple mechanisms
Modern healthcare is a complex, socio-technical system in which a HIT incident mainly involves the dimensions of human-computer interface, human behaviour and performance, workflow and communication, clinical content, and culture.30,31 Multiple mechanisms are likely to be involved in one single event, including technical and human factors. 32 For example, a technical failure in a network or server could have caused human error due to the need for the manual process of patient data, as analysed in our results. The human failure, thus, in return, may incur patients not being identified correctly, wrong patient details being entered by the radiographer, or patient details not being entered at all. Such erroneous patient data travels through various systems and different stages of the clinical workflow, making the situation more complicated for the patients, healthcare professionals, and the organisation as a whole. Once incorrect clinical information is in the system, it is likely to be considered true because of “automation bias” manifesting as a “tendency to use automated cues as a heuristic replacement for vigilant information seeking and processing”. 33 This necessitates continual ingenuity and robust mechanisms to activate the cross-checking mechanisms at each phase of the clinical workflow to prevent and detect these incidents.
It may seem simple to identify that the software was unavailable or prescriptions were lost; however, the exact underlying system issue and the mechanism behind those issues are difficult to determine. Even though immediate contributing factors associated with human and technical failures were apparent in most reports, the underlying mechanism of these factors was not reported. It is possible the reporters might not have understood the real reason behind those failures or not been aware of any such mechanism.
Organisational and patient outcomes
The system issues, especially those in the Health Information System, e-prescribing System, and Patient Administration System, raised concerns regarding data reliability and patient safety. In these cases, healthcare professionals read wrong patient information, and medical decisions were made based on false grounds. According to Kutney-Lee et al., 34 poorer EHR usability was associated with not only a heavier workload and more burnout for nurses but also higher surgical patient mortality and readmission.
The results also indicated that system issues may deteriorate data quality and lead to privacy concerns. Data quality is of clear importance to healthcare, and completeness and correctness are the two most common dimensions in which data is assessed. 35 However, it was evident in this study that these two dimensions had been jeopardised by various identified system issues. In addition, system issues also introduced human workarounds outside the system, such as using USBs for information transportation, which further pose threats to data quality and reliability and lead to the possibility of clinical information leaks.
Preventive and corrective strategies
The following preventive and corrective strategies to mitigate the HIT system issues are based on the findings of this study, our own and new deductions after merging data from several sources, and reflections emerging from the literature and government reports. These suggestions are listed in accordance with the various stages of the HIT lifecycle where problems occur and have the potential to improve the current challenges encountered by Swedish healthcare. ➢ ➢ ➢ ➢ ➢
Strengths and limitations of the study
Despite the small sample size, incidents collected from different healthcare actors supplemented those from voluntary regional incident reports and presented a more holistic picture of what went wrong in routine clinical practice concerning HIT in Swedish healthcare. Taking into account the COVID-19 pandemic situation, holding distance interviews via email, audio conference, or telephone was more feasible than face-to-face meetings. However, there was still a need to bring different sources together to provide a more comprehensive understanding of the characteristics, contributing factors, and consequences of HIT-related incidents. Therefore, it is always desirable that these reports are supplemented with additional information from relevant healthcare stakeholders 8 and other sources, including medicolegal files, complaints, and root cause analysis of similar sentinel events. 49
No data saturation or theory triangulation was performed due to the lack of resources and time. The study rather aimed to collect 80–150 incident reports from a variety of sources, adequate for determining the ongoing challenges encountered in Swedish healthcare in terms of HIT systems. However, the interview questions were tested with six students in the school workshops.
Factors related to a system, software, and server were the most frequent contributors, particularly when the types of HIT issues (in nature) themselves are system, software, and server-related (through thematic analysis). This characteristic indicates the recursive nature of the things that have gone wrong and the recurrent association between the types of issues and particular contributing factors. This manifestation of the recursive nature confirms an appraisal of internal validation of the reporting and analysis process. On the other hand, the ICPS was originally framed without considering the HIT system; therefore, a slight modification of the “action taken” nomenclature “HIT system”, instead of “medical device” has been employed in this study.
The application of HIT-CS and the thematic analytical method to the same set of incidents allowed the relative strengths to be assessed and compared. With both deductive and inductive methods, information that was not evident using deductive analysis could be extracted, providing comprehensive information responding to the research questions. However, the limitation came with the subjective nature of the qualitative approach. 50 The incidents collected by the interviews were limited due to participants’ personal impressions or bias toward reporting incidents that appeared interesting or unusual. The content knowledge of the reporters also limited the reports; for example, no difference was specified between system interface or integration issues; thereby, such problems were grouped in the same category.
Moreover, the categories listed here have remained as reported. If additional details had been obtainable, some of these problems could have been allocated to other categories, and some could have been unified into similar groups. For example, the term “error” was used to indicate system malfunction or failures as it was in the reports. Since data were analysed manually using both deductive and inductive analysis methods, there was a concern that the perspectives gained in the deductive analysis may have influenced the result of the following inductive analysis. Therefore, external checks and double coding with the supervisor were adopted during the data analysis to minimise subjective bias.
Exploring the system issues that occurred in Swedish healthcare requires continuous effort since the new systems are continuously being tested and implemented into clinical practice, and solutions for issues that occur can only be fixed when they are reported and analysed. It is crucial for future studies to look into larger-scale data in a longer time period and propose solutions for identified issues to improve the overall eHealth system.
Conclusion
This study shed light on various system issues occurring in day-to-day clinical practice, particularly in Swedish healthcare, through the lens of multiple qualitative approaches. The findings indicated that system integration issues among various HIT systems (e.g., EMR and e-prescribing systems) posed serious patient safety risks, causing patient and staff/organisational harm. Choosing and commissioning suitable systems in the first place, designing out “error-prone” features for a safe user interface, ensuring contingency procedures are in place, implementing a clinical decision support system, and responding to the incidents in a timely manner can mitigate the risks and improve the quality of the HIT systems.
Footnotes
Acknowledgements
The authors wish to thank Sofia Backåberg and Pauline Johansson, Senior Lecturers at Linnaeus University, for their help and support.
Authors’ contribution
DP and MSRJ researched literature, obtained ethical advice, and analysed data. DP was involved in protocol development and writing, reviewing and editing the drafts of the manuscript. Both MSRJ and EN reviewed and edited the manuscript and approved the final version.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The work was part of a Master’s degree project at the eHealth Institute, Linnaeus University, Sweden. A publishing grant was received from Linnaeus University as a part of the University Library’s research support.
