Using the SEIPS Model to Assess a Maternal Hemorrhage Risk Alert Tool

Abstract

Deaths resulting from maternal hemorrhage (MH), for example, excessive blood loss related to childbirth, are preventable if the hemorrhage is identified in a timely manner. One approach to support timely identification is the use of alerts. However, alerts with low utility have negative, unanticipated consequences for both clinician and patient outcomes. This study has two objectives: (1) use the Systems Engineering Initiative for Patient Safety model to describe how the MH risk alert tool is used in the care process, and (2) evaluate the utility of the MH risk alert tool from the perspective of the care team. We performed observations to understand the care process and fielded a survey to evaluate the alarm utility. Our results showed that the MH risk alert tool is used throughout the care process, both antepartum and postpartum. Additionally, our results showed that workflow integration, usefulness, and trustworthiness of the alert need to be improved.

Introduction

Excessive blood loss related to childbirth, for example, maternal hemorrhage (MH), is a leading cause of maternal morbidity and mortality in the United States (ACOG, 2017; Andrikopoulou & D’Alton, 2019). One approach to support patient care during MH events is the use of alerts. An alert is an electronic reminder or prompt, often displayed as a pop-up in an electronic health record (EHR; Backman et al., 2017). For example, a clinician may receive an alert if a patient has certain health characteristics that are risk factors for a hemorrhage, such as a high body mass index or nulliparity. Alerts are commonly used in health care to prompt patient care actions or to provide clinicians with information that can be used for decision-making (Dexheimer et al., 2008). However, alerts with low utility have negative, unanticipated consequences for both clinician and patient outcomes (Backman et al., 2017; Khajouei & Jaspers, 2010).

There are five dimensions of alert utility developed based on (Salwei et al., 2021; Schumacher & Lowry, 2010; Xiao et al., 2014; Xie et al., 2020)—timeliness, workflow integration, usefulness, usability, and trust. Timeliness refers to the alert being integrated into the temporal nature of work—it should occur at a time when the clinician needs to perform an action or make a decision (Salwei et al., 2021). Workflow integration refers to the temporal integration of the alert within the current work system, that is, people, tasks, other tools and technologies, physical environment, and organizations (Salwei et al., 2021). Usefulness refers to the alert containing a feature or a function that is needed to carry out a specific task, whereas usability refers to the alert being easy to use (Schumacher & Lowry, 2010). Trust refers to the willingness to believe or depend upon the information that is presented in the alert (Xie et al., 2020).

A macroergonomic approach can be used to evaluate and improve the alert design. The Systems Engineering Initiative for Patient Safety (SEIPS) model describes the interacting work system elements that influence care processes (Carayon et al., 2006, 2020). We use the SEIPS model as a framework to assess a MH risk alert tool embedded in the EHR. The MH risk alert tool consists of two components: a risk algorithm and an alert. The input to the algorithm is a questionnaire about common risk factors of MH, which is completed by a clinician. The algorithm then determines the risk level, that is, standard, medium, high, or highest risk. An alert then appears in the EHR. The alert describes the risk level, the risk factors the patient has, and the recommended patient care tasks, foe example, ordering hemorrhage medications.

This study has two objectives: (1) to describe how the MH risk alert tool is used in the care process and (2) to evaluate the utility of the MH risk alert tool.

Methods

This study is part of a larger project to reduce maternal morbidity and mortality resulting from MH by redesigning processes and technology to better support the patient and care team. This study took place in the Department of Obstetrics and Gynecology (OB/GYN) in an academic hospital in the Midwest and was approved by the hospital Institutional Review Board. We performed observations to understand the care process and fielded a survey to evaluate the utility of the MH risk alert tool.

Observations

Two researchers performed 26 observations of direct patient care, during which we used a structured observation tool to document the work system of each interaction. We identified the clinical role, the tasks, the tools and technologies, the physical and external environment, and the organization. After completing observations, we used the SEIPS-based process modeling method (Wooldridge et al., 2017, 2020, 2022) to model the care process. Two clinician collaborators validated the accuracy of the process maps.

Survey

We designed a survey with short-answer and Likert-style questions. In the short answer questions, participants described their last interaction with the MH risk alert tool, for example, risk level, workflow, and tasks performed. In the Likert-style questions, participants rated their perception of the utility dimensions, that is, timeliness, workflow integration, usefulness, usability, and trust. We invited 10 attending obstetricians, 10 resident obstetricians, and 112 registered nurses from the OB/GYN department to participate—51 clinicians responded to our survey (38.6% response rate). We thematically analyzed the short answer responses and grouped them into four categories—acknowledge the alert, dismiss the alert, notify a physician, and perform care tasks. We calculated the mean response of the Likert-style questions.

Results

The MH risk alert tool is used in four phases: outpatient clinic, admission, labor and delivery, and postpartum care.

• Outpatient Clinic: Two weeks before delivery, the care team used the tool to calculate risk. For a “high” or “highest” risk, the team placed order sets so that hemorrhage medication and blood were reserved once the patient arrived at the hospital.

• Admission: When a patient presented at the hospital for delivery, the care team used the tool to re-calculate the risk. If the risk was “high” or “highest,” and order sets were not previously placed at the outpatient clinic, the team placed order sets for hemorrhage medication and blood.

• Labor and Delivery: The tool was not used.

• Postpartum Care: Following delivery, the tool was used to re-calculate risk. Depending on the risk level, the care team created a frequency plan to monitor the patient.

The survey revealed that 69% of participants interacted with the tool when they first opened the EHR. After viewing the risk level, participants reported the following actions:

• 39% acknowledged the alert;

• 27% dismissed the alert;

• 25% notified a physician;

• 8% performed care tasks.

Those participants who dismissed the alert cited several reasons, including that it appeared at a time when they were too busy to address it and that the information provided in the alert was redundant or confusing. The questions related to the alert utility revealed that participants perceived the alert as early or on time, and somewhat easy to use. However, the alert did not fit well in their workflow, did not provide useful features, and the participants did not fully trust the alert.

Discussion

Our results showed that the MH risk alert tool is used throughout the care process at the labor and delivery site, both antepartum (in the outpatient clinic and admission) and postpartum (in postpartum care). Antepartum, the MH risk alert tool is used to estimate the likelihood that the patient will experience a hemorrhage during labor and delivery. Depending on the risk level returned by the tool, the care team may perform patient care tasks in anticipation of a MH event. These anticipation-related patient care tasks support the timely management of future MH events. Postpartum, the MH risk alert tool is used to estimate the likelihood that the patient will experience a hemorrhage during recovery. The care team then uses the risk level returned by the tool to develop a frequency plan to monitor the patient. Interestingly, the MH risk alert tool is not used intrapartum (during labor and delivery). Instead, the care team uses clinical judgment to identify if the MH risk is increasing. As this was a preliminary analysis, the next steps will explore how the MH risk alert tool can be applied intrapartum. For example, a recent study by Zheutlin et al. (2022) identified that an intrapartum monitoring tool could increase the accuracy when assessing the risk of MH.

Additionally, our results showed that the utility of the MH risk alert tool could be improved, specifically, the workflow integration, usefulness, and trust dimensions. As this was a preliminary analysis, the next steps will include additional data collection. We will use semi-structured interviews to understand what work system factors impede or support MH anticipation, identification, and response. After analyzing the interview data, we will redesign the MH alert, including identifying when and where in the care process the MH risk alert tool should be integrated (workflow integration), what information should be presented (usefulness), and how sensitive the risk assessment should be so that the tool is perceived as trustworthy by clinicians (trust).

A limitation of our study is that 39 of the 51 participants were registered nurses. Our survey results may not accurately represent the perception of all clinicians, for example, attending obstetricians and resident obstetricians. Another limitation is that we asked participants to describe their last interaction with the MH risk alert tool, which may have introduced recall bias. We chose this approach, rather than observing the participants interact with the tool, to ensure we did not interrupt patient care.

Conclusion

This study used the SEIPS model as a framework to describe how the MH risk alert tool is used in the care process and to evaluate its utility. Our results showed that the MH risk alert tool is used antepartum and postpartum but not intrapartum. Additionally, our results showed that the workflow integration, usefulness, and trust of the MH risk alert tool need to be improved. As this was a preliminary analysis, the next steps include redesigning the MH risk alert tool.