Using simulation to improve pharmacy operators’ handling of cytotoxic spills

Abstract

Introduction

International Society of Oncology Pharmacy Practitioners guidelines recommend having standard operating procedures (SOPs) and initial and yearly retraining programs on cytotoxic spill handling for pharmacy operators (POs). This study aimed to create a simulation-based training (SBT) program on this subject and evaluate its impact on POs’ real-life performance.

Methods

Randomly formed pairs of POs underwent a 2.5-hour training program, including two simulation exercises (a broken cytotoxic vial on the floor and a leaking cytotoxic bag) in a simulated pharmacy production unit. Each participant applied the cytotoxic spill handling SOPs. The PO and trainer-pharmacist did a debriefing after each exercise. Satisfaction was recorded on a 0-to-100% scale. A 20-item questionnaire assessed general knowledge about cytotoxic spill handling before and after the training. One month before and one month after the training, the POs underwent a real-life test when the trainer broke a fake cytotoxic vial in the cytotoxic storage area. Their performance in applying the SOPs was assessed on a 20-point checklist, and the time to handle the spill was recorded.

Results

Twelve POs participated. Mean satisfaction score was 98.9%. Mean knowledge score improved from 10.8/20 (SD = 2.0) before training to 14.5/20 (SD = 1.6) after training (p < 0.05). Mean real-life SOP performance improved from 78.6% (SD = 7.4%) to 97.1% (SD = 5.2%) (p < 0.05). Mean time to handle cytotoxic spills decreased from 17.3 minutes (SD = 3.6 minutes) to 11.9 minutes (SD = 1.5 minutes) (p < 0.05).

Conclusion

POs improved their knowledge and real-life competencies for handling cytotoxic spills. This training will be included in POs’ initial and continuing training programs.

Introduction

Vial breakages or spills are common risky situations in cytotoxic production units. They lead to a high risk of contaminating pharmacy operators (POs) with these hazardous products and to a risk of drug dispersion in the environment if handling is done incorrectly. Considering that our pharmacy's chemotherapy unit employs 16 POs producing 25,000 cytotoxic preparations per year, they face a non-negligible risk of exposure.

As recommended by the International Society of Oncology Pharmacy Practitioners (ISOPP) guidelines, the handling of cytotoxic spills should be taught during POs’ initial training. ¹ This usually consists of a 1-hour presentation by the trainer responsible, who explains the standard operating procedures (SOPs) and shows the trainees what is included in the spill kit. The ISOPP guidelines also recommend annual refresher training in the form of a spill simulation exercise, which does not occur in our hospital.^1,2 During recent cytotoxic spills (fewer than five per year), POs were observed to be distraught, stressed and slow to react, with difficulties applying the SOPs. They admitted to being uncomfortable and lacking confidence about handling these spills.

Simulation is a training technique that can replace or amplify real-life activities through guided experiences, often immersive in nature, that evoke or replicate substantial aspects of the real world in a fully interactive manner. ³ The present study's objectives were to create an SBT program on handling cytotoxic spills and to evaluate its impact on a population of POs using four criteria: their satisfaction with the training program, improvements to their general knowledge of handling cytotoxic spills, their performance in real-life conditions, and the time they took to handle spills completely.

Methods

The study design is presented in Figure 1.

Figure 1.

Study design diagram.

Simulation-based training (SBT)

Training took place in a fake pharmacy production unit setting consisting of a room used for logistics (selecting products and hand hygiene), a room for changing clothes, and a cleanroom containing an isolator and a vertical laminar airflow hood. Briefings and debriefings took place in another room with direct access to the cleanroom area; it also contained a screen connected to two cameras in the logistics room and one camera in the cleanroom so that activities in the entire simulation could be watched at the same time. Each training group comprised two participants, randomly paired according to the unit's day-to-day human resources requirements. SBT took 2.5 h on one afternoon.

Participants underwent a 15-minute briefing starting with a reminder of the key learning objective, which was to review the SOPs for managing cytotoxic vial breakages. This was followed by a short presentation by the trainer-pharmacist on the types of workers exposed, modes of contamination, hazard classification, the nature of the hazard, protective measures, how to correctly use gloves, waste management, and a reminder of the SOPs for cytotoxic vial breakage, with or without cutaneous or ocular exposure. Participants were encouraged to react to the briefing if they needed clarifications.

In the first 15-minute simulation exercise, the trainer announced the scenario thus: “It is 7 a.m. You are the first to arrive at the chemotherapy unit, and you enter the logistics room to organize the day.” On entering the logistics room, the two participants discovered a broken vial on the floor (placed there by the trainer before their arrival). One of the two participants had to manage the spill using the spill kit. The other one returned to the control room with the trainer to follow the scenario's development on the screen. The participants could also communicate with each other using microphones whenever they wanted to and ask the trainer questions if needed.

In the second 20-minute simulation exercise, the scenario was announced thus: “It is now 10 a.m. It is time for the morning shift hand-over, and you have to replace your colleagues inside the cleanroom to prepare some chemotherapies.” The two participants had to apply the pre-cleanroom entry SOPs in the preparation area (perform hand hygiene, change clothes). Once inside the cleanroom, they discovered the trainer, also dressed in cleanroom clothing, playing the role of a PO in a state of complete panic. The fake operator said, “I have a problem”, and showed them a fake leaking cytotoxic bag in one hand and a needle puncture mark on the tip of one finger of the other hand (represented by a red pen mark on the glove). The second PO—the one who had not handled the vial breakage—had to use the cytotoxic spill kit, while the first PO accompanied the fake operator outside the cleanroom. Dealing with the needle-stick injury was not part of the training exercise but added an element of stress to the context.

After each simulation exercise—so, twice per training session—a debriefing took place where participants could express their feelings about the situation they had just lived through (the emotions phase). They also analyzed their spill-handling practices and what they could have done better with the input of their colleague and the trainer-pharmacist (the analysis phase). Finally, each PO summarized what they had learned (the synthesis phase). ⁴

Evaluation of the training

The SBT program's success was assessed according to Kirkpatrick's four-level scale for evaluating training programs. ⁵ The four levels are Reaction (measuring whether participants found the training relevant to their role, engaging, and useful), Learning (measuring whether participants acquired the knowledge that the training program focused on), Behavior (measuring post-training behavioral changes to see whether participants were applying what they learnt in their daily practice), and Results (measuring the training's impact at an operational level). We did not evaluate the fourth level, however.

Participants completed a mandatory, non-anonymous, 11-item satisfaction survey, on a 0-to-100% scale, to evaluate Reaction. Before and after their training, participants also completed a mandatory, non-anonymous, 20-question survey (1 point per question) to evaluate their Learning about various critical elements of cytotoxic exposure and handling cytotoxic spills. This was inspired by the Foch Hospital's questionnaire (Appendix 1). ⁶

PO Behavior was evaluated using real-life simulation tests 1 month before and 1 month after their SBT session. During these tests, operators had to handle a fake surprise cytotoxic vial breakage during their daily practice. The trainer-pharmacist broke a transparent water-filled glass vial with a fake chemotherapy label marked irinotecan on the floor of the cytotoxic storage area. Each time, the trainer discreetly called over a different operator, thus: “I think we have a problem. Could you please come and help?” The PO had to clean the area by applying the cytotoxic spill handling SOPs. If the PO was reluctant to handle the spill alone and wanted to get some help, the trainer-pharmacist offered that help. If the PO was afraid of being contaminated by the spill and wanted to leave the area, the trainer explained that it was a simulation and invited the PO to apply the SOP anyway. The trainer-pharmacist stood back to evaluate the operator according to a checklist prepared using the SOPs (Table 1). Each successfully completed step scored one point, with a maximum total of 20 points. A full 20 points was equivalent to a 100% score. The trainer also recorded the time needed to handle the spill, from its discovery to the PO leaving the area. The test was repeated one month after the training session using identical conditions.

Table 1.

SOPs.

Step	Objective	Points
Isolation	Evacuate people from the room (answer to the trainer's question: “Imagine there are other people in the room. What would you tell them?”)	1
	Inform a supervisor (pharmacist/pharmacy operator responsible for training) (answer to the trainer's question: “Would you warn someone about the incident?”)	1
	Take the spill kit	1
	Remove and prepare the equipment near the contaminated area	1
	Demarcate the area (warning tape + panel)	1
Total points for the isolation step		5
Protection	Put on the FFP3 mask	1
	Put on the overcoat	1
	Put on the overshoes	1
	Put on the protective glasses	1
	Put on a pair of nitrile gloves and a pair of chemical protection gloves	1 (0 if one type of glove is missing)
Total points for the protection step		5
Cleaning	Disperse the absorbent powder over all the liquid	1
	Pick up the pieces of glass and the powder using the dustpan and squeegee (if there are small pieces of glass, use the pliers)	1
	Clean the contaminated area using an absorbent wipe moistened with water and using spiral movements from the outside to the inside	1 (0 if the movement is not from the outside to the inside)
	Wash with water a total of 3 times	1 (0 if < 3, 1 if ≥ 3)
	Clean with disinfectant and dry the zone	1
Total points for the cleaning step		5
Elimination	Discard all soiled equipment as you go (dustpan, squeegee, absorbent wipe) and the warning tape and panel	1
	Discard the protective clothing (possibly retaining the mask)	1
	Close the kit's blue container or waste bag using the seal and dispose of it before leaving the room	1
	Do a handwash (answer to the trainer's question: “What would you do after handling the spill?”)	1
	Wait approximately 10 minutes before re-entering the cleanroom (answer to the trainer's question: “How long would you wait before re-entering the cleanroom if the incident occurred there?”)	1
Total points for the elimination step		5

Data were collected on a Microsoft Excel® spreadsheet (Microsoft Corporation, Redmond, USA). Differences between pre-training and post-training test scores for knowledge and performance were assessed using Student's t-tests for paired samples (with α = 0.05). The statistical analysis was done using BiostaTGV software (https://biostatgv.sentiweb.fr/), and boxplots were drawn using Datalab software (https://datatab.net/).

Results

Twelve POs participated in the present study, which was conducted over 4 months (from July to October 2023). Their mean post-training satisfaction score (the Reaction level of Kirkpatrick's pyramid) was 98.9% (Table 2). Several positive comments were given about the SBT itself, emphasizing the fact that it was fun and stimulating, necessary for maintaining POs’ professional skills, and updated knowledge on cytotoxic spill handling in a judgment-free environment that promoted collaboration. Other positive comments focused on the use of SBT itself, highlighting that simulation enabled better memorization and recall, that practice made it easier to integrate learning, and that the briefing was essential to bring the SOPs back to mind and put them in context. On the other hand, one participant underlined the fact that some of the questions in the knowledge survey were difficult to understand, and another regretted that it was not an electronic survey.

Table 2.

Results of the satisfaction survey (n = 12).

Question	Score (%)
This simulation is relevant to my professional practice	100
This experience will improve my professional practice	98.3
This simulation model is a relevant educational tool for operators’ continuing training	100
The simulation updated my knowledge of cytotoxic spill handling	98.2
The fun aspects added value for reviewing procedures compared to ‘traditional’ teaching	100
Total simulation training time was of the correct/adequate length	95
The simulation was well organized	96.7
The briefing and debriefing sessions were well facilitated	100
There was a good general atmosphere throughout the simulation	100
The simulation benefitted me	100
I will recommend this training program to colleagues	100
Mean	98.9

Knowledge scores (the Learning level of Kirkpatrick's pyramid) improved from 10.8/20 (SD = 2.0) before the SBT session to 14.5/20 (SD = 1.6) after it (p < 0.05) (Figure 2). Performance scores improved from 78.6% (SD = 7.4%) in the pre-SBT session test to 97.1% (SD = 5.2%) one month afterwards (p < 0.05) (Figure 3). Before the training, the main errors committed were forgetting a piece of personal protective equipment (PPE), such as the mask or the overshoes, and not following the recommendation of cleaning from the outside to the inside. ⁷ Time to handle the cytotoxic spill decreased from 17.3 minutes (SD = 3.6 minutes) before the training program to 11.9 minutes (SD = 1.5 minutes) after it (p < 0.05) (Figure 4). These two results represent the Behavior level of Kirkpatrick's pyramid.

Figure 2.

Improvement in knowledge questionnaire scores (in points), before and after the SBT session (n = 12).

Figure 3.

Improvement in the real-life performance test scores (in %), 1 month before and after the SBT session (n = 12).

Figure 4.

Improvement in time to handle the cytotoxic spill (in minutes), 1 month before and after the SBT session (n = 12).

Discussion

To the best of our knowledge, this was the first study to evaluate the impact of an SBT program to improve real-life cytotoxic spill handling by POs. Indeed, the Behavior level of Kirkpatrick's pyramid has not been previously attained on this subject. The Foch Hospital, in France, provides cytotoxic spill handling training similar to ours to POs from French-speaking countries, but although their knowledge is assessed before and after training, that training's impact on their real-life performance is not. ⁶

Other healthcare professionals could be interested in participating in this type of training. Indeed, simulated chemotherapy spills are widely used to increase the competencies and confidence of oncology nurses. For example, using three simulation scenarios involving leaking 5-fluorouracil, doxorubicin, and cyclophosphamide infusions, DeMacedo increased 29 nurses’ knowledge of the contents of a chemotherapy spill kit and their feelings of being prepared to deal with any future spills. ⁸ In Linnard-Palmer's study, pediatric oncology nurses had to use a spill kit to handle a pool of a simulated, red-colored chemotherapy spill representing doxorubicin. ⁹ The group's training rating scores demonstrated a high level of satisfaction, and they self-reported increased knowledge, confidence, and spill skills performance. ⁹ However, these results concentrated mostly on satisfaction with training or self-reported outcomes; they did not formally assess skills using a real-life evaluation. Once again, previously published studies did not manage to attain the Behavior level of Kirkpatrick's pyramid.

Our results for participants’ satisfaction with their training were similar to those found in the literature, and all our POs agreed that an SBT program was a relevant educational tool for their continuing professional training. As an appreciated and relevant means of training POs, SBT should be used to train other skills and improve knowledge on other topics.

Our POs improved their knowledge about cytotoxic spill handling (the Learning level) by almost 4/20 points, which means that, in theory, they should have a better understanding of how they should react. However, the present study's most valuable results were the SBT program's impact on their real-life performance, which improved by almost 20%, and on the time they needed to handle the spill, which decreased by an average of more than 5 minutes. Thus, after training, operators were acting in a safer way and spending less time in contact with cytotoxic drugs, which improved their personal safety.

Most cytotoxic drugs are transparent, making it complicated to verify whether all the necessary areas have been decontaminated. A solution for simulating environmental contamination by cytotoxic drugs is to use fluorescent markers. Semi-quantitative fluorescent methods have been widely used in environmental contamination evaluation studies involving nurses, ¹⁰ for verifying the effectiveness of closed-system drug transfer devices,^11–13 or for measuring the competency of POs handling cytotoxic drugs. ¹⁴ The concentration of fluorescein in water used is usually 10 g/L.^10,15 We did not use fluorescein, however, but considered that three cleaning operations were necessary, as per the European Society of Hospital Pharmaceutical Technologies video training. ¹⁵

Indeed, several videos exist presenting the procedures for handling cytotoxic spills.^15–17 One of them even explains how to make an “at-home” simulation by providing the list of necessary equipment. Video is an interesting medium with which to reach many people, but it is dependent on the will of the trainer to ensure the implementation of what is explained in the video. Video may also be complicated to update if SOPs change—removing out-of-date information from the internet is difficult. Based on the feedback from the POs who participated in our study, learning by doing is very powerful and watching a video probably does not have the same impact on knowledge and performance.

On the other hand, this type of SBT program is more resource-intensive than a classroom course or a simple demonstration. Firstly, it requires specific simulation material. The costs of the supplies needed for our simulation exercise were covered by our pharmacy's budget but, whenever possible, we used or re-used expired or unused open supplies to reduce waste and study expenditure. Secondly, it requires participant availability and, mostly, time on the trainer-pharmacist's schedule. A comparison of the relative impacts of SBT and a traditional classroom course or videos on POs’ performances in handling cytotoxic spills would be very interesting.

Limitations

The present study's main methodological limitation was its small population of 12 POs. Also, the two scenarios involved a liquid cytotoxic (from a broken vial and a leaking bag) and none involved a cytotoxic powder. This choice was made because approximately 90% of the cytotoxic vials in our unit contain liquids. Also, the second real-life evaluation (1 month after the training session) was no longer a surprise, as operators immediately understood that it was a post-training test, even though it had not been announced earlier in the program. This could have caused a non-evaluable bias in our results.

Conclusion

This fun SBT was highly appreciated by the POs who took part in it. It highlighted the added value of this kind of training compared to other, more traditional methods. POs improved their knowledge and their real-life performance when handling cytotoxic vial breakages. They also dealt with incidents faster, allowing production to start up again sooner. Our goal is to include this SBT program in our university hospital pharmacy's routine initial and continuing professional training for POs, as recommended by the ISOPP guidelines, and to provide it to POs in other French-speaking hospitals. There was also a request from our hospital's oncology nurses to adapt this training to them, especially using the scenario involving a leaking cytotoxic bag.