Abstract
Background:
Hospitalized children are prone to experience harm from medication errors (MEs). Strategies to prevent MEs can be developed from identified malfunctioning practices and conditions in the medication use process. In this study, we aimed to identify MEs and potentially unsafe medication practices (PUMPs) in hospitalized children, and to assess the potential harm of these, using raters of different professions.
Methods:
A 1-week observation using an undisguised technique was conducted on four paediatric hospital wards. One observer followed ward staff during medication prescribing, preparation and administration. MEs and PUMPs were documented using field notes. Three raters including a physician, a nurse and a clinical pharmacist assessed the potential harm of each ME and PUMP using a six-point Likert scale. Agreement was analysed using Fleiss’ Kappa.
Results:
A total of 16 MEs and 809 PUMPs were identified involving a preparation and administration error rate of 8%. No actual harm to patients was observed during the study. Raters assessed the potential harm of 318 unique MEs and PUMPs. Only slight agreement was found (Kappa = 0.26–0.33). A 4-hour delay in the administration of intravenous cefuroxime received the highest harm score. Observations involving no information during prescribing and variations in medication preparation were considered potentially fatal for medications such as digoxin, morphine, enoxaparin and insulin.
Conclusions:
MEs and potentially unsafe practices and conditions may affect medication safety of hospitalized children. However, observed MEs did not result in any harm. The agreement among raters assessing the potential harm of observations was low. Alternative methods to determine the clinical relevance of errors are needed.
Introduction
Medication errors (MEs) in hospitalized children are associated with increased risk of harm.1–2 Hospitalized children, especially those treated in the intensive care units or under the age of 2 years may be particularly prone to MEs.1–4 In order to improve medication safety, MEs and practices which potentially contribute to MEs must be explored. Different approaches, including incident reporting, chart review and observations have previously been used to detect MEs in paediatric hospital settings.5,6 Direct observation is considered more efficient and accurate in identifying medication administration errors. 7 Since not all MEs result in actual harm, it is advantageous to target MEs with the highest risk of harm. Studies have previously used assessment of actual or potential harm of MEs to identify those most relevant to prevent.8–10 In this study, we aimed to identify MEs and potentially unsafe medication practices (PUMPs) involving practices or conditions with the opportunity to cause MEs in hospitalized children, and to assess the potential harm of these using raters of different professions.
Method
Direct observations of ward staff during medication prescribing, preparation and administration were performed on four neonatal and paediatric wards at a Danish tertiary hospital (>1000 beds). Identified MEs and PUMPs were subsequently assessed for potential harm by a physician, a nurse and a clinical pharmacist.
Setting and participants
Four wards were included: a ward for infants up to age 12 months with non-infectious medical and surgical non-intensive diseases (infants’ ward), a newborns’ ward, a neonatal intensive care unit (NICU) and a paediatric intensive care unit (PICU) (Table 1). The selection of wards was based on the complexity of care provided and the high volume of ME reports from each ward. Participants included ward staff, either trained or undergoing training, who undertook medication prescribing, preparation or administration on the wards. Physicians and nurses were randomly selected by the observer on three of the wards. However, on the infants’ ward, a nurse manager selected the nurses to participate in observations. Before each observation, the physician or nurse was verbally informed of the study and asked for permission to accompany them during medication prescribing, preparation and administration.
Characteristics of the four participating wards.
Medications used on the ward (Anatomical Therapeutic Classification System codes A01–V07AB). The standard assortment on the newborns’ ward and PICU also included medications used to treat adults.
NICU, neonatal intensive care unit; PICU, paediatric intensive care unit.
On the wards, prescribing was documented by neonatologists (infants’ ward, newborns’ ward and NICU) or anaesthesiologists (PICU) using a computerized physician order entry (CPOE) system. None of the systems had incorporated barcode-assisted medication administration technology (BCMA).
Nurses were responsible for preparing and administering medication on the wards. Medication preparation was performed in a standardized medication room situated on each ward, which also stored stock medications.
Data collection
Each ward was observed for 7 nonconsecutive days from March to May 2016. The majority of observations were conducted during day shifts, but at least one night shift was included in the observation round on each ward. The observer, a trained pharmacist (RMR), followed ward staff in their daily routines involving medications. Policies involving the medication practices on each ward were deliberately not read by the observer, to avoid a narrow view on what constitutes an ME and a PUMP and potentially ensure that as many MEs and PUMPs were identified. During the data collection, the observer only interacted with ward staff if potentially serious errors were observed (n = 17). These were not considered MEs because the observer interfered before any ME occurred.
Field notes were used to document observed MEs and PUMPs. MEs were defined as ‘any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the healthcare professional’. 11 PUMPs were defined as ‘practices or conditions with opportunities to cause MEs’. For each identified ME or PUMP, information about the involved stage of the medication process (prescribing, preparation and administration), the type of practice, the medication (generic name, brand name, concentration, route of administration, etc.) and whether the practice resulted in an ME was documented. MEs were categorized using a modified version of the National Coordinating Council for Medication Error Reporting and Prevention taxonomy. 12 In this study, wrong-time errors were defined as the administration of a medication more than an hour before or after the scheduled administration time. 13 Additionally, PUMPs were categorized through thematic analysis of the field notes. The observer did not have access to patient records or medication charts which implied that MEs resulting from PUMPs, such as no dose checks, could not be verified. Potential harm was defined as the possible patient outcome resulting from MEs or PUMPs. An example of this might be that morphine overdoses have the potential to cause respiratory depression in paediatric inpatients.
After the observational study, local patient safety managers on each ward were contacted to confirm if any MEs were reported during the observational period.
Assessment of potential harm of potentially unsafe medication practices
Potential harm of identified MEs and PUMPs were assessed by three raters using a questionnaire (Supplement 1). The raters were a physician, a nurse and a clinical pharmacist recruited from one of the participating wards and the hospital pharmacy. The location of the observations in the questionnaire was blinded to the raters. In the questionnaire, the raters were presented with a description of each type of ME or PUMP with information about the medication(s) involved (generic name, strength and route of administration). The potential harm of each practice was assessed using a modified version of the scoring system used in the national mandatory incident reporting system, the Danish Patient Safety Database, to categorize incident reports. 14 Scores were as follows. (1) No harm: potentially no harm. (2) Mild: potentially mild transient harm that does not require any intensified treatment or care. (3) Moderate: potentially transient harm that requires intensified treatment. (4) Severe: potentially permanent harm that requires increased treatment or other harm requiring acute treatment. (5) Death: potentially fatal. (6) Do not know. The raters were asked to base their assessment on the following assumptions that: (a) the practice observed resulted in an ME reaching the patient; (b) the practice resulted in a one-time event; (c) the other medications and the disease of the patient could be disregarded; and (d) the medication processes followed standard procedures on the ward unless otherwise described. Furthermore, raters were allowed to make one assumption about each case, that is, if the condition of the child was critical then the ME could potentially be fatal. The observer was present during the assessment to assist completion of the questionnaire.
Prior to data collection, the questionnaire was pilot tested by three clinical pharmacists, two physicians, including a neonatologist, and a paediatric nurse.
Analysis
Results from raters assessing the potential harm of observations were analysed using descriptive statistics. The rate of medication preparation and administration errors was defined as the number of MEs observed from the total number of medication doses which were observed during preparation and administration.
Fleiss’ Kappa with linear weights was used to describe the strength of agreement among raters. We reported two kappa analyses: (a) one kappa where ‘do not know’ responses were analysed as missing; and (b) one kappa where observations with ‘do not know’ responses were disregarded. Kappa values were interpreted according to the criteria by Landis and Koch. 15
Further, medications involved in MEs and PUMPs were compared with medication use on each of the four wards. Medication use was measured by the number of units, that is, a container with 300 tablets constituting 300 units. Infrequently used medications were defined as medications which constituted less than 0.5% of medication consumption in each children’s ward. Data involving medication purchases from January to December 2016 were provided by Amgros I/S.
Ethics and approval of study
This study was approved by the clinical managers of the four wards. According to Danish law, no ethics approval was required, as the observational study focused on the working procedures of ward staff only. 16 No personal information about participants or patients was collected.
Results
In total, 182 h of observations were completed on the four paediatric and neonatal wards. Medication preparation was the main focus of this study. However, at least one physician was observed during medication prescribing on each ward. Observations involved 39 nurses, 5 physicians and 235 medication doses. A total of 16 MEs and 809 PUMPs were identified involving 200 observed medication doses (Table 2). No MEs were reported by staff from the wards during the observational period.
Potentially unsafe medication practices on four hospital wards (n = 809) in relation to the medication process, types of practices, therapeutic subgroups and drug substances.
The number ⩾25 refers to the total number of potentially unsafe medication practices observed on the four wards involving therapeutic subgroups (Anatomical Therapeutic Chemical Classification System second level) and drug substances (Anatomical Therapeutic Chemical Classification System fifth level).
CPOE, computerized physician order entry; N/A, not applicable; NICU, neonatal intensive care unit; PICU, paediatric intensive care unit.
Medication errors
The identified preparation and administration error rate was 8% (16 MEs in 200 medication doses). MEs were observed on the NICU (n = 9), the newborns’ ward (n = 5) and the infants’ ward (n = 2). Types of error involved delayed medication preparation and administration (n = 10) and wrong injection rate (n = 6). On the infants’ ward and NICU, nine MEs involving delayed medication preparations and administrations including caffeine (n = 2), erythropoietin, furosemide, digoxin, cefuroxime, propranolol, ferrous fumarate, sodium glycerophosphate and vitamin D were observed. On the NICU, the delays resulted from the nurse having to complete other nursing tasks, such as weighing the neonate, before the medication could be administered. On the newborns’ ward, five MEs involved too rapid administration of intravenous solutions including ampicillin (n = 4, injection rate ⩽ 1 min instead of 5–10 min) and gentamicin (n = 1, injection rate = 1 min instead of at least 3 min). The injection rate was not specified in any of the prescriptions. None of the MEs resulted in actual patient harm.
In total, 17 potential MEs were intercepted by the observer before reaching the patient. These involved no cleaning of ampoules/vials/plastic bottles before broaching (n = 11) and no shaking of oral suspensions including nystatin (n = 5) and sulfamethoxazole/trimethoprim (n=1) and no medication preparation and administration of oral propranolol due to lack of knowledge about the emergency CPOE system.
Potentially unsafe medication practices
On average, four PUMPs were identified per dose observed. The majority of PUMPs were observed during preparation of medications (94%).
A total of 40 different types of PUMPs were identified. Five types constituted 80% of PUMPs. These were no check of expiry date (24%), an inappropriate default setting of the CPOE system implemented on three of the wards which showed the previous day’s prescriptions (18%), no recalculation of a prescribed dose (17%), no double check (12%) and double-check variations (8%). Variations in double checks involved no verification of the patient, medication, dose, route of administration or time. Most double checks involved a second nurse verifying the volume of medicine drawn up in a syringe. Further, 14 PUMPs (2%) involved nurses preparing oral suspensions from tablets involving propranolol (n = 6), metolazone (n = 5), and spironolactone (n = 3) suspended in water from which an amount was drawn up.
The most common therapeutic subgroups and drug substances involved in the PUMPs included (a) antibacterials for systemic use: ampicillin, cefuroxime and gentamicin given by intravenous injection; (b) diuretics: furosemide and metolazone given orally; and (c) blood substitutes and perfusion solutions: sodium chloride, magnesium chloride, sodium bicarbonate and sodium glycerophosphate given orally (Table 3).
Potentially unsafe medication practices most commonly observed on four hospital wards.
Most commonly observed PUMPs involve those that constituted ⩾2% of PUMPS observed in each ward.
ATC, Anatomical Therapeutical Chemical Classification System; CPOE, computerized physician order entry; NICU, neonatal intensive care unit; PICU, paediatric intensive care unit; PUMPs, potentially unsafe medication practices.
Assessing potential harm of medication errors and potentially unsafe medication practices
A total of 318 unique MEs and PUMPs were assessed for potential harm (Figure 1). The raters were a neonatologist, paediatric nurse and clinical pharmacist with 26, 28 and 6 years of professional experience, respectively. Both kappa values showed fair agreement for data where ‘do not know’ responses were considered missing (Fleiss’ Kappa = 0.26, n = 318) and for data where observations with ‘do not know’ responses were excluded (Fleiss’ Kappa = 0.33, n = 205), respectively. The highest number of identical scores was observed for the physician and clinical pharmacist (39%, n = 125) compared with the physician and the nurse (32%, n = 101) and the clinical pharmacist and the nurse (29%, n = 91).
Assessment of potential harm of medication errors and potentially unsafe medication practices (n = 318).
Total agreement was observed in 61 practices (19%) where raters scored two observations as severe and 59 observations as no harm. No double check during the preparation of dalteparin for intravenous injection and enoxaparin for subcutaneous injection was considered to cause severe harm resulting from an overdose. Preparation and administration of an expired medication was not considered as harmless for only 37% (21/57) of medications observed.
Of the 16 MEs, a 4-hour delay in the administration of intravenous cefuroxime received the highest harm score. In this case, the nurse reported the ME to potentially result in severe harm, whereas the pharmacist reported the ME to potentially result in moderate harm and the physician assessed the incident to be harmless. Late administration of vitamin D, ferrous fumarate, erythropoietin, sodium glycerophosphate, propranolol and caffeine were generally considered harmless by all raters.
In total, six PUMPs were rated to potentially cause death by two of the raters. Potentially deadly prescribing practices (n = 3) involved situations where (a) the standardized medication template in the CPOE system did not contain information about the concentration of an oral solution containing digoxin; (b) the dose of morphine for intravenous administration was prescribed in ml and mg/ml, but the amount (mg) was omitted; and (c) the dose of enoxaparin was prescribed in mg and mg/ml, but the volume (ml) was omitted. Potentially deadly medication preparation practices (n = 3) involved no recalculation of a prescribed dose of insulin for intravenous administration and digoxin for oral administration and no labelling of a syringe containing digoxin for oral administration involving information about the medication and patient identification.
Comparing medication errors and potentially unsafe medication practices with medication use
Medications involved in MEs including ampicillin on the newborns’ ward, and caffeine, ampicillin, sodium glycerophosphate, propranolol and cefuroxime on the NICU were also frequently used (Table 4).
Top 10 medications used on four participating hospital wards.
Medication units, i.e. ampoules, tablets, etc. were purchased by the four wards from January to December 2016.
ATC, Anatomical Therapeutical Chemical Classification System; NICU, neonatal intensive care unit; PICU, paediatric intensive care unit.
Medications involved in PUMPs comprised both the most and less frequently used medications ranging from 0.004% to 30% of medication units sold on each of the four wards. PUMPs involving enoxaparin given by subcutaneous injection were frequently observed on the infants’ ward and raters considered incidents involving this medication to be fatal. However, the medication was less frequently used (0.04% of medication units purchased). In general, PUMPs involving medications associated with high harm scores were infrequently used on the four wards. Thus, digoxin, enoxaparin and insulin constituted less than 0.05% of the medication units used in the ward where the PUMPs were observed.
Discussion
We identified 16 MEs and 809 PUMPs involving medication prescribing, preparation and administration on four paediatric hospital wards. No harm to patients was observed during the study. There was great variation in assessment of potential harm by three clinical experts. In general, infrequently used medications involved in MEs and PUMPs had generally higher harm scores than commonly used medications.
We identified a medication administration error rate of 8%. Other studies using direct observations have previously reported medication administration error rates ranging from 8.4% to 44.3% in similar settings.9,17,18 The error rate in our study may be underestimated as we had no access to patient information to verify the appropriateness of doses. Variations in error rates among published studies are likely to be due to differences in methodology, ME definitions, practices used in the settings and medication use at the time of data collection. 9
Implications for future practices to prevent medication errors
In our study, delayed administration of medications was the most common type of ME. Similarly, other studies have reported high rates of these errors indicating that current practices do not ensure timely administration of medications.17,18 A relatively large delay of administering intravenous cefuroxime was considered potentially harmful by the raters. Late or no administration of antibacterial drugs may be a particular concern in immunocompromised children or children with severe infections where timely administration leads to optimal therapeutic effects. 19 Nursing practices should facilitate timely administration of time-sensitive medications.
MEs involving too rapid injection of gentamicin and ampicillin were considered to cause mild harm by the raters. However, administration of gentamicin boluses over 1 min has not previously been associated with any adverse effects in infants. 20 Minor errors have been reported involving bolus administration of ampicillin, but rapid vancomycin resulting in severe hypotension and flushing of the upper body is considered more severe. 21 In the worst case, rapid injection of medications can result in cardiac arrest, shock and loss of consciousness, and nurses must be made aware of these consequences. 22
Considerable variations in medication double checks were observed on the two wards which had implemented this practice. After the study, it was discovered that a standardized written policy did not exist on either of these wards, which is likely to explain the differences observed. The Institute for Safe Medicine Practices and others recommend using double checks to improve medication safety.23–25 In a systematic review, Alsulami and colleagues found a significant reduction in medication administration errors when two nurses checked the medication before administration. 23 However, a standardized policy should be implemented and staff educated in the procedure. The policy should list high-risk situations and high-alert medications to be included in the double check. Further, staff should be made aware of situations that allow deviation from the procedure. Also, double checks should be truly independent. This requires that the second nurse does not communicate with or observe calculations performed by the other nurse. 24 However, shortage of staff may prevent double checking from being implemented.
On three of the wards, several PUMPs involving the CPOE system were identified. Two raters considered omitting the concentration of digoxin, the amount of morphine and the volume of enoxaparin to potentially result in harm, requiring intensified treatment. Also, participating nurses commented that not including the amount of medication to be given in the prescription was considered a serious threat. However, hospital policy was that physicians only had to indicate the volume of medicines to be given when prescribing. The CPOE system could potentially be optimized by developing standardized templates of the most commonly used medications to ensure accurate information about the concentration, amount of the medication (mg and ml) to be administered and injection or infusion rates of intravenous medications. However, standardized templates can also induce medication prescribing errors, which highlight the importance of designing and testing these if they are to be implemented. 26
Clinical pharmacists may contribute to medication safety in advising physicians about medication and dose selection and teaching nurses on preparation and administration of medicines. Also, interventions such as BCMA and intravenous administration technology have been suggested as possible strategies to reduce MEs in hospitalized children.27,28 However, more robust studies are needed to examine the potential effects of these interventions. 27 Further, even though MEs involving wrong patient or wrong medication were not observed in our study, barcode scanning could potentially play an important role in improving medication safety in these children’s wards.
Challenges involving assessment of potential harm
Three raters were involved in the assessment of potential harm of MEs and PUMPs and great variation was observed among their ratings. Like us, others have reported low kappa values when assessing the harm of MEs in hospitalized children.9,29 In our study, assessment of potential harm is likely influenced by subjective judgement, professional background and the fact that raters had to assume the patient outcome for the majority of the practices. We cannot determine whether variation is caused by the professional background or individual differences of the raters, as only one rater was recruited from each discipline. Further, assessing the potential harm was not a routine task for any of the raters. Training prior to the assessment only involved assessing the potential harm of one theoretical example.
Expert ratings of severity of errors or clinical relevance of interventions are extensively used in quality improvement work. Low kappa values like the ones we report in our study indicate the difficulty in using the method to target areas for improving medication safety.9,10,30 However, if these methods are to be used, they should be validated and tested for reliability. 31 Few studies have previously used validated methods.32–34
We chose to use a scoring system similar to the one in the Danish national mandatory reporting system. Raters were presumed to be familiar with the categories that would ease the scoring processes. However, the categories are originally used to score actual patient harm and are likely not suitable to classify potential harm. 31
Strengths and limitations
One strength of our study is the use of direct observations which is considered reliable and accurate to identify errors during medication preparation and administration. 31 However, the presence of the observer may have affected the behaviour of ward staff. This is known as the Hawthorne effect in which participants are likely to overperform. Further, ward staff might also have been distracted by the observer, which could have increased the rates of MEs and PUMPs. Dean and Barker have previously reported that observation of nurses during medication preparation and administration did not significantly affect the error rate. 35 We used an undisguised-observational approach due to ethical considerations. Another limitation is the generalizability of our results. First, observations were performed in a single university hospital over a short 7-day period. The short period of time increased the probability to miss rare, but serious MEs. However, our aim was not to obtain a total overview of all MEs in hospitalized children, but the most commonly observed ME and PUMPs which could potentially be targeted by interventions. Second, recruitment of more experienced nurses on the infants’ ward may have prevented us from identifying MEs and caused a lower number of observed PUMPs. However, prevented MEs such as not shaking oral solutions were observed on this ward.
Conclusion
MEs are likely to pose a threat to medication safety in hospitalized children. However, we did not identify any actual harmful medication events during a 1-week observation study on four paediatric hospital wards. Several potentially unsafe practices and conditions with opportunities to cause errors were identified, but with low agreement among raters. These practices and conditions should be further investigated to determine targets for safety improvements. Potential strategies to improve medication safety include reorganizing nursing work processes to allow timely administration and optimization of prescribing and double-check procedures. Efforts should be made to explore alternative methods to assess potential harm of MEs.
Footnotes
Acknowledgements
The authors would like to thank the nurses and physicians who participated in the observational study. We thank the three raters for their hard work and dedication during the assessment of the observations. Also, we would like to thank Sune Lindgaard at Amgros I/S for providing data involving medication purchases from the four wards. Lastly, we thank Ulrich Halekoh from the University of Southern Denmark for providing expert statistical guidance.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Conflict of interest statement
The authors declare that there is no conflict of interest.