What is the impact of introducing inpatient electronic prescribing on prescribing errors? A naturalistic stepped wedge study in an English teaching hospital

Abstract

Most studies evaluating the impact of electronic prescribing on prescribing safety have used comparatively weak study designs such as uncontrolled before-and-after studies. This study aimed to apply a more robust naturalistic stepped wedge study design to compare the prevalence and types of prescribing errors for electronic prescribing and paper prescribing. Data were collected weekly during a phased electronic prescribing implementation across 20 wards in a large English hospital. We identified 511 (7.8%) erroneous orders in 6523 paper medication orders, and 312 (6.0%) in 5237 electronic prescribing orders. Logistic regression suggested no statistically significant effect of electronic prescribing use or of study week; patient and ward had significant effects. Errors involving incorrect doses and illegible or incomplete orders were less common with electronic prescribing; those involving duplication, omission, incorrect drug and incorrect formulation were more common. Actions are needed to mitigate these error types; future studies should give more consideration to the effects of patient and ward.

Keywords

electronic prescribing hospitals patient safety prescribing errors stepped wedge study

Background

Medication errors are a global priority,¹ with prescribing errors in the hospital setting a significant concern. An international systematic review reported a median prescribing error rate of 7 per cent of medication orders, and 52 errors per 100 admissions.² Studies in the United Kingdom have reported prescribing errors in 7.5–14.7 per cent of inpatient medication orders.^3–6 While not all such errors result in patient harm, all can create additional work and reduce patients’ confidence in their care.⁴ Many initiatives have been proposed in an attempt to reduce such errors; in particular, electronic prescribing (eP) systems, allowing ordering or prescribing of medication electronically with varying degrees of clinical decision support, have been widely advocated to improve prescribing safety.⁷

Studies of the impact of eP on prescribing errors generally demonstrate significant reductions in medication errors.^8–11 However, the majority of studies have used comparatively weak study designs, such as uncontrolled before-and-after studies, rather than more robust designs such as a stepped wedge study¹² where different clinical areas change from paper to eP at different times. There are few UK studies, with the majority of studies originating in the United States.¹³ Of the few UK studies published, most are in specific clinical areas.^14–17 The English Government is committed to increasing use of eP in hospitals with its use expected to increase significantly in coming years.^7,18–19 It is therefore important to understand the impact of introducing eP using a robust study design in the UK setting, where practices differ in some respects from those in the United States. In particular, UK hospitals do not mandate pharmacist verification of each medication order prior to any doses being administered, unit dose drug distribution is not used and there is greater use of ward stock and patients’ own medication supplies.

Our aim was to evaluate the impact of introducing eP on prescribing errors in an English teaching hospital, using a stepped wedge study design. Objectives were to compare the following for paper-based prescribing and eP: (1) prescribing error rates, (2) types of prescribing errors, (3) the number of doses given (or omitted) before erroneous medication orders were corrected and (4) to make recommendations as needed to improve prescribing safety in English hospitals in the context of eP.

Methods

Study design and setting

We studied a 500-bed London teaching hospital introducing eP as part of a wider implementation of a commercially available electronic health record and electronic ordering system (Cerner Millennium). The hospital implemented eP during 2016 in all 20 adult non-critical care inpatient areas during a phased roll-out according to the organisation’s eP implementation plan. We therefore used a naturalistic stepped wedge study design²⁰ based on the organisation’s implementation plan, rather than a cluster randomised stepped wedge design in which the order in which clinical areas implemented eP would have been randomised.¹²

At the start of the study, all inpatient prescribing was paper-based. Prescribers hand-wrote medication orders on pre-formatted paper drug-charts. This involved writing the drug name, dose, route, frequency, time of administration and any additional instructions, and signing and dating each medication order. Following eP implementation, medication was ordered using the eP system by specifying these details using either drop-down menus or free text. Clinical decision support in operation at the time of the study comprised pre-programmed suggested dosing regimens (‘order sentences’) for commonly used drugs, lists of multiple orders typically required for particular situations (‘care sets’), patient-specific drug-allergy notifications and reminder alerts to encourage timely review of medication such as anti-infectives. For anti-infective medications, prescribers were also required to state its indication and duration. Drug interaction checking and dose range alerts were not in operation at the time of data collection.

Pharmacists provided a typical UK ward pharmacy service, with most wards visited by a pharmacist for 1–3 h each weekday. Pharmacists checked inpatient medication orders to ensure they were clear, legal and clinically appropriate. Standard practice was that any prescribing errors identified were rectified as needed, usually by contacting a prescriber.

Data collection

Following briefings in team meetings and via email, ward pharmacists were asked to collect data once a week on all included wards for 16 weeks from 13 January 2016 to 28 April 2016 inclusive. At the start of the data collection period, all prescribing was paper-based; data collection continued as wards went live with eP in five stages at weeks 6, 7, 10, 11 and 12 (Table 1).

Table 1.

Roll out of electronic prescribing (eP) by week.

	Week 1	Week 6	Week 7	Week 10	Week 11	Week 12
Number of wards using paper	20	18	14	10	6	0
Number of wards using eP	0	2	6	10	14	20

Data were collected as part of pharmacists’ usual clinical duties; data were ideally collected on Wednesdays, but we allowed some flexibility for pharmacists to collect data on Tuesday or Thursday instead if necessary.

A data collection form was developed based on previous work;⁴ this was given to pharmacists together with a written and verbal briefing. Data were recorded for all new ‘regular’ and ‘when required’ medication orders that had not already been screened by a pharmacist or discontinued. Drugs prescribed on supplementary paper drug charts, oxygen, ‘once only’ doses and continuous intravenous infusions were excluded to simplify data collection, as these types of medication order are less common and/or not consistently screened by pharmacists. Recorded data comprised the bed number, number of previously unscreened medication orders and, for any errors identified, brief details of drugs involved and the error type. As a measure of system resilience, we also asked pharmacists to record the number of doses given or omitted before the error was corrected.⁴ Each individual prescribing error was classified into one of 13 mutually exclusive types;⁴ each medication order could be associated with more than one error. If a patient was an inpatient for at least 2 consecutive weeks, they could be included more than once.

Sample size calculation

Based on 20 wards being included and 40 new medication orders being screened each day on each ward,⁴ we estimated that we would be able to include 800 medication orders at each of the 16 weekly time points. If the baseline prescribing error rate was 10 per cent,⁴ we anticipated identifying 100 prescribing errors at each time point. Based on a simple two group comparison with 6400 orders on each of paper and eP, this was calculated to be a sufficient sample to identify a reduction to 8.5 per cent as being statistically significant using the 95 per cent confidence interval (CI) for the difference between two proportions using alpha of 0.05 and beta of 0.8.²¹

Definitions

A medication order was defined as one drug prescribed for an inpatient; there are usually multiple medication orders on each patient’s inpatient drug chart or electronic prescription.⁴

The definition of a prescribing error was

A prescribing error occurs when, as a result of a prescribing decision or prescription writing process, there is an unintentional, significant: reduction in the probability of treatment being timely and effective or increase in the risk of harm when compared to generally accepted practice.²²

This definition is accompanied by a list of situations that should be included and excluded as prescribing errors.

Analysis

Data were entered into an Excel database. Errors that did not meet the study definition of an error were removed; pharmacists’ classifications of error types were checked by the authors and recoded where necessary to ensure consistency of classification.

Univariate analysis was first conducted without taking into account clustering by patient or ward, and without controlling for time. The denominator for calculating the error rate was the number of newly written regular and ‘when required’ medication orders screened by ward pharmacists, plus any medication orders omitted.^4,23 Ninety-five per cent CI were presented for prescribing error rates with non-overlapping CIs taken to indicate statistical significance.²⁴

We then fitted a multivariable three-level fixed effects generalised linear model (logistic) using STATA IC version 15. The dependent variable was whether or not each medication order was erroneous; independent variables were ‘paper versus eP’ (categorical) and ‘study week’ (linear). The three levels of hierarchy were to account for clustering of medication orders within patients and clustering of patients within wards. Covariates were removed from the model if the likelihood ratio test of the hierarchical model was not significant (p > 0.05). Random intercept logistic regression analysis confirmed there to be no significant effect of study week (p = 0.153) together with a non-significant likelihood ratio test. The fixed-effect variable ‘study week’ was therefore removed from the model. The final model contained only ‘paper versus eP’ as a fixed-effect variable. Types of error and number of doses given before error correction were summarised descriptively.

Ethical considerations

The study was approved locally as a service evaluation; ethics approval was not required. No patient or prescriber identifiers were recorded. Pharmacists’ names were recorded to permit follow up of any queries on data collection forms and then replaced with codes prior to analysis.

Results

Pre- and post-eP data were collected for 20 wards in total, by 39 different pharmacists. Overall, 241 data collection forms were completed of a possible of 320 (75%). Return rates by week ranged from 55 (week 11) to 95 per cent (weeks 2 and 4).

Prevalence of prescribing errors

A total of 11,586 newly written medication orders were reviewed by ward pharmacists during the study, of which 9184 (79.3%) were for regular medication and 2402 (20.7%) for ‘when required’. There were 174 prescribing omissions also identified, giving a total denominator of 11,760 for calculating the error rate. Of the 11,760 medication orders prescribed or omitted, 6523 (55.4%) were on paper and 5237 (44.5%) on eP. These relate to pharmacists reviewing charts for 4036 patients during the study period. There were 831 individual prescribing errors recorded in 823 (7.0%) of 11,760 medication orders, concerning 563 (13.9%) of 4036 patients.

Of the 6523 medication orders written on paper, 511 were erroneous, giving an error rate of 7.8 per cent (95% CI: 7.1%–8.5%); most erroneous orders had one error except for eight which had two errors. Of the 5237 eP orders, 312 were erroneous, giving an error rate of 6.0 per cent (95% CI: 5.4%–6.7%); all erroneous eP orders had only one error. The non-overlapping CIs suggest a significant difference, with a lower error rate for eP. Figure 1 presents prescribing error rates by weeks post-implementation on the ward concerned, suggesting no obvious learning effect in the 11 weeks following go-live.

Figure 1.

Prescribing error rates for electronic medication orders presented by weeks since electronic prescribing (eP) went live on the ward concerned.

The final three-level random intercept model suggested no statistically significant effect of eP use once clustering by patient and ward was taken into account (odds ratio 0.938 eP versus paper; 95% CI: 0.727–1.210; p = 0.621). Ward and patient (clustered within ward) both had significant effects (Table 2).

Table 2.

Results of three-level random intercept logistic regression analysis.

Variable	Odds ratio for effect on erroneous medication orders (95% confidence interval)
Electronic prescribing (with paper-based prescribing as base category)	0.938 (0.727–1.210)
Ward variance	0.366 (0.172–0.775)
Patient (within ward) variance	2.166 (1.690–2.775)

A post hoc descriptive analysis of error rates by ward is presented in Figure 2, confirming that both error rates and the effect of eP varied by ward. The most dramatic error reductions associated with eP occurred on wards 1, 6, 10 and 19 which were day surgery, medical admissions, orthopaedics and neurology. The most dramatic increases occurred on wards 11, 14 and 15 which were gastroenterology, medicine for the elderly and acute stroke, respectively. The intraclass correlation coefficients were 0.063 (95% CI: 0.031–0.124) for ward and 0.435 (95% CI: 0.376–0.496) for patient (within ward), suggesting that the majority of this variation occurred at the patient level.

Figure 2.

Analysis by ward.

Types of prescribing errors

A breakdown of the different error types for paper and eP is given in Figure 3, suggesting increases in duplication of the same drug, omission, incorrect drug and incorrect formulation with eP, and decreases in errors involving drugs that were not indicated or contraindicated, incorrect doses, illegible and incomplete orders with eP. Prevalence of other types of errors was similar.

Figure 3.

Types of prescribing errors identified using both paper and electronic prescribing (eP).

Rectification of errors

Pharmacists recorded the number of doses given or omitted before the error was corrected for 624 of 823 (75.8%) erroneous medication orders. These data suggest that erroneous doses were more likely to be rectified in a timely fashion for paper-based prescribing (Table 3).

Table 3.

Summary of findings in relation to number of doses given (or omitted) before erroneous errors were corrected.

	Number of erroneous orders for which these data recorded (%)	Mean number of doses given or omitted before error corrected (range)	Number of orders in which error corrected before any doses were administered or omitted (%)
Paper prescribing	389 of 513 (75.8%)	1.2 (0–8)	158 (40.6%)
Electronic prescribing	235 of 310 (75.8%)	1.8 (0–26)	102 (43.4%)

Discussion

Key findings

Overall, errors were identified in 7.7 per cent of paper medication orders and 6.0 per cent with eP. While univariate analysis suggests that this indicates a decrease with eP, multivariable analysis taking into account clustering by patient and ward suggests no statistically significant difference. Instead, the effects of ward and patient (within ward) were significant, suggesting that factors other than type of prescribing system have a significant impact on prescribing safety and the effect of eP. Our findings also confirm that certain types of errors are more likely with eP with others less likely.

Interpretation and comparison with existing literature

Previous studies of the effect of eP on prescribing errors generally suggest a reduction in prescribing errors. However such studies have not taken into account the effect of ward or patient, precluding comparison with our findings in this respect. A recent study of medication administration errors²⁵ suggests clustering by ward to be an important factor; our study suggests that ward may also be important for prescribing errors.

In line with previous studies, our findings indicate that eP changes the types of prescribing errors that occur.¹⁷ Perhaps unsurprisingly, there was a reduction in illegible and incomplete medication orders. Prescribing errors associated with the wrong dose also decreased, which is likely to be associated with the use of order sentences and order sets that are pre-populated with commonly used dosing regimens. However, other error types increased. As in our study, previous research suggests an increase in errors associated with duplication of medication and the failure to discontinue drugs that are no longer needed.^26–28 New errors have also been attributed to factors associated with the eP system itself and its usability, such as mis-selection from drop-down menus, inability to view all medication concomitantly and failure to change default settings.^28–30

A previous study in the context of paper prescribing suggested that 0.9 doses were administered (or omitted) prior to errors being corrected,⁴ which compares to our figure of 1.2 for paper prescribing. However we also identified a higher figure of 2.0 with eP; it is not clear why errors were corrected less swiftly with eP.

Strengths and limitations

The main strength of this study was the application of a stepped wedge study design, which is considered more robust in comparison to uncontrolled before-and-after studies. In contrast to previous work in this area, we also took into account clustering by ward and patient. While slightly smaller than anticipated, we still achieved a large sample size of more than 11,000 medication orders.

Limitations include the return rate for pharmacists’ data collection forms, which ranged from 55 to 95 per cent; weeks where the response rate was particularly low coincided with the introduction of eP on the ward concerned, suggesting that this affected data collection. Informal feedback suggested that failure to return data collection forms was due to forgetting to complete the forms or being too busy to record the data. The quality, accuracy and completeness of data collected may also vary depending on individual pharmacists. For example, data regarding the number of doses given or omitted before errors were rectified was recorded for only 76 per cent of errors. It was not possible to take into account any effect of prescriber since we did not record the prescriber for medication orders written or for errors identified, and although we had data on the pharmacist identifying each error, we were unable to take this into account in the model due to the large numbers of different pharmacists who collected data (n = 39) and considerable correlation of pharmacist with ward. Finally, we did not assess the likely clinical significance of errors as pharmacists were asked for only very brief error descriptions to facilitate data collection, with no details of patients’ clinical conditions.

Implications for practice and policy

Our findings suggest that one of the purported benefits of introducing eP, namely a reduction in prescribing errors, may not be realised in practice. Instead, we found a more nuanced effect with different effects on different wards, as well as different types of error. While other benefits in terms of legibility, clear audit trails and reduced reliance on paper storage mean that eP is likely to become ubiquitous, organisations introducing eP should take action to identify those error types that increase with eP and how they can be prevented. Based on our data, duplication of therapy is a key area on which to focus; a parallel qualitative study suggests that screen design is an important consideration in this respect, with eP screens often precluding viewing of all prescribed medication at one time.³⁰ Other eP-related factors that may influence prescribing errors include usability and system training.^29–30 We recommend that organisations supplement implementation advice in published tools^31–32 with local data on specific risks.

Implications for research

Our findings suggest ward to be a significant predictor of both prescribing error rates and the effect of eP. Subsequent research in this field should therefore take this into account in study design and analysis, as others have suggested elsewhere.³³ It is not clear why eP increased the error rate on some wards, decreased it on others and had no apparent effect on others. While possible reasons include the effect of different prescribers and different pharmacists and/or wider cultural issues around medication safety, our data suggest that patient factors may be a significant contributor to this variation. It is not clear whether this is due to differences in clinical conditions, medications, or prescribers. Further research, both qualitative and quantitative, should therefore explore these areas, taking into account the sociotechnical context as well as other factors that can affect prescribing errors in the context of eP.²⁹ Recent research also suggests that local implementation factors affect outcomes relating to safety and efficiency, even where the same commercial system is implemented.³⁴ Given that our findings suggest that errors were corrected less swiftly with eP, this is also an area for further investigation.

Conclusion

Our analysis suggests that eP has no statistically significant effect on the rate of erroneous medication orders, but that ward and patient have significant effects. Errors involving incorrect doses and illegible or incomplete orders were less common with eP; those involving duplication, omission, incorrect drug and incorrect formulation were more common. Actions are needed to mitigate these error types; future studies should give more consideration to the effect of ward and patient factors on prescribing error rates and the effect of eP.

Footnotes

Acknowledgements

The authors thank Helen Bell for assistance with data analysis and Ofran Almossawi for statistical advice.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article represents independent research funded by the National Institute for Health Research (NIHR) Imperial Patient Safety Translational Research Centre and the NIHR Health Protection Research Unit in Healthcare Associated Infections and Antimicrobial Resistance at Imperial College London, in partnership with Public Health England (PHE). The views expressed are those of the authors and not necessarily those of the National Health Service (NHS), the NIHR, PHE or the Department of Health and Care.

Bryony Dean Franklin supervises a PhD student part funded by Cerner, a supplier of hospital electronic health record systems including electronic prescribing (eP). Bryony Dean Franklin and Seetal Puaar received funding from Pfizer for delivering teaching at a one-off symposium on medication safety unrelated to this study.

References

World Health Organization. The third WHO global patient safety challenge: medication without harm, http://www.who.int/patientsafety/medication-safety/en/ (2017, accessed 10 January 2018).

Lewis

Dornan

Taylor

, et al. Prevalence, incidence and nature of prescribing errors in hospital inpatients: a systematic review. Drug Safety 2009; 32(5): 379–389.

Dornan

Ashcroft

Heathfield

, et al. An in depth investigation into causes of prescribing errors by foundation trainees in relation to their medical education: EQUIP study. London: General Medical Council, 2009.

Franklin

Reynolds

Shebl

, et al. Prescribing errors in hospital inpatients: a three-centre study of their prevalence, types and causes. Postgrad Med J 2011; 87: 739–745.

Ryan

Ross

Davey

, et al. Prevalence and causes of prescribing errors: the prescribing outcomes for trainee doctors engaged in clinical training (PROTECT) study. PLoS ONE 2014; 9(1): e79802.

Seden

Kirkham

Kennedy

, et al. Cross-sectional study of prescribing errors in patients admitted to nine hospitals across North West England. BMJ Open 2013; 3: e002036.

Department of Health. The NHS plan: a plan for investment, a plan for reform, http://webarchive.nationalarchives.gov.uk/20130107105354/ http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/@ps/documents/digitalasset/dh_118522.pdf (2000, accessed 22 May 2018).

Radley

Wasserman

Olsho

LEW

, et al. Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems. J Am Med Inform Assn 2013; 20: 470–476.

Ammenwerth

Schnell-Inderst

Machan

, et al. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. J Am Med Inform Assn 2008; 15: 585–600.

10.

Nuckols

Smith-Spangler

Morton

, et al. The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 2014; 3: 56.

11.

Shamliyan

Duval

, et al. Just what the doctor ordered: review of the evidence of the impact of computerized physician order entry system on medication errors. Health Serv Res 2008; 43(1 Pt 1): 32–53.

12.

Brown

Lilford

RJ.

The stepped wedge trial design: a systematic review. BMC Med Res Methodol 2006; 6: 54.

13.

Ahmed

Garfield

Jani

, et al. Impact of electronic prescribing on patient safety in hospitals: implications for the UK. Pharm J 2016, https://www.pharmaceutical-journal.com/research/review-article/impact-of-electronic-prescribing-on-patient-safety-in-hospitals-implications-for-the-uk/20201013.article

14.

Evans

Benham

Garrard

CS.

A comparison of handwritten and computer-assisted prescriptions in an intensive care unit. Crit Care 1998; 2: 73.

15.

Mitchell

Usher

Gray

, et al. Evaluation and audit of a pilot of electronic prescribing and drug administration. J Inform Technol Healthcare 2004; 2(1): 19–29.

16.

Shulman

Singer

Gildstone

, et al. Medication errors: a prospective cohort study of hand-written and computerised physician order entry in the intensive care unit. Crit Care 2005; 9(5): R516–R521.

17.

Franklin

O’Grady

Donyai

, et al. The impact of a closed-loop electronic prescribing and administration system on prescribing errors, administration errors and staff time: a before and after study. Qual Saf Health Care 2007; 16: 279–284.

18.

National Information Board. Personalised health and care 2020: using data and technology to transform outcomes for patients and citizens, a framework for action, www.gov.uk/government/uploads/system/uploads/attachment_data/file/384650/NIB_Report.pdf (2014, accessed 22 May 2018).

19.

NHS England. Global digital exemplars, https://www.england.nhs.uk/digitaltechnology/info-revolution/exemplars/ (2017, accessed 22 May 2018).

20.

Franklin

Reynolds

Sadler

, et al. The effect of the electronic transmission of prescriptions on dispensing errors and prescription enhancements made in English community pharmacies: a naturalistic stepped wedge study. BMJ Qual Saf 2014; 23: 629–638.

21.

Metcalfe

Statistical power: still an essential element in sample size calculation. BMJ 2009; 339: b3985.

22.

Dean

Barber

Schachter

What is a prescribing error?

Qual Health Care 2000; 9(4): 232–237.

23.

Allan

Baker

KN.

Fundamentals of medication error research. Am J Hosp Pharm 1990; 47: 555–571.

24.

Gardner

Altman

DG.

Statistics with confidence: confidence intervals and statistical guidelines. London: BMJ Publications, 1989.

25.

Westbrook

Hooper

, et al. Effectiveness of a ‘do not interrupt’ bundled intervention to reduce interruptions during medication administration: a cluster randomized controlled feasibility study. BMJ Qual Saf 2017; 26: 734–742.

26.

Reckmann

Westbrook

Koh

, et al. Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review. J Am Med Inform Assn 2009; 16: 613–623.

27.

Ranji

Rennke

Wachter

RM.

Computerised provider order entry combined with clinical decision support systems to improve medication safety: a narrative review. BMJ Qual Saf 2014; 23: 773–780.

28.

Ash

Sittig

Poon

, et al. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assn 2007; 14(4): 415–423.

29.

Brown

Mulcaster

Triffitt

, et al. A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care. J Am Med Inform Assn 2017; 24(2): 432–440.

30.

Puaar

Franklin

BD.

Impact of an inpatient electronic prescribing system on prescribing error causation: a qualitative evaluation in an English hospital. BMJ Qual Saf 2018; 27: 529–538.

31.

ONC change package for improving EHR usability, https://www.healthit.gov/sites/default/files/playbook/pdf/usability-change-plan.pdf (accessed 31 July 2018).

32.

ePrescribing Toolkit for NHS Hospitals, http://www.eprescribingtoolkit.com/tools/ (accessed 31 July 2018).

33.

Westbrook

Raban

, et al. Stepped-wedge cluster randomised controlled trial to assess the effectiveness of an electronic medication management system to reduce medication errors, adverse drug events and average length of stay at two paediatric hospitals: a study protocol. BMJ Open 2016; 6: e011811.

34.

Ratwani

Savage

Will

, et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assn 2018; 25: 1197–1201.