Abstract
Objectives
Interventions to drive quality of care using strategies to empower clinical staff with personal and local performance indicators should lead to measurable improvements in outcome. The current evidence for the linkage of quality metrics and patient outcome usually relies on surrogate measures. There is a continuing need for the development of specific measurable indicators of quality of care, which have demonstrable links to patient outcome.
Design
A single centre naturalistic study to validate electronically collected drug administration and documentation measures of quality improvement, during the introduction of a quality improvement programme.
Setting
A large academic teaching hospital in the UK.
Participants
Staff involved in administration of prescribed medication using an in-house developed prescribing information and communication system (PICS).
Main outcome measures
Mortality data from nationally collected ‘hospital episode statistics’ and markers of the precision of medicines administration and documentation ‘uncharted’ and ‘missed’ drug doses.
Results
A 16.2% reduction in local mortality rate was demonstrated (P=0.013) compared to static mortality rates throughout England. Similar temporal reductions were seen in the rates of missed and uncharted doses of medication at the hospital over the same period.
Conclusions
When monitored as part of a quality improvement strategy, ‘missed’ and ‘uncharted’ drug dose information may reflect wider institutional changes and act as a valid indicator of quality of healthcare more generally.
DECLARATIONS
All authors work within the University Hospital Birmingham NHS Foundation Trust which is collaborating with CSE Healthcare Systems to commercialize the PICS system in the UK; All authors also declare that there are no spouses, partners, or children with relationships with commercial entities that might have an interest in the submitted work; No non-financial interests that may be relevant to the submitted work
JJC is funded by the National Institute for Health Research (NIHR) through the Collaborations for Leadership in Applied Health Research and Care for Birmingham and Black Country (CLAHRC-BBC) programme. All other authors have received no specific funding for this work
Review by hospital management board
Carmen Tsang
Introduction
Over the past decade there has been an increasing emphasis on providing safe care to hospitalized patients. Much of the interest was sparked from healthcare-related incidents highlighted in key reports from the 1980s and 1990s. 1 3 There have been many attempts to institute provider changes with the aim of ‘quality improvement’ in various areas-including infection control, medication management, evidence-based practice, and patient surveillance. Most interventions have the ultimate aim of reducing morbidity and mortality, but outcome-based research is often difficult to perform and surrogate outcome measures, for example length of stay or hospital readmissions, are frequently used to measure the effectiveness of interventions.
Medication errors have been an area of significant focus in the quality and safety movement. Although there is much focus on incorrect drug or dosing errors, it is likely that late or omitted administration of prescribed medication whilst in hospital is more prevalent and carries similar risks of adverse outcome. Missed doses can cause direct harm through omission of newly instituted medications such as antibiotics, or may cause morbidity through the failure to maintain regular treatment such as antiparkinsons medication. There has been a clinical, economic, academic, technological and societal thrust to reduce preventable drug errors in order to improve patient care. Despite this, the evidence suggests that medication delivery and administration systems in hospitals and skilled nursing facilities continue to operate sub-optimally. 4 Care providers have been encouraged to seek strategies to tackle system-related errors in order to provide higher quality services, which encompass safe, effective, patient-centred, timely, efficient and equitable care. 5 Health information technology is increasingly used to provide patient-specific clinical information. 6 The increasing introduction of computerized prescribing over the last decade has impacted favourably on the rates of medication errors. 7 9 However, as systems have evolved, not only has the clinical decision support underpinning such systems improved, but the system data capture, audit and presentation capabilities have also progressed. Systems can be interrogated to review detailed information on each prescription and administration, or analyzed to uncover evolving trends. 10 Data gleaned from the analysis of such systems can be made visible to managers and care providers to improve patient care. Clinical dashboards, which provide access to summary data on care processes with a clear, user-friendly interface, are one example of this technology in practice. 11 Interventions to drive quality of care using strategies to empower clinical staff with personal and local performance indicators should lead to measurable improvements in outcome. Evidence supporting the use of these interventions is currently sparse and mainly limited to the primary care setting. 12 We present observational data recorded over the period of the introduction of a multi-faceted Trust-wide strategy focusing on quality of care, demonstrating that specific measurable indicators of quality of care-namely markers of the precision of medicines administration and documentation-can be linked to improvements in mortality.
Methods
Setting and study population
The study was undertaken following approval by the Trust's research governance unit at University Hospitals Birmingham NHS Foundation Trust. Situated in the West Midlands in England, the 1250 bedded academic teaching hospital consists of a staff of over 7,000 employees delivering general, specialist and emergency services to the local population. The hospital does not include inpatient obstetrics, gynaecology or paediatric services.
The development of the Trust's vision, values and strategy to ‘Deliver the Best in Care’ has been explicitly led by the Board of Directors, acknowledging the evidence that hospitals whose Boards are actively engaged with the quality of care have been shown to achieve better outcomes. 13 The strategy is based on a Trust-wide drive to reduce errors to a minimum. The underpinning hypothesis is that reducing errors in care delivery will improve outcomes, even if the individual errors themselves may not be judged as significant. It is recognized that structured change must be multi-faceted in order to be effective. 14
The multi-faceted strategy employed by the Trust to drive quality of care consists of:
An in-house developed Prescribing Information and Communication System (PICS) with advanced clinical decision support, to minimize errors at the point of decision making. System changes have been introduced within the user interface over the study period to alert healthcare professionals to late or missed doses. Information included on clinical dashboard interface by ward level
In order to drive personal accountability and responsibility the trust introduced Chief Executive Officer (CEO)-led, themed ‘care omissions’ root cause analysis (RCA) monthly meetings, initiated in February 2010. Cases are selected for these meetings using interrogation of the prescribing database in PICS, mainly, but not exclusively, based on medicines management issues. The responsible clinical management team undertakes a detailed breakdown of the whole episode of care, identifying suboptimal areas using a modification of the national Department of Health RCA documentation. This process allows the Board's expectations of acceptable performance to be widely shared and understood, as well as identifying systematic failures and occasionally individual poor performance.
Mortality rates are closely monitored and benchmarked, using England's Hospital Episode Statistics (HES) data. However, the Trust does neither widely publicize nor use these negative patient outcomes internally within the organization. Patient mortality is clearly important to the hospital, but its use as a focus for the error reduction strategy could weaken the message that all errors matter.
The prescribing system includes a medication administration section where every dose can and should be charted as administered or omitted-failure to chart either of these options is classed as an ‘uncharted’ dose. The rate of uncharted doses of medication was not publicized during the study period and were not specifically focused on as part of the care omissions meetings. This enabled the number of uncharted doses to be monitored by the executive team to identify if staff were failing to officially omit drugs in an attempt to modify their clinical area's performance figures.
Data Capture
For the purposes of this analysis, comparison has been made between national mortality rates of emergency admissions within 30 days of any nondiagnostic interventional procedure (calculated as total deaths divided by total deaths plus discharges). This measure includes all therapeutic, interventional and surgical procedures. Obstetrics, gynaecology and paediatrics services were excluded from the analysis to reflect the patient population investigated. Data were analyzed over a period of four years from April 2007 to February 2011, spanning the implementation of the error reduction strategy. Twelve-month moving averages were used in order to remove seasonal variation. All mortality data were extracted from national Hospital Episode Statistics (HES). Fields used from the national HES dataset were: admission method (emergencies >13), primary procedure not null to identify procedure, discharge method and death code (code 4 for method of discharge or 79 for discharge destination). We sourced HES datasets for England from the NHS Information Centre.
The medication administration records within the audit database of our electronic prescribing system, PICS, were interrogated to obtain data on every medication dose over the same time period. Data for administered doses, omitted doses and uncharted doses were extracted. Frequency of missed antibiotic doses was analyzed separately to act as a measure of the adherence to therapeutic interventions likely to be linked to short-term outcome.
Data Assurance
The mortality statistic chosen for comparison was deaths in hospital within 30 days of nondiagnostic interventional procedures for emergency admissions. This statistic has been used as a core performance parameter within the NHS for a number of years following its publication within the NHS Executive's ‘Performance Assessment Framework’ document, published in 1999, 15 and more recently in the UK ‘National Clinical and Health Outcomes Knowledge base’. The coding relating to this mortality statistic has remained stable over time and the hospital undergoes regular independent monitoring of its coding practices.
Analysis
The primary analysis considered the trend in mortality rates over time for England as a whole and for the hospital Trust separately. Trends in missed and uncharted dose rates with respect to time were calculated. All of these correlations were analysed using Spearman's Rho. P values of <0.05 were taken to be significant. All analyses were undertaken using Statistical Package for the Social Sciences (SPSS) version 19 software.
Results
The results of the data analysis demonstrate a significant fall in the mortality rate at the University Hospitals Birmingham NHS Foundation Trust (UHBFT) over the period of analysis. The 12-month moving average for mortality between April 2007 and March 2008 is 5.44% and for March 2010 to February 2011 is 4.56%, representing a reduction of 16.2% of the initial mortality rate. This fall in mortality is not mirrored by national rates in the rest of England, which remained unchanged (Figure 1). Over the period of analysis (April 2007 to February 2011), the local mortality rate trend over time significantly decreases (Spearman's Rho -0.360 (P =0.013)) whereas national mortality rates for England show no significant change (Spearman's Rho 0.022 (P =0.886)). An analysis of discharge destinations for our hospital showed no change in the proportions of patients discharged to different places (for example nursing home versus own place of residence) suggesting no increase in dependent living at the expense of mortality (data not shown).
Time series plot of 12-month moving average mortality rates for England (crosses) and University Hospital Birmingham (diamonds). Trust intervention time points labelled
There is a demonstrable temporal decline in rates of missed doses of antibiotics, non-antibiotics and non-charted medication doses over the period of intervention (Table 2). When missed antibiotics and non-charted drug doses are plotted with mortality, similar temporal patterns are seen (Figure 2).
Time series plot of 12-month moving average University Hospital Birmingham mortality rates (solid squares), missed doses of antibiotic (diamonds), missed non-antibiotic (triangles), uncharted doses (clear squares), by month. Trust intervention time points labelled Data on mortality rates and markers of the precision of medicines administration and documentation over time (significant figures shown in bold)
Discussion
This naturalistic study captures a period during which a multi-faceted approach to quality improvement was implemented in a large teaching hospital with well developed informatics systems allowing continuous automated audit. A strong relationship was identified between temporal reductions in missed medication doses and emergency case mortality, with a significant improvement on baseline hospital and national mortality statistics. Direct correlations between markers of improvement in service delivery and hard markers of benefit, such as morbidity or mortality, are rare in the literature.
A focus on personal responsibility in healthcare delivery, with emphasis on timely drug delivery has been successfully implemented. It is recognized that more accurate drug administration is not the sole determinant of quality in this area, as other initiatives have demonstrated reductions in harm including avoidance of interruptions during nursing drug rounds. 16 The wider agenda about more vigilant caregivers also correlates with better outcomes seen when, for example, patients are entered into research trials, or focused safety checklists are employed. 17 The surveillance of electronic health records is reminiscent of the management interventions in the original Hawthorne experiments, now more famously known as the ‘Hawthorne effect’ in experimental interventions. 18 It is recognized that nursing staff at the point of care facilitate early detection of problems and can instigate actions that minimize patient adverse events. 19 There is also evidence that board level engagement with quality issues-including a quality committee, a defined strategy and the use of clinical dashboards with appropriate indicators-is associated with a reduction in risk-adjusted mortality. 20
As with all purely observational data, we are unable to directly attribute causation between missed drug dose rates and emergency case mortality. The data demonstrate the strongest temporal link with a reduction in the rate of uncharted doses, rather than either missed doses of all drugs or antibiotics specifically; with the initial fall in mortality occurring before any fall in missed antibiotic doses. Uncharted doses are likely to closely represent drug omissions. A small proportion of omissions may reflect improvements in the documentation of dose administration by nursing staff, although the potential disciplinary action arising from failure to document an administered drug makes this unlikely to account for a large proportion of recorded omissions. A reduction in the rate of uncharted doses without a concurrent rise in missed antibiotic doses should represent an increase in antibiotic administration. There is strong evidence that the timely administration of antibiotics is associated with reduced mortality, 21 and so it is plausible that our reported improvements in antibiotic administration rates have impacted here. The observational nature of our study design did not allow the inclusion of a control population. We are therefore unable to directly attribute improvements in mortality to the Trust's quality improvement strategy, and other unidentified confounding variables may have also influenced outcome.
Uncharted doses were not specifically focused on by the error reduction campaign, and so the reduction in their number is likely to represent an institution-wide cultural and behavioural change in attitude to errors previously perceived as unimportant. The demonstrable fall in mortality may therefore reflect more multi-factorial effects leading to a global reduction in errors.
These data have identified that markers of the precision of medicines administration and documentation could be used as performance indicators suitable for the continuous monitoring of patient care. Selecting sensitive indicators for quality of care is difficult,22,23 but we propose that ‘missed doses’ may represent patient vigilance more generally and could perform as a marker of quality of care. With the spread of electronic prescribing systems and informatics tools allowing complete data aggregation and visualization of such quality measures, many more healthcare institutions will be able to test this hypothesis.
Footnotes
Acknowledgements
None