Identifying medication harm in hospitalised patients: a bimodal,targeted approach

Medication harm is in the spotlight with the World Health Organization’s (WHO) Global Patient Safety challenge. ¹ Defined as “any negative patient outcome or injury related to a medication”, ² it is a costly problem for healthcare systems. Mortality, secondary to medication harm, is reported at 0.3% of all hospital patients.^3,4 Recent local studies in two Australian hospitals linked medication harm to anticoagulants, insulin and antihypertensives,^5,6 which is similar to findings from other health systems.^7,8

The WHO has set a target of reducing medication harm by 50%, in particular by expediting digital solutions. In response there has been a flurry of publications on predictive modelling to help with early detection and prediction of those at high risk.^5,9–11

Identifying medication harm is challenging however, as it occurs in combination with other causal factors and spans all hospital events. The difficulty of defining causality is reflected in the wide ranges of reported rates of inpatient medication harm, ranging from 1.6% to 35% of admissions.^8,12,13 Incident reporting by clinicians is the traditional method of ascertaining medication harm. However, under-reporting is the norm and incident reports reflect only the tip of the medication-harm iceberg. Further, incident reporting systems focus on errors and do not include patient harm arising from appropriate medicines use. The gold standard method is a prospective appraisal of patient medical records, laboratory tests and interviews with patients and care providers.^14,15 This approach is considered the most reliable as it is likely to detect more incidents than retrospective methods, but it is resource intensive and not feasible in large patient cohorts.

Targeted review of medical records using coded data or trigger tools (TTs) to identify which records should be reviewed is one method to address this.^16,17 This is increasingly viable with the availability of digital healthcare data for triggers. Identification of medication harm using routinely collected patient data is important as it is a major opportunity for improving clinical care. We propose a bimodal, targeted approach combining triggers and diagnostic codes to identify inpatient medication harm. We also discuss how machine learning (ML) and clinicians working at the coal face can improve medication-harm detection.

Targeted retrospective medical record review in the hospital setting

Traditionally epidemiological evidence of medication harm is from studies using retrospective medical record review, for example the Harvard Medical Practice Study and other similar studies.^18,19 These studies retrospectively reviewed all patient records to detect the presence of harm. However, this non-targeted review (i.e. screening every patient record) is resource intensive and the determination of causality is difficult. ¹⁴ Methods to reduce the volume of records and improve validity are needed.

Two methods that use a targeted method to facilitate a structured approach to medical record review include the use of triggers^20,21 and clinical coding data. ²² The Adverse Drug Event Trigger Tool (ADE TT) described by Rozich ²⁰ and later refined by the Institute for Health Care Improvement enables targeted medical record review. ²³ The ADE TT contains a set of triggers which signal that medication harm may have occurred. ²³ Triggers include administration of antidotes or out-of-range laboratory results, for example, administration of protamine sulphate and/or a supratherapeutic activated partial thromboplastin time (aPTT), signalling a potential bleed due to heparin therapy.

There are now several TTs available. One study evaluated 8 different TT methods in 1115 adult inpatients and found low sensitivities, ranging from 2% to 16%, but high specificities of 99%. ²⁴ A systematic review of automated ADE detection in electronic health records using TTs identified 11 studies (7 of which were in a paediatric population), with a median positive predictive (PPV) value of 40%. ²⁵ Another TT for identifying medication harm in older adult inpatients (TRIGGER-CHRON) was evaluated across 12 Spanish hospitals and reported an overall PPV of 22%. ²⁶ The differences in findings may in part be due to patient cohorts (i.e. age), and variations in trigger sets. It should be noted that TTs with high PPVs are possible through the selection of individual triggers with higher PPVs, however this comes at a cost of losing clinically important but rare adverse events. ²⁷ Whilst a wide range of PPVs have been reported, findings indicate that there is the potential to use TTs to drive medication-harm detection and spearhead patient safety initiatives. The advent of electronic health records (EHRs) is well suited to the TT methodology, which can be systematically applied to hospital records (e.g. by specifying predetermined threshold changes for laboratory tests), using automated algorithms, to detect harm in real time and at an institutional level. ²⁸ Work to update and standardise the triggers is needed.

A second method is the use of coding data. International Classification of Disease (ICD) codes are allocated for every hospital separation, primarily for reimbursement purposes. ICD-10-Y codes (medication-related codes) can be coupled with diagnosis codes (e.g. codes for bleeding or hypoglycaemia) as prompts for medical record review in large data sets. ²⁹ Similar to TTs, studies evaluating ICD codes report variable accuracy, with sensitivities ranging from 6% to 56% and specificity ranging from 95% to 99%.^17,22,30 For example, a Canadian study reported sensitivities of 9–83% when evaluating ICD codes at four tertiary hospitals. ³¹ The wide range of sensitivities suggest that coding practices (i.e. extent of clinician documentation and the skill of coders) are likely to determine the effectiveness of coding.

One potential approach to improve validity is to use a bimodal approach using both ADE TT and coding in the same EHRs. This can be streamlined to flag patient records for potential medication harm for clinical review to establish causality. Furthermore, triggers can be adapted to local circumstances, and for special populations (e.g. older adults). ³² Validated causality assignment tools, such as the WHO Uppsala Monitoring Centre criteria, should be used to standardise assessments. ³³ A review of the literature found no studies evaluating this bimodal approach.

ML and automated harm detection

ML methods are becoming increasingly popular in the healthcare setting to assist clinicians with diagnosis and prognosis.^34,35 Whilst in their infancy, studies have explored the use of ML to predict or detect medication harm. A recent study described the development of multiple risk prediction models. ³⁶ The authors utilised a de-identified dataset from a Swedish hospital. A series of ICD-10 codes related to the diagnosis of medication harm were used as the outcomes for models (e.g. I95.2 = drug-related hypotension). Whilst some models achieved high area under the curve (AUC) for their predictive performance (ranging from 0.8 to 0.9), there was no clinical verification of the accuracy of the coded outcome data (i.e. causality assessment to ensure codes were correctly allocated). ³⁶ An Australian study also reported using ML and ICD codes to detect medication harm in a tertiary hospital. The automated algorithm demonstrated promising performance with an AUC of 0.803. ³⁷

Whilst ML offers an exciting opportunity to detect harm on a large scale, its success is dependent on the availability of high-quality data. This is challenging with outcomes that are sparse and difficult to verify, such as medication harm. Furthermore compared with conventional statistical methods, the inner workings of ML models are not transparent and they can lack face validity.^38,39 This modelling approach can be abstract to clinicians, known as the so-called ‘black box’, it creates challenges in detection of bias, overfitting and for external evaluation and user acceptance testing. ⁴⁰

Conclusion

For harm-mitigation strategies to be successful a pragmatic standardised approach to detection is essential. Leveraging a targeted, bimodal method of medical record review enables healthcare professionals to capitalise on the availability of EHRs to assist with the detection of medication harm.