Abstract
Introduction:
The handling of drug–drug interactions regarding QTc-prolongation (QT-DDIs) is not well defined. A clinical decision support (CDS) tool will support risk management of QT-DDIs. Therefore, we studied the effect of a CDS tool on the proportion of QT-DDIs for which an intervention was considered by pharmacists.
Methods:
An intervention study was performed using a pre- and post-design in 20 community pharmacies in The Netherlands. All QT-DDIs that occurred during a before- and after-period of three months were included. The impact of the use of a CDS tool to support the handling of QT-DDIs was studied. For each QT-DDI, handling of the QT-DDI and patient characteristics were extracted from the pharmacy information system. Primary outcome was the proportion of QT-DDIs with an intervention. Secondary outcomes were the type of interventions and the time associated with handling QT-DDIs. Logistic regression analysis was used to analyse the primary outcome.
Results:
Two hundred and forty-four QT-DDIs pre-CDS tool and 157 QT-DDIs post-CDS tool were included. Pharmacists intervened in 43.0% and 35.7% of the QT-DDIs pre- and post-CDS tool respectively (odds ratio 0.74; 95% confidence interval 0.49–1.11). Substitution of interacting agents was the most frequent intervention. Pharmacists spent 20.8 ± 3.5 min (mean ± SD) on handling QT-DDIs pre-CDS tool, which was reduced to 14.9 ± 2.4 min (mean ± SD) post-CDS tool. Of these, 4.5 ± 0.7 min (mean ± SD) were spent on the CDS tool.
Conclusion:
The CDS tool might be a first step to developing a tool to manage QT-DDIs
Plain Language Summary
Keywords
Introduction
Drug–drug interactions (DDIs) regarding QTc-prolongation are common in daily practice due to the high number of drugs known for prolonging the QTc-interval. Currently, over 50 drugs are associated with causing Torsade de Points (TdP) by prolonging the QTc-interval, according to the CredibleMeds® QT-drug lists of the Arizona Center for Education and Research on Therapeutics (AzCERT). 1 This number has been increasing over the years as new drugs are added to the QT-drug list due to monthly reviews of AzCERT. 2 QTc-prolongation is used as a surrogate marker for the risk of TdP, a ventricular tachycardia which may ultimately lead to ventricular fibrillation or sudden cardiac death.3–5 Although QTc-prolongation is not the perfect marker for arrhythmia risk as many other risk factors play a role in developing TdP, it has become the primary safety parameter among health care professionals, because it is still the most validated marker for the proarrhythmic potency of drugs.5–7
Although a QTc-prolonging drug in itself will rarely induce clinically relevant QTc-prolongation (>500 ms), a combination of QTc-prolonging drugs in patients with multiple risk factors can result in QTc-intervals above 500 ms.8,9
The Dutch drug database ‘G-standard’, which contains information for clinical decision support, describes the current guidelines for risk management of drug safety alerts. In The Netherlands, clinical decision support (CDS) systems in primary and secondary care generate QT-DDI alerts when two QTc-prolonging drugs with a known risk of TdP are combined. More than 40% of the processed drug prescriptions lead to drug safety alerts. 10 However, the specificity of the alerts generated by CDS systems is very low, resulting in a low number of interventions. At the moment, there is a complete lack of discrimination when handling these QT-DDIs. Many of the generated QT-DDI alerts do not require an intervention. With the increasing number of QTc-prolonging drugs, alert fatigue could be imposed on physicians. Low specificity alerts contribute to non-compliance with current guidelines.11–15 To decrease the alert burden, more advanced clinical rules including clinical parameters such as patient characteristics and laboratory values are used to improve the specificity of the alerts and decrease the alert rate.16–20
In the case of QT-DDI alerts, an advanced clinical rule should be able to discriminate low- and high-risk patients for developing QTc-prolongation. QT-DDI alerts are redundant in patients with no other risk factors for QTc-prolongation. In high-risk patients, QTc-prolonging drugs should be either substituted or routine electrocardiogram (ECG) monitoring is required. This clinical rule should be developed irrespective of the QTc-prolonging drugs involved in the QT-DDIs, due to insufficient data on the absolute effect on QTc-prolongation of these drugs. A risk profile of each individual patient will improve accuracy of QT-DDI alerts and support the risk management of QT-DDIs. 8
Therefore, a CDS prediction tool was developed to assess the risk of QT-DDIs for developing QTc-prolongation (Figure 1).21,22 The main aim of this study was to determine the effect of such a CDS tool on the interventions made by pharmacists in primary care. We also explored the usability of the CDS tool in clinical practice.

Clinical decision support tool to assess the risk of QT-DDIs.
Methods
Development of a prediction model
This CDS prediction model was developed by performing a prospective observational study in 107 patients using two or more QTc-prolonging drugs with a known risk of TdP
1
to identify risk factors for QTc-prolongation.
21
A standard 12-lead resting ECG was recorded at the estimated time of peak concentration (
Study design
An intervention study was performed using a pre- and post-design in 20 community pharmacies in The Netherlands. We implemented the use of the CDS tool (consisting a paper-based flowchart) to study the impact on the handling of QT-DDIs.
All QT-DDIs that occurred during a pre- and post-CDS tool period of 3 months were included. The QTc-prolonging drugs involved in the QT-DDIs are listed at the CredibleMeds® QT-drug list with a known risk of TdP (Supplemental material Table S1 online). Only pharmacies using the pharmacy information system Pharmacom® (by TSS Pharma Partners) in the region of Rotterdam (Rijnmond) were included to ensure conformity in data capture and data extraction. The medical ethics review board of the Erasmus University Medical Centre approved the protocol and waived the requirement for obtaining informed consent (MEC-2015-513). The study was conducted according to the principles of the Declaration of Helsinki.
Study population
All QT-DDIs including QTc-prolonging drugs with a known risk of TdP that occurred in the community pharmacies during a study period of 3 months before and 3 months after implementation of the CDS tool were included. QT-DDIs of patients younger than 18 years old were excluded.
Outcome measures
The primary outcome measure of this study was the proportion of QT-DDIs in which pharmacists intervened. An intervention was defined as a consultation with prescribers to discuss the clinical relevance of the QT-DDI, and proposal for further actions to be taken (hereafter referred to as intervention). Secondary outcome measures were the types of interventions made by pharmacists. The interventions for the QT-DDIs were subsequently categorized in (I) dispensing both drugs on account of the prescriber, (II) ECG monitoring, (III) substitution of one of the interacting agents, (IV) dose adjustments or (V) (temporarily) stopping one of the interacting agents. We have also studied the difference between first-time prescriptions and repeat-prescriptions on the handling and types of interventions. Another secondary outcome measure was the time spent on handling QT-DDIs. Finally, the CDS tool was evaluated by pharmacists on usability in clinical practice.
Data collection
For all QT-DDIs, the following variables were collected: the management of the QT-DDI including interventions, the interacting drugs and the dosages of interacting drugs.
For all patients: age, gender, comorbidities (registered as drug–disease interactions) were collected. Concomitant drug use was retrieved from the medication history up to 1 year prior to the QT-DDI alert. The following laboratory values were collected, if registered in Pharmacom®: renal function, liver function parameters and electrolyte serum levels. Patient data were handled confidentially and were extracted anonymously according to the Dutch Personal Data Protection Act (Wbp). All patients with QT-DDI alerts were captured in an electronic clinical data management system (OpenClinica, LLC, Waltham, United States).
CDS tool
The CDS tool was implemented in the participating community pharmacies after a 3 month baseline analysis as a tool to support the electronic handling of QT-DDI alerts. The CDS tool consisted of a paper-based flowchart identifying patients that were at increased risk for developing QTc-prolongation as is shown in Figure 1. The criteria of step 1 of the flowchart were based on a literature review and an expert panel with two cardiologists with expertise in electrophysiology and publications in the field.23–30 Before implementation, the pharmacists were trained in using the CDS tool. The CDS tool was expected to be used for all QT-DDI alerts during the post-implementation period. When patients scored ⩾6 using the tool, an intervention by pharmacists was recommended. The risk scores and types of interventions were documented using a paper form.
Usability clinical decision support tool
After a period of 3 months, the tool was evaluated by the pharmacists on usability in clinical practice using the System Usability Scale (SUS) of Brooke.28,29 The SUS is based on 3 usability measures suggested by the International Organization for Standardization (ISO)-9241-11: effectiveness, efficiency and satisfaction. The SUS consists of a 10-item questionnaire covering subjective items of usability using a five-point Likert scale with a degree from total disagreement (1) to total agreement (5). For evaluation of the CDS tool, 10 items were formulated so that they were compatible for the CDS tool (Supplemental Table S2). Items 1, 3, 6, 7 and 9 were positively formulated and items 2, 4, 5, 8 and 10 were negatively formulated. The SUS score was calculated, first by recalculating the score of each item (1–5) to a range from 0–4 using the following formula; for the positively formulated items: scale position minus 1, and for the negatively formulated items: 5 minus scale position. Second, the sum of these scores was multiplied by 2.5 to obtain the overall SUS score. 31 Total SUS scores range from 0 to 100; if SUS scores are <60 the system is considered to be unacceptable, 60–70 is acceptable, 70–80 is good, 80–90 is very good and >90 is excellent.32,33
Statistical analysis
Based on information provided by the Stevens Institute for Research (SIR) in Leiden, The Netherlands, Dutch community pharmacies on average dispense QTc-prolonging drugs 140 times a year with a QT-DDI (interaction code 6297). That results in 0.5 QT-DDIs per day per pharmacy in which the DDI-causing drug is dispensed. A QT-DDI does not always result in dispensing, so we estimate that one QT-DDI per day per pharmacy will be handled. So, in 3 months (pre and post measurement) approximately 45 QT-DDIs per pharmacy were expected to be monitored, which would result in a total number of 900 QT-DDIs in 20 community pharmacies. With an alpha of 0.05 and a power of 80%, a difference in proportion of QT-DDIs with intervention of 60% (pre)
The primary outcome was determined by dividing the number of QT-DDIs with an intervention by the total number of QT-DDIs. Univariate logistic regression analysis was used to analyse the primary outcome between the measurements before and after implementation of the CDS tool. If a variation in interaction characteristics occurred between the before- and after-period, the primary outcome was adjusted using multivariate logistic regression analysis. Odds ratios (ORs) and their 95% confidence intervals (CIs) were reported. Secondary outcome measures were analysed using descriptive statistics.
Results
Study population
The baseline characteristics of the participating community pharmacies are presented in Table 1. The after-period included 16 community pharmacies because four pharmacists, and therefore four pharmacies, discontinued their participation because of construction work of the pharmacy and shortages in personnel. A total of 928 QT-DDI alerts were generated during the pre- and post-CDS tool phases, of which 401 QT-DDIs were included for analysis. In the before-period, 244 QT-DDIs belonging to 233 patients were included for analysis. Six QT-DDIs were excluded because they occurred in patients <18 years old (Figure 2). In the after-period, a total of 157 QT-DDIs of 149 patients were included, as shown in Figure 3. Only 23 patients were included in both the before- and after-periods; in 17 patients the QT-DDIs were identical.
Baseline characteristics of the participating community pharmacies.
EPR, electronic patient record; FTE, fulltime-equivalent; GP, general practitioner; HKZ, Harmonization quality assessment in Health Sector; SD, standard deviation.

Flowchart of QT-DDI inclusions in before-period.

Flowchart of QT-DDI inclusions in after-period.
The baseline patient characteristics of patients in the before- and after-periods are shown in Table 2. The two groups did not significantly differ in patient characteristics. The median (interquartile range) age of the total cohort was 65 (28) years and most QT-DDI alerts belonged to female patients (64.1%). From only a limited numbers of patients the renal function parameters (10.2%) and potassium levels (0.8%) could be extracted from Pharmacom®. In clinical practice, pharmacy information systems are frequently linked to the information system of the general practitioners (GPs). Unfortunately, it was not possible to retrieve these variables from the GP information system due to privacy legislation.
Baseline characteristics of study population.
Independent
Chi-square test.
Fisher’s exact test.
BMI, body mass index; COPD, chronic obstructive pulmonary disease; CVA, cerebrovascular accident; eGFR, estimated glomerular filtration rate; IQR, interquartile range; MDRD, Modification of Diet in Renal Disease; TIA, transient ischemic attack.
There was no significant difference in the proportion of first-time prescriptions in the QT-DDIs in the before-period (62.5%) compared with the after-period (73.9%,
Top-10 QT-DDIs.
Missing values 1 and 2.
QT-DDI, QT drug–drug interaction.
Outcome measures
There was no significant difference in the proportion of QT-DDIs for which an intervention was made after implementing the CDS tool; 43.0% before and 35.7% after implementation (OR 0.74, 95% CI 0.49–1.11;
An interacting agent was substituted in 41.0% of the interventions in the before-period and in 46.4% in the after-period. In 37.7% of the QT-DDIs with intervention, the QTc-prolonging drugs were dispensed in the before-period, and in 26.8% of the QT-DDIs in the after-period. Almost 12% of all alerts were incorrect QT-alerts because one of the interacting agents was already stopped or the combination was not given simultaneously. In 5.2% of the QT-DDIs, pharmacists did not properly document their actions. The variety of interventions performed by pharmacists are presented in Table 4.
Type of intervention.
ECG, electrocardiogram.
After implementation, 157 QT-DDI alerts were handled electronically. However, pharmacists completed the paper forms in only 30.6% of the QT-DDI alerts. Of these, 30 QT-DDIs (63.5%) could be dispensed without intervention according to the flowchart of the CDS tool. Pharmacists dispensed the QT-DDI with no intervention, and therefore adhered to the flowchart in 19 cases (63.3%). In the remaining cases (
When we focus on the criteria of step 1 of the flowchart, the proportion of interventions when QT-DDIs included amiodarone did not differ between the two periods (48.1%
Intervention rate of alerts divided into first-time (FP) and repeat (RP) prescriptions.
Chi square test.
Time spent by pharmacists
In the before-period, nine pharmacists documented the time spent on the management of QT-DDIs in 56 QT-DDIs. On average, pharmacists spent 21 min on the management of QT-DDIs, as shown in Table 6. Consultation with the prescriber was the most time-consuming and took 10 min on average. In five cases the handling of QT-DDIs encompassed more than 1 day, because the pharmacists could not reach the prescriber. Nevertheless, these days were not counted in the overall time as this does not represent the time pharmacists actually spent on QT-DDIs.
Time for handling QT-DDIs in community pharmacies before implementation of the clinical decision support tool.
FP, first-time prescription; PA, pharmacy assistant; PIS, Pharmacy Information System; QT-DDI, QT drug–drug interaction; RP, repeat prescription.
After implementation, time management was documented in 48 QT-DDIs (Table 7). Pharmacists selected a variety of QT-DDIs. Based on these QT-DDI, pharmacists spent on average 15 min on the management of QT-DDIs, approximately 6 min less than in the before-period. Time spent on consultation with the prescriber was reduced to 4 min. Of these 15 min, nearly 5 min were spent on completing the CDS flowchart.
Time for handling QT-DDIs in community pharmacies after implementation of the clinical decision support tool.
FP, first-time prescription; PA, pharmacy assistant; PIS, Pharmacy Information System; QT-DDI, QT drug–drug interaction; RP, repeat prescription.
Usability CDS prediction tool
The mean ± SD SUS score of the CDS tool was 74.1 ± 19.1 (14 pharmacists). The highest (maximum of 4) mean score per question was 3.5 and was scored on question 1 regarding satisfaction; pharmacists would like to use the CDS tool in clinical practice. The lowest (minimum of 0) mean score per question was scored on question 4 regarding reliability; pharmacists expected to need the support of different literature sources, besides the tool, to safely manage QT-DDIs. Three pharmacists ranked the CDS tool <60 and considered the tool as ‘unacceptable’, mainly because the tool was time-consuming in their opinion. The overall suggestion of most pharmacists was to eventually integrate the clinical decision support tool into the pharmacy information system.
Discussion
To our knowledge, this was the first study on a CDS tool to support the handling of QT-DDIs in community pharmacies. Our study has shown that the implementation of an advanced CDS tool did, not significantly, reduce the proportion of QT-DDIs for which an intervention was made (43.0–35.7%,
At first, we hypothesized that an advanced CDS tool would result in more interventions, but more than 70% of the QT-DDI alerts did not require an intervention according to the CDS tool and could be considered as irrelevant. Several other studies show that specification of alert triggers by advanced clinical decision rules could decrease the alert rate up to 90%.34–37 In total, 75% of QT-DDIs were handled with the CDS tool. Pharmacists did not comply with the flowchart of the CDS tool when the QT-DDIs included amiodarone, because they did not feel comfortable dispensing QT-DDIs with amiodarone. Therefore, if the tool is implemented in clinical practice, education on the risks of QTc-prolongation and TdP can be useful to achieve more compliance by pharmacists.
Repeat prescriptions for chronic medication are common in primary care. However, many QT-DDIs are only relevant at the start of therapy, and are, therefore, more likely to be followed by an intervention. 11 In our study, 50% of the first-time prescription QT-DDIs were followed by an intervention and 21% of the repeat-prescription QT-DDIs were followed by an intervention, as is shown in Table 7. The CDS tool enables reassessment of repeat-prescription QT-DDIs when the condition of a patient might change during chronic treatment where a patient might become a high-risk patient.
Van der Sijs
A strength of this study was the pre- and post-design of the study which enables us to study the current handling of QT-DDIs. Also, the 20 community pharmacies included in this study represent the general Dutch community pharmacies according to the Dutch Foundation for Pharmaceutical Statistics (SFK) Facts and Figures of 2015.
This study also has some potential limitations. First, the tool was paper-based and was not integrated into the electronic CDS system of the community pharmacies. Consequently, the documentation of the QT-DDIs was limited because work processes in community pharmacies are fully digitalized. Only 48 forms were completed
Second, apart from the criteria incorporated in step 1 of the flowchart, the tool does not stratify the various QT-DDIs. Although the QTc-prolonging drugs have different pharmacological pathways for inducing QTc-prolongation, it is relatively unknown whether combining QTc-prolonging drugs with different pharmacological pathways has an additive or synergistic effect on the extent of QTc-prolongation.39,40 Therefore, in this study we assumed the synergistic effect of the QTc-prolonging drugs to be similar. Also, the number of laboratory values retrieved from the pharmacy information systems was low. In reality, this number of available laboratory values is higher because pharmacy information systems are frequently linked to information systems of GPs. In 58.1% of the included pharmacies, the pharmacy information system was linked to the GP information system, as shown in Table 1. Additionally, a potassium level of <2.5 mmol L−1 scores two points according to the risk model; however, it is unlikely that an outpatient would have a value this low and any measurement this low would likely have been managed in an inpatient setting. Although the aim of the present study was to evaluate how such a tool would perform in a primary care setting, the tool was also developed for use in hospitals, and in such a setting a low potassium level may occur.
In total, four pharmacies dropped out of the study due to construction work of the pharmacy and shortages of personnel. These could potentially lead to biased results, but as these pharmacists dropped out before the implementation of the tool, this is probably not the case.
For future perspectives, this clinical decision support tool deserves further investigation to assess its effect when it is integrated in the pharmacy information system. Such a study should be performed in large patient groups with clinically relevant endpoints such as QTc-prolongation before implementation in clinical practice can be recommended. Ideally, the system will then automatically calculate a risk score for the individual patient and only generate alerts if the risk score is >6, resulting in more specific alerts.
In conclusion, these results suggest that the clinical decision support tool might be an effective tool to manage QT-DDIs
Supplemental Material
sj-pdf-1-taw-10.1177_2042098621996098 – Supplemental material for The use of a clinical decision support tool to assess the risk of QT drug–drug interactions in community pharmacies
Supplemental material, sj-pdf-1-taw-10.1177_2042098621996098 for The use of a clinical decision support tool to assess the risk of QT drug–drug interactions in community pharmacies by Florine A. Berger, Heleen van der Sijs, Teun van Gelder and Patricia M. L. A. van den Bemt in Therapeutic Advances in Drug Safety
Footnotes
Acknowledgements
The authors thank all pharmacists who participated in this research project. We also thank the SIR Institute for Pharmacy Practice and Policy (Leiden, The Netherlands) and Health Base Foundation (Houten, The Netherlands) for their contribution to this research project.
Author contributions
PvdB and TvG designed the research study. FB contributed to the conduct of the study, where FB collected data from the participating pharmacies. Data analysis was performed by FB. The results were analysed, interpreted, and discussed by FB, HvdS, PvdB, TvG. FB drafted the manuscript and all co-authors revised and approved the final version of the manuscript.
Conflict of interest statement
The authors declare that there is no conflict of interest.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was supported by a grant from the Royal Dutch Pharmacists Association (Koninklijke Nederlandse Maatschappij ter bevordering der Pharmacie, KNMP holding B.V.).
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
