Ultrasound-Assisted Catheter-Directed Thrombolysis for the Management of Pulmonary Embolism: A Single Center Experience in a Community Hospital

Abstract

Current guidelines recommend anticoagulation alone for low-risk pulmonary embolism (PE) with the addition of systemic thrombolysis for high-risk PE. However, treatment recommendations for intermediate-risk PE are not well-defined. Due to bleeding risks associated with systemic thrombolysis, ultrasound-assisted catheter-directed thrombolysis (USAT) has evolved as a promising treatment modality. USAT is thought to decrease the rate of major bleeding by using localized delivery with lower thrombolytic dosages. Currently, there is little guidance on the implementation of USAT in the real-world clinical setting. This study was designed to evaluate our experience with USAT at this single community hospital with a newly initiated Pulmonary Embolism Response Team (PERT). All patients identified by the PERT with an acute PE diagnosed by a computed tomography (CT) scan from January 2021 to January 2023 were included. During the study period, there were 89 PERT activations with 40 patients (1 high-risk and 37 intermediate-risk PE) receiving USAT with alteplase administered at a fixed rate of 1 mg/h per catheter for 6 h. The primary efficacy outcome was the change in Pulmonary Embolism Severity Index (PESI) score within 48 h after USAT. The primary safety outcome was major bleeding within 72 h. The mean age was 57.4 ± 17.4 years and 50% (n = 20) were male, 17.5% (n = 7) had active malignancy, and 20% (n = 8) had a history of prior deep vein thrombosis (DVT) or PE. The mean PESI score decreased from baseline to 48 h post-USAT (84.7 vs 74.9; p = 0.025) and there were no major bleeding events. The overall hospital length of stay was 7.5 ± 9.8 days and ICU length of stay was 2.2 ± 2.8 days. This study outlined our experience at this single community hospital which resulted in an improvement in PESI scores and no major bleeding events observed.

Keywords

thrombosis catheter-directed therapy pulmonary embolism

Introduction

After coronary artery disease and stroke, acute pulmonary embolism (PE) ranks third among the most common causes of death from cardiovascular disease.¹ The incidence of PE is approximately 60-120 per 100 000 in the United States with a mortality rate of approximately 30% if untreated, compared to 2-11% when treated with anticoagulation.^2,3 In addition to the acute risks associated with PE, approximately one-third of patients will have long-term complications such as PE reoccurrence, chronic thromboembolic disease, pulmonary hypertension, atrial fibrillation, and decreased exercise capacity and quality of life.³ An acute PE can be classified into high-risk, intermediate-risk, and low-risk based on hemodynamic stability and presence of right ventricular (RV) strain or elevated troponin levels.^4,5 Barring any contraindication, therapeutic anticoagulation is the primary treatment for PE with systemic thrombolysis being reserved for patients with high-risk PE (hemodynamic instability).^4–7 While systemic thrombolysis has been shown to reduce mortality for high-risk PE, its use for intermediate-risk PE (hemodynamically stable with RV dysfunction) remains inconclusive given an increase in bleeding events.⁸

In an attempt to mitigate bleeding risks associated with systemic thrombolysis, catheter-directed therapies (CDT) have evolved for the treatment of both intermediate-risk and high-risk PE. Four main types of catheter-based therapies are currently available: (1) mechanical thrombectomy without thrombolysis, (2) pharmacomechanical CDT, (3) standard CDT, and (4) ultrasound-assisted catheter-directed thrombolysis (USAT).⁹ It is theorized that adding targeted ultrasound waves to CDT facilitates the separation of fibrin strands to allow better penetration, improved efficacy and therefore a lower thrombolytic dose.¹⁰ Of the several catheters that are FDA approved for the treatment of acute PE, ultrasound-assisted catheter-directed thrombolysis (USAT) is the most widely studied.^8,11 The Ultrasound-Assisted, Catheter-Directed Thrombolysis for Acute Intermediate-Risk Pulmonary Embolism (ULTIMA) trial demonstrated that fixed-dose USAT using the EkoSonic™ Endovascular System (EKOS) plus anticoagulation improved RV function to a greater extent than anticoagulation alone without causing major bleeding.¹² In the Submassive and Massive Pulmonary Embolism Treatment With Ultrasound Accelerated Thrombolysis Therapy (SEATTLE II) trial, the use of this device was associated with decreased RV size, reduced pulmonary hypertension, and decreased thrombus burden with no reported intracranial hemorrhage in patients with acute PE.¹³ In 2018, varying dosing strategies were investigated in the Optimum Dose and Duration of Acoustic Pulse Thrombolysis Procedure in Acute Intermediate-Risk Pulmonary Embolism (OPTALYSE PE) trial which demonstrated that lower USAT fibrinolytic doses and shorter duration regimens were as effective as the regimens used in previous studies.¹⁴ The Higher-Risk Pulmonary Embolism Thrombolysis (HI-PEITHO) trial is an ongoing study evaluating the clinical outcomes of USAT using the EKOS catheter plus anticoagulation versus anticoagulation alone in intermediate-risk PE patients.¹⁵ While most evidence on the safety and efficacy of USAT originates from its use in intermediate-risk PE, USAT may be considered in high-risk PE patients in whom systemic thrombolysis is contraindicated or has failed.^4–7,16 Overall, the results of established trials and ongoing studies have suggested that USAT is a promising strategy to relieve obstruction quickly and restore pulmonary circulation in acute PE patients.^17,18

Due to the high morbidity and mortality rates associated with acute PE and the various treatment modalities available, many institutions have established dedicated PE response teams (PERT) to rapidly evaluate patients and individualize treatment options.¹⁹ In December 2020, a PERT program was established at MemorialCare Heart and Vascular Institute at Long Beach Medical Center, which adopted using the USAT in select patients with dosing strategies as described in the OPTYLASE PE trial. This study evaluates our experience with USAT at this single community hospital with a newly initiated PERT.

Methods

Patient Selection

The data collection protocol was approved by the Institutional Review Board at Long Beach Medical Center. We conducted a retrospective, single-arm study of patients with an acute PE receiving USAT at Long Beach Medical Center between January 2021 to January 2023. Long Beach Medical Center is a 453-bed community hospital and level I trauma center serving the patients of Long Beach, CA. Baseline demographics, comorbidities, presenting characteristics, treatment details, and clinical outcomes were retrospectively captured from the electronic medical record and hospital communications system. Patients were included if they were at least 18 years or older, had a computed tomography (CT) scan confirming PE, and received CDT with the EKOS catheter. Patients were excluded if they received CDT with a catheter other than the EKOS catheter or had incomplete medical records (eg, transfer to an outside hospital).

PERT Activation

The diagnosis of acute PE was made by the treating physician and verified by CT imaging. PERT was activated in patients with presumed high-risk or intermediate-risk PE by messaging the PERT on-call physicians using the hospital's communication system. PERT physicians collaborated with the ordering physician to gather pertinent information and determine available diagnostic and treatment options. During this discussion, the PERT provided recommendations and assisted with arranging advanced therapies, if appropriate.

Treatment

The USAT procedure was performed in an angiographic suite by interventional cardiologists and catheters were placed per manufacturer recommendations. For unilateral PE, one catheter was placed in the involved vessel. For bilateral PE, two catheters were placed: one in each of the involved vessels. After the procedure, the patients were transferred to the intensive care unit (ICU) for close monitoring and continued on alteplase 1 mg/h per catheter for a total of 6 h (total 6 mg vs 12; 1 vs 2 lungs). During the thrombolytic infusion, the systemic heparin rate was reduced to 500 units/h and resumed full therapeutic dosing after the removal of catheters.

Outcomes

The primary efficacy outcome was the change in Pulmonary Embolism Severity Index (PESI) score pre- and post-USAT.^4,20 PESI scores were stratified into class I (very low mortality risk, 0-1.6%), class II (low mortality risk, 1.7-3.5%), class III (moderate mortality risk, 3.2-7.1%), class IV (high mortality risk, 4.0-11.4%), or class V (very high mortality risk, 10.0-24.5%).⁴ Class I and II were considered low mortality risk. Pre-USAT PESI score was measured at baseline, and post-USAT PESI score was measured 48 h after the start of USAT or at hospital discharge, if sooner. The primary safety outcome was the frequency of major bleeding within 72 h after the start of USAT as defined by the International Society on Thrombosis and Haemostasis (ISTH) criteria: (1) fatal bleeding, (2) symptomatic bleeding in a critical area or organ such as intracranial, intraspinal, intraocular, retroperitoneal, intra-articular, pericardial, or intramuscular with compartment syndrome, or (3) bleeding causing a decrease in hemoglobin of ≥2 g/dl or bleeding leading to transfusion of ≥2 U of whole blood or red blood cells.²¹ Alternatively, nonmajor bleeding was defined as a bleeding event that is not a major bleed as defined by ISTH.²¹ Secondary outcomes included hospital length of stay, ICU length of stay, recurrent PE at 30 days, hospital readmission at 30 days, and all-cause mortality. The efficiency of the PERT protocol was assessed through treatment time intervals for patients admitted from the emergency department with a confirmed diagnosis of PE.

Statistical Analysis

Statistical analyses were performed using Excel software and SPSS Statistics Version 29.0. Our sample size determination was performed using G*Power software for our primary efficacy outcome: change in PESI score pre- and post-USAT.²² Assuming an expected mean difference of 10 and standard deviation of 27, which was calculated from a sample of the retrospective data, an estimated sample size of 60 patients would be required to achieve an α level of 0.05 and a power of 0.80. Continuous variables were presented as mean ± standard deviation (SD) and categorical data were shown as number and percent. To compare PESI scores pre- and post-procedure, a paired t-test was used. Findings with a p < 0.05 were deemed statistically significant.

Results

There were a total of 89 PERT activations and 47 patients with orders for CDT during the study period. Of these patients, a total of 40 patients with high-risk (2.5%, n = 1) and intermediate-risk (97.5%, n = 39) PE were included. The mean age was 57.4 ± 17.4 years, 50% (n = 20) were male, 17.5% (n = 7) had active malignancy, and 20% (n = 8) had a history of prior deep vein thrombosis (DVT) or PE. All patients had a baseline RV/LV ratio ≥ 0.9 as estimated by CT scan. The demographics, comorbidities, risk factors, and presenting characteristics are summarized in Table 1.

Table 1.

Baseline Characteristics.

Age—years^a	57.4 ± 17.4
Male sex	20 (50)
Weight—kg^a	97.6 ± 28.0
Ethnicity
White	18 (45)
Black or African American	16 (40)
Hispanic	3 (7.5)
Asian	2 (5)
Other	1 (2.5)
Comorbidities
Obesity (BMI > 30 kg/m²)	22 (55)
Renal insufficiency	1 (2.5)
Hypertension	19 (47.5)
Diabetes mellitus	6 (15)
Congestive heart failure	2 (5)
History of VTE	8 (20)
Active malignancy	7 (17.5)
Receiving chemotherapy	4 (10)
Recent surgery	5 (12.5)
Oral contraceptive use	4 (10)
Anticoagulation use	4 (10)
PE classification
Intermediate-risk	39 (97.5)
High-risk	1 (2.5)
Presenting characteristics
Heart rate^a	104 ± 18
Systolic blood pressure^a	132 ± 23
Cardiac arrest	2 (5)
Intubated	2 (5)
Mode of hospital arrival
Emergency department	29 (72.5)
Transfer-in	0 (0)
In-hospital	11 (27.5)

Abbreviations: BMI, body mass index; VTE, venous thromboembolism; PE, pulmonary embolism; RV/LV, right ventricle to left ventricle; HR, heart rate; SBP, systolic blood pressure.

Mean ± standard deviation.

All patients were initiated on therapeutic anticoagulation with unfractionated heparin titrated to an aPTT goal of 65-100 s (correlating to anti-Xa levels of 0.3 IU/mL and 0.7 IU/mL) managed by pharmacists per the institution-approved dosing guideline. This included a heparin bolus of 80 units/kg followed by heparin at a rate of 18 units/kg/h for patients less than 70 years old or 15 units/kg/h for patients 70 years of age or older. A clinical pharmacist performed empiric dose adjustments based on baseline coagulation results and risk factors for bleeding in accordance with the approved dosing guideline. After the fibrinolytic infusion, all patients were resumed on full therapeutic heparin dosing.

Following USAT, the average PESI score decreased from 84.7 ± 36.2 at baseline to 74.9 ± 36.7 at 48 h (p = 0.025). At baseline, the average PESI score was in class II (low mortality risk, 1.7-3.5%) and remained in class II at 48 h of the procedure. Therefore, the average overall risk stratification did not change. However, 27.5% (n = 11) of all patients were re-classified into a lower PESI class from baseline. Of the patients in PESI class II at baseline, 18.2% (n = 2) were re-classified into class I. Of the patients in PESI class III at baseline, 80% (n = 4) were re-classified into either class I or II. Of the patients in PESI class IV at baseline, 42.9% (n = 3) were re-classified into either class II or III. Of the patients in PESI class V at baseline, 40% (n = 2) were re-classified into class II. Overall, there was an increase in the number of patients stratified into the low mortality classes (class I and II) from 57.5% (n = 23) at baseline to 77.5% (n = 31) at 48 h. The primary outcome results are summarized in Figure 1 and Table 2.

Figure 1.

Change in PESI score from baseline to 48 h.

Table 2.

Summary of Initial and Resultant PESI Classes from Baseline to 48 h.

Initial PESI Class, N (%)	Resultant PESI Class 1	Resultant PESI Class 2	Resultant PESI Class 3	Resultant PESI Class 4	Resultant PESI Class 5
Class 1 12 (30%)	12 (100%)	0 (0%)	0 (0%)	0 (0%)	0 (0%)
Class 2 11 (27.5%)	2 (18.2%)	9 (81.8%)	0 (0%)	0 (0%)	0 (0%)
Class 3 5 (12.5%)	1 (20%)	3 (60%)	1 (20%)	0 (0%)	0 (0%)
Class 4 7 (17.5%)	0 (0%)	2 (28.6%)	1 (14.3%)	3 (42.9%)	1 (14.3%)
Class 5 5 (12.5%)	0 (0%)	2 (40%)	0 (0%)	0 (0%)	3 (60%)

Class I (very low mortality risk, 0-1.6%), Class II (low mortality risk, 1.7-3.5%), Class III (moderate mortality risk, 3.2-7.1%), Class IV (high mortality risk, 4.0-11.4%), Class V (very high mortality risk, 10.0-24.5%).

The overall hospital length of stay was 7.5 ± 9.8 days, and the ICU length of stay was 2.2 ± 2.8 days. For patients admitted from the ED with a confirmed diagnosis of PE, the hospital length of stay was 5.0 ± 4.2 days and ICU length of stay was 1.8 ± 1.8 days. One patient had a recurrent PE at 30 days. Three patients did not survive to discharge for underlying causes of death not attributed to an acute PE (pancreaticobiliary adenocarcinoma, septic shock, neutropenic sepsis).

There were no major bleeding events, but there were 4 documented non-major bleeding events. Non-major bleeding events included two patients with hematuria, one with bleeding from a surgical drain, and one with bloody oral secretions.

The duration of unfractionated heparin and the choice of a long-term anticoagulation agent were determined at the discretion of the treating physician. For patients admitted from the ED with a confirmed diagnosis of PE, the door-to-unfractionated heparin time was 4.7 ± 3.9 h, door-to-CT time was 3.8 ± 3.3 h, door-to-PERT activation time was 8.5 ± 8.1 h, PERT activation to USAT time was 5.7 ± 9.6 h, and completion of USAT to initiation of oral anticoagulant time was 73.6 ± 119.0 h.

Discussion

In the current study performed at a large community hospital with a newly initiated PERT, we found that USAT was associated with a significant reduction in PESI scores at 48 h. PESI scores are most useful in predicting 30-day mortality in patients with acute PE. In a study by Moores et al, PESI scores that were recalculated 48 h after admission correlated well with observed 30-day mortality.²⁰ The investigators reclassified 83 (27.3%) patients as low risk (class I and II) at 48 h. The 30-day mortality rate in these patients was 1.2% versus 11.3% in those who remained at high risk.²⁰ In our study, 20% (n = 8) of patients were reclassified at 48 h to lower risk strata. These preliminary findings suggest that our USAT protocol is associated with a reduction in PESI score from baseline.

In terms of safety, there were no major bleeding events reported for patients included in the current study. USAT is thought to reduce overall thrombolytic exposure by allowing direct administration to the pulmonary arteries. Earlier trials, including the ULTIMA and SEATTLE II trials, used 12-24 h durations of thrombolytic therapy.^12,13 Our dosing strategy was adopted from the OPTALYSE PE trial in which patients were randomized into 1 of 4 treatment groups and received alteplase doses ranging from 4 to 12 mg per lung and infusion durations between 2 to 6 h. There were five major bleeding events, one in arm 3, which used our current dosing strategy of 1 mg/h/catheter infusion. Improvements in RV/LV ratio were demonstrated across all four groups.¹⁴ Considering the lack of difference between arms, a 6-h duration was chosen for our institution's protocol. Overall, our safety findings indicate a better safety profile than previously published trials which may be attributed to our smaller sample size and single-centered population.

Hospital length of stay and treatment time outcomes were longer than expected, even for patients admitted from the ED with an acute PE diagnosis. A CDT approach is postulated to reduce the length of stay due to a lower risk of bleeding complications. Unfortunately, we did not collect data to evaluate for confounding factors that may have attributed to a longer length of stay. In this study, the duration of heparin therapy after USAT was at the physician's discretion. Future investigation can be directed toward exploring opportunities for earlier transitioning to oral anticoagulation therapy to reduce hospital length of stay.

In previous USAT studies, the study populations comprised largely of patients with intermediate-risk PE.^12–14 In the current study, one high-risk PE patient was included. Although systemic thrombolysis is the standard of care in high-risk patients,^4–7 our high-risk patient was not a candidate for systemic thrombolytics: initially presenting with altered mental status. While this patient demonstrated improvement in their PESI score and no bleeding event, the high-risk PE population is heterogeneous and further research into the role of USAT in this patient population is warranted.

During the time of this study, USAT was the primary CDT option for all PE cases for which the PERT was activated, while other treatment options included anticoagulation alone and systemic thrombolysis. Patients were selected to receive USAT if they had a high-risk PE or intermediate-risk PE with high-risk features (elevated troponin or brain natriuretic peptide [BNP]) with significant thrombus on CT imaging. Since the culmination of this study, mechanical thrombectomy has been utilized an alternative treatment option for the intermediate-risk and high-risk PE population at this community hospital. Therefore, future plans to compare USAT to other PE treatment modalities, such as mechanical thrombectomy should be considered.

There are several limitations to our study. This retrospective analysis involved a small study group that did not meet power to detect our primary outcome. Additionally, our primary efficacy endpoint is not a recognized marker for PE improvement. The PESI score relies on several patient characteristics, some of which will produce inflated baseline scores that are unaffected by the treatment approach taken, such as age and sex. Therefore, evaluating the PESI score may not reflect a true improvement in PE outcomes. Further, this study did not adjust for confounding factors that may have affected clinical outcomes and length of stay. The assessment of time is also limited by the inclusion of patients who were admitted for a non-related illness and developed an acute PE during the hospital admission. Lastly, the evaluation of a single treatment regimen without a comparator group likely involves inherent selection bias. While the implementation of this USAT protocol demonstrated favorable outcomes based on the retrospective data presented, the applicability of these results to other treatment centers remains unclear. Further investigation is warranted in prospective, multi-centered, and randomized trials to guide the implementation of USAT in the real-world clinical setting.

Conclusions

Implementing USAT at this large community hospital demonstrated improvement in PESI scores and no major bleeding events. Furthermore, randomized controlled trials with comparator groups would be critical in expanding our knowledge of USAT and its role in the real-world clinical setting.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jasmine Ventenilla

References

Goldhaber

Bounameaux

. Pulmonary embolism and deep vein thrombosis. Lancet. 2012;379(9828):1835-1846.

Freund

Cohen-Aubart

Bloom

. Acute pulmonary embolism: a review. JAMA. 2022;328(13):1336-1345.

Data and statistics on venous thromboembolism. Centers for Disease Control and Prevention. February 16, 2023. Accessed May 29, 2023. https://www.cdc.gov/ncbddd/dvt/data.html

Konstantinides

Meyer

Becattini

, et al. 2019 ESC guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the European respiratory society (ERS). Eur Heart J. 2020;41(4):543-603.

Jaff

McMurtry

Archer

, et al. Management of massive and submassive pulmonary embolism, iliofemoral deep vein thrombosis, and chronic thromboembolic pulmonary hypertension: a scientific statement from the American Heart Association. Circulation. 2011;123(16):1788-1830.

Ortel

Neumann

Ageno

, et al. American Society of Hematology 2020 guidelines for management of venous thromboembolism: treatment of deep vein thrombosis and pulmonary embolism. Blood Adv. 2020;4(19):4693-4738.

Stevens

Woller

Kreuziger

, et al. Antithrombotic therapy for VTE disease: second update of the CHEST guideline and expert panel report. Chest. 2021;160(6):e545-e608.

Meyer

Vicaut

Danays

, et al. Fibrinolysis for patients with intermediate-risk pulmonary embolism. N Engl J Med. 2014;370(15):1402-1411.

Singh

Shafi

Rali

Panaro

Lakhter

Bashir

. Contemporary Catheter-based treatment options for management of acute pulmonary embolism. Curr Treat Options Cardiovasc Med. 2021;23(7):44.

10.

Garcia

. Endovascular management of acute pulmonary embolism using the ultrasound-enhanced EkoSonic system. Semin Intervent Radiol. 2015;32(4):384-387.

11.

Kline

Rodino

Dorszynski

Murray

Cicci

Iyer

. Ultrasound-assisted catheter-directed thrombolysis versus systemic anticoagulation alone for submassive pulmonary embolism. J Thromb Thrombolysis. 2021;52(1):130-137.

12.

Kucher

Boekstegers

Müller

, et al. Randomized, controlled trial of ultrasound-assisted catheter-directed thrombolysis for acute intermediate-risk pulmonary embolism. Circulation. 2014;129(4):479-486.

13.

Piazza

Hohlfelder

Jaff

, et al. A prospective, single-arm, multicenter trial of ultrasound-facilitated, catheter-directed, low-dose fibrinolysis for acute massive and submassive pulmonary embolism: the SEATTLE II study. JACC Cardiovasc Interv. 2015;8(10):1382-1392.

14.

Tapson

Sterling

Jones

, et al. A randomized trial of the optimum duration of acoustic pulse thrombolysis procedure in acute intermediate-risk pulmonary embolism: the OPTALYSE PE trial. JACC Cardiovasc Interv. 2018;11(14):1401-1410.

15.

Klok

Piazza

Sharp

ASP

, et al. Ultrasound-facilitated, catheter-directed thrombolysis vs anticoagulation alone for acute intermediate-high-risk pulmonary embolism: rationale and design of the HI-PEITHO study. Am Heart J. 2022;251:43-53.

16.

Kuebel

Winter

Martin

, et al. Systemic thrombolytic and ultrasound-assisted catheter-directed thrombolysis for treatment of acute pulmonary embolism: a 7-year, multicenter experience. J Thromb Thrombolysis. 2023;55(3):545-552.

17.

Kaymaz

Akbal

Tanboga

, et al. Ultrasound-assisted Catheter-directed thrombolysis in high-risk and intermediate-high-risk pulmonary embolism: a meta-analysis. Curr Vasc Pharmacol. 2018;16(2):179-189.

18.

Pei

Liu

Yaqoob

, et al. Meta-analysis of Catheter directed ultrasound-assisted thrombolysis in pulmonary embolism. Am J Cardiol. 2019;124(9):1470-1477.

19.

Rosovsky

Zhao

Sista

Rivera-Lebron

Kabrhel

. Pulmonary embolism response teams: purpose, evidence for efficacy, and future research directions. Res Pract Thromb Haemost. 2019;3(3):315-330. Published 2019 Jun 9.

20.

Moores

Zamarro

Gómez

, et al. Changes in PESI scores predict mortality in intermediate-risk patients with acute pulmonary embolism. Eur Respir J. 2013;41(2):354-359.

21.

Kaatz

Ahmad

Spyropoulos

Schulman

; Subcommittee on Control of Anticoagulation. Definition of clinically relevant non-major bleeding in studies of anticoagulants in atrial fibrillation and venous thromboembolic disease in non-surgical patients: communication from the SSC of the ISTH. J Thromb Haemost. 2015;13(11):2119-2126.

22.

Faul

Erdfelder

Buchner

Lang

. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods. 2009;41(4):1149-1160. doi:https://doi.org/10.3758/BRM.41.4.1149.