Abstract
Purpose:
Previous studies have shown that activated clotting time (ACT)–guided heparinization leads to better anticoagulation levels during noncardiac arterial procedures (NCAP) than a standardized bolus of 5000 IU. Better anticoagulation should potentially result in lower incidence of thrombo-embolic complications (TEC). Comparative investigations on clinical outcomes of these heparinization strategies are scarce. This study investigated clinical outcomes of ACT-guided heparinization with a starting dose of 100 IU/kg in comparison with a single standardized bolus of 5000 IU heparin during NCAP.
Materials and methods:
Analysis from a prospectively collected database of patients undergoing NCAP in 2 vascular centers was performed. Patients receiving ACT-guided heparinization were matched 1:1 with patients receiving 5000 IU heparin using propensity score matching (PSM). Primary outcomes were TEC, bleeding complications, and mortality within 30 days of procedure or during the same admission.
Results:
A total of 759 patients (5000 IU heparin: 213 patients, ACT-guided heparinization: 546 patients) were included. Propensity score matching resulted in 209 patients in each treatment group. After PSM, the groups were comparable, with the exception of a higher prevalence of peripheral arterial disease in the ACT-guided heparinization group (103 patients, 49% vs 82 patients, 39%, p=0.039). The target ACT (>200 seconds) was reached in 198 patients (95%) of the ACT-guided group versus 71 patients (34%) of the 5000 IU group (p<0.001), indicating successful execution of the ACT-guided protocol. Incidence of TEC (13 patients, 6.2% vs 10 patients, 4.8%, p=0.52), mortality (3 patients, 1.4% vs 0 patients, p=0.25), and bleeding complications (32 patients, 15% vs 25 patients, 12%, p=0.32) did not differ between patients receiving ACT-guided heparinization and 5000 IU heparin. Protamine was administered in 118 patients (57%) in the ACT group versus 11 patients (5.3%) in the 5000 IU group (p<0.001), but did not influence incidence of TEC (17 patients, 5.9% vs 6 patients, 4.7%, p=0.61) or bleeding complications (34 patients, 12% vs 22 patients, 17%, p=0.14).
Conclusion:
No difference in TEC, bleeding complications, or mortality was found between ACT-guided heparinization and a single bolus of 5000 IU heparin during NCAP.
Clinical Impact
Previous studies have shown that activated clotting time (ACT)-guided heparinization leads to better anticoagulation levels during non-cardiac arterial procedures (NCAP) then a standardized bolus of 5000 IU. Comparative investigations on clinical outcomes are scarce. This study focussed on clinical outcomes of both protocols in NCAP in a propensity score matched cohort. Thrombo-embolic complications (TEC), bleeding complications and mortality within 30 days after NCAP or during the same admission were comparable between groups. Future studies should focus on optimizing ACT-guided protocols, specifically in patients with a high risk of TEC and bleeding complications.
Introduction
Unfractionated heparin is administered to prevent thrombo-embolic complications (TEC) during noncardiac arterial procedures (NCAP).1–4 The anticoagulant effect of heparin in an individual patient is unpredictable because of the nonlinear dose-response and elimination curve. 5 Differences in heparin efficacy are known between sexes, different brands, and even different batches of the same brand.6,7 To lower the incidence of TEC without increasing the bleeding risk, activated clotting time (ACT) is increasingly used to monitor the heparin anticoagulation effect and adjust heparin dosing in NCAP.8,9 Nevertheless, most vascular surgeons currently do not monitor heparin anticoagulation and use a standardized dose of 5000 IU heparin, independent of patient characteristics. 1
Previous studies have demonstrated the feasibility of ACT-guided heparinization.10,11 Compared with a standard bolus of 5000 IU, ACT-guided heparinization leads to better anticoagulation levels and more patients reaching the target ACT. 11 A target ACT of 200 to 250 seconds seems optimal to lower TEC incidence, with limited occurrence of bleeding complications in NCAP. 11 However, studies comparing the clinical outcomes of ACT-guided heparinization with a standardized heparin dose have not yet been conducted.
This study aimed to compare the clinical outcomes of ACT-guided heparinization with a standardized dose of 5000 IU heparin during NCAP.
Materials and Methods
Study Design and Patient Population
The MANCO-registry (Measuring Activated clotting time during Non-Cardiac arterial prOcedures, clinicaltrials.gov: NCT03426293) is an ongoing, prospective, multicenter cohort study designed to optimize heparinization strategies during NCAP using ACT. This study was an analysis of the MANCO-registry. The MANCO-protocol was reviewed and approved by the Medical Ethics Committee Noord-Holland (the Netherlands). An opt-out procedure for informed consent was performed. Included patients in the registry were older than 18 years, undergoing NCAP. Exclusion criteria were acute intervention, an eGFR < 30 mL/min (based on the 2012 KDIGO guidelines), allergy for heparin or known history of heparin-induced thrombocytopenia (HIT), and a known history for coagulation disorders or patients who received heparin prior to surgical procedure. 12 Patients were included in a university hospital (Amsterdam University Medical Center, location Vumc) and a large vascular teaching hospital (Dijklander Hospital) in the Netherlands from December 2016 until March 2022. Until January 2018 and June 2018 in the respective hospitals, patients received a standardized bolus of 5000 IU heparin. In January 2018, the ACT-guided heparinization protocol was initiated at the university hospital, and in June 2018 at the vascular teaching hospital. Patients from both treatment groups were compared.
Heparinization Strategies
Heparin was administered 3 to 5 minutes before arterial cross-clamping in case of open procedures or 3 to 5 minutes before sheath introduction (ie, before arterial puncture) when endovascular procedures were performed. Patients in the 5000 IU group received an initial dose of 5000 IU heparin. Additional heparin dosages and protamine could be administered at the vascular surgeons’ discretion. Eventual additional heparin dosages were not based on ACT. Patients in the ACT-guided heparinization group received a starting dose of 100 IU/kg heparin, followed by ACT-guided heparinization. The target ACT was >200 seconds. Follow-up heparin administrations were based on ACT. ACT measurements were performed 5 minutes after heparin administration and every 30 minutes if the ACT was >200 seconds. At the end of the procedure, when the blood flow was re-established, the target ACT was <180 seconds. Protamine could be given at the vascular surgeons’ discretion if the ACT was >180 seconds. The protocol is described in detail in previous literature.10,13
Activated Clotting Time Measurements and Data Collection
ACT measurements were performed following a standardized protocol using the Hemostasis management system plus (HMS plus; Medtronic, Minneapolis, Minnesota) with high-range ACT cartridges. ACT measurements and data collection are described in detail in previous literature. 10
Outcomes
Primary outcomes were TEC, bleeding complications, and all-cause mortality within 30 days or during the primary admission after NCAP. Primary outcomes were assessed separately. TEC that were registered included myocardial infarction, cerebrovascular infarction / transient ischemic attack (TIA), deep venous thrombosis, pulmonary embolism, thrombo-embolism, bowel ischemia, thrombo-embolic renal insufficiency, unexpected tissue loss / amputation, spinal cord ischemia, and graft thrombosis. Perprocedural blood loss was registered, and bleeding complications were classified according to the European multicentre study on Coronary Artery Bypass Grafting classification (E-CABG). Grades were class I: transfusion of 2 to 4 units of red blood cells (RBC) or transfusion of platelets, fresh frozen plasma, or prothrombin complex concentrate; class II: transfusion of 5 to 10 units of RBC or reoperation for bleeding; and class III: transfusion of >10 units of RBC.14,15 The E-CABG grade 1 or higher was registered as bleeding complication. Influence of protamine administration on TEC and bleeding complications incidence was investigated.
Propensity Score Matching
Propensity score matching (PSM) was performed using a matching software package “MatchIt” in “R” version 3.4.4 (RStudio, Boston, Massachusetts). Patients receiving 5000 IU heparin and patients receiving ACT-guided heparinization were matched based on propensity score. Propensity score was calculated in a generalized linear model, based on 18 included covariates. These 18 covariates were selected based on reported associations with TEC, bleeding complications, or mortality in previous literature.16–24 Included covariates and related studies are depicted in Supplementary Table 1. Missing data occurred in 20 of 759 patients (total: 2.6%, 5000 IU group: 1.9%, ACT-guided heparinization group: 2.9%). Due to the limited occurrence of missing data, which were considered to be missing at random, and which were not disproportionally skewed, patients with missing data were excluded and only complete case analyses were performed. 25 PSM was performed matching patients in a 1:1 ratio. A nearest neighbor matching model without replacement of cases was used. 26
Statistical Analysis
Evaluation of matching quality and postmatching analyses were performed using SPSS Statistics for Windows version 28 (IBM Corp, Armonk, New York). Normality of continuous variables was tested using Kolmogorov-Smirnov tests. Parametric data were presented as mean with standard deviations (SD), while nonparametric data were presented as median with interquartile ranges (IQR). Missing data were excluded per separate analysis. Comparative analyses were performed using the unpaired Student t test or the Mann-Whitney U test for parametric and nonparametric variables respectively. Categorical variables were expressed as frequencies and percentages. Comparison analyses were performed using Pearson χ2 test or Fisher exact test when expected counts of more than 20% of cells were less than 5. A p value <0.05 was considered statistically significant.
Results
Patient Characteristics and PSM
In total, 759 patients were included (5000 IU heparin: 213 patients, ACT-guided heparinization: 546 patients). PSM resulted in 209 patients in both groups. The groups were adequately matched based on matching covariates, with the exception of a higher prevalence of peripheral arterial disease (PAD) in the ACT-guided heparinization group (103 patients, 49% vs 82 patients, 39%, p=0.039). Baseline characteristics and evaluation of matching quality are depicted in Table 1.
Baseline Characteristics of Patients Receiving 5000 IU Heparin and Patients Receiving ACT-Guided Heparinization During Noncardiac Arterial Procedures Before and After Propensity Score Matching.
Abbreviations: ACT, activated clotting time; ASA, acetylsalicylic acid; ASA-score, American Society of Anesthesiologists classification; BMI, body mass index; eGFR, estimated glomerular filtration rate; fem-pop, femoropopliteal bypass; FFX, femoral-femoral crossover bypass; IU, international units; IQR, interquartile range; PTA, percutaneous transluminal angioplasty.
Continuous data are presented as medians with IQR. Categorical variables are presented as counts with percentages. Bold p values indicate a p value <0.05.
In the 5000 IU group, 23 patients (11%) received additional heparin (median additional heparin: 2500 IU, IQR: 1750). Median total heparin dose in the 5000 IU group was 5000 IU (IQR: –) versus 8500 IU (IQR: 3050) in the ACT-guided group (p<0.001). In the 5000 IU group, an ACT >200 seconds was reached by 71 patients (34%) versus 198 patients (95%) in the ACT-guided group (p<0.001), indicating the ACT-guided heparinization protocol was successfully executed.
Outcomes
TEC occurred in 13 patients (6.2%) who received ACT-guided heparinization versus 10 patients (4.8%) receiving 5000 IU heparin (p=0.52, Table 2). In 7 patients who suffered from thrombo-embolism or graft thrombosis, one or multiple technical causes were found that potentially contributed to development of TEC (5000 IU heparin: 2 patients vs ACT-guided heparinization: 5 patients). Technical contributors found were iatrogenic dissection in 5 patients, dislodgement of calcified plaque in 2 patients, inflow stenosis in 2 patients, and brachial artery thrombo-embolic occlusion after brachial artery catheter insertion in 1 patient. Three patients experienced unexpected tissue loss (5000 IU heparin: 1 patient vs ACT-guided heparinization: 2 patients). The patient who experienced unexpected tissue loss in the 5000 IU group underwent transmetatarsal foot amputation. Both patients in the ACT-guided group who experienced unexpected tissue loss underwent transtibial amputation, one after experiencing bypass graft thrombosis.
Primary Outcomes of 209 Patients Receiving 5000 IU Heparin and 209 Patients Receiving ACT-Guided Heparinization Within 30 Days After Noncardiac Arterial Procedure or During Primary Admission.
Abbreviations: ACT, activated clotting time, IU, international units, TEC, thrombo-embolic complications.
Continuous data are presented as medians with interquartile ranges (IQR). Categorical variables are presented as counts with percentages.
Bleeding complications are classified according to the European multicentre study on coronary artery bypass grafting classification (E-CABG). Grades are class I: transfusion of 2 to 4 units of red blood cells (RBC) or transfusion of platelets, fresh frozen plasma, or prothrombin complex concentrate; class II: transfusion of 5 to 10 units of RBC or reoperation for bleeding; and class III: transfusion of >10 units of RBC.
Bleeding complications occurred in 32 patients (15%) receiving ACT-guided heparinization versus 25 patients (12%) receiving 5000 IU heparin (p=0.32). Median perprocedural blood loss was 200 mL (IQR: 400) in both treatment groups (Table 2). Forty-three of the patients (19%) undergoing an open surgical procedure experienced bleeding complications, compared with 6 patients (4.6%) undergoing an endovascular procedure (p<0.001). Bleeding complications between endovascular and hybrid NCAP (4.6% vs 7 patients, 12%, p=0.12) and between open and hybrid NCAP (19% vs 12%, p=0.17) did not differ. TEC did not differ between open, endovascular and hybrid NCAP (18 patients, 7.9% vs 3 patients, 2.3% vs 2 patients, 3.3% respectively, p=0.058). Due to the low number of events per procedure, subgroup analyses for specific NCAP were not performed.
Three patients (1.4%) receiving ACT-guided heparinization died within 30 days after surgery or during the same admission. All 3 patients were treated for aortoiliac occlusive disease and died from pneumonia, mesenteric ischemia, or unknown cause. Mortality did not occur in patients where 5000 IU heparin was administered (p=0.25, Table 2).
Peripheral Arterial Disease
As incidence of PAD more frequently occurred in the ACT-guided group than the 5000 IU group, its influence on occurrence of TEC and bleeding complications was investigated. In the whole cohort, PAD did not significantly alter TEC incidence (PAD: 13 patients, 7.0% vs no PAD: 10 patients, 4.3%, p=0.28) or the incidence of bleeding complications (PAD: 30 patients, 16% vs no PAD: 26 patients, 11%, p=0.13).
Protamine
Protamine was administered more frequently in the ACT-guided heparinization group (118 patients, 57%) than the 5000 IU group (11 patients, 5.3%, p<0.001). Overall, median protamine dose was 2500 IU (IQR: 500). The incidence of TEC in the whole group was not influenced by protamine administration (no protamine: 17 patients, 5.9% vs protamine: 6 patients, 4.7%. p=0.61). In the whole cohort, bleeding complications did not differ in patients who received protamine (no protamine: 34 patients, 12% vs protamine: 22 patients, 17%, p=0.14).
Discussion
TEC lead to significant mortality and morbidity in NCAP.27,28 Hence, efforts should be made to minimize TEC as much as possible during these procedures. The unpredictable and variable effect of heparin in individual patients has urged the development of ACT-guided heparinization. 29 Previous literature suggested that weight-based ACT-guided heparinization targeting an ACT >200 seconds could contribute to reduction of TEC.10,30 In the current study, clinical outcomes of ACT-guided heparinization were compared with a single, standardized dose of 5000 IU heparin during NCAP. PSM resulted in comparative groups, based on a large set of confounders associated with TEC, bleeding complications, and mortality. In the ACT-guided cohort, 95% of patients reached an ACT >200 seconds, indicating ACT-guided heparinization was performed adequately. In contrast, 34% of the patients in the 5000 IU group reached this ACT. Unexpectedly, comparative analysis showed that the incidence of TEC, mortality, and bleeding complications between patients receiving ACT-guided heparinization and 5000 IU heparin did not differ.
The present findings indicate that maintaining substantial and consistent heparin levels targeting an ACT > 200 seconds through ACT-guided heparinization might not significantly improve clinical outcomes across a diverse range of NCAP. This is unexpected, because preliminary data of the MANCO-registry suggested potential improvement of clinical outcomes using this antithrombotic protocol. In these studies, a TEC incidence of 9% in patients receiving 5000 IU heparin and 4.3% in patients receiving ACT-guided heparinization was reported.10,30 In these separate studies, no comparative analysis on heparinization strategies was performed, nor was there correction for confounders. Another previous MANCO analysis found a TEC incidence of 8.7% when receiving 5000 IU heparin and 5.7% when applying ACT-guided heparinization. 31 This study was not designed as a comparative study and was specifically evaluating interventions for PAD. Therefore, the current evaluation of the MANCO registry was undertaken, comparing a standardized heparin dose with ACT-guided heparinization in PSM-balanced groups in a general vascular surgery population. Apparently, the potential difference found in earlier publications is not sustained after correction with PSM.
TEC may occur due to thrombus formation alongside intravascular sheaths or catheters or blood flow stasis after arterial cross-clamping. Heparin is administered to prevent this thrombus formation and thereby TEC. In addition, technical complications like iatrogenic dissections or dislodgement of debris may induce TEC that are impossible to prevent with any form of anticoagulation. 32 Although a 0% incidence of TEC is aimed for, some will continue to occur due to the complexity of the procedures in patients with often polyvascular extensive atherosclerosis.32,33 In 7 patients in the current study (ACT group: 5 patients vs 5000 IU group: 2 patients), additional technical factors were found after TEC occurrence. These additional factors could have compromised arterial flow and thereby caused TEC.
Mortality was low in comparison with previous literature describing NCAP and did not significantly differ between patients receiving ACT-guided heparinization and 5000 IU heparin.17,18,27 Importantly, ACT-guided heparinization did not increase bleeding complications. Based on the results of this study, the current ACT-guided heparinization protocol is safe to use in patients during NCAP.
Using a universal heparinization strategy across NCAP potentially has benefits because, in general, homogeneity in clinical protocols showed to improve patient outcomes, improve consistency in care, and, if publicly available, increase transparency.34,35 As the results of the current study did not demonstrate a beneficial effect of a standardized ACT-guided heparinization when applied during a diverse range of NCAP, it would be interesting to investigate specific procedures. Previous studies investigating optimal heparinization strategies for specific NCAP are scarce. In a study investigating percutaneous peripheral vascular interventions, comparisons using PSM were made between patients receiving a total heparin dose >60 IU/kg or <60 IU/kg, and between patients reaching a peak ACT >250 seconds or ACT <250 seconds. Increased transfusion rates were described in patients receiving >60 IU/kg heparin (4.7% vs 3.2%, p=0.016), and more patients receiving >60 IU/kg heparin or reaching an ACT >250 seconds experienced a drop in hemoglobin level >3 g/dL (7.0% vs 5.1%, p=0.01 and 10.0% vs 5.8%, p=0.03 respectively). A composite endpoint of death, myocardial infarction or TIA/stroke, and thrombotic and embolic complications did not differ between the studied groups. 16 Another study investigated the effect of very high levels of anticoagulation during carotid artery stenting. The authors compared patients reaching an ACT of 250 to 299 seconds with patients reaching 300 to 350 seconds. A peak ACT of 250 to 299 seconds was associated with a lower combined event rate of death, myocardial infarction and stroke (2.9% vs 10.4%, p<0.001), and a longer event-free survival (p<0.001). 36 Both of these studies did not compare distinct regimes of heparin dosing and ACT-monitoring but looked at the anticoagulation levels only; introducing bias as one of the important characteristics of heparin is the unpredictable effect. Another study investigated anticoagulant effects of a fixed dose of 5000 IU heparin during 140 carotid endarterectomies, using ACT. Signs of cerebral injury or wound hematomas were not found in patients who received between 80 and 90 IU/kg heparin in contrast to higher or lower heparin dosages. The authors suggested using a weight-based heparin dose of 85 IU/kg based on these findings. 37 However, different heparinization strategies could not directly be compared due to a limited sample size. Investigating optimal heparin administration for specific NCAP has the potential to further reduce complications and should be part of future research. Previously conducted studies may offer guidance toward strategies tailored to specific NCAP.
Particularly, procedures with high risk of TEC and bleeding complications could benefit from using ACT-guided monitoring. According to previous literature, open aortic repair and peripheral bypass surgery are at risk for TEC and bleeding complications compared with other NCAP.16,38–41 The increased risk of bleeding complications in these procedures was also found in the current study, in which bleeding complications frequently occurred in patients undergoing open surgery. Administering 100 IU/kg heparin to prevent TEC, and use ACT to monitor the heparin effect to prevent overdosing and underdosing could hold particular significance in these high-risk procedures. A previous Delphi study on antithrombotic strategies in fenestrated or branched aneurysm repair also recommended using ACT-guided heparinization, especially when the procedure duration exceeds 2 hours. Administering 70 to 100 IU/kg, targeting an ACT of 250 to 300 s was agreed on by most clinical experts. 42 As the Delphi design implicates, these recommendations are based on expert opinion, and evidence to support these recommendations is still required. However, the potential risk of graft thrombosis in small-caliber branches or stent grafts within target vessels could justify the advice to maintain a substantial and stable heparin anticoagulation level during these procedures. Increased attention to heparin efficacy could be warranted during procedures involving extended sheath introductions or clamping times, as the heparin effect in NCAP is highly variable and gradually declines within 3 minutes after administration. 43 Future research could demonstrate that ACT-guided heparinization is reserved for high risk and prolonged NCAP. Therefore, the results of the first randomized trial in open abdominal aortic aneurysm surgery are highly awaited for. 13
Apart from investigating specific procedures, more research is needed in defining the optimal protocol for ACT-guided heparinization. Based on prior research, the target ACT in the current study was >200 seconds. 10 Future studies should investigate if this is indeed the optimal target ACT and if an interval of 30 minutes between ACT measurements is adequate.
Strengths and Limitations
This is the first study focussing on clinical outcomes of ACT-guided heparinization in comparison with a standardized dose of 5000 IU heparin during NCAP. Both heparinization strategies were executed adequately, increasing validity of this comparison. PSM facilitated balancing groups based on a large number of confounding variables that potentially affected primary outcomes. The study was performed in both a peripheral vascular teaching hospital and a university hospital, which resulted in a broad cohort of included patients undergoing NCAP. Results are therefore transferable to the general vascular surgery population.
Several limitations of this study should be acknowledged. Despite that a substantial number of patients could be included, a very large sample size will be needed to draw definitive conclusions on clinical outcomes of ACT-guided heparinization in NCAP. Nevertheless, this is the first study indicating that ACT-guided heparinization with a target ACT of >200 seconds does not improve clinical outcomes when applied in a general vascular surgery population. Data from this study serve as a potential foundation for future studies with larger sample sizes and provide direction toward studies on specific NCAP types. Historical controls from a prospective database were used in this study, which could have introduced bias. To minimize this risk, all patients were included in a short time period (December 2016 until March 2022), reducing potential influence of methodological changes over time. Thereby, as a hospital-wide change in anticoagulation strategy in both participating hospitals was accomplished, selection bias was mitigated. Several limitations of PSM should also be mentioned. Using a large set of covariates could have caused intervention groups not being balanced on incidence of PAD (39% vs 49%, p=0.039). However, matching intervention groups based on as many confounding variables as possible that could be predictive of clinical outcomes will minimize bias and is in accordance with recommendations in literature. 44 Also, PAD did not influence occurrence of TEC or bleeding complications and intervention groups were balanced on all other variables. Therefore, a 10% difference in PAD incidence was accepted.
Conclusion
Propensity score matched analyses showed comparable incidence of TEC, bleeding complications, and mortality between patients treated with ACT-guided heparinization and standardized 5000 IU heparin during NCAP. Future, preferably randomized studies, should encompass optimizing and testing ACT-guided heparinization protocols, specifically in patients with a high risk of TEC and bleeding complications.
Supplemental Material
sj-docx-1-jet-10.1177_15266028241278137 – Supplemental material for Clinical Outcomes of 5000 IU Heparin Versus Activated Clotting Time–Guided Heparinization During Noncardiac Arterial Procedures: A Propensity Score Matched Analysis
Supplemental material, sj-docx-1-jet-10.1177_15266028241278137 for Clinical Outcomes of 5000 IU Heparin Versus Activated Clotting Time–Guided Heparinization During Noncardiac Arterial Procedures: A Propensity Score Matched Analysis by Max Hoebink, Liliane C. Roosendaal, Marie-José Beverloo, Arno M. Wiersema, T. van der Ploeg, Thomas A. H. Steunenberg, Kak Khee Yeung and Vincent Jongkind in Journal of Endovascular Therapy
Footnotes
Author Contributions
This manuscript has been seen and approved by all authors.
Data Availability
Study data were not made publicly available.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: One of the authors of this article is the Editor in Chief of the journal. To avoid a conflict of interest, an alternate member of the Editorial Board handled this submission.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was partially supported by an unrestricted research grant from Medtronic. Medtronic had no involvement in the collection, analysis and interpretation of data, writing of the report or the decision to submit the manuscript for publication.
Ethical Approval
The study protocol was reviewed and approved on January 16, 2017, by the Medical Ethics Committee Noord-Holland (the Netherlands).
Informed Consent
A written opt-out procedure for informed consent was performed.
Trial Registration Number
Clinicaltrials.gov: NCT03426293
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
