Validation of the RIETE,Kuijer,and HAS-BLED Models to Assess 3-Month Bleeding Risk in Anticoagulated Patients Diagnosed with Venous Thromboembolic Disease

Abstract

Objective

To evaluate the discriminative ability and calibration of the RIETE, Kuijer, and HAS-BLED models for predicting 3-month bleeding risk in patients anticoagulated for venous thromboembolism (VTE).

Methods

External validation study of a prediction model based on a retrospective cohort of patients with VTE seen at the Hospital Universitario San Ignacio, Bogotá (Colombia) between July 2021 and June 2023. The calibration of the scales was evaluated using the Hosmer-Lemeshow test and the ratio of observed to expected events (ROE) within each risk category. Discriminatory ability was assessed using the area under the curve (AUC) of a ROC curve.

Results

We analyzed 470 patients (median age 65 years, female sex 59.3%) with a diagnosis of deep vein thrombosis in most cases (57.4%), 5.7% bleeding events were observed. Regarding calibration, adequate calibration cannot be ruled out given the limited number of events. The discriminatory ability was limited with an area under the curve (AUC) of 0.48 (CI 0.37–0.59) for Kuijer Score, 0.58 (CI 0.47–0.70) for HAS-BLED and 0.64 (CI 0.51–0.76) for RIETE.

Conclusion

The Kuijer, HAS-BLED, and RIETE models in patients with VTE generally do not adequately estimate the risk of bleeding at three months, with a low ability to discriminate high-risk patients. Cautious interpretation is recommended until further evidence is available.

Keywords

anticoagulation risk bleeding prediction models HAS-BLED RIETE Kuijer

Introduction

Venous thromboembolism (VTE) is the third most common acute cardiovascular syndrome, surpassed only by myocardial infarction and stroke. The annual incidence of deep vein thrombosis (DVT) is estimated to be as high as 162 cases per 100,000 population, while the incidence of pulmonary embolism (PE) is estimated to be as high as 115 cases per 100,000 population.^1,2

The mainstay of treatment for venous thromboembolism is anticoagulation, and any delay in initiating anticoagulation may have a negative impact on prognosis.³ This includes an increased risk of recurrence, embolization and death, particularly during a critical period of 3 to 6 months after the initial event.⁴ However, it is important to remember that anticoagulant treatment carries a risk of bleeding, which is more pronounced in the first months.^5,6 Therefore, it is essential to have tools to assess the individual risk of patients.

Several models have been developed to predict the risk of bleeding in patients initiating anticoagulation therapy, including those developed by Kuijer, RIETE, and HAS-BLED (Supplementary Table 1). Kuijer et al developed a system based on readily available factors such as age, sex, and presence of malignancy, and their results showed that it was possible to discriminate the subgroup of patients at high risk of bleeding complications.⁷ Subsequently, the RIETE model was developed with variables such as age over 75 years, cancer, recent bleeding, creatinine level over 1.2 mg/dl, and anemia, among others.⁸ Finally, HAS-BLED was initially developed to assess risk in patients with atrial fibrillation.⁹ All of these have shown acceptable performance in the populations in which they were initially developed, but they have not been externally validated to determine their applicability in the Latin American population, where there may be significant differences in different aspects such as the frequency of use of direct anticoagulants, accessibility to the health system, educational or socioeconomic level of the patients, characteristics that have been associated with the frequency of hemorrhagic complications.¹⁰

The purpose of this study is to validate the RIETE, Kuijer, and HAS-BLED models for estimating 90-day bleeding risk in patients with VTE initiating anticoagulation management, to evaluate the ability of the models to discriminate patients at higher risk, and to evaluate the relationship between observed and predicted events (calibration) in a cohort of patients managed in a referral hospital in Colombia.

Methods

Study Design and Participants

This study is an external validation of three prognostic models based on a retrospective cohort. Patients were included in the institutional anticoagulation registry of Hospital Universitario San Ignacio, which systematically records information on all patients initiating anticoagulation management at the institution. Inclusion criteria were patients over 18 years of age, treated at the Hospital Universitario San Ignacio between July 2021 and June 2023, with an indication for anticoagulation due to acute pulmonary embolism and/or deep vein thrombosis, using anticoagulation with warfarin, direct oral anticoagulants or low-molecular-weight heparin, and with at least three months of follow-up. Patients who did not give verbal consent to participate by telephone, those who discontinued anticoagulation for medical reasons, and those who died before hospital discharge were excluded. The institutional research ethics committee approved the study (approval number FM-CIE-0171–22). Study data were collected and managed using REDCap electronic data capture tools hosted at Hospital Universitario San Ignacio.^11,12

Sociodemographic data, comorbidities, diagnostic test reports, anticoagulation use, and pharmacologic choice were collected from the institutional anticoagulation registry, where information is systematically collected at the point of care using standardized instruments, and then periodic audits of the data collection process are performed to identify areas for improvement and ensure data quality. When missing information was identified, it was supplemented by a retrospective review of institutional electronic medical records. Bleeding outcomes were assessed by telephone follow-up three months after hospital discharge. Missing information on patients lost to follow-up was obtained by reviewing the institutional electronic medical record and the single database of members of the Unique Database for the General System of Social Security in Health (ie, Base de Datos Única para el Sistema General de Seguridad Social en Salud (BDUA-SGSSS)). The information recorded on this website is reported by governmental entities as part of the mandatory reporting to the Colombian Health Social Security System.

The RIETE, Kuijer, and HAS-BLED scales, which have been validated in VTE,^7,10,13 were used to estimate bleeding risk. The data required to calculate the scales corresponded to the time of diagnosis of VTE. Major bleeding was defined as bleeding requiring hospitalization, hemoglobin drop > 2 g/dL, transfusion requirement of ≥ 2 units of red blood cells, critical site bleeding (intracranial, intraspinal, intraocular, retroperitoneal, intraarticular, pericardial, or intramuscular with compartment syndrome), or fatal bleeding according to the International Society on Thrombosis and Hemostasis guidelines.¹⁴ Non-major bleeding was defined as any bleeding that did not meet the previously described criteria.

Statistical Analysis

Absolute and relative frequencies were used to describe qualitative variables. Measures of central tendency and dispersion were calculated for quantitative variables, mean and standard deviation for variables with normal distribution, and median with interquartile range for variables with non-normal distribution. The Shappiro-Wilk test was used to assess the normality assumption.

The RIETE, Kuijer, and HAS-BLED scales were validated by assessing their calibration and discriminatory ability.¹⁵ Calibration was assessed by comparing the number of bleeding events observed at 3-month follow-up with the number of events predicted by each scale within each risk category. The expected proportions of bleeding events were obtained from the original studies in which authors Kuijer, Ruiz, and Brown presented each prognostic model.^6,7,9 Expected events were calculated by multiplying the expected proportions by the number of patients in each class. The hypothesis of adequate calibration was evaluated using the Hosmer-Lemeshow statistical test.¹⁶ Discriminatory ability (ability of the prognostic model to discriminate between patients with different outcomes) was assessed with a receiver operating characteristic (ROC) curve, considering an adequate area under the curve (AUC) greater than 0.7. Results were reported according to the recommendations of the PROBAST tool.¹⁷

Statistical analysis was performed using STATA software (Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC).

Results

Figure 1 shows the patient selection process. Table 1 summarizes the demographic and clinical characteristics of the 470 patients finally included in the analysis. The median age was 65 years, with a higher prevalence of female sex (59.3%). The main diagnosis was deep vein thrombosis (57.4%), of which 67.4% was distal. Active cancer was the most common comorbidity (30.0%). Most patients were treated with direct oral anticoagulants (58.7%), followed by low molecular weight heparin (38.5%) and warfarin (2.5%). When comparing patients who presented with bleeding to those who did not, we found statistically significant differences in the proportion of patients with active cancer (p 0.034), renal disease (p 0.004), and alcohol consumption (p < 0.001). We identified 5.1% of patients at high risk of bleeding according to RIETE, 29.5% according to Kuijer, and 3.6% according to HAS-BLED.

Figure 1.

Selection of patients included in the study.

Table 1.

Sociodemographic Characteristics of Patients with VTE.

Variable	n: 470	Bleeding n = 27	No bleeding n = 443	p value
Sex female, n (%)	279 (59.3)	17(63.0)	262(59.1)	0.695
Age in years, median (IQR)	65 (52–76)	64(52–75)	65(52–75)	0.859
Indication of anticoagulation n (%)
DVT	270 (57.4)	21(77.8)	249(56.2)	0.028
PE	266 (56.6)	12(44.4)	254(57.4)	0.189
GFR ml/min, n (%)
< 30	12 (2.6)	3(12.5)	9(2.1)
30 – 60	48 (10.7)	6(25.0)	42(9.9)	<0.001
> 60	387 (86.5)	15(62.5)	372(87.9)	<0.001
Diabetes mellitus, n (%)	68 (14.4)	7(25.9)	61(13.8)	0.081
Uncontrolled hypertension, n (%)	44 (9.4)	4(14.8)	40(9.0)	0.316
Hepatic disease, n (%)	5 (1.1)	1(3.7)	4(0.9)	0.168
Renal disease*, n (%)	12 (2.5)	3(11.1)	9(2.0)	0.004
Previous stroke, n (%)	15 (3.2)	1(3.7)	14(3.2)	0.876
Previous major bleeding, n (%)	9 (1.9)	1(3.7)	8(1.8)	0.485
Alcohol consumption, n (%)	3 (0.6)	2(7.41)	1(0.23)	<0.001
Medications, n (%)
ASA	55 (11.7)	4(14.8)	51(11.5)	0.604
P2Y12 inhibitors	3 (0.6)	0(0)	3(0.7)	0.668
NSAIDS	14 (2.9)	2(7.4)	12(2.7)	0.368
Active cancer, n (%)	141 (30.0)	13(48.2)	128(28.9)	0.034
Previous VTE, n (%)	10 (2.1)	1(3.7)	9(2.0)	0.559
Anticoagulant of discharge, n (%)
Warfarin	12 (2.5)	1(3.7)	11(2.5)	0.702
LMWH	181 (38.5)	18(66.7)	163(36.8)	0.002
Apixaban	203 (43.1)	6 (22.2)	197(44.5)	0.023
Rivaroxaban	70 (14.9)	2(7.4)	68(15.3)	0.260
Dabigatran	4 (0.8)	0(0)	4(0.9)	0.620

IQR - Interquartile range; DVT: deep vein thrombosis; PE: pulmonary embolism; GFR: glomerular filtration rate; ASA: acetylsalicylic acid; NSAID: nonsteroidal anti-inflammatory drug; VTE: venous thromboembolic disease; LMWH: low molecular weight heparin; *Defined as creatinine > 2. 26 mg/dl, renal replacement therapy or renal transplantation according to HAS-BLED scale.

In terms of outcomes of interest, there were 27 events classified as “any bleeding” (5.7%). Of these, 35.7% were major bleeding events, with gastrointestinal bleeding being the most common site (22%), followed by intracranial bleeding (7.4%). We found a statistically significant difference in the anticoagulant used in the bleeding subgroup, with a higher use of LMWH (66.7% vs 36.8%, p:0.002) and lower use of apixaban (22.2% vs 44.5%, p:0.023).

Table 2 shows the calibration of the models for predicting total bleeding. Evaluation of the calibration of HAS-BLED and RIETE showed that the observed bleeding events were higher than expected, with ratios of observed to expected events of 2.5 and 2.8, respectively (Table 2A and B), suggesting that the scale underestimates risk. This finding holds for each of the risk categories. For the Kuijer score, the expected bleeding events were higher than the observed ones, with an observed-to-expected ratio of 0.51, which could indicate an overestimation of risk.

Table 2.

Calibration of Kuijer, HAS-BLED, and RIETE Prognostic Models as Predictors of 90-day Total Bleeding in Patients with Venous Thromboembolic Disease.

Score	N	% total	Expected events	Observed events	Expected proportion	Observed proportion	Ratio O/E
A. HAS-BLED
0	194	41.3	3.6	9	1.9	4.64	2.4
1	179	38.1	4.2	8	2.4	4.47	1.8
2	80	17.0	2.1	8	2.7	10	3.7
3	14	3.0	0.43	1	3.1	7.4	2.3
≥ 4	3	0.6	0.15	1	5.0	33.3	6.7
Total	470	100	10.48	27	2.2	5.7	2.5

The expected proportion of bleeding was obtained from the original study by Brown.⁹ Hosmer-Lemeshow test = 1.68; p = 0.43.

Score	n	% total	Expected events	Observed events	Expected proportion	Observed proportion	Ratio O/E
B. RIETE
0	67	14.2	0.2	3	0.3	4.4	14.6
1–4	379	80.6	9.8	20	2.6	5.2	2.0
≥ 4	24	5.11	1.7	4	7.3	16.6	2.2
Total	470	100	11.7	27	2	5.7	2.8

The expected proportion of bleeding was obtained from the original study by Ruíz-Giménez.⁷ Hosmer-Lemeshow test = 0.00; p = 1.

Score	n	% total	Expected events	Observed events	Expected proportion	Observed proportion	Ratio O/E
C. Kuijer
0	59	12.5	0	2	0	3.3	NE
1–3	272	57.8	16.3	18	6	6.6	1.1
≥3	139	29.5	36.1	7	26	5.0	0.19
Total	470	100	52.4	27	11	5.7	0.51

The expected proportion of bleeding was obtained from the original study.⁶ Hosmer-Lemeshow test = 0.99; p = 0.3196.

NE: Not evaluable. Expected events were calculated by multiplying the expected proportions by the number of patients in each class.

The discrimination ability of the three prediction models was low, with an area under the curve (AUC) of 0.48 (CI 0.37–0.59) for the Kuijer score, AUC of 0.58 (CI 0.47–0.70) for HAS-BLED, and AUC of 0.64 (CI 0.51–0.76) for RIETE. (Figure 2). A subgroup analysis considering only patients treated with LMWH showed similar results with an AUC of 0.66 (CI 0.51–0.82) for RIETE, which is higher than those reported for HAS-BLED 0.55 (CI 0.41–0.69) and Kuijer 0.50 (CI 0.36–0.65).

Figure 2.

Discrimination capacity of the Kuijer, HAS-BLED, and RIETE models as predictors of total bleeding at 90 days in patients with venous thromboembolic disease.

In addition, we evaluated the sensitivity and specificity of the cut-off points proposed by the original articles and found a sensitivity of 33.33% and a specificity of 93.68% for RIETE (≥4), a sensitivity of 7.41% and a specificity of 96.61% for HASBLED (≥3), and a sensitivity of 48.15% and a specificity of 45.82% for Kuijer (≥3). The optimal cut-off point proposed for each scale was also evaluated and found to be 2.75 for RIETE (sensitivity 48%, specificity 78%, AUC 0.63), 1.5 for HASBLED (sensitivity 37%, specificity 80%, AUC 0.59) and 2.9 for Kuijer (sensitivity 26%, specificity 71%, AUC 0.48).

Given the high proportion of patients lost to follow-up (26.1%), a comparison was made between this group and the patients finally enrolled in the study. No significant differences in clinical characteristics or treatment received were found between the groups, except for a higher incidence of cancer in patients lost to follow-up (46.3% vs 30.0%, p < 0.01). (Supplementary Table 2). Within this subgroup, the mortality rate was 32.5%; however, only 4 of the 54 events were mortality due to fatal bleeding.

Discussion

The present study evaluates the predictive ability of the RIETE, Kuijer, and HAS-BLED scales to predict total bleeding at 3 months of anticoagulation therapy in patients with VTE. Our results suggest that the RIETE and HAS-BLED prediction models underestimate bleeding risk, whereas the Kuijer model overestimates it. In addition, they suggest that the three prediction models evaluated have a poor ability to discriminate high-risk patients.

Our population, unlike other studies, has a high proportion of patients with cancer, which may influence the bleeding risk of patients, as patients with cancer have a higher risk of bleeding.¹⁸ This also explains the high percentage of patients treated with LMWH. The differences seen between patients who bled and those who did not are to be expected, as chronic kidney disease, alcohol consumption and cancer are known risk factors for bleeding.

There have been several validations of the HAS-BLED score in patients with atrial fibrillation. One of them, a study by Pisters et al, concluded that the HAS-BLED scale was effective in predicting the risk of major bleeding, with an AUC of 0.72.⁹ In another complementary analysis, Apostolakis et al found a moderate ability to predict the risk of major bleeding, with an AUC of 0.67.^5,19 However, in the case of venous thromboembolic disease, the available evidence is limited, and our results are consistent with those of Kooiman J et al, who demonstrated an AUC of 0.55, suggesting that for this indication the model has a low ability to discriminate patients at high risk of bleeding and therefore its clinical utility is limited.⁶

The predictive ability of the HAS-BLED score in assessing bleeding risk is inconsistent. In one specific case, a study found that a HAS-BLED score ≥3 correlated with an increased risk of major bleeding; however, actual rates of major bleeding within this group varied significantly, suggesting poor calibration of the score.²⁰ Conversely, another study found poor performance of the HAS-BLED score in elderly patients, which may indicate a possible underestimation of bleeding risk in this population.²⁰

When evaluating the RIETE scale, Tchen et al showed an AUC of 0.63 for discriminating patients at increased risk for major bleeding,¹³ similar to our results for total bleeding. Nieto et al used the RIETE database to validate the risk score for major bleeding during the first 3 months of anticoagulation. Their results showed that the score had superior performance in predicting fatal gastrointestinal bleeding, with a C-statistic of 0.869, compared with intracranial bleeding. The discrepancy with our results may be due to the fact that the population had greater variability between the bleeding and non-bleeding groups, including the description and presence of comorbidities, which was more pronounced in the bleeding group.²¹

Evidence regarding calibration is also mixed, with one study finding that RIETE underestimates risk in older patients (≥ 80 years). In contrast, another study showed that a score ≥4 established a clear cut-off for those at high risk of major bleeding, with higher positive predictive values but very low sensitivity, suggesting a possible overestimation of bleeding risk.²⁰

Among the available data on Kuijer, the study by Zhang et al stands out, which evaluated and compared four scores, including Kuijer and RIETE. The results showed that none of them showed satisfactory performance in predicting clinically relevant bleeding events during 3 months of anticoagulation therapy in Chinese patients with pulmonary embolism. Specifically, the area under the curve for Kuijer was 0.57 and 0.56 for RIETE, values that are consistent with our findings.²²

The risk scores used in this study have several limitations. The Kuijer and RIETE models were developed exclusively in populations receiving oral anticoagulant therapy, which limits their external validity in populations treated with low-molecular-weight heparin. In addition, these scores focus on major bleeding events and do not address the occurrence of clinically relevant bleeding. As for the HAS-BLED score, it does not consider variables such as cancer, which limits its usefulness in this population, although, as we have shown, it is the most common comorbidity.^10,22 Our population had a high percentage of LMWH use; however, when we performed an analysis in these patients only, the performance of the scales was similar and there was no statistically significant difference between them.

Based on the available evidence, no score was found with an AUC greater than 0.7, making it difficult to definitively determine which scale is the most effective. It should be noted that RIETE may come closer to meeting this criterion, although a more detailed evaluation is needed.¹³ The cut-off points proposed by the scales show a high specificity at the expense of a very low sensitivity; however, lowering the cut-off points to the optimal ones proposed by our analysis, although it would partially improve the sensitivity, significantly worsens the specificity.

A strength of this study is the close follow-up and direct telephone contact with our patients. This specificity probably explains the detection of more events and, in part, the remarkable discrepancy observed between the number of events detected and those expected.

It is crucial to highlight the limitations of the study, including the assessment of outcomes based on patient self-report, which could introduce misclassification bias. To minimize this risk of bias, we retrospectively reviewed the clinical history of the patients. A second limitation is that although we included a large sample, the number of events was relatively low, with only 27 events for all bleeding types, which limits the precision of our estimates. Larger, multicenter studies will be needed to verify our findings. Finally, the high loss to follow-up rate of 26.1% limits the assessment of events of interest. However, as shown in Supplementary Table 2, no significant differences in baseline patient characteristics were observed, except for the presence of active neoplasia as a comorbidity. When evaluating bleeding as a cause of death in the excluded patients, we found that it was 2.4%, which is relatively low and consistent with what has been previously reported by our group,²³ so we expect that the bleeding rate is not significantly higher, and even if there really was such an increase in bleeding, we would expect a greater underestimation of risk by the RIETE and HASBLED scales, without changing the direction of the conclusions of our study.

In conclusion, the results presented indicate that the RIETE, HAS-BLED, and Kuijer prediction models have insufficient discriminatory ability and poor calibration as estimators of total bleeding at 3 months in patients with VTE. These models have limited utility in our population. If, for example, the use of RIETE were to be considered, recalibration would be required to correct for overestimation of risk. Further prospective research is needed to develop a new and better prediction rule to estimate the risk of bleeding in patients with VTE. In the meantime, clinical decisions should be made with caution, using clinical judgment on a case-by-case basis by the treating physician.

Supplemental Material

sj-docx-1-cat-10.1177_10760296241271351 - Supplemental material for Validation of the RIETE, Kuijer, and HAS-BLED Models to Assess 3-Month Bleeding Risk in Anticoagulated Patients Diagnosed with Venous Thromboembolic Disease

Supplemental material, sj-docx-1-cat-10.1177_10760296241271351 for Validation of the RIETE, Kuijer, and HAS-BLED Models to Assess 3-Month Bleeding Risk in Anticoagulated Patients Diagnosed with Venous Thromboembolic Disease by Stephanie Ortiz Gómez, MD, Paula Ruiz-Talero, MD, Oscar Muñoz, MD, PhD, and Luis Manue Hoyos Pumarejo, MD in Clinical and Applied Thrombosis/Hemostasis

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Stephanie Ortiz Gómez

Paula Ruiz-Talero

Supplemental Material

Supplemental material for this article is available online.

References

Konstantinides

Meyer

Bueno

, et al. 2019 ESC guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the European respiratory society (ERS). Eur Heart J. 2020;41(4):543‐603.

Jimenez-Hernandez

Ruiz-Artacho

. Enfermedad tromboembólica venosa (ETEV) en Urgencias. Sociedad Española de Medicina de Urgencias y Emergencias; 2017.

Yamashita

Morimoto

Kimura

. Venous thromboembolism: Recent advancement and future perspective. J Cardiol [Internet]. 2022;79(1):79‐89. Available from: 10.1016/j.jjcc.2021.08.026.

Khan

Tritschler

Kahn

Rodger

. Venous thromboembolism. The Lancet [Internet]. 2021;398(10294):64‐77. Available from: 10.1016/S0140-6736(20)32658-1.

Klok

Huisman

. How I assess and manage the risk of bleeding in patients treated for venous thromboembolism. Blood. 2020;135(10):724‐734.

Klok

Kooiman

Huisman

Konstantinides

Lankeit

. Predicting anticoagulant-related bleeding in patients with venous thromboembolism: A clinically oriented review. Eur Respir J. 2015;45(1):201‐210.

Kuijer

PMM

Hutten

Prins

Büller

. Prediction of the risk of bleeding during anticoagulant treatment for venous thromboembolism. Arch Intern Med. 1999;159(5):457‐460.

Ruíz-Giménez

Suárez

González

, et al. Predictive variables for major bleeding events in patients presenting with documented acute venous thromboembolism. Findings from the RIETE Registry. Thromb Haemost. 2008;100(1):26‐31.

Pisters

Lane

Nieuwlaat

, et al. A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation: The euro heart survey. Chest [Internet]. 2010;138(5):1093‐1100. Available from: 10.1378/chest.10-0134.

10.

Brown

Goodin

Lip

GYH

Adams

. Risk stratification for bleeding complications in patients with venous thromboembolism: Application of the HAS-BLED bleeding score during the first 6 months of anticoagulant treatment. J Am Heart Assoc. 2018;7(6):1‐9.

11.

Harris

Taylor

Minor

, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019;95:103208.

12.

Harris

Taylor

Thielke

Payne

Gonzalez

Conde

. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377‐381.

13.

Tchen

Ryba

Patel

Cavanaugh

Sullivan

. Validation of bleeding risk prediction scores for patients with Major bleeding on direct oral anticoagulants. Ann Pharmacother. 2020;54(12):1175‐1184.

14.

Kaatz

Ahmad

Spyropoulos

Schulman

. Definition of clinically relevant non-major bleeding in studies of anticoagulants in atrial fibrillation and venous thromboembolic disease in non-surgical patients: Communication from the SSC of the ISTH. Journal of Thrombosis and Haemostasis [Internet]. 2015;13(11):2119‐2126. Available from: 10.1111/jth.13140.

15.

Altman

Vergouwe

Royston

Moons

KGM

. Prognosis and prognostic research: Validating a prognostic model. Br Med J. 2009;338(7708):1432‐1435.

16.

Kupper

Hosmer

Lemeshow

. Applied logistic regression. J Am Stat Assoc. 1990;85(411):901.

17.

Moons

KGM

Wolff

Riley

, et al. PROBAST: A tool to assess risk of bias and applicability of prediction model studies: Explanation and elaboration. Ann Intern Med. 2019;170(1):W1‐33.

18.

Vedovati

Giustozzi

Munoz

, et al. Risk factors for recurrence and major bleeding in patients with cancer-associated venous thromboembolism. Eur J Intern Med. 2023;112:29‐36.

19.

Klok

Hösel

Clemens

, et al. Prediction of bleeding events in patients with venous thromboembolism on stable anticoagulation treatment. European Respiratory Journal [Internet]. 2016;48(5):1369‐1376. Available from: 10.1183/13993003.00280-2016.

20.

Nopp

. Bleeding risk assessment in patients with venous thromboembolism. Hamostaseologie. 2021;41(4):267‐274.

21.

Nieto

Solano

Trapero Iglesias

, et al. Validation of a score for predicting fatal bleeding in patients receiving anticoagulation for venous thromboembolism. Thromb Res [Internet]. 2013;132(2):175‐179. Available from: 10.1016/j.thromres.2013.06.019.

22.

Zhang

Lei

Zhai

, et al. Comparison of prediction value of four bleeding risk scores for pulmonary embolism with anticoagulation: A real-world study in Chinese patients. Clinical Respiratory Journal. 2019;13(3):139‐147.

23.

Reyes

Ruiz-Talero

Arenas

Hernández-Flórez

Muñoz

. Factors associated with in-hospital and outpatient survival of patients with different types of stage IV cancer and venous thromboembolism. J Int Med Res. 2023;51(12).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB