Inter-observer reliability of a risk assessment model for venous thromboembolism in acutely-ill medical hospitalized patients: Results from a prospective cohort study
Restricted accessResearch articleFirst published online December, 2021
Inter-observer reliability of a risk assessment model for venous thromboembolism in acutely-ill medical hospitalized patients: Results from a prospective cohort study
To analyze the inter-observer reliability of risk for venous thromboembolism (VTE) in a population of adult acutely-ill medical patients.
Methods
In this prospective cohort study, we collected risk factors and risk classification for VTE using RAM IMPROVE7. Kappa statistics was used to evaluate inter-observer reliability between lead clinicians and trained researchers. We evaluated occurrence of VTE in patients with mismatched classification.
Results
We included 2,380 patients, median age 70 years (interquartile range [IQR], 58-79), 56.2% female. Adjusted Kappa for VTE risk factors ranged from substantial (0.64, 95% confidence interval [CI], 0.61-0.67) for “immobilization”, to almost perfect (0.98; 95% CI 0.97-0.99) for “thrombophilia”; risk classification was 0.64 (95% CI 0.60-0.67). Divergent risk classification occurred in 434 patients (18.2%) of whom seven (1.6%) developed VTE.
Conclusion
Despite substantial to almost perfect reliability between observers for risk factors and risk classification, lead clinicians tended to underestimate the risk for VTE.
PannucciCJObiAAlvarezR, et al. Inadequate venous thromboembolism risk stratification predicts venous thromboembolic events in surgical intensive care unit patients. J Am Coll Surg2014; 218: 898–904.
2.
SpyropoulosACAndersonFAFitzGeraldG, et al. Predictive and associative models to identify hospitalized medical patients at risk for VTE. Chest2011; 140: 706–714.
3.
AndersonFAWheelerHBGoldbergRJ, et al. A population-based perspective of the hospital incidence and case-fatality rates of deep vein thrombosis and pulmonary embolism. The Worcester DVT Study. Arch Intern Med1991; 151: 933–938.
4.
RaskobGEAngchaisuksiriPBlancoAN, et al. Thrombosis: a major contributor to global disease burden. Arterioscler Thromb Vasc Biol2014; 34: 2363–2371.
5.
JhaAKLarizgoitiaIAudera-LopezC, et al. The global burden of unsafe medical care: analytic modelling of observational studies. BMJ Qual Saf2013; 22: 809–815.
6.
TapsonVF.Prophylaxis strategies for patients with acute venous thromboembolism. Am J Manag Care2001; 7: S524–531.
7.
CohenATAlikhanRArcelusJI, et al. Assessment of venous thromboembolism risk and the benefits of thromboprophylaxis in medical patients. Thromb Haemost2005; 94: 750–759.
8.
KucherNKooSQuirozR, et al. Electronic alerts to prevent venous thromboembolism among hospitalized patients. N Engl J Med2005; 352: 969–977.
9.
SpyropoulosACMcGinnTKhoranaAA.The use of weighted and scored risk assessment models for venous thromboembolism. Thromb Haemost2012; 108: 1072–1076.
10.
BarbarSNoventaFRossettoV, et al. A risk assessment model for the identification of hospitalized medical patients at risk for venous thromboembolism: the Padua prediction score. J Thromb Haemost2010; 8: 2450–2457.
11.
NendazMSpirkDKucherN, et al. Multicentre validation of the Geneva Risk Score for hospitalised medical patients at risk of venous thromboembolism. Explicit ASsessment of Thromboembolic RIsk and Prophylaxis for Medical PATients in SwitzErland (ESTIMATE). Thromb Haemost2014; 111: 531–538.
12.
de BastosMBarretoSMCaiafaJS, et al. Derivation of a risk assessment model for hospital-acquired venous thrombosis: the NAVAL score. J Thromb Thrombolysis2016; 41: 628–635.
13.
ZakaiNAWrightJCushmanM.Risk factors for venous thrombosis in medical inpatients: validation of a thrombosis risk score. J Thromb Haemost2004; 2: 2156–2161.
14.
BlondonMSpirkDKucherN, et al. Comparative performance of clinical risk assessment models for hospital-acquired venous thromboembolism in medical patients. Thromb Haemost2018; 118: 082–089.
15.
KahnSRLimWDunnAS, et al. Prevention of VTE in nonsurgical patients: antithrombotic therapy and prevention of thrombosis, 9th ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest2012; 141: e195S–e226S.
16.
SchulmanSAngerasUBergqvistD, et al. Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients. J Thromb Haemost2005; 3: 692–694.
17.
VargheseJKleineMGessnerSI, et al. Medical effect of venous thromboembolism prophylaxis systems and common input categories: preliminary findings from a systematic review. Stud Health Technol Inform2017; 245: 1175–1179.
18.
VargheseJKleineMGessnerSI, et al. Effects of computerized decision support system implementations on patient outcomes in inpatient care: a systematic review. J Am Med Inform Assoc2018; 25: 593–602.
19.
YangYWangXHuangY, et al. Ontology-based venous thromboembolism risk assessment model developing from medical records. BMC Med Inform Decis Mak2019; 19: 151.
DonnerAEliasziwM.A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation. Stat Med1992; 11: 1511–1519.
23.
RosenbergDEichornAAlarconM, et al. External validation of the risk assessment model of the International Medical Prevention Registry on Venous Thromboembolism (IMPROVE) for medical patients in a tertiary health system. J Am Heart Assoc2014; 3: e001152.
24.
MahanCELiuYTurpieAG, et al. External validation of a risk assessment model for venous thromboembolism in the hospitalised acutely-ill medical patient (VTE-VALOURR). Thromb Haemost2014; 112: 692–699.
25.
SchunemannHJCushmanMBurnettAE, et al. American Society of Hematology 2018 guidelines for management of venous thromboembolism: prophylaxis for hospitalized and nonhospitalized medical patients. Blood Adv2018; 2: 3198–3225.
26.
PetrieASabinC.Medical statistics at a glance. 3rd ed. Oxford: Wiley-Blackwell, 2009.
27.
ByrtTBishopJCarlinJB.Bias, prevalence and kappa. J Clin Epidemiol1993; 46: 423–429.
28.
CicchettiDV.Multiple comparison methods: establishing guidelines for their valid application in neuropsychological research. J Clin Exp Neuropsychol1994; 16: 155–161.
29.
GuyattGHAklEACrowtherM, et al. Executive summary: antithrombotic therapy and prevention of thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest2012; 141: 7S–47S.
30.
FuentesHEPazLHAl-OgailiA, et al. Validation of a patient-completed Caprini risk score for venous thromboembolism risk assessment. TH Open2017; 1: e106–e112.
