Validation of a Patient-Completed Caprini Risk Assessment Tool for Spanish,Arabic,and Polish Speakers

Abstract

Targeted prophylaxis for venous thromboembolism (VTE) using the Caprini risk score (CRS) is effective reducing postoperative VTE. Despite its availability as preventive strategy, risk scoring remains underutilized. Critics to the CRS contend the time it takes to complete, and its limitation to English language. Aim is to create and validate patient-completed CRS tools for Spanish, Arabic, and Polish speakers. We translated the first patient-completed CRS to Spanish, Arabic, and Polish. We conducted a pilot study followed by the validation study. Using PASS version 11, we determined that a sample size of 37 achieved a power of 80%, to detect a difference of 0.1 between the null hypothesis correlation of 0.5 and the alternative hypothesis correlation of 0.7 using a 2-sided hypothesis test, significance level of .05. We tabulated and categorized scores using SPSS version 23 to estimate κ, linear correlation, and Bland Altman test. κ value >0.8 was defined as “almost perfect agreement.” From 129 recruited patients, 50 (39%) spoke Spanish, 40 (31%) spoke Arabic, and 39 (30%) spoke Polish; average age 51 (16.69) years, 58 (45%) were men, with less than college education (67%). Mean (standard deviation) CRS was 5 (3.90), the majority (63%) above moderate VTE risk. We report excellent agreement comparing physician and patient results (κ = 0.93) and high correlation 0.97 (P < .01) for the overall score. Bland Altman did not show trend for extreme values. We created and validated the first Spanish, Arabic, and Polish versions of the patient-completed CRS, with excellent correlation and agreement when compared to CRS-trained physician-completed form. Based on these results, the physician needs to calculate the body mass index. Completing the form was not time-consuming.

Introduction

Venous thromboembolism (VTE), comprising deep vein thrombosis and pulmonary embolism (PE), is a largely preventable global cause of morbidity and mortality. It represents the second most common postoperative complication and the third most common cause of excess mortality and cost in perioperative patients. ¹ Nonfatal VTE events are responsible for one-third of disability-adjusted life years. ^2,3 Hence, VTE prevention is the most important strategy for improving hospitalized patients’ safety and reducing VTE-related complications. ^4,5 Surprisingly, despite the available strategies for VTE prevention, adequate target prophylaxis remains underutilized. ^{6 –8}

The need and timing of thromboprophylaxis are based upon balancing both patient and procedure-specific risks for bleeding and thrombosis, and the use of VTE prediction tool is recommended to aid on this decision. Among the existent VTE risk assessment models, the 2005 Caprini risk score (CRS) is the most widely used and validated model. ^{9 –15} It weights independent risk factors for the individual, summing up a total score that correlates with the risk of postoperative VTE. ¹⁶ Implementation of the CRS has lowered the incidence of postoperative VTE; in fact, the 9th American College of Chest Physician (ACCP) guidelines recommend the use of this model for risk stratification in nonorthopedic surgical patients. ¹⁷ However, critiques to the CRS include relative complexity for reliable use, interpreter dependence, limited to one language, and time-consuming for health-care providers. ¹⁸

There is a known association between low health literacy and worse medical outcomes. ^19,20 This, combined with limited English proficiency, constitutes an even greater barrier to health care, comprising a vulnerable group with high prevalence of poor health status in the nonnative speaker population. ^21,22 Our study was designed taking advantage of the patient-centered communication ^{23 –25} and focused on the subject of VTE prevention. Moreover, we expanded our reach to 3 widely spoken languages worldwide (Spanish, Polish, and Arabic). ^{26 –28} In the absence of patient-centered VTE risk assessment instruments, we aim to create and validate patient-completed versions of the CRS in these 3 languages.

Methods

Patients and Methods

We prospectively recruited consecutive Spanish, Arabic, and Polish native-speaking patients and their relatives (>18 years old) at John H. Stroger Hospital from October 2016 through March 2017. We included patients admitted to a medical or a surgical unit and excluded patients with inability to read or write, altered mental status, visual disorders, and acquired/congenital cognition impairment. A 3 step methodology was used for creation and validation on each language.

Step 1: Standardized translation

We recently designed and validated the first patient-completed CRS with almost perfect agreement compared to a physician-completed score. ²⁹ Considering body mass index (BMI) was ineffectively estimated by patients, this should be calculated by the physician to obtain the final score.

We translated the patient-completed CRS from English to Spanish, Arabic, and Polish following a standardized process. First, 3 language-native authors independently translated each form into Spanish (L.H.P., D.O., and X.A.), Arabic (A.A., M.I., and A.I.), and Polish (W.K., A.L., and L.P.). Then, a fourth author created a unified version for Spanish (H.F.), Arabic (F.I.), and Polish (A.I.). Finally, all translators for each language reviewed the last form for accuracy. (Appendix)

Step 2: Pilot study

We conducted a pilot study to identify additional challenges specific to each language. In this phase, we conducted a standardized interview on hospitalized patients and their family members. During the first part of the interview, patients calculated their CRS using the form in their native language. Subsequently, a native-speaker physician blinded to the patient’s answers, scored the CRS for the same patient. At the end of the interview, we tabulated both forms for analysis.

Step 3: Validation

In a 15-week process, we prospectively enrolled patients admitted to the medical and surgical units. Patients were interviewed following the standardized process detailed above. Each rater received a training session on the CRS by one of the senior authors prior to starting the validation process. The local institutional review board approved this study and waived signed consent.

Statistical Analysis

We categorized CRS into very low, low, moderate, and high risk, as proposed in the 9th edition of the ACCP Evidence–Based Clinical Practice Guidelines, ¹⁸ and measured agreement level using Cohen κ. Using PASS version 11, we determined that a sample size of 37 achieved a power of 80%, to detect difference of 0.1 between the null hypothesis correlation of 0.5 and the alternative hypothesis correlation of 0.7 using a 2-sided hypothesis test with a significance level of .05. κ statistic values of 0.4 or less are considered as poor, 0.41-0.60 as moderate, 0.61-0.80 as substantial (good), and 0.81-1 as almost perfect (excellent) agreement. ³⁰ We calculated Spearman correlation coefficient to assess validity and correlation of the overall scores. Also, to quantify agreement between patient–physician cumulative CRS, we used the Bland Altman method. All statistical analysis was conducted in SPSS, version 22 (IBM Corp, Armonk, New York)

Results

In the pilot phase, we recruited a total of 83 patients, 33 (40%) spoke Spanish, 15 (18%) spoke Arabic, and 35 (42%) spoke Polish. Patients found difficulties adding up each item to obtain the cumulative score. However, in the interim analysis, this mathematical error did not affect the agreement level between physician- and patient-completed CRS. Therefore, no changes were made to the forms prior to the validation phase.

In the validation phase, we enrolled a total of 129 patients, 50 (39%) spoke Spanish, 40 (31%) spoke Arabic, and 39 (30%) spoke Polish. The Spanish-speaking group (n = 50) had a mean (standard deviation [SD]) age of 48 (15.8) years; 22 (44%) were men, with less than college education in its majority (76%). The Arabic-speaking group (n = 39) had a mean (SD) age of 43(15.6) years; 20 (50%) were men, with less than college education (50%). The Polish-speaking group (n = 39) had a mean(SD) age of 62(12.7) years; 16 (41%) were men, with less than college education (74%). The mean (SD) CRS calculated by the physician were 5 (4.37), 4(3.85), and 4 (3.12) for Spanish, Arabic, and Polish, respectively. When combined, the majority (63%) were classified above moderate risk of VTE based on the CRS (Table 1). Patients spent a median of 6 minutes (3-8) filling the form.

Table 1.

Patient Characteristics.

Variables	Cohort
n	129
Language	n (%)
Spanish	50 (39.00)
Arabic	40 (31.00)
Polish	39 (30).00
Age (SD); range	51(16.7); 17-91
Spanish	48(15.8); 18-88
Arabic	43(15.6); 17-82
Polish	62(12.7); 29-91
Gender
Women	71 (55.00)
Men	58 (45.00)
Education level
No education	3 (2.3)
Elementary	40 (31)
High School	44 (34.1)
College	40 (31)
Postgraduate	2 (1.6)
Patient-completed score, mean (range)	5.00 (0-18)
Physician-completed score, mean (range)	5.00 (0-18)

Abbreviation: SD, standard deviation.

The agreement level was excellent when the CRS was categorized following the ACCP guideline recommendations (κ = 0.93). Similar results were obtained when we stratified the analysis by languages, there was an excellent agreement level for Spanish (κ = 1.00), Arabic (κ = 0.93), and Polish (κ = 0.85) forms. Spearman correlation coefficients between patient- and physician-completed forms were 0.97 (P < .01) for the entire cohort (Figure 1), 0.98 (P < .01) for Spanish, 0.95 (P < .01) for Polish, and 0.99 (P < .01) for Arabic. The Bland Altman plot did not show any trend for extreme values (Figure 2).

Figure 1.

Spearman correlation.

Figure 2.

Bland Altman.

Discussion

We have created and validated the first Spanish, Arabic, and Polish versions of a patient-completed CRS, the most widely used perioperative VTE risk assessment model resulting not only in excellent correlation but also in excellent agreement when compared to CRS-trained physician-completed form. Our results may facilitate the implementation of a patient-driven risk assessment for Spanish-, Arabic-, and Polish-speaking communities in the world.

Patient-reported questionnaires have been found useful, valid, and well suited when health issues of complexity are explored in medical and surgical patients. ^31,32 Jolly et al validated a self-reported instrument to assess disease impact named The Lupus Impact Tracker ³³ ; capturing unique information prior to the medical encounter and incorporating patient’s perspective for disease management. The incorporation of a patient-completed CRS is in line with this approach.

Providing a good estimate of the risk of thrombosis may positively influence physicians in the selection of appropriate prophylaxis and aid in reducing the burden of VTE. Although the individualized risk assessment approach might be currently time consuming, it is effective and more importantly strongly advocated. ^34,35,18

The 2005 Caprini Risk Assessment Model (RAM) ¹⁷ has been extensively validated in terms of its predictive value for VTE posthospitalizations, enabling adequate extended postdischarge prophylaxis when warranted. ^10,11,36,37  It is the most widely used RAM and currently serves as the guideline for thromboprophylaxis decision-making in nonorthopedic surgical patients. ¹⁸ However, the scoring for VTE risk stratification using CRS has been obtained by the physician or care provider since its inception. Cassidy et al successfully implemented a thromboprophylaxis protocol based on the CRS in surgical patients. This protocol aimed to dictate the type and duration of VTE prophylaxis by incorporating recommendations to the electronic medical records. Using this protocol, patients with scores less than 5 efficiently received a mechanical thromboprophylaxis avoiding bleeding complications. There was a proven decrement of PE rate at 30 days (1.1%-0.5%). ³⁸ Similar results were observed in a recent meta-analysis of 13 studies by Pannucci et al. Among 14 776 patients, those with scores less than 6 did not get any significant VTE reduction by using pharmacological thromboprophylaxis, indicating that these patients can be safely spared from pharmacological prophylaxis. ³⁹

Strengths of our study include the lower average level of education in the patients assessed, which may enhance the external validity of our results. In addition, our methodology included a rigorous standardized translation process. The availability of the patient-completed CRS in 3 commonly spoken languages in the globe strengthens its potential reach and applicability. Moreover, despite the criticized cumbersome nature of the score, ¹⁹ in our study patients required an average of 6 minutes to fill the form with an excellent agreement level.

Limitations to our study include that this was a single-center study and that we did not plan to determine correlation with VTE incidence. Conversely, the demographics at our institution allowed for a diverse nationality enabling us to use native speakers of each of the languages from both the patient and the physician standpoint. The new score needs confirmation of the BMI by the treating physician. There is substantial evidence reporting inappropriate obesity estimation when BMI calculation is based on patient-reported height and weight. ⁴⁰ Because such data would be readily available in the medical records and are necessary to define the intensity of prevention, we do not think this should extinguish the applicability of our score.

We believe that implementing a patient-completed RAM would be a favorable way to promote self-advocacy in collaboration with the hospital team and provide the appropriate prophylaxis with a less time-consuming decision process. Taking advantage of a self-completed questionnaire, the incorporation of the patient CRS may precede the patient–physician encounter, and the results as well as interpretation be discussed during the patient–physician interaction. This strategy may apply for hospitalized patients as well as ambulatory encounters during perioperative risk evaluations.

The patient CRS is not meant to supplant the final physician’s oversight for the intensity and duration of thromboprophylaxis, but to assist in simplifying the risk stratification of patients, accounting for BMI and bleeding risk.