Abstract
Our objective was to assess the utility of an assessment battery capturing health literacy (HL) and biopsychosocial determinants of health in predicting 30-day readmission in comparison to a currently well-adopted readmission risk calculator. We also sought to capture the distribution of inpatient HL, with emphasis on inadequate and marginal HL (an intermediate HL level). A prospective observational study was conducted to obtain HL and biopsychosocial data on general medicine inpatients admitted to the UCLA health system. Five hundred thirty-seven subjects were tracked prospectively for 30-day readmission after index hospitalization. HL was significantly better at predicting readmission compared to LACE + (Length, admission acuity, comorbidities, emergency room visits) alone (P = .013). A multivariate model including education, insurance, and language comfort was a strong predictor of adequate HL (P < .001). In conclusion, HL offered significant improvement in risk stratification in comparison to LACE + alone. Patients with marginal HL were high-risk, albeit difficult to characterize. Incorporating robust HL and biopsychosocial determinant assessments may allow hospital systems to allocate educational resources towards at-risk patients, thereby mitigating readmission risk.
Keywords
Introduction
In response to national guidelines by the Center for Medicare and Medicaid Services to focus efforts on reducing hospital readmissions, healthcare systems seek to optimize transitions of care. Population-specific factors for poor care transitions have incorporated measures to predict 30-day readmission rates among hospitalized patients as a means to reduce unplanned readmissions (1). Metrics including the LACE + score (a standardized readmission risk tool including length of stay [L], acuity of admission [A], comorbidities [C], emergency room visits [E]) (2), have assisted providers in directing care towards high-risk patients for future readmission by utilizing clinical and nonclinical parameters (2,3). However, such tools are limited in specific populations. For example, the LACE + score was demonstrated to be a poor predictor of readmission when evaluating geriatric patients (4). It was also ineffective at predicting risk for hospitalized heart failure patients with limited health knowledge (5). Closer investigations of readmission risk stratification tools within various hospital populations are necessary to determine the appropriate allocation of resources.
Among other studied parameters, functional health literacy (used interchangeably with HL in this study) has been shown to correlate with length of stay (LOS) and readmission rates (6,7). As per the Institute of Medicine, HL is defined as “the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions”(8). Recent studies have investigated HL's ability to predict readmissions for patients with specific disease states (9–13). In contrast to the LACE + , HL was shown to significantly predict mortality in patients with heart failure in a rural setting (11). Additionally, high HL was strongly associated with higher self-care confidence in hospitalized heart failure patients (9). Studies also suggest associations between HL and auxiliary risk factors for readmission, such as fall tendency in geriatric patients (14).
Multiple studies have utilized abbreviated tools such as the Rapid Estimate of Adult Literacy in Medicine and the Brief Health Literacy Screen (BHLS) to expediently assess patients, providing a binary output (High or Low) for HL with varying degrees of success (15–17). Jaffee et al. (18) demonstrated that the binary HL stratification tool, BHLS, could be used to elicit an association between LOS and HL. However, there is a paucity of studies utilizing validated tools that stratify HL beyond 2 functional categories, to assess HL’s impacts with greater nuance and granularity. One such validated tool, the test of functional health literacy in adults (TOFHLA) includes a marginal HL category, defined as an intermediate level of HL between adequate and inadequate, with an objective range of scores that define its reproducibly. Statistical analyses in the literature depict marginal HL as equivalent to a 7th or 8th grade reading, comprehension level (19). However, due to a tendency to bifurcate functional HL level in creating shorter screening tools, the authors of this study theorize that patients with intermediate levels of HL remain uncharacterized. Thus, such patients are difficult to risk stratify. Moreover, few studies have investigated the impact of psychosocial determinants such as patient self-efficacy and language comfort on HL and readmission risk prediction (20–23).
Given the latent utility of HL assessment as an aid to risk stratify patients at risk for poor transitions of care, we evaluated the potential of using robust HL measurements in a risk assessment of our inpatient population. Within the scope of patient-centered readmissions work, our first objective was to assess the combined power of HL and psychosocial determinants in predicting 30-day readmission in general medicine inpatients. To fulfill this objective, we conducted a prospective observational study utilizing an assessment battery of psychosocial factors related to patient transitions of care and the validated assessment tool, TOFHLA, which measures HL with increased granularity by including a marginal subcategory. We also compared the assessment battery's risk-predictive capacity to that of the LACE + score. Our second objective was to better summarize the distribution of marginal and inadequate HL within our inpatient population, to better identify features of the population with marginal HL. Through this study, we hope to understand how an extensive assessment of HL and psychosocial determinants of health might fit into operational improvements in transitions of care workflows.
Methods
This prospective cohort study was conducted at 2 medical centers within the UCLA Health System: Ronald Reagan UCLA Medical Center (a tertiary care, academic teaching hospital) and UCLA Medical Center-Santa Monica (a teaching-affiliate community hospital). The study was conducted from February 16, 2018, to August 8, 2018. The protocol was submitted to the institutional review board (IRB) and was determined to be exempt from full IRB review.
As part of the protocol, trained quality improvement (Q.I.) personnel interviewed adult general medicine inpatients meeting inclusion criteria (Appendix Table 1a) at the bedside, utilizing the TOFHLA (Appendix Table 2). The TOFHLA evaluates functional HL, assessing numeracy and reading comprehension through a modified cloze procedure (24). For the purposes of this study, numeracy was defined as the ability to understand and use quantitative information, as per the definition within the administration manual of the TOFHLA tool (Appendix Figure 1). Correct response frequency was translated to one of the TOFHLA’s 3 HL proficiencies: Inadequate (score: 0-16), Marginal (score: 17−22), and Adequate (score: 23−36). Descriptive criteria for each level of HL, as stated in the official administration manual, are provided in the Appendix. Patients unable to read or speak English or Spanish at any fluency were excluded from the study, as the TOFHLA tool is validated only in English and Spanish. Patients that could speak English or Spanish at any fluency, but preferred another language were asked whether they prefer a translator or preferred to interact with care providers in English or Spanish. Patients that preferred a translator were excluded from the study population. Patients that could speak English or Spanish at any fluency but preferred another language were also accounted for by the language preference question (Appendix Figure 2). Excluded patients were not given a HL status. Patients with permanent caregivers (for any medical purposes) that always needed to be present within the hospital room were also excluded from the study (see Appendix Table 1b for further details on exclusion criteria).
TOFHLA sample passage as given to subject.
Study design—Patient language preferences were subclassified and utilized to determine the study population using the schematic above.
Study Inclusion Criteria.
Study exclusion criteria
TOFHLA Categorization Levels Based on Health Literacy Fluency.
A cluster of psychosocial metrics: patient education level, self-rated health status, language comfort, native language, internet access to health content, ethnicity, gender, age, insurance status, and self-efficacy was included in the assessment battery. Self-efficacy was interpreted as the patient's belief in their abilities to interpret health information, perform health care tasks, and exercise control over their healthcare activities. We assessed this using a validated proxy measure termed the “Ask, Understand, Remember Assessment (AURA) tool.” The AURA consists of 4 statements at a 6th-grade reading level and includes 4 tiers of agreement (25) (Appendix Figure 3). Education level was measured using the International Standard Classification of Education (26) (Appendix Table 3). Patient self-rated health status was obtained using a 4-point scale (Appendix Table 4). Patients were subsequently asked to report their preferred language and their level of comfort with English using a validated 4-point scale (27) (Appendix Table 5). Patient access to internet health content was evaluated using a binary (yes/no) response (28). Patient ethnicity was self-reported and matched to a category in the US Census 6-category ethnicity table (29) (Appendix Table 6). Demographic data including patient gender, insurance provider, and age were elicited from the patient and confirmed utilizing the electronic medical record. LACE + scores, as calculated at the time of index hospitalization discharge, were also recorded from the electronic medical record.
Ask, Understand, Remember Assessment (AURA) tool scoring and interpretation index. Source: Clayman et al., 2010.
International Standard Classification of Education. Eight Classifications.
Self-Rated Health Status Using 5-Point Likert Scale Standardized Question.
Standardized 4-Point Scale to Assess English Language.
U.S. Census 6-Category Ethnicity Categorization.
Upon completion of bedside assessments, responses were securely stored in a database referred to as REDCap (Research Electronic Data Capture), a clinical research workflow-optimized, HIPAA compliant data storage platform (30). Following index discharge, enrolled patients placed in cohorts corresponding to their HL as defined by the TOFHLA were tracked prospectively, utilizing the electronic medical record to determine their 30-day readmission status within the UCLA Health hospital system. Patients readmitted for planned organ transplantation, maintenance chemotherapy, planned procedures, or obstetrical delivery were considered planned readmissions and excluded from the analysis.
Statistical Analysis
Two sample t-tests were used to compare continuous variables between groups, and Fisher's exact test was used to compare categorical variables. TOFHLA was analyzed as a binary variable (adequate vs inadequate/marginal). Unadjusted and multivariable logistic regression models were also used to analyze associations between patient characteristics and all 3 TOFHLA categories, as well as readmissions. Associations were summarized using odds ratios (ORs) and their 95% confidence intervals (CI). To evaluate whether TOFHLA adds predictive value to LACE + score predicting 30-day readmission (y/n), the area under the curve (AUC) from the receiver operating characteristic (ROC) curves of the 2 logistic models using LACE + score and combined LACE + score and TOFHLA, were compared using DeLong’s test. All tests were 2-sided, and P-values <.05 were considered statistically significant. The data sets were assessed using completed case analysis, in which subject entries with incomplete or missing parameters were omitted from analysis, to ensure statistical uniformity. Analyses were performed using SAS, Copyright I 2002-2012 by SAS Institute Inc., Cary, NC, USA.
Results
Five hundred thirty-seven patients met the inclusion criteria and were subsequently evaluated by trained Q.I. personnel to partake in our study. Table 1 is a summary of the descriptive statistics obtained for the patient cohorts.
Univariate Statistics Summary of Patients’ Demographics and TOFHLA Scores.
Sample Characteristics
The median LOS was 7 days (IQR = 4-13 days), and the mean LACE + score was 58.97 (SD = 14.7). As per the LACE + score interpretation guidelines(2), LACE + scores were defined as having low (score ≤ 28), medium (score between and including 29-58), and high (score ≥59) risk for readmission. This suggests that our patient population had several comorbidities, and at baseline, was on the border between medium and high risk for readmission. When tracking patients prospectively, it was determined that 107 (19.9%) patients had an unplanned readmission within the UCLA health system, 30-days after index discharge.
Parametric Associational Analysis
A ROC curve was used to assess the LACE + score's ability to discriminate between 30-day unplanned readmitted and nonreadmitted patients. A second ROC curve was also generated by combining TOFHLA and LACE + to evaluate any improvement in the discriminatory ability of the model. The AUC for LACE + was 0.55, 95% CI: (0.49-0.61) (Figure 1), while the combined TOFHLA and LACE + score model had an AUC of 0.61; 95%CI: (0.55-0.68). ΔAUC was calculated by determining the difference between TOFHLA combined with LACE + AUC and TOFHLA and LACE + AUC alone. The P-value was calculated using DeLong's test. The combined model performed significantly better than LACE + alone, ΔAUC = 0.062, 95%CI (0.013, 0.111); P = .013 (Figure 1).
ROC curve comparison between TOFHLA and LACE + scores in terms of readmission predictive power. LACE + score alone (AUC = 0.55, 95% CI: 0.49-0.61) with a model including LACE + score and TOFHLA adequate score (y/n) (AUC = 0.062; 95% CI: 0.013-0.111).
Predictors of TOFHLA
With the observed improvement in predictive performance by combining TOHFLA with LACE + and given that the feasibility of administering the TOFHLA tool was not yet determined, we sought to ascertain whether a patient’s TOHFLA score could be predicted in terms of commonly captured demographic and clinical characteristics. We first determined individual associations between HL and multiple psychosocial factors to help characterize patient HL Fisher’s exact test revealed a significant association between adequate HL and high education level (P < .001). Fisher's exact test also demonstrated a significant association between inadequate HL and low education (less than high school P = .045). AURA scores were used to effectively assess a patient’s self-efficacy. A Spearman correlation illustrated a significant correlation between HL and AURA score (rho = 0.12; P = .018) as well as HL and self-rated health status (rho = 0.09; P = .044). In multivariate analysis, the strongest predictors of Adequate TOHFLA (y/n) included: age (P < .001), gender (P = .03), education (Less than High School, High school and some college, and College and Postgraduate Degree) (P < .001), insurance (Medical, Medicare, PPO, and Other) (P = .05), and preferred language English (y/n) (P = .009) (Appendix Table 7). The selected metrics were designated as predictors of TOFHLA score based on their significant individual associations in comparison to other elicited metrics. Given their significant individual predictive power, we sought to determine the combined performance of these metrics in predicting HL. ROC analysis was conducted to achieve this and revealed that the AUC for the multivariable model was 0.77, 95%CI (0.72-0.82) (Figure 2). However, these combined characteristics (stated above) did not yield a significant correlation with readmission (AUC 0.18, 95%CI [0.10-0.27]).
ROC curve of multivariable model investigation, a combination of demographic as TOFHLA predictors. AUC = 0.77, 95% CI (0.72-0.82).
Univariate (Unadjusted) and Logistic Regression Model (Adjusted) Model Using TOFHLA Proficiencies as Outcome Variables. Two Sample t-Tests Were Used to Compare Continuous Variables Between Groups, and Fisher’s Exact Test was Used to Compare Categorical Variables.
Characterization of Marginal HL
In diving deeper into the granularity of the TOFHLA tool, we observed that marginal HL had a higher association with readmission, compared to inadequate HL (OR 3.28, 95% CI [1.85−5.38]), P < .001). Inadequate HL also had a higher association with readmission when compared to adequate HL (OR 1.94, 95% CI [1.048-3.603], P < .001). The mean education level for marginal HL was at the level of “some college coursework without a degree.” However, identifying marginal HL using education as a proxy measure may not account for all marginal HL patients, due to variance in education levels (in contrast to adequate and inadequate HL, where the education level is significantly correlated, P < .001 and P = .045, respectively).
The language preference metric revealed that 31% of the total subject population had chosen a non-English primary language. Patients with marginal HL had a higher “non-English language” preference (39.9%). However, a majority (60.1%) of marginal HL patients were comfortable with English instructions, in contrast to the 71% of patients with inadequate HL choosing a non-English primary language.
Basic Feasibility Analysis
In a post-hoc feasibility analysis, we examined the time taken to administer the TOFHLA and capture other metrics to determine the feasibility of fitting this assessment battery into existing hospital system workflows. The total administration time for the TOFHLA assessment (mean time of 12.8 min per patient [SD: 2.82]) and psychosocial determinant assessments (mean total time of 3.0 min [SD: 1.1]) was 15.8 min. This time does not include pre-administration tasks to determine patient eligibility (using inclusion and exclusion criteria). Four printed pages of assessment battery administration-associated materials were required per patient. Data input into the REDCap database was conducted manually upon conclusion of assessments and took approximately 15 min for every 5 patients surveyed (mean input time: 15.3 min, SD: 3.6).
Discussion
Our first objective proposed a comparison between the combined predictive power of the TOFHLA and psychosocial parameters to that of the LACE + score regarding readmission risk. The ROC curve (Figure 1) for the TOFHLA and LACE + model performed significantly better than the LACE + alone in predicting readmission (P = .013). A multivariable regression utilized education, ethnicity, preferred language (all cited as having an impact on HL) and other metrics including insurance status and language preference did not yield significant correlation with readmission risk (C-statistic 0.18) but had a strong ROC (C-statistic = 0.77) when estimating HL (23,26) (Figure 2). This indicates that the collection of disparate metrics may not capture the breadth of psychosocial influences tied to HL that affect readmission risk directly and indirectly. A cross-sectional analysis conducted by Fiscella et al. depicted direct, individual associations between variables such as ethnicity, income and relative risk of influenza infection (23), but further work is needed to appreciate how these variables portend marginal HL as a risk factor for readmission. The data validates that the LACE + tool alone may not sufficiently identify patients at high risk of readmission. The ROC analysis (Figure 2) also supports the position of utilizing the set of metrics including age, gender, education level, insurance, and preferred language as a proxy for TOFHLA score. However, the inability of this model to significantly predict readmission restricts us from incorporating these metrics solely into clinical care plans. Other models using pre-existing social determinant parameters available in a patient's chart have been developed to eliminate the use of additional tools. For example, the area deprivation index (ADI), developed by the United States federal census office, was utilized by Hu et al. to appreciate the statistically significant influences of environment (such as a patient’s neighborhood) on readmission likelihood (31). A promising body of research harnessing machine learning to titrate the weightage of individual social determinants on readmission risk is also underway (32,33).
Metrics such as the AURA reflect how social parameters such as self-efficacy have a quantifiable impact on the quality of life. The significant positive association between AURA score and HL (P = .018) indicates that inadequate HL and lower self-efficacy may increase barriers to communication and compliance. Metrics such as self-rated health status also remain unquantified, possibly due to the difficulty in data capture. However, the significant positive association between HL and self-rated health status (P = .044) suggests that a patient's perception of health status may adversely affect their HL and may have impacts on readmission risk. Patients with low HL may invariably enter the hospital setting with a low self-efficacy and thus perceive their health condition as suboptimal due to their inability to effectively utilize health resources and navigate the healthcare environment (Figure 1). This is consistent with the literature for specific patient populations. Taal et al. (34) demonstrated an association between self-efficacy and health status amongst patients with rheumatoid arthritis, concluding that patient education should focus on strengthening self-efficacy to improve self-management. Furthermore, Salbach et al. (35) observed a positive association between self-efficacy and health status in post-stroke patients in a randomized trial.
The complexity in identifying marginal HL patients, as noted in the results, may force care teams to default marginal HL patients to the adequate category, as such patients may have some competency that masks any limitations without a deeper investigation. However, the data suggests that marginal HL patients should be defaulted to the inadequate HL category, due to the greater similitude in population profiles and higher readmission risk. Specifically, as it pertains to discharge teaching, it is plausible that the marginally health literate patient will understand some post-discharge instructions but misunderstand details that are necessary for complex medication administration.
The analysis also revealed that the optimal period for assessment battery administration was well before patient discharge, to effectively organize educational interventions delivery around the patient’s predicted LOS. Additionally, incorporation of the assessment battery into a clinical care management plan upon admission would have allowed for timely delivery of assessments, reducing interference with transition of care procedures. Volunteer Q.I. personnel also reduced the burden on nursing and clerical staff by administering the assessment battery during their standard volunteer shift hours. Given that physicians and allied health care providers could utilize collected data to make educational resource allocation decisions in real time, the intended user base for the assessment battery data was determined to be care-team constituents responsible for choosing, and delivering educational interventions to patients during discharge planning.
As part of the next steps to inform change, the results were presented to health system leadership focused on readmission reduction as well as nursing leadership focused on patient education. Further pilots utilizing the study results are in development to help administer patient education to at risk populations at integral times to improve patient outcome and experience.
Limitations
This study should be interpreted in view of its limitations. While we were able to enroll patients at 2 study sites, our results may not be generalizable to other patient populations. Additionally, paper administration of the assessment battery might be cumbersome without dedicated Q.I. personnel due to burdening health systems with resource constraints. Utilizing an electronic medium may augment administration time and ease, and reducing data upload time. Moreover, readmissions of patients within other departments were not captured, potentially affecting the predictive capacity of the metrics discussed. Future iterations will include piloting this assessment battery in other service lines.
Conclusion
The superior predictive capacity of HL demonstrates its value in readmission risk assessment. Known psychosocial determinants such as ethnicity, language preference, and education level fail to characterize marginal HL effectively, highlighting the latent need to increase the granularity of HL assessment tools used in stratifying readmission risk. However, these determinants, along with others such as AURA score and self-rated health status, associate significantly with adequate HL, underscoring the multifactorial composition of HL as a socio-clinical marker of patient health. Further interventions may focus on optimizing HL tool administration and integration into operational workflows for discharge planning in general medicine and step-down unit patients. Additionally, broadening the linguistic scope of the TOFHLA may capture HL in high readmission risk patients with no English or Spanish fluency.
Finally, the utilization of trained volunteer Q.I. personnel and electronic assessment mediums could augment prospective data collection. The data on psychosocial determinants presented here could act as a nidus for the assembly of larger datasets that more accurately capture marginal HL. Through the assessment of HL and the collection of significantly associated psychosocial determinants, hospital systems may allocate educational resources towards marginal HL patients and reduce the risk of unplanned readmissions in this patient population.
Footnotes
Acknowledgments
The authors would like to thank Q.I. personnel who implemented data collection for the study.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health (NIH) (grant number UL1TR001881).
Informed Consent
Verbal informed consent was obtained from the patients for their anonymized information to be published in this article.
Appendix
Descriptive criteria for each category of health literacy according to TOFHLA administration protocol: