Rapid Independent Health Literacy Assessment: A Pilot Study Among Native English-speaking and Low English Proficiency Patients

Abstract

Objectives:

Health literacy is an individual’s capacity to obtain, process, and understand basic health information needed to make appropriate health decisions. Failure to understand and correctly execute a plan of care often leads to poor health outcomes. Determining patient health literacy allows health care providers to tailor their plan of care instructions, increasing the probability of understanding, and adherence. Several validated health literacy tests have been developed to assess the health literacy of individuals and ethnic groups. However, because a proctor is required to administer these tests, their usefulness in clinical settings is limited. The issue of health literacy is especially relevant within minority groups. This research focused on producing a translatable assessment that can be administered quickly without a proctor.

Methods:

We developed a 15-question instrument (the RIHLA) in English using the Delphi method with a panel of bilingual experts and translated it into Spanish. Internal reliability was assessed using Cronbach’s alpha for 3 groups: Native English-speaking College students (NESC), Native English-speaking patients (NES), and Limited English Proficient Spanish-speaking patients (LEP). External validity was assessed using Pearson’s correlation coefficient to compare our instrument to a previously validated, proctored instrument measuring health literacy (the SAHL-E).

Results:

Four hundred fifteen subjects completed the RIHLA. Of these, 192 (46.3%) were NESC, 208 (50.1%) were NES, and 15 (3.6%) were LEP. The mean number of correct answers was 11.2, 11.6, and 8.3 respectively with the LEP group scoring lower (P < .01). Cronbach’s alpha was >.70 for each group. Moderate correlation between the RIHLA and the previously validated instrument was present (P < .01) with Pearson’s r = .47 (95% CI: 0.18-0.69).

Conclusion:

The RIHLA is a non-proctored assessment tool that may provide a measure of patients’ health literacy in multiple languages. Further studies with larger sample sizes are necessary to confirm the reliability, validity, and generalizability to a wider population.

Background

The 2003 National Assessment of Adult Literacy showed that 36% of the adult population has basic or below basic literacy, making it difficult to integrate information from complex texts and perform calculations requiring 2 or more steps. ¹ Most adults read at an eighth-grade level but most health care materials are written at a 10th-grade level, making it difficult for many patients to comprehend. ² A person’s health literacy is defined as “an individual’s capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.”^3,4 A recent systemic review expanded on this idea, stating that health literacy is comprised of “variable sets of key dimensions, each appearing as a cluster of related abilities, skills, commitments, and knowledge that enable a person to approach health information competently and effectively and to derive at health promoting decisions and actions.” ⁵ Furthermore, health literacy is requisite to successfully navigate the health care system, including accessing care and filing insurance paperwork. ⁴

The association between Limited Health Literacy (LHL) and poor health outcomes is shown in multiple studies.^2,6,7 Dreger and Tremback ⁸ found that patients with LHL are less likely to comply with instructions and therefore suffer unnecessary complications. Almost 60% of patients discharged with a cardiac condition didn’t understand the indication, dose, or frequency of their cardiac medications, resulting in an alarming rate of non-compliance with dismissal instructions. ⁹ LHL patients are more likely to skip important preventive measures ¹⁰ and they tend to be more ill when entering the healthcare system. ¹¹ They have increased numbers of preventable hospitalizations.^12-15 LHL patients tend to utilize services that treat complications of disease while failing to take advantage of services designed to prevent complications.^10,12,13,15 Parikh et al. ¹⁶ found that those with LHL skills reported a sense of shame about their skill level. They may hide reading or vocabulary difficulties to maintain their dignity, resulting in unintentional non-compliance with treatment recommendations. ¹⁷ Because of these factors, LHL is associated with an estimated $69 billion in excess health care costs annually. ⁴

Several health literacy tests and pilots have been developed. All Aspects of Health Literacy Scale (AAHLS) was piloted as a tool to assist primary healthcare workers in assessing specific local health knowledge and literacy skills in an inner London borough. ¹⁸ Another pilot study was developed to assess an individual’s “ability to obtain and use health information from print [and]. . . non-print sources.” ¹⁹ The Wide Range Achievement Test (WRAT) and the Rapid Estimate of Adult Literacy in Medicine (REALM) use word recognition to evaluate an individual’s health literacy.^20,21 Both tests are highly correlated and take less than 5 min to administer, making either test applicable to a primary care setting. ²² However, they do not assess comprehension and there is not a non-English equivalent. The Test of Functional Health Literacy in Adults (TOFHLA) was developed to assess comprehension of prescription instructions via 17 numerical items and 3 prose passages. ²³ It also correlates with the WRAT and REALM tests, and a Spanish language version is available. ²³ The Newest Vital Sign (NVS) assesses health literacy by asking 6 questions about a nutritional label. ²⁴ NVS requires mathematical calculations, but is quick to administer and is highly correlated with TOFHLA. ²⁴ The Short Assessment of Health Literacy for Spanish-speaking Adults (SAHLSA) utilizes word pronunciation and comprehension to assess health literacy in Spanish-speaking patients. ²⁵ SAHLSA has an English equivalent known as the SAHL-E. Both tests are comparable, with proven validity and reliability. ²⁶ It requires a proctor to administer the test and is only moderately correlated with TOFHLA-S. ²⁷

When patients do not speak English proficiently, health literacy is impacted.^22,28 Over 6 million US residents cannot speak English and millions more have limited fluency. ²⁹ Although Spanish speakers were less likely than English speakers to understand discharge instructions or to keep follow-up appointments, those with high health literacy were more likely to follow up (59% vs 36%). ³⁰

A validated health literacy assessment instrument integrated into the clinical practice would enable clinical care teams to recognize LHL, customize advice, and minimize impact on outcomes. Spanish speakers are prevalent in the United States and present special challenges regarding health literacy. ³¹ As noted above, many health literacy assessment instruments do not have a Spanish language equivalent, and the SAHLSA requires a proctor to administer, making it difficult to utilize widely in a busy clinical environment. Thus, a simple dual language (English and Spanish) health literacy assessment instrument that can be administered online or in paper form without a proctor in a short period of time, is needed. This project aimed to develop and pilot a dual language health literacy assessment instrument that can be administered easily without a proctor. Its focus was examining the domain of health knowledge to assess a crucial part of health literacy.

Methods

Development

The Rapid Independent Health Literacy Assessment (RIHLA) was designed to measure patient’s health literacy and general understanding of their health. A multiple-choice format based on existing assessments such as the SAHLSA, WRAT, and REALM was selected to allow for easy development and administration without a proctor. ²⁵ Complexity of the survey was minimized to facilitate translation of the English version to other languages.

We elected to use a survey as a recent systematic review of health literacy assessment tools found that “all tools. . . adopted a questionnaire approach.” ³² Three basic domains of healthcare were focused on to help guide question development: taking medications, common diseases/illnesses and their treatment, and overall health. From these domains, questions were selected that assessed knowledge pertaining to prevalent issues in medical care today, such as hypertension, stroke, heart disease, and medication non-compliance.^33-35 Each multiple-choice question had a correct answer, 1 distractor chosen to represent common mistakes, and an “I don’t know” option. Simple instructions were given, encouraging respondents to answer honestly and utilize the “I don’t know” option instead of guessing.

Fifteen multiple-choice questions were developed using the Delphi method for the RIHLA pilot tool.^36,37 The Delphi panel consisted of 2 second-year medical students, an established family medicine physician, and 2 native English-speaking patients. Both medical students had an undergraduate background in accessing health literacy. All 3 of the medical experts were also fluent in Spanish. Panel selection of topic experts and overall size conformed to accepted Delphi standards. ³⁸ Initial questions were developed based on common issues the patient population faced in the clinic where the questionnaire was developed. Multiple rounds of questionnaires were sent out to the panel. Face-to-face meetings and email communication between members of the panel were conducted until they reached a consensus on the questions to be used.

Following development of the instrument in English (RIHLA-E), it was translated into Spanish (RIHLA-S) by a group of certified medical translators. The RIHLA-S and RIHLA-E were then verified for accuracy and psychometric equivalence with the help of a third-party, English-to-Spanish professional translator. The translation process produced what should be considered a linguistic equivalent of the RIHLA-E in another language, namely Spanish. The final RIHLA version is shown in Figure 1. Institutional Review Board approval was obtained prior to testing the instrument.

Figure 1.

RIHLA instrument.

Setting

Validation of the survey instrument was conducted at a local university and medical clinic located in southeast Idaho. The clinic is a multi-specialty group that serves eastern Idaho. The specialties include family, internal medicine, pediatrics, and OBGYN services. ³⁹ The city where the clinic is located has a population of 28 337 people, with 6.2% being Hispanic or Latino. Median household income in 2016 was $26 445, with the poverty level reported as 43.2%. The percentage of homes that speak a language other than English was estimated at 11.5%, ⁴⁰ the majority of which speak Spanish.

Reliability

The instrument was administered to 3 groups: Native English-speaking College (NESC) students from the university, Native English-speaking (NES) patients at the medical clinic, and Limited English Proficient (LEP) Spanish-speaking patients from the same clinic. Patients were grouped based upon their self-reported primary language. The NESC group received the questionnaire via hardcopy. The questionnaire was administered in 2 different forms to the patients presenting sequentially for appointments at the clinic. It was administered via hardcopy if the patient was new and had not set up an online patient portal or via email if the patient had an online portal. Distribution methods limited knowledge of acceptance rates among the groups, as students or patients could ignore the questionnaire without consequence or reporting.

Written informed consent was obtained via 2 slightly different methods depending on whether or not the patient was new. If the subject was a new patient, the informed consent was included in the initial paperwork. Subjects had to sign the informed consent in order for their questionnaire responses to be included. If the patient was established at the clinic, an electronic version of the informed consent was sent to their online patient portal before the survey was released.

Following consent in their native language (either English or Spanish), the questionnaire was administered to those who chose to participate. Reliability within each group was assessed by Cronbach’s alpha, with values above .70 considered adequate reliability.^41,42 To obtain a 95% confidence interval width of less than 0.1 with an assumed Cronbach’s alpha of .80 on a 15-item questionnaire, 207 respondents needed to be sampled. ⁴³ A summary statistic consisting of the number of correct responses (max = 15) was computed. We assumed the distribution of scores would be approximately normal and therefore used a t-test with P < .05 considered significant to assess whether the LEP summary scores were different than NES and NESC summary scores.

Validity

Both the RIHLA-E and the SAHL-E were administered to a NESC subgroup as a control to enable comparison of the RIHLA survey to a previously validated survey. We assumed any correlation between RIHLA-E and SAHL-E would be approximately linear. A summary statistic consisting of the number of correct responses was computed for each instrument, resulting in a maximum score of 15 for the RIHLA-E and 18 for SAHL-E. Pearson’s correlation coefficient with 95% confidence intervals was computed using R version 3.0.2. To test whether correlation exists, the coefficient was compared to the alternative hypothesis that it is not equal to 0 with P-values <.05 considered significant.

Results

The RIHLA questionnaire was completed by 415 subjects. Of these 192 (46.3%) were NESC, 208 (50.1%) were NES, and 15 (3.6%) were LEP. The percent of respondents in each group answering correctly for each question is shown in Table 1. Due to the limited responses of the LEP group, reliability should be interpreted carefully.

Table 1.

Percent Correct for Each Question Broken Down by Group (NESC and NES on RIHLA-E and LEP on RIHLA-S).

Question	NESC (%)	NES (%)	LEP (%)
1	96	97	50
2	69	77	43
3	92	95	64
4	95	98	93
5	66	70	57
6	35	38	29
7	65	66	36
8	64	63	64
9	53	57	50
10	86	83	79
11	93	96	36
12	88	79	100
13	65	85	14
14	58	58	64
15	97	98	50

In general, findings suggest limited internal reliability with Cronbach’s alpha >.7 for each group as seen in Table 2. The LEP summary scores were statistically different from both NES (P < .01) and NESC (P < .01) summary scores as seen in Table 2, possibly reflecting less health knowledge in the LEP group.

Table 2.

Descriptive Statistics for the RIHLA by Group.

Group	n	Mean # of correct responses	Standard deviation	95% confidence interval	Cronbach’s alpha	P-value*
NESC	192	11.2	2.6	11.2, 11.9	.72	<.01
NES	208	11.6	2.7	10.8, 11.5	.74	<.01
LEP	15	8.3	2.9	6.82, 9.76	.70	—

*

Correct responses compared to the LEP group using a t-test.

Validity

A total of 37 students in the NESC group completed both the RIHLA-E and SAHL-E instruments. Only the NESC group received both questionnaires. Moderate correlation was present (P < .01) with Pearson’s r = .47 (95% CI: 0.18-0.69). Figure 1 shows a scatter plot of the SAHL-E versus RIHLA-E summary scores with an ordinary least squares best fit line and 95% confidence interval. This suggests that the RIHLA-E assessment is valid to a degree, as the SAHL-E assessment has previous proven validity. It was assumed that if the RIHLA-E was valid, then an accurate translation to the RIHLA-S should maintain the validity. This was not confirmed via statistical analysis.

Discussion

Patients with lower RIHLA scores may be more likely to have difficulty understanding and executing care plans due to limited health literacy. This also may be exacerbated by a provider if assumptions are made about health literacy levels of their patients. By utilizing the RIHLA score, clinicians can recognize those patients with lower basic health literacy and tailor their recommendations, follow-up arrangements, and patient education to overcome health literacy barriers and prevent potential harms.

It is interesting that across all 3 groups surveyed, multiple questions resulted in higher than expected incorrect responses, namely questions 2, 5 to 9, and 13 to 14, which account for approximately half of the questions. Further investigation is needed to determine the cause of this, but this finding might suggest that general medical literacy among all people is lacking, regardless of the language spoken. It hints at a need for community engagement from medical personnel and patient education in all aspects of medical care, which are ongoing issues today. Questions were ultimately selected based on common issues seen in the clinic in which the tool was to be tested. These issues were presumed to be common issues in other clinics as well. Although some questions were considered more challenging than others, the mean correct responses for the RIHLA-E groups remained around 11 with similar standard deviations. Gross deviation in total correct responses (ex. <50% correct on the whole survey) would not be expected with adequate medical literacy.

The RIHLA-E shows moderate correlation with the SAHL-E instrument, but unlike the SAHL-E, it does not require a proctor to administer. Most primary care clinics are unable to provide proctors, so this gives the RIHLA a significant advantage toward adoption. While time data was not obtained as part of this pilot study, completion of the survey generally was done while patients were waiting to be roomed. Many medical home practices already administer similar-length questionnaires to assess their patients and have integrated such use into collaborative care management. ⁴⁴ Thus, the RIHLA instrument could be adopted into the workflow of most primary care clinics.

The RIHLA demonstrated limited internal reliability via statistical analysis and validity after comparison between the RIHLA-E and SAHL-E assessments using the NESC group as a control. It was assumed that validity and reliability would be maintained with accurate translation of the instrument into the RIHLA-S. These findings suggest the design and nature of the RIHLA allows for easy language translation, making primary care assessment of health literacy in particularly vulnerable minority groups possible. However, assessment of reliability and validity of the RIHLA-S within the Spanish-speaking population was limited due to small sample size and large confidence intervals. Further research should investigate if this questionnaire maintains reliability and validity across languages.

The RIHLA survey makes its mark in that it does not need a proctor to be administered and it has the potential for easy translation into more languages than Spanish alone. The RIHLA sentence structure is simple, its length is short, and the general health topics addressed typically have direct cross-language equivalents, all of which allow easier translation. The authors would suggest additional translations after further studies with larger populations confirming validity and reliability of the instrument at large.

Moreover, the flexibility of the survey (how it was constructed with local community member and physician input) allows possible addition or subtraction of certain questions that are more or less prevalent in certain communities, like obesity or diabetes. Thus, additional RIHLA iterations may be plausible after further validation and reliability testing of the current tool. The nature of a pilot study leaves many doors open, as is the case with the RIHLA survey.

Limitations

Study power for the Spanish language version of the RIHLA was limited. The clinic used served a small Spanish-speaking populace, which made it difficult to gather ample subjects for the RIHLA-S. However, despite larger confidence intervals, the internal reliability seems to be similar to both other groups. The clinical and college populations studied were relatively homogenous and from a single region. Furthermore, the college students are by definition, more highly educated and younger than typical family medicine patients. Thus, continued testing in a more economically, geographically, and culturally diverse population is recommended. It is important to note that the survey is specific to modern medical culture, which might limit its use outside of westernized countries. Additionally, translation into various languages may be difficult due to barriers such as lack of linguistic vocabulary equivalence with English medical terminology. It is worth noting that reliability and validity of this pilot study were limited due to sample size. Any results pertain solely to the groups mentioned in this study and may only hint at reliability and validity for the larger general population. Further research should expand on these findings and determine reliability and validity of the instrument with a larger sample size, especially within the translated version. Selection bias may be at play as the NESC group took the survey via hard copy while the other 2 groups received the survey via 2 different methods depending on patients having an online patient portal or not. The use of personal technology inherently selects against those that either don’t have as much access or don’t navigate technology well. Finally, due to the anonymous nature of the questionnaire, basic demographic information was not collected.

Conclusion

The RIHLA is a non-proctored assessment tool that provides a measure of a patients’ health knowledge, which is a significant contributor to their overall health literacy. Its design makes cross-language translation easier, allowing potential identification of low health literacy patients in diverse primary care populations. Further study is required to determine if the translated versions have the same validity as the English versions.