Abstract
Background:
Limited health literacy is common among persons infected with HIV and has been linked to poor mental and physical health outcomes, but there are no well-validated screening measures of health literacy in this vulnerable clinical population. The present study evaluates the usefulness of the Newest Vital Sign (NVS) as a brief measure of health literacy in HIV disease.
Methods:
Seventy-eight HIV+ adults were administered the NVS, Rapid Estimate of Adult Literacy in Medicine (REALM), and Single Item Literacy Screener (SILS). Main criterion variables included plasma HIV viral load, medication management capacity, self-efficacy for medication management, and perceived relationships with healthcare providers.
Results:
The NVS showed good internal consistency and moderate correlations with the REALM and SILS. Rates of limited health literacy were highest on the NVS (30.3%) as compared to SILS (6.6%) and REALM (9.2%). A series of regressions controlling for education showed that the NVS was incrementally predictive of viral load, medication management capacity and self-efficacy, and relationships with healthcare providers, above and beyond the REALM and SILS.
Conclusion:
The NVS shows evidence of reliability, convergent validity, and incremental criterion-related validity and thus may serve as useful screening tool for assessing health literacy in HIV disease.
Keywords
Introduction
Health literacy is a dynamic construct that has been defined as “the degree to which individuals can obtain, process, understand, and communicate about health-related information needed to make informed medical decisions.” 1 It is a concept that is distinguishable from general literacy, academic skills, and global neurocognitive functions, 2 although it often entails many of the same skills that support these abilities, such as word knowledge, numeracy, and problem-solving. It is estimated that one-third to one-half of the US population has either marginal or low health literacy. 3 A growing body of literature shows that suboptimal health literacy is associated with poorer functional and disease outcomes. Specifically, low health literacy has been consistently linked to difficulties with health-care activities (eg, capacity to take medications correctly, understand, and interpret health messages), poorer mental health (eg, symptoms of depression 4 ), less successful disease control, and increased risk of hospitalization and mortality, particularly in the elderly individuals. 5 The financial burden of low health literacy in the United States is estimated to fall between US$100 and US$250 billion annually. 6
Despite its obvious relevance to populations with chronic medical conditions, health literacy received relatively little attention in the context of HIV disease. The lack of attention to health literacy among individuals with HIV is surprising, given that HIV disproportionately affects populations at high risk of poor health literacy, including racial/ethnic minorities 7 and individuals of lower socioeconomic status. 3 For example, in the United States, African Americans constitute nearly 45% of the HIV-positive population and have high mortality rates. 8 In fact, it has been suggested that poor health literacy may at least partly explain the racial health disparities commonly seen in HIV. 9 Individuals with HIV that evidence limited health literacy have less knowledge about their disease, 10 are more likely to be nonadherent to their medication regimen, have lower self-efficacy for health behaviors, 9 and higher rates of HIV-associated neurocognitive impairment. 11 Evidence for the association between health literacy and clinical markers of HIV severity (eg, CD4 count, viral load, and AIDS diagnoses) is weak and conflicting. 4,12 Indeed, some studies report significant relationships between lower health literacy and HIV disease severity, 13 whereas other studies show null results. 14 However, some limitations exist within this literature, including selection bias toward highly functional persons and psychometric issues regarding the measurement of health literacy. For example, many of these studies utilized the Rapid Estimate of Adult Literacy in Medicine (REALM) 15 and Test of Functional Health Literacy in Adults (TOFHLA), 16 which sometimes have ceiling effects and may not measure health literacy, per se, but rather general reading ability. 17 Such limitations beg the question as to whether the relationship between health literacy and outcomes in HIV is dependent on sample characteristics, a function of the measurement tool, or is simply weak and clinically nonmeaningful.
The assessment of health literacy is constantly evolving, and currently many tools are available. Health literacy tests vary in administration time, format (eg, written versus spoken), utilization across settings (eg, primary care versus research laboratory), type of literacy measured (eg, print literacy, oral literacy, numeracy), and approach to measurement (eg, self-report vs objective assessment). 18 Currently, there is no gold standard for the screening or measurement of health literacy, and each test is associated with its own strengths and weaknesses.
One potentially useful measure of health literacy, the Newest Vital Sign (NVS), 19 has gained popularity in recent years. The NVS is a brief, 6-item screening tool that assesses an individual’s ability to understand and apply information displayed on a nutrition label for ice cream. The NVS has demonstrated good reliability and convergent validity with well-validated and commonly used measures of health literacy 19 such as TOFHLA. Moreover, it shows evidence of validity in various demographic populations, including Spanish speakers and diverse age ranges. 20 The relatively easy and quick (3-minute) administration has made NVS suitable for primary care and clinic-based settings as well as the research laboratory. In line with the current direction of dynamic and increasingly holistic operationalization of health literacy tools, 18 the NVS arguably taps into multiple different but important aspects of print literacy, numeracy, and oral literacy (ie, communication skills). The NVS has been studied across several medical conditions including kidney disease, 21 diabetes, 22 chronic pain, 23 multiple sclerosis, 24 and Parkinson disease. 25 Across many of these studies, the NVS has been useful in predicting important health outcomes such as hospitalization 24,25 and disease control. 22
To our knowledge, only 2 studies utilized the NVS in the context of HIV disease. One study by Morgan et al 11 found that, among 24 individuals with HIV-associated neurocognitive disorders, lower scores on the NVS were correlated with greater severity of neurocognitive dysfunction, most notably working memory (rs = .65) and verbal fluency (rs = .70). In a study of 46 HIV-positive individuals, lower NVS scores were related to worse performance on Web-based measures of online health-care skills, including pharmacy refills (r s = .42) and health records navigation (r s= .46). 26 Therefore, this early research suggests that the NVS may be valuable tool in HIV. However, little is known about NVS in relation to disease outcomes and traditional measures of health-related behaviors in HIV disease. To date, there are no studies that have taken a clinical outcomes-focused approach to evaluating the independent or incremental utility of the NVS in HIV. Therefore, the primary purpose of the study was to investigate the relationship between the NVS and various health outcomes and behaviors in a sample of HIV-positive adults. Moreover, we aimed to compare the usefulness of the NVS in comparison to 2 widely used measures of health literacy: REALM and Single Item Literacy Screener (SILS). It was hypothesized that NVS would predict HIV disease outcomes and health behaviors above and beyond these other measures of health literacy.
Methods
Participants
This study included 78 persons with HIV disease from the San Diego community and local HIV clinics, whose demographic and disease characteristics are presented in Table 1. HIV serostatus was determined by enzyme-linked immunosorbent assays and confirmed by Western blot tests or by MedMira rapid HIV-antibody test. Participants were excluded if they had histories of severe psychiatric disorders, chronic medical or neurological conditions (eg, traumatic brain injury with loss of consciousness >30 minutes and active opportunistic infections), non-HIV-related dementias, or an estimated premorbid IQ <70 based on NIH toolbox 27 or Wechsler Test of Adult Reading. 28 Individuals were also excluded if they met Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) criteria 29 for substance use disorders within 1 month of evaluation or if they had tested positive for urine toxicology screening for illicit drugs (except marijuana) on the day of testing. A diagnosis of an HIV-associated neurocognitive disorder was assigned if criteria set forth by Antinori et al 30 were met based on a comprehensive neuropsychological and neuromedical research evaluation. 31
Demographic, Health, and Medical Characteristics of the Study Sample.a
Abbreviation: BERMA, Beliefs Related to Medication Adherence.
aMean (standard deviation).
Materials and Procedure
Health Literacy Assessment
The NVS is a 6-item performance-based measure of health literacy. 19 The total score is derived by summing the correct answers about information contained on a nutrition label for ice cream, with a possible score range from 0 to 6. Sample items are “If you eat the entire container, how many calories will you eat?” and “Pretend you are allergic to the following substances: penicillin, peanuts, latex gloves, and bee stings. Is it safe for you to eat this ice cream?” The NVS manual recommends that total scores between 0 and 1 reflect limited health literacy, scores between 2 and 3 reflect possible limited literacy, and scores between 4 and 6 indicate adequate health literacy. In this study, scores below 4 were considered as indicative of limited health literacy.
The SILS 32 is a 1-item self-report measure that is used to assess the amount of assistance one needs with printed material. Specifically, the item asks “How often do you need to have someone help you when you read instructions, pamphlets, or other written material from your doctor or pharmacy?” and is rated on a 5-point Likert-type scale. A score >2 indicates some difficulty with reading printed material and implies limited health literacy.
The REALM 33 is a 66-item screening measure that is used to assess an individual’s capacity to recognize and pronounce health-related words (eg, “pill” and “anemia”). It is scored on a scale ranging from 0 (no words read correctly) to 66 (all words read correctly). Scores between 0 and 44 suggest a reading level of up to sixth grade and indicate limited health literacy. Scores between 45 and 60 suggest a reading level of seventh to eighth grade and are indicative of marginal health literacy. Scores between 61 and 66 suggest a reading level of ninth grade and above and are indicative of adequate health literacy. In this study, all scores below 61 were considered as indicative of limited health literacy.
Medical Evaluation
The neuromedical evaluation consists of a blood draw and thorough assessment of comorbidities and current medications. Participants were screened for various comorbid health conditions including hepatitis C coinfection, diabetes, chronic pulmonary disease, mild liver disease, and transient ischemic attacks. Cardiovascular risk was assessed based on the presence of a history of myocardial infarction, congestive heart failure, or hyperlipidemia. A comorbidity composite was derived for each participant using the total comorbidities or risk factors present (eg, out of a possible 9 conditions). Current CD4 count and viral load were derived from plasma samples. Nadir CD4 count, AIDS diagnosis, and estimated duration of infection were derived from clinician interviews.
Psychiatric Evaluation
Current and lifetime affective (ie, depression and anxiety) and substance use disorder were determined using the Composite International Diagnostic Interview (version 2.1). Participants also completed the Profile of Mood States (POMS) 34 to assess current level of affective stress. The Total Mood Disturbance Score, which ranges from 0 to 200, was used, wherein higher values indicated greater affective distress.
Health Behavior and Attitudes
An adaptation of the Medication Management Test–Revised (MMT-R) 35 was used as a performance-based measure of medication management skills. 26,36 The main component of the task is pill dispensing, whereby participants are given a prescription for several medications and asked to dispense the appropriate amount of pills for 1 day in a pill organizer. It is scored on a scale of 0 to 13, and higher scores indicate better performance. Due to a ceiling effect in our sample (median score = 13), continuous scores were transformed into a dichotomous pass/fail variable. Scores of 13 (ie, the median for this sample) were classified as passing, while scores <13 (i.e., committing any error) were classified as failing.
The Beliefs Related to Medication Adherence (BERMA) 37 is a self-report measure of general medication management. The 23-item “dealing with health professionals” subscale assesses the relationship 1 perceives with his or her medical providers. Examples of questions include “I have difficulty talking openly with my physician” and “I have difficulty asking my pharmacist questions about my medicines.” The 24-item “memory for medications” subscale assesses perceptions of one’s ability to remember and follow his or her own medication regimen and includes items such as “I am good at remembering to take my medications” and “If I am put on the spot to remember why I am taking my medications, I know I will have difficulty doing it.” All subscale items are rated on a 5-point Likert-type scale ranging from 1 (strongly disagree) to 5 (strongly agree). Items are reversed so that higher sores indicate more positive beliefs. The possible range of scores for the dealing with health professionals scale is 23 to 115, while the memory for medications scale ranges from 24 to 120.
Data Analysis
Statistical analyses were conducted using a JMP pro software package (version 12.1.0). Bivariate relationships were assessed using Pearson r for normally distributed variables, Spearman ρ for nonnormally distributed variables, and Pearson chi-square tests for categorical variables. Nominal logistic and multiple linear regression models were used to investigate the relationship between health literacy measures and various behavioral and health outcomes. Since educational attainment is strongly related to health literacy, 38 level of education was chosen as an a priori covariate for all models. Additionally, since one of the research aims was to assess the predictive value of the NVS compared to the SILS and REALM, all 3 tests of health literacy were forced into every model. Therefore, the models that were evaluated for every criterion variable (ie, plasma viral load, MMT-R, and the 2 BERMA scales) included education, NVS, SILS, and REALM as predictors. In an effort to enhance the clinical relevance of these data, we followed these multivariable models with simple univariate tests comparing persons with low and high NVS scores (using the above-referenced cut points) on each outcome. Cohen’s d effect size estimates or simple odds ratios with 95% confidence intervals were generated to accompany these follow-up analyses. Critical α was set to .05 for all statistical tests.
Results
Comparison of Health Literacy Measures
Cronbach α for the 6 individual items of the NVS was 0.7, suggesting good internal consistency. The range of NVS total scores was 0 to 6, with a median of 4 (interquartile range 3-6). Moderate correlations were observed for the NVS total score with the REALM (r s =−.47; P < .001) and with the SILS (r s = −.39; P = .001), thus demonstrating good convergent validity. Using the dichotomous thresholds listed earlier, the frequencies of low health literacy were compared across the 3 health literacy measures (see Figure 1). Fourteen percent of the sample (n = 11) was classified as having limited health literacy by the SILS (6.6%), the REALM (9.2%), or both (1.3%). In contrast, 30.3% (n = 30) of the sample was classified as having limited health literacy by the NVS. Thus, the NVS identified 118% (n = 13) more cases of limited health literacy than the REALM and SILS combined.

Health literacy classifications across 3 commonly used measures in 76 participants with HIV. Adequate = The score on that measure fell above the established cut points for poor health literacy. Limited – Overlapping = The score on that measure and at least 1 other measure fell below the established cut points (ie, 2 or more tests classified the individual as having poor health literacy). Limited – Unique = The score on only that measure fell below the established cut points and the other measures classified that individual as having adequate health literacy. NVS indicates Newest Vital Sign; SILS, Single Item Literacy Screener; REALM, Rapid Estimate of Adult Literacy in Medicine.
Predictors of Health Behavior, Attitudes, and Outcomes
A summary of the primary statistical models examining the association between the NVS and health behaviors, attitudes, and viral load in this HIV+ sample is presented in Table 2. The multiple regression model predicting HIV plasma viral load was significant (F 4, 69 = 2.68; P = .04), and the NVS emerged as the lone significant predictor, t(69) = −3.10; P = <.01. Univariate follow-up tests showed that HIV-positive persons with low NVS scores had significantly higher viral loads (M=1.87 log10 copies per mL; standard deviation [SD]=.7) than those with higher NVS scores (M= 1.62 log 10 copies per mL; SD = .08). This finding was associated with a medium effect size (d = .67 [0.16-1.18]). Figure 2 illustrates group differences between good (ie, adequate) and poor (ie, limited) NVS performers across each outcome measure.
Summary of Multivariable Regression Analyses Predicting Health Behaviors and Outcomes.
Abbreviations: BERMA, Beliefs Related to Medication Adherence; MMT-R, Medication Management Test–Revised; NVS, Newest Vital Sign; REALM, Rapid Estimate of Adult Health Literacy in Medicine; SILS, Single Item Literacy Screener; B, unstandardized regression coefficient; SE B, unstandardized standard error.
a P < .01.
b P < .05.
c F ratio of the analysis of variance.
d P < .1.
eχ2 statistic.

Poor health literacy as defined by the Newest Vital Sign (NVS) is associated with worse health behaviors and outcomes in 76 participants with HIV disease. The dark boxes show participants with adequate health literacy on the NVS (n = 53) and the light boxes show participants who scored in the range of poor health literacy on the NVS (n = 23). On the left-hand y-axis are average sample-based z scores for HIV RNA and 2 scales of the Beliefs Related to Medication Adherence (BERMA) questionnaire. On the BERMA, lower scores reflect poorer self-efficacy for managing medications and dealing with health professionals. Standard errors are provided. On the right-hand y-axis is the frequency of pass rates on the Medication Management Test–Revised (MMT-R).
The logistic regression model predicting MMT-R failures was significant (χ2 = 16.58; P ≤ .01), and both the NVS and SILS emerged as significant predictors of performance (Ps < .05). Participants with low NVS scores were 2.4 (0.8-6.7) times more likely to fail the MMT-R than those with good NVS performance. By way of comparison, participants who reported elevations on the SILS (ie, poor ability to read health material on their own) were 1.8 (0.2-17.1) times more likely to fail the MMT-R versus those with normal SILS scores.
The model predicting BERMA Memory for Medications neared significance (F 4, 70 = 2.10; P = .09), although the NVS emerged as a significant predictor, t (68) = 2.25, P = .03, within this model. Univariate follow-up tests showed that HIV-positive persons with low NVS scores had a lower BERMA Memory for Medications score (M = 71.78; SD = 13.29) than those with higher NVS scores (M = 80.17, SD = 11.17). This finding was associated with a medium effect size (d = .68 [−0.0 to 1.15]).
The model predicting BERMA Dealing with Health Professionals was significant F 4, 68 = 3.68; P = .009, and, again, the NVS emerged as the sole predictor, t(68) = 2.98; P = .004. Univariate follow-up tests showed that HIV-positive persons with low NVS scores had lower BERMA dealing with health professionals score (M = 85.14; SD = 15.86) than those with higher NVS scores (M = 96.62, SD = 10.90). This finding was associated with a large effect size (d = .79 [0.27 to 1.3]).
Finally, performance on the NVS, SILS, or REALM was not related to the presence of (ie, >0) or total number of comorbid medical conditions, current CD4 count, nadir CD4 count, an AIDS diagnosis, or the presence of an HIV-associated neurocognitive disorder (all Ps > .05).
Discussion
The primary objective of this study was to assess the utility of NVS in predicting clinically relevant health behaviors and outcomes in persons infected with HIV. Findings provided strong support for the internal consistency, sensitivity, and construct validity of the NVS in HIV disease. Approximately one-third of the HIV-positive sample performed below standard NVS cut scores for adequate health literacy, which was nearly 120% more than that classified by 2 more commonly used health literacy screeners (ie, SILS and REALM). Additionally, the NVS was a significant predictor of plasma viral load, objective medication management capacity, self-efficacy for medication management, and perceived attitudes toward health professionals. Speaking to its potential incremental validity in this regard, the NVS emerged as an independent predictor of these outcomes relative to 2 other measures of health literacy (eg, SILS and REALM) and educational attainment. The implications of these various findings are discussed in detail subsequently.
Overall, the NVS demonstrated good psychometric properties in this HIV-positive sample. Internal consistency (Cronbach α = .7) was found to be comparable with the original report using a clinical sample of persons in primary care (Cronbach α = .76) 19 and is adequate by conventional standards. 39 Moreover, the NVS correlated moderately well both the SILS and the REALM, which is consistent with prior reports from a primary care setting 9 and suggests good convergent validity with established screening measures of health literacy. In conceptual terms, better NVS performance was associated with better recognition and pronunciation of health-related words and better perceived self-sufficiency in reading health-related material. Taken together, our results therefore suggest that the psychometrics of the NVS operate similarly in an HIV-positive population, as they do in other clinical populations in which it has been studied.
Highlighting the sensitivity of the NVS, performance failures on this measure were at 30%, which was the highest among the 3 health literacy screening tools administered in this study. This finding is consistent with other reports in clinical populations such as diabetes 40 and older adults engaged in primary care, 41 which also suggest that the NVS may have superior sensitivity to health illiteracy as compared to other screening measures. Importantly, more than half of the individuals classified as having limited health literacy by the NVS were not captured by the other 2 screening measures (Figure 1), despite the moderately strong correlations with the SILS and REALM. Post hoc analyses revealed that individuals who failed only the NVS (ie, Limited-Unique NVS group; Figure 1) did not differ substantially from those who had demonstrated limited health literacy across more than 1 measure with regard to demographic variables, disease severity, presence of HIV-associated neurocognitive disorder, or other medical and psychiatric comorbidities (all Ps > .05). However, the sample size of these post hoc analyses was small (n = 23), and there was likely limited power to detect anything less than a very large effect. Nevertheless, these findings suggest that the SILS or the REALM are not grossly insensitive to a particular subgroup. Rather, it is perhaps more likely that the NVS is more fully capturing the multidimensional construct of health literacy in HIV (ie, numeracy, prose literacy, and oral literacy), which therefore increases its sensitivity.
One notable finding was a significant relationship between NVS and plasma viral load. As noted previously, the current literature on health literacy and clinical biomarkers in HIV is weak and conflicting. 4 In this study, the association between NVS and viral load was moderately strong (d = .67), which suggests that previous null findings could be partially due to differences in health literacy measurement. That is, many previous studies employed the REALM, TOFHLA, or S-TOFHLA as measures of health literacy. In this sample, it may be that certain aspects of health literacy assessed by the NVS, such as numeracy, are influencing viral load above and beyond word reading or self-sufficiency in reading health material. Numeracy skills are an integral component of medication management capacity 42 and are also associated with medication adherence 43 which in turn influences HIV viral load. While this study found significant relationships between NVS performance and medication management capacity as well as plasma viral load, future research may wish to expand upon these findings and assess whether a causal relationship exists between components of health literacy (eg, numeracy), medication management capacity, medication adherence, and viral load. Furthermore, the NVS is a more objectively difficult task relative to S-TOFHLA, 9,40 which may explain, in part, why the NVS may be a more sensitive measure to outcomes, despite both tools assessing broadly similar domains.
One important aspect of health literacy is the ability to obtain and communicate health-related information, which is a skill that often requires interaction with various health-care personnel—from learning administration and side effects of medication at the pharmacy counter, to communicating changes in symptoms to one’s primary care provider. This study identified that HIV-positive adults with low health literacy are more likely to have negative beliefs about interacting with medical professionals (Figure 2). This finding also echoes previous NVS literature that found that parents with low health literacy contacted their child’s pediatrician with less continuity and were confused about the process by which their children were to receive services. 44 In HIV disease, it is unclear whether beliefs about dealing with health-care professionals result in reduced interaction with the health-care system, whether similar downstream effects of low health literacy (eg, poor continuity in care) are present or the magnitude to which health literacy interventions impact patient–clinician interactions.
Of further clinical relevance, the NVS showed incremental validity in predicting health behaviors and health outcomes above and beyond education and 2 other commonly used tools: the SILS and REALM. The SILS is a simple self-report measure of assistance needed for reading health-related material and the REALM, being a word reading task, maps onto basic decoding skills. In contrast, the NVS assesses more complex cognitive functions associated with reading comprehension, numeracy, and problem-solving and may be more sensitive to health illiteracy by virtue of tapping into more aspects of neurocognitive functioning. 11,25 Thus, it stands to reason that the assessment of health literacy using the NVS may provide an added degree of clinical utility to providers in health-related environments at little cost, in terms of both time (ie, the NVS takes only 3 minutes to administer) and expense. Further research is needed to assess whether such information impacts patient care, such as aiding clinicians in identifying good candidates for health literacy interventions or persons at risk for disengaging from health care.
There are several limitations to the study that are worth noting. Our sample was predominantly highly educated Caucasian men, which may limit the external validity of the findings. The prevalence of limited health literacy in our sample may therefore be an underestimate. Nevertheless, we were able to identify significant health literacy vulnerabilities in an otherwise low-risk population. Although no formal gold standard exists for measuring health literacy, other more extensively studied tools, such as the TOFHLA, were not included for comparison. Additionally, this sample performed exceptionally well on the objective measure of medication management capacity, which resulted in a high ceiling effect and consequently limited the statistical analyses. Moreover, measures of medication management capacity and beliefs about memory for medication were included but measures of objective medication adherence (eg, electronic monitoring) were not. Similarly, as previously noted, a measure of attitudes toward health-care professionals, but actual number of interactions, or measures of retention in care, were not assessed. One avenue of future research is to understand the role that health literacy plays in the relationship between functional capacity to engage in health behaviors and actual real-world behaviors. While measures of functional capacity and indicators of real-world behavior were measured in this study, the cross-sectional nature of data limited any implications about causality that can be made.
The present findings add to the growing body of literature of the utility of the NVS in assessing health literacy in clinical populations. In summary, the results of the study suggest that NVS is a reliable tool with emerging evidence of construct validity in an HIV-positive population. These findings can aid physicians and health personnel in clinical practice and guide future research on the study of the impact and remediation of limited health literacy on individuals with HIV disease.
Footnotes
Acknowledgments
We are grateful to Erin Morgan, Donald Franklin, Jessica Beltran, and Stephanie Corkran for their help to the study management, data collection, and data processing
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: grants R01-MH073419, R21-MH098607, and P30-MH62512 from the National Institute of Health.
