The Importance of Measuring and Analyzing Disparities in Vaccine Hesitancy by Race and Ethnicity

A manuscript recently published in the Journal of Primary Care & Community Health entitled, “Intention to Receive COVID-19 Vaccine by U.S. Health Sciences University Employees” ¹ is an example of the breadth of research that seeks to understand and identify solutions to curb the COVID-19 pandemic in near real-time. The publication of science has accelerated during the COVID-19 pandemic. ² The manuscript mentioned above is an illustration of how the accelerated review process could result in the publication of studies that suffer from important methodological and measurement deficiencies. Due to the undetected methodological problems described below, the findings are less than useful (at best) and potentially misleading (at worst). The primary source of these concerns lies in the omission of critical measures that influence vaccine hesitancy, including race, ethnicity, and occupation, and over-generalization of findings to health care providers when the population was not limited to clinical staff. Each of these concerns is described in detail below.

First, 3 critical measures (race, ethnicity, and occupation) were not analyzed despite their clear associations with vaccine hesitancy. ³ The intersection between these 3 measures is especially pronounced in university settings (ie, people who identify as White are overrepresented in faculty positions and people who identify as Black or African American and Hispanic or Latinx are overrepresented in administrative support and maintenance positions).^4,5 Among health care workers, those who identified as African American or Black were half as likely to be vaccinated when compared to Whites. ⁶ Despite identifying that “demographic questions” (p. 1) were administered to participants, respondents’ race and ethnicity were, surprisingly, never mentioned in the manuscript. Excluding this critical confounder from multivariate models may result in mis-specified and biased estimates that lead to incorrectly attributed conclusions. The omission is especially problematic because these measures are the best proxies to explain disparities observed in clinical trial participation, medical interventions, and reduced uptake of public health interventions among persons of color; particularly, African American/Black, Hispanic/Latinx, and indigenous communities. ⁷ The American Medical Association reinforced the notion that race and ethnicity must be examined, as “removing race and ethnicity from research may conceal health disparities. Thus, inclusion of race and ethnicity in reports of medical research to address and further elucidate health disparities and inequities remains important at this time” (p. 3).^8,9

The authors adapted the “SAGE Working Group on Vaccine Hesitancy” questionnaire, which does not include measures of race or ethnicity. ¹⁰ The systematic exclusion of race and ethnicity in this questionnaire limits our collection of data to identify disparities in vaccine uptake, our ability to address social, economic, and environmental barriers and implement culturally appropriate interventions. Without knowledge on “what works” and for whom, African American/Black, Hispanic/Latinx, and indigenous communities will continue to be disproportionately affected by COVID-19.

Similarly, the measurement of participants’ occupational status lacked precision. The authors classified participants as faculty, staff, “resident/fellow/post-doc”, or some “other” role. However, some critical distinctions are missing from this measurement structure given the study site (a university—based health science center) and the subject matter. For example, information on job type (administrative support, physician, medical assistant, research faculty, etc.) and roles (clinical or non-clinical; with or without patient interaction) would identify whether participants were routinely interacting with patients. The imprecise classification structure used in this manuscript does not account for differences in hesitancy between clinical and non-clinical faculty, research assistants that work with patients versus administrative assistants who are working at home, and post-doctoral research associate or medical fellow/trainee (with patient interaction). The omission of clinical activity as a covariate could explain the counterintuitive findings, such as “Men had higher levels of intention to receive the vaccine [compared to women]”. ¹¹ The finding that faculty were 2.7 times as likely to report being “likely to get a vaccine” compared to staff is likely to represent the association between confounders that were omitted from the model, such as socioeconomic status, race, ethnicity, and patient interaction.

Second, the occupational setting plays a key role in one’s risk of COVID-19 exposure and is essential to develop and tailor messages and incentives to encourage vaccine uptake. The limited measures on demographics and occupational role used in the present study did not permit a thorough assessment to identify key opportunities for tailored messaging. Other measures known to influence vaccination uptake and hesitancy include the presence of children or high-risk individuals in the home—another indicator that was not assessed in the present study.

In summary, we highlighted several measurement problems that impact the internal validity of the findings in a manuscript published on vaccine hesitancy during the COVID-19 pandemic. Most of the missing measures we identified concern the omission of racial, ethnic, and occupational diversity measures, which not only introduce the problem of confounding in multivariate models, but also the limited value of research using the SAGE questionnaire to reduce disparities in vaccination. We urge scientists and funding agencies to discuss equitable measurement and the importance of high-quality research, especially for topics that have life and death policy implications, such as vaccine hesitancy.