Abstract
Objective
Our objective was to develop and test a digital decision-aid tool to help vaccine-hesitant participants make informed decisions regarding COVID-19 vaccination.
Methods
A multidisciplinary team at Emory University developed an online decision-aid tool that presented information about the virus, vaccine risks/benefits, and vaccination resources. This tool was provided to patients in an Atlanta primary care clinic who were not fully vaccinated/boosted. Participants were then surveyed to assess their knowledge and opinions.
Results
The tool was administered to 40 patients (55% Black, 70% female, and 45% aged 50 – 69) between January-June2024. Nearly all (97.5%) found the tool useful for improving their understanding of vaccine risks and benefits, with 70% rating it “very” or “extremely” useful. After using the tool, 47.5% of participants reported that the tool had increased their willingness to get vaccinated, and 37.5% indicated they had decided to get vaccinated/boosted. Notably, 22.2% of participants who had never previously received a COVID-19 vaccine stated their intent to get vaccinated after using the tool.
Conclusions
This pilot study demonstrated that a digital decision-aid tool can improve knowledge and perceptions surrounding COVID-19 vaccination to encourage healthy decision-making. As such, the tool represents a promising adaptable model for self-administered decision aids in broader healthcare contexts.
Introduction
Improving COVID-19 vaccination rates is urgent due to the unique transmission dynamics of the SARS-CoV-2 virus. COVID-19 is caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which can be transmitted through contact, droplet, and airborne routes. SARS-CoV-2 readily spreads through aerosols in poorly ventilated indoor environments, contributing to prolonged community transmission. 1 Aerosol transmission occurs for SARS-CoV-2 via small respiratory droplets (less than 10 μm) that remain airborne and viable for up to 3 hours and 4.8 meters away. 1 This aerosol transmission paradox, where particles of various sizes remain airborne, renders traditional public health guidance less effective, as it primarily targets larger droplet transmission. Thus, conventional prevention measures such as social distancing alone are insufficient, and vaccination remains an essential mitigation strategy, especially in indoor healthcare and community settings with airborne transmission risk. 1
Beyond its rapid spread, vaccine uptake was further hindered by the novel nature of mRNA-based vaccines. Unlike conventional vaccines, mRNA vaccines function by administering a small amount of mRNA that codes for a specific viral protein. Cells use this mRNA to produce the viral antigen, which the immune system recognizes and responds to by producing antibodies and developing immunological memory for rapid response upon future exposure to the virus. 2 While mRNA vaccines offer considerable advantages, this novel mechanism of action represented a significant departure from traditional vaccine approaches. Despite robust safety and efficacy data, the unfamiliarity of mRNA technology became a point of skepticism and contributed to vaccine hesitancy, uncertainty, and decreased uptake in some populations. 2
Digital health tools are transforming primary care by enhancing patient engagement, personalizing health interventions, and reducing clinician burden.3–5 The increasing integration of web-based decision-aids, mobile health (mHealth) applications, and telemedicine platforms into clinical workflows has already demonstrated significant potential for improving healthcare accessibility and equity.6–8 The use of digital health approaches in healthcare delivery has been found to improve information sharing between patients and providers, as well as to treat hard-to-reach populations. 9 Further, scoping reviews have found digital health tools to have tangible positive impacts on health outcomes, such as chronic disease management. 10
Vaccine hesitancy is a pressing challenge within primary care and was identified as one of the top ten threats to global health by the WHO in 2019. 11 While traditional in-person counselling remains critical, digital tools offer an opportunity to provide patients with accurate, tailored vaccine information in a scalable and efficient manner to combat hesitancy. This study introduces a novel, self-administered digital decision-aid tool designed to support patients in a primary care setting who express hesitancy regarding COVID-19 vaccines. COVID-19 vaccine hesitancy persists in the United States, with only 22.6% of the adult population having received the updated 2024-25 COVID-19 vaccine as of February 2025. 12 This lack of uptake is particularly stark when compared to the flu vaccination rate of 44.3% for the same period. 12 The issue is even more pronounced among certain racial and ethnic minority groups, with only 15.3% uptake among Hispanic Americans and 17.2% among non-Hispanic Black Americans. 13 Further, regional disparities in vaccination trends also exist. For example, Georgia, where this study takes place, has one of the lowest rates of COVID-19 vaccination in the country, with only 13.8% of residents having received any dose of the updated 2024-25 COVID-19 vaccine. 13
Unlike conventional vaccine communication strategies that typically rely on clinician-led discussions, this decision-aid tool enables patients to independently engage with structured, evidence-based vaccine information. Designed to address the empirically supported drivers of vaccine hesitancy, such as fear of side effects, doubts about vaccine effectiveness, and skepticism about vaccine necessity, patients can interact with the decision-aid via their own devices in the waiting room before their medical appointment. 14 This approach has the potential to improve patient decision-making while minimizing the burden on healthcare providers.
This study evaluates the decision-aid’s impact on COVID-19 vaccine knowledge, perceptions, and intent to vaccinate among primary care patients in Atlanta, Georgia. The immediate goal was to decrease COVID-19 vaccine hesitancy, with the broader objective of advancing digital health solutions to address primary care challenges. By exploring this intervention within the context of digital primary care, we examine how self-administered decision-support tools can enhance vaccine literacy and empower individuals to make informed health choices. The findings contribute to the growing body of evidence supporting the integration of digital health tools in primary care and their role in reducing vaccine hesitancy.
Methods
Tool development
From December 2022 to December 2023, a multidisciplinary team with expertise in computer science, epidemiology, and clinical medicine at Emory University developed an online visual COVID-19 decision-aid tool. The tool drew inspiration from the Australian National Centre for Immunization Research and Surveillance’s tool - “Decision aid: Should I get the COVID-19 vaccine?” which launched in 2021. 15 That decision-aid and the one developed in this project were guided by International Patient Decision Aid Standards, which stress that the primary goal of such tools is to reduce decisional conflict and that decision-aids are successful when patients feel they have made the right decision for their circumstances, based on their values. 16 Following these guidelines, the tool was designed to present facts about COVID-19 and vaccination that are unbiased, giving equal attention to benefits and risks associated with all options, as well as helping patients clarify how their values related to each option and providing a way for patients to communicate these values with their healthcare practitioners. 17
The design of the tool was also guided by evidence and research from a combination of the fields of epidemiology, behavioral psychology, and marketing. For example, the decision to use a multi-tier final option for vaccine decision-making, rather than the traditional two-option choice of either accepting or refusing the vaccine, was derived from marketing research on the efficacy of promoting compromise options to increase decision confidence in health settings. 18
Epidemiological data for the tool was sourced through an internal systematic review of studies on COVID-19 vaccine risks and benefits. This included data from vaccine manufacturers, post-marketing surveillance on side effects, FDA reports on efficacy, and community prevalence rates. These data sources were cross-validated and integrated into a live system to create personalized risk and benefit models. However, the final version was not based on a previously validated questionnaire. Given the pilot nature of the study, the survey was designed specifically for this tool and has not undergone formal psychometric validation. Computer scientists on the research team developed the tool and subsequently integrated it into Emory’s COVID-19 Health Equity Dashboard, using open-source libraries like React JS for front-end development. The tool featured several interactive modules: information on COVID-19 risks, a personalized risk assessment calculator, details on approved vaccines, and the advantages and disadvantages of vaccination. Each module was developed in parallel and equipped with anonymized interaction logs for future research on its effectiveness. Formative user feedback was gathered to iteratively refine the tool’s interface and content.
Participants, setting, & study design
This study was a cross-sectional, one-group post-test-only design conducted at the Emory Dunwoody Family Clinic in Atlanta, Georgia, from January to June 2024. Adult patients of the clinic were recruited in person via flyers posted and distributed in the clinic’s waiting room.
Inclusion and exclusion criteria
Eligible participants were (1) 18 years or older, (2) English-speaking, and (3) not fully vaccinated and boosted for COVID-19 at the time of participation, according to Centers for Disease Control and Prevention (CDC) recommendations. Individuals were excluded if they had known contraindications to COVID-19 vaccination or were unable to provide informed consent.
Procedure
Potential participants independently screened, consented, viewed the decision-aid, and completed a survey via QR code included on flyers in the waiting room of the clinic. The QR code first directed patients to a consent form and eligibility screening questionnaire, and participants who met all criteria were then taken to the web-based decision-aid tool (available at https://covid19.emory.edu/decision-aid/about and in Supplementary Material). Participants were guided to independently navigate the site as per on-screen instructions. Figure 1 provides a visual of the first four webpages of the decision-aid tool as experienced by participants. Once finished moving through the decision-aid tool, they completed a 15-question survey that assessed their knowledge and opinions about COVID-19 vaccines and a demographic survey. Screen captures from the decision-aid tool (desktop version).
Measures
Two primary survey instruments were used to assess the tool’s impact. The 15-question post-tool survey asked participants about the tool’s impact on their beliefs and plans related to COVID-19 vaccination, their vaccine knowledge after using the tool, and the general usability of the tool. The questionnaire specifically asked participants if they decided to get the COVID-19 booster or vaccine after using the tool to evaluate the tool’s ability to reduce vaccine hesitancy. A range of question types was used, including multiple choice, open response, and sliding-scale questions. A 16-question demographic survey that followed the primary questionnaire asked about participant characteristics such as age, gender, race, and occupation. Additionally, this section included questions about participants’ general experiences and beliefs surrounding healthcare authorities and vaccinations. Both survey instruments can be viewed at https://covid19.emory.edu/decision-aid/about. (See Supplemental Material).
Data analysis
Descriptive statistical analysis was conducted using Microsoft Excel, summarizing categorical variables through frequencies and percentages. Given the cross-sectional nature of the study, inferential statistical testing was not performed. As such, potential confounders, such as other previous belief-influencing experiences and factors, were acknowledged as study limitations. Missing data were handled by excluding incomplete responses, with three participants removed from the final analysis.
Results
Participant characteristics
Demographic characteristics of participants.
The majority of participants agreed that vaccines, in general, are important for the health of others in their communities (80.0%) and found childhood vaccines to be important and effective (72.5%). However, nearly half of the participants (47.5%) reported having heard negative stories about the COVID-19 vaccination that caused them to worry about receiving the vaccine, and 40.0% noted that they had serious concerns about possible reactions they might have after receiving the COVID-19 vaccine. Trust in information sources varied, with only 57.5% of participants stating that they trust the information they receive from healthcare authorities and experts regarding the COVID-19 vaccine. Further, survey responses indicated a gap in COVID-19 vaccine education, as 40.0% of participants indicated they had not received information or education about COVID-19 vaccines before participating in this study.
Participants had varying levels of vaccination, with 40.0% only having received one booster, 37.5% having received the primary vaccine series but no boosters, and 22.5% having never received any vaccination against COVID-19.
Decision-aid effectiveness
After using the decision-aid tool, participants were asked whether they had reached any decisions regarding COVID-19 vaccination. Fifteen participants (37.5%) indicated they had decided to get vaccinated or boosted, while 8 participants (20.0%) stated they had made the decision not to.
Fifteen participants (37.5%) had not yet reached a decision, with 27.5% stating they needed to discuss the decision further with family or healthcare providers and 10.0% indicating they needed more information before deciding. While these responses do not necessarily indicate movement toward vaccination, they do reflect continued engagement with the decision-making process rather than outright rejection of future vaccination. The remaining two participants (5%) responded “other” to this question.
Among those who were completely unvaccinated before using the tool, 22.2% reported deciding to get vaccinated, while 44.4% were still deliberating. Although it is unclear whether these participants will ultimately pursue vaccination, their responses suggest the tool at a minimum played a role in prompting reflection on the decision. Figure 2 provides an overview of decisions to vaccinate by vaccination level. COVID-19 Vaccination Decisions After using Decision-aid Tool by Previous Level 2 of Vaccination.
In a similar vein to the question on decision-making, participants ranked the likelihood that they would become fully vaccinated on a scale from 0 to 10 after using the tool. The average of the responses was 7.4, indicating that many participants were more likely than not to receive the next booster or vaccine after using the tool.
Assessing the impact of the decision-aid tool on participants’ willingness to receive COVID-19 vaccination presents inherent challenges due to the study’s design. Without a pre-test or control group, it is not possible to definitively attribute decisions made or changes in willingness solely to participants’ interaction with the decision-aid tool. However, data does suggest that the tool played a role in shaping participants’ decision-making process, whether by increasing willingness, reinforcing existing stances, or prompting further consideration. Participants were asked to identify if and to what degree using the decision-aid tool specifically had changed their willingness to get vaccinated. Nineteen participants (47.5%) stated that the tool had increased their willingness to get vaccinated, while one individual (2.5%) reported decreased willingness. Half of the participants (50%) reported that their willingness had remained the same. It is important to note that a stable willingness to vaccinate does not necessarily indicate that the tool was ineffective. It is possible that the tool reinforced participants’ existing stance, helping them reach a confident decision – even if that decision did not change.
Other responses offer further insights into the tool’s effectiveness. Thirty-five participants (87.5%) affirmed that the tool provided enough information to make an informed decision, including 17 (85%) of those who reported no change in their willingness to vaccinate after using the tool. Similarly, 85% of all participants stated that the tool successfully addressed their concerns or questions about COVID-19 vaccination, with 52.5% reporting that all their questions were fully answered.
Participants’ likelihood to recommend the tool to others is another meaningful measure of its perceived value. More than half (52.5%) stated they would be very likely to recommend the decision-aid tool, including 33% of those whose personal decision remained unchanged. This suggests that even individuals who did not alter their own stance found the tool to be a useful resource generally.
Tool utility and user-friendliness
Beyond influencing vaccination decisions, the tool was widely recognized as useful for improving participants’ understanding of COVID-19 vaccination risks and benefits. Nearly all participants (97.5%) found the tool to be useful, with 70% rating it as either “extremely” or “very” useful. Information about the risks of remaining unvaccinated was particularly compelling, with 70% of participants identifying this information as impactful. Additionally, 11 participants cited risk-related data as the most influential factor in their vaccination decision-making, with one participant specifically noting that the “ratio showing hospitalization rates for unvaccinated versus fully vaccinated individuals” was particularly persuasive.
Post decision-aid survey questions & responses.
Discussion
In terms of the immediate goal of reducing COVID-19 vaccine hesitancy and keeping in mind the limitations of the study, these results are promising. The findings suggest that the self-administered digital decision-aid tool helped reduce decisional conflict regarding COVID-19 vaccination for this user group, and this reduction trended favorably toward acceptance or interest in further consideration of COVID-19 vaccination. By providing a structured, interactive format for exploring unbiased epidemiological information, participants were able to make informed decisions independently, potentially reducing reliance on clinician-led discussions. This approach aligns with broader trends in digital primary care, where tools such as telemedicine platforms and mobile health applications are increasingly leveraged to improve healthcare accessibility and outcomes. 19
The decision-aid demonstrated promising usability and effectiveness, with most participants reporting increased knowledge of the COVID-19 vaccine and 19 participants expressing increased willingness to get vaccinated as a direct result of using the tool. The integration of personalized risk assessments and real-time vaccine efficacy data proved particularly impactful in addressing participant concerns, according to participant feedback.
Reducing vaccine hesitancy is often not an immediate process; rather, it involves addressing concerns, building trust, and providing accessible, reliable information. 20 Decision-aid tools like this can play a key role in that process by helping patients clarify their questions and concerns over time. While not all participants reached a final decision on vaccination immediately after using the tool, the structured engagement with evidence-based information represents a meaningful step toward informed decision-making. Additionally, tools like this may serve as a starting point for critical discussions between patients and healthcare providers, allowing for more productive conversations about vaccination in a clinical setting. The fact that many participants indicated a desire to further discuss their concerns with their provider suggests that the decision-aid may function to facilitate deeper engagement and enhance patient-provider interactions.
The tool also proved to be a valuable asset in terms of time efficiency. Participants were able to complete the decision-aid in the waiting room, often spending well under 30 minutes in total, which likely reduced the need for extensive clinician time. At the same time, many users indicated a desire to discuss their concerns further with a healthcare provider after using the tool. This suggests that the tool prompted important conversations that may not have otherwise occurred. These conversations are likely to be informed by the information presented in the decision-aid, sparking more tailored lines of questioning and ultimately improving both the efficiency and quality of patient-provider communication.
This study contributes to a growing body of research on decision-aids for vaccination. Previous studies have demonstrated the efficacy of decision-aid tools for HPV vaccination and other immunizations, with notable success in engaging diverse populations.21–23 Our study expands on these efforts by demonstrating the potential of digital decision-aids in addressing COVID-19 vaccine hesitancy among adults in primary care settings, providing further evidence of their value in improving vaccination rates across various demographics. A critical element of this decision-aid is its focus on the consideration of patient values. It is a core principle in decision-aid discourse that users make decisions they find right for them, grounded in their personal values and context. This was an essential consideration in the design of our tool, as reflected in the survey instruments we included. Questions prompting participants to reflect on topics such as the efficacy of childhood vaccines or the importance of vaccines for community health gave users an opportunity to consider their values while using the decision-aid. This process may help ensure they reach decisions they feel confident in, while also generating data that can inform future iterations of the tool and create additional opportunities for customization. 24 Our neutral approach to presenting information and response options is something other tools may look to adopt, as it enables individuals to make informed, healthy decisions without influence that can result from the perception of a designed agenda. Beyond COVID-19 vaccination, personalized digital health interventions may be particularly effective in reaching populations with longstanding vaccine concerns. The structure of this decision-aid tool – incorporating risk assessment, interactive learning, and unbiased information – could be adapted for other vaccines, including influenza, HPV, and childhood immunizations. Given the increasing role of digital health solutions in primary care, decision aids like this could be expanded to promote a range of preventative health measures and public health interventions. With further development, such tools could be customized to provide guidance on other medical decisions, ensuring their continued relevance in patient-centered healthcare.
This tool represents a valuable addition to the digital health space, not only as a vaccine-related intervention but also as part of a broader movement toward using technology to improve patient engagement and decision-making. The results highlight the potential efficacy and usability of such tools, reinforcing the role of digital health interventions in enhancing preventative care and supporting healthcare systems that are more efficient and patient-centered.
It is also important to note the demographics of this study, which provide insight into historically underrepresented patient populations, such as Black Americans. It is crucial to inform future advancements in digital health interventions with the perspectives of understudied populations to ensure equitable healthcare delivery in an increasingly tech-centered healthcare landscape.
Given the increasing adoption of digital health solutions in primary care, the early successes of this tool can inform the development and implementation of other technology-based decision-making tools for promoting positive health behaviors.
Future decision-aids
Looking ahead, as vaccine technology evolves, the materials to enhance literacy in primary care settings must change as well. One such strategy for COVID-19 is the heterologous prime-boost vaccination approach, where patients receive different COVID-19 vaccines for their initial and booster doses. Research demonstrates that when individuals primed with AstraZeneca vaccine received a Moderna booster, neutralizing antibody levels increased 115-fold and showed greater effectiveness against variants such as Beta. 25 Furthermore, the development of nanovaccines, much like mRNA vaccines, will require comprehensive decision-aid education. Nanovaccines utilize nanoparticles and enable rapid and long-lasting immunity while offering site-specific delivery, increased bioavailability, and reduced adverse effects. 26 Thus, nanovaccines can be tailored to individual immune profiles, and be combined with other vaccines to target multiple pathogens. 26 These aforementioned advancements will require future decision-aids to include clear explanations to help vaccine-hesitant individuals understand the scientific rationale, address concerns, and reduce hesitancy by demonstrating the effectiveness of modern vaccination strategies/technology.
Finally, the growing role of artificial intelligence (AI) in health communication presents both opportunities and challenges for decision-aid development. Tools like ChatGPT demonstrate capacity to provide health information but have significant limitations: lack of medical citations, outdated training data, inability to account for individual patient factors, and absence of clinical oversight. 27 Currently, evidence-based approaches like decision-aids offer distinct advantages over unstructured AI tools, however, as AI advances, decision-aids must evolve and potentially integrate validated AI capabilities while maintaining rigorous evidence-based frameworks to remain relevant.
Limitations
While the results of this study are encouraging, several limitations must be considered. First, the sample size was relatively small, which limits the generalizability of the findings. Additionally, besides basic vaccination status, the study lacked a detailed pre-test measurement to establish the other factors that may have influenced participants’ thought processes related to vaccination, making it difficult to isolate the tool’s specific impact. The inability to verify participants’ actual vaccination status post-decision also presents a challenge, as it is unclear whether participants who expressed intent to vaccinate followed through on their decision. 27
Another limitation was the absence of a control group, which would have helped isolate the effects of the decision-aid from other confounding variables. The study was conducted in a clinic setting among individuals already engaged in the healthcare system, which may not be representative of the broader community population, particularly those who are not regularly in contact with healthcare providers.
Lastly, the questionnaires were not formally validated instruments, which may limit the reliability and generalizability of our measured outcomes. However, the questions were developed with input from clinical staff, informed by prior studies on COVID-19 vaccine hesitancy, and designed to align with the tool’s underlying behavioral framework.
Despite these limitations, the study still provides valuable insights into the potential of digital decision-aids in reducing vaccine hesitancy. The findings suggest that such tools can be effective in improving vaccine literacy, encouraging informed decision-making, and sparking important conversations between patients and healthcare providers. Future iterations of this tool, with larger and more diverse sample sizes and control environments, will help further validate its effectiveness and expand its applicability to other vaccination campaigns or preventative health measures.
Conclusion
This pilot study shows that a self-administered digital decision-aid can support vaccine-hesitant patients in making informed choices about COVID-19 vaccination. Most participants (97.5%) reported improved understanding of vaccine risks and benefits, and 22.2% of previously unvaccinated individuals chose to pursue vaccination after using the tool. The high engagement among a predominantly Black population in a low-uptake region highlights the potential of such interventions to reach underserved communities. While limited by sample size and design, these findings demonstrate the feasibility of scalable digital decision-support tools as complements to clinical counseling. Future studies should assess their long-term impact and adaptability to emerging vaccine technologies.
Supplemental material
Supplemental material -Self-administered digital vaccination decision-aids in primary care: A pilot study of COVID-19 vaccination in Atlanta, GA
Supplemental material for Self-administered digital vaccination decision-aids in primary care: A pilot study of COVID-19 vaccination in Atlanta, GA by Jera White, Tej Shah, Emily Wall, Shivani A. Patel, Kevin Wu, Yuxuan Shi, Yuxi Zhang, Ziwei Dong and Ambar Kulshreshtha in Digital Health.
Footnotes
Acknowledgements
The authors thank all participating clinicians and patients for their time and invaluable input. Additionally, we thank Dr. Jane Frawley from the Sydney University of Technology School of Public Health for her foundational work on this subject.
Ethical considerations
Approval was obtained from the Institutional Review Board of Emory University (STUDY00004904).
Consent to participate
Written informed consent was received before the participants viewed the tool or completed any associated study questionnaires.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by Emory University’s Health Innovation Grant Program, supported by the National Center for Advancing Translational Science of the National Institutes of Health under Award Number UL1TRoo2378. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
