Chatbots are increasingly used to correct health misinformation. However, few studies have investigated whether and how certain design features could enhance their effectiveness. We developed four chatbots and conducted an experiment that examined whether chatbots with interactive and empathetic conversational cues could outperform a basic chatbot at correcting unvaccinated participants’ COVID-19 vaccination misperceptions and increasing their vaccination intentions. Perceived chatbot interactivity was associated with lower levels of misperception, which in turn were linked to greater vaccination intention. Perceived chatbot empathy did not reduce misperception, yet was directly and positively associated with vaccination intention. Implications of these findings are discussed.
AltayS.HacquinA.-S.ChevallierC.MercierH. (2021). Information delivered by a chatbot has a positive impact on COVID-19 vaccines attitudes and intentions. Journal of Experimental Psychology: Applied, 29(1), 52–62. https://doi.org/10.31234/osf.io/eb2gt
AraujoT. (2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051
6.
AslettK.SandersonZ.GodelW.PersilyN.NaglerJ.TuckerJ. A. (2024). Online searches to evaluate misinformation can increase its perceived veracity. Nature, 625(7995), 548–556. https://doi.org/10.1038/s41586-023-06883-y
7.
BodeL.VragaE. K. (2015). In related news, that was wrong: The correction of misinformation through related stories functionality in social media. Journal of Communication, 65(4), 619–638. https://doi.org/10.1111/jcom.12166
8.
BodeL.VragaE. K.TullyM. (2020). Do the right thing: Tone may not affect correction of misinformation on social media. Harvard Kennedy School Misinformation Review, 1, Article e26. https://doi.org/10.37016/mr-2020-026
9.
BorahP.SuY.XiaoX.Lai LeeD. K. (2022). Incidental news exposure and COVID-19 misperceptions: A moderated-mediation model. Computers in Human Behavior, 129, Article 107173. https://doi.org/10.1016/j.chb.2021.107173
10.
BozicJ.TazlO. A.WotawaF. (2019). Chatbot testing using AI planning. In 2019 IEEE international conference on artificial intelligence testing (AITEST) (pp. 37–44). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/AITest.2019.00-10
11.
BrehmJ. W. (1966). A theory of psychological reactance. Academic Press.
12.
BrehmS. S.BrehmJ. W. (2013). Psychological reactance: A theory of freedom and control. Academic Press.
13.
BridgmanA.MerkleyE.LoewenP. J.OwenT.RuthsD.TeichmannL.ZhilinO. (2020). The causes and consequences of COVID-19 misperceptions: Understanding the role of news and social media. Harvard Kennedy School Misinformation Review, 1(3), Article 028. https://doi.org/10.37016/mr-2020-028
ChanM.HuP. K. F.MakM. (2022). Mediation analysis and warranted inferences in media and communication research: Examining research design in communication journals from 1996 to 2017. Journalism & Mass Communication Quarterly, 99(2), 463–486. https://doi.org/10.1177/1077699020961519
18.
ChuaA. Y. K.BanerjeeS. (2015). How businesses draw attention on Facebook through incentives, vividness and interactivity. IAENG International Journal of Computer Science, 42(3), 275–281.
19.
ChungM.LimY. (2022). When Health Organization Answers the Question: Differential Effects of Dialogic Messages in Website and Twitter through Social Presence and Psychological Distance. Health Communication, 37(6), 685–695. https://doi.org/10.1080/10410236.2020.1864098
20.
CorritoreC. L.KracherB.WiedenbeckS. (2003). On-line trust: Concepts, evolving themes, a model. International Journal of Human-Computer Studies, 58(6), 737–758. https://doi.org/10.1016/S1071-5819(03)00041-7
21.
CramerD.JowettS. (2010). Perceived empathy, accurate empathy and relationship satisfaction in heterosexual couples. Journal of Social and Personal Relationships, 27(3), 327–349. https://doi.org/10.1177/0265407509348384
22.
DadaczynskiK.OkanO.MesserM.LeungA. Y. M.RosárioR.DarlingtonE.RathmannK. (2021). Digital health literacy and web-based information-seeking behaviors of university students in Germany during the COVID-19 pandemic: Cross-sectional survey study. Journal of Medical Internet Research, 23(1), Article e24097. https://doi.org/10.2196/24097
de GraafM. M. A.Ben AllouchS. (2013). Exploring influencing variables for the acceptance of social robots. Robotics and Autonomous Systems, 61(12), 1476–1486. https://doi.org/10.1016/j.robot.2013.07.007
DillardJ. P.ShenL. (2005). On the nature of reactance and its role in persuasive health communication. Communication Monographs, 72(2), 144–168. https://doi.org/10.1080/03637750500111815
27.
DurizzoK.AsieduE.Van der MerweA.Van NiekerkA.GüntherI. (2021). Managing the COVID-19 pandemic in poor urban neighborhoods: The case of Accra and Johannesburg. World Development, 137, Article 105175. https://doi.org/10.1016/j.worlddev.2020.105175
28.
EckerU. K. H.LewandowskyS.CookJ.SchmidP.FazioL. K.BrashierN.KendeouP.VragaE. K.AmazeenM. A. (2022). The psychological drivers of misinformation belief and its resistance to correction. Nature Reviews Psychology, 1(1), Article 1. https://doi.org/10.1038/s44159-021-00006-y
FangT.FuX. (2020). Development status and marketing strategy of smart speakers. In KantolaJ. I.NazirS.SalminenV. (Eds.), Advances in human factors, business management and leadership (pp. 553–562). Springer. https://doi.org/10.1007/978-3-030-50791-6_71
FengB. (2014). When should advice be given? Assessing the role of sequential placement of advice in supportive interactions in two cultures. Communication Research, 41(7), 913–934. https://doi.org/10.1177/0093650212456203
33.
GambinoA.FoxJ.RatanR. A. (2020). Building a stronger CASA: Extending the computers are social actors paradigm. Human-Machine Communication, 1, 71–85. https://doi.org/10.3316/INFORMIT.097034846749023
34.
GeriN.WinerA.ZaksB. (2017). Challenging the six-minute myth of online video lectures: Can interactivity expand the attention span of learners?Online Journal of Applied Knowledge Management, 5(1), 101–111. https://doi.org/10.36965/OJAKM.2017.5(1)101-111
35.
Gesser-EdelsburgA.DiamantA.HijaziR.MeschG. S. (2018). Correcting misinformation by health organizations during measles outbreaks: A controlled experiment. Plos ONE, 13(12), 1–23a9h.
36.
GoE.SundarS. S. (2019). Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior, 97, 304–316. https://doi.org/10.1016/j.chb.2019.01.020
GriolD.CarbóJ.MolinaJ. M. (2013). An automatic dialog simulation technique to develop and evaluate interactive conversational agents. Applied Artificial Intelligence, 27(9), 759–780. https://doi.org/10.1080/08839514.2013.835230
39.
GuoX.JinH.QiT. (2023). How does social presence influence public crisis information sharing intention? Situational pressure perspective. Frontiers in Public Health, 11, 1124876. https://doi.org/10.3389/fpubh.2023.1124876
HayesA. F.CaiL. (2007). Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation. Behavior Research Methods, 39(4), 709–722. https://doi.org/10.3758/BF03192961
43.
HoA.HancockJ.MinerA. S. (2018). Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. Journal of Communication, 68(4), 712–733. https://doi.org/10.1093/joc/jqy026
44.
HuX. (2023). Empowering Consumers in Interactive Marketing: Examining the Role of Perceived Control. In WangC. L. (Ed.), The Palgrave Handbook of Interactive Marketing (pp. 117–147). Springer International Publishing. https://doi.org/10.1007/978-3-031-14961-0_6
45.
JiangS. (2022). The roles of worry, social media information overload, and social media fatigue in hindering health fact-checking. Social Media + Society, 8(3), Article 113070. https://doi.org/10.1177/20563051221113070
46.
JohnsonV.ButterfussR.KimJ.OrcuttE.HarschR.KendeouP. (2022). The “Fauci effect”: Reducing COVID-19 misconceptions and vaccine hesitancy using an authentic multimodal intervention. Contemporary Educational Psychology, 70, Article 102084. https://doi.org/10.1016/j.cedpsych.2022.102084
47.
JohnstonR.JonesK.ManleyD. (2018). Confounding and collinearity in regression analysis: A cautionary tale and an alternative procedure, illustrated by studies of British voting behaviour. Quality & Quantity, 52(4), 1957–1976. https://doi.org/10.1007/s11135-017-0584-6
48.
KearsleyG.ShneidermanB. (1998). Engagement theory: A framework for technology-based teaching and learning. Educational Technology, 38(5), 20–23.
KimJ.NanX. (2016). Effects of consideration of future consequences and temporal framing on acceptance of the HPV vaccine among young adults. Health Communication, 31(9), 1089–1096. https://doi.org/10.1080/10410236.2015.1038774
51.
KimS. C.VragaE. K.CookJ. (2021). An eye tracking approach to understanding misinformation and correction strategies on social media: The mediating role of attention and credibility to reduce HPV vaccine misperceptions. Health Communication, 36(13), 1687–1696. https://doi.org/10.1080/10410236.2020.1787933
52.
KrauseN. M.FreilingI.BeetsB.BrossardD. (2020). Fact-checking as risk communication: The multi-layered risk of misinformation in times of COVID-19. Journal of Risk Research, 23(7–8), 1052–1059. https://doi.org/10.1080/13669877.2020.1756385
53.
KrepsS. E.GoldfarbJ. L.BrownsteinJ. S.KrinerD. L. (2021). The relationship between U.S. adults’ misconceptions about COVID-19 vaccines and vaccination preferences. Vaccines, 9(8), Article 8. https://doi.org/10.3390/vaccines9080901
54.
LeeS. K.SunJ.JangS.ConnellyS. (2022). Misinformation of COVID-19 vaccines and vaccine hesitancy. Scientific Reports, 12(1), Article 1. https://doi.org/10.1038/s41598-022-17430-6
LiuB.SundarS. S. (2018). Should machines express sympathy and empathy? Experiments with a health advice chatbot. Cyberpsychology, Behavior, and Social Networking, 21(10), 625–636. https://doi.org/10.1089/cyber.2018.0110
57.
LiuR.LiG. M. (2021). Hesitancy in the time of coronavirus: Temporal, spatial, and sociodemographic variations in COVID-19 vaccine hesitancy. SSM: Population Health, 15, Article 100896. https://doi.org/10.1016/j.ssmph.2021.100896
58.
MakranskyG.PetersenG. B. (2019). Investigating the process of learning with desktop virtual reality: A structural equation modeling approach. Computers & Education, 134, 15–30. https://doi.org/10.1016/j.compedu.2019.02.002
59.
MuldoonJ.ApostolidisP. (2023). “Neither work nor leisure”: Motivations of microworkers in the United Kingdom on three digital platforms. New Media & Society. Advance online publication. https://doi.org/10.1177/14614448231183942
60.
NassC.MoonY. (2000). Machines and mindlessness: Social responses to computers. Journal of Social Issues, 56(1), 81–103. https://doi.org/10.1111/0022-4537.00153
61.
NassC.SteuerJ.TauberE. R. (1994). Computers are social actors. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 72–78). Association for Computing Machinery. https://doi.org/10.1145/191666.191703
62.
NgaiC. S. B.SinghR. G.LuW.YaoL.KoonA. C. (2023). Exploring the relationship between trust-building strategies and public engagement on social media during the COVID-19 outbreak. Health Communication, 38(10), 2141–2157. https://doi.org/10.1080/10410236.2022.2055261
63.
NuzhathT.TasnimS.SanjwalR. K.TrishaN. F.RahmanM.MahmudS. M. F.ArmanA.ChakrabortyS.HossainM. M. (2020). COVID-19 vaccination hesitancy, misinformation and conspiracy theories on social media: A content analysis of Twitter data. SocArXiv. https://doi.org/10.31235/osf.io/vc9jb
64.
NyhanB.ReiflerJ. (2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2), 303–330. https://doi.org/10.1007/s11109-010-9112-2
65.
O’BrienH. L.TomsE. G. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology, 59(6), 938–955. https://doi.org/10.1002/asi.20801
66.
O’KeefeD. J. (2003). Message properties, mediating states, and manipulation checks: Claims, evidence, and data analysis in experimental persuasive message effects research. Communication Theory, 13(3), 251–274. https://doi.org/10.1111/j.1468-2885.2003.tb00292.x
67.
ParnasD. L. (2017). The real risks of artificial intelligence. Communications of the ACM, 60(10), 27–31. https://doi.org/10.1145/3132724
68.
PeerE.RothschildD. M.EverndenZ.GordonA.DamerE. (2021). Data quality of platforms and panels for online behavioral research (SSRN scholarly paper ID 3765448). Social Science Research Network. https://doi.org/10.2139/ssrn.3765448
69.
Pérez-FuentesM.delC.Gázquez LinaresJ. J.Molero JuradoM.delM.Simón MárquezM.delM.Martos MartínezÁ. (2020). The mediating role of cognitive and affective empathy in the relationship of mindfulness with engagement in nursing. BMC Public Health, 20(1), 16. https://doi.org/10.1186/s12889-019-8129-7
70.
PicklesK.CoppT.Meyerowitz-KatzG.DoddR. H.BonnerC.NickelB.SteffensM. S.SealeH.CvejicE.TabaM.ChauB.McCafferyK. J. (2022). COVID-19 vaccine misperceptions in a community sample of adults aged 18–49 years in Australia. International Journal of Environmental Research and Public Health, 19(11), Article 11. https://doi.org/10.3390/ijerph19116883
RadziwillN. M.BentonM. C. (2017). Evaluating quality of chatbots and intelligent conversational agents (arXiv:. 1704.04579). arXiv. https://doi.org/10.48550/arXiv.1704.04579
73.
RappA.CurtiL.BoldiA. (2021). The human side of human-chatbot interaction: A systematic literature review of ten years of research on text-based chatbots. International Journal of Human-Computer Studies, 151, Article 102630. https://doi.org/10.1016/j.ijhcs.2021.102630
74.
SamboC. F.HowardM.KopelmanM.WilliamsS.FotopoulouA. (2010). Knowing you care: Effects of perceived empathy and attachment style on pain perception. Pain, 151(3), 687–693. https://doi.org/10.1016/j.pain.2010.08.035
75.
SchodtK. B.QuirozS. I.WheelerB.HallD. L.SilvaY. N. (2021). Cyberbullying and mental health in adults: The moderating role of social media use and gender. Frontiers in Psychiatry, 12, Article 674298. https://www.frontiersin.org/articles/10.3389/fpsyt.2021.674298
76.
SchuetzS. W.SykesT. A.VenkateshV. (2021). Combating COVID-19 fake news on social media through fact checking: Antecedents and consequences. European Journal of Information Systems, 30(4), 376–388. https://doi.org/10.1080/0960085X.2021.1895682
77.
StangerN.KavussanuM.RingC. (2012). Put yourself in their boots: Effects of empathy on emotion and aggression. Journal of Sport and Exercise Psychology, 34(2), 208–222. https://doi.org/10.1123/jsep.34.2.208
78.
StolerJ.KlofstadC. A.EndersA. M.UscinskiJ. E. (2022). Sociopolitical and psychological correlates of COVID-19 vaccine hesitancy in the United States during summer 2021. Social Science & Medicine, 306, Article 115112. https://doi.org/10.1016/j.socscimed.2022.115112
79.
SundarS. S. (2008). The MAIN model: A heuristic approach to understanding technology effects on credibility. Digital Media, Youth, and Credibility, 2008, 73–100. https://doi.org/10.1162/dmal.9780262562324.073
80.
SundarS. S.BellurS.OhJ.JiaH.KimH.-S. (2016). Theoretical importance of contingency in human-computer interaction: Effects of message interactivity on user engagement. Communication Research, 43(5), 595–625. https://doi.org/10.1177/0093650214534962
81.
Swire-ThompsonB.CookJ.ButlerL. H.SandersonJ. A.LewandowskyS.EckerU. K. H. (2021). Correction format has a limited role when debunking misinformation. Cognitive Research: Principles and Implications, 6(1), Article 83. https://doi.org/10.1186/s41235-021-00346-6
82.
ToaderD.-C.BocaG.ToaderR.MăcelaruM.ToaderC.IghianD.RădulescuA. T. (2020). The effect of social presence and chatbot errors on trust. Sustainability, 12(1), Article 1. https://doi.org/10.3390/su12010256
83.
TrevorsG. J.MuisK. R.PekrunR.SinatraG. M.WinneP. H. (2016). Identity and epistemic emotions during knowledge revision: A potential account for the backfire effect. Discourse Processes, 53(5–6), 339–370. https://doi.org/10.1080/0163853X.2015.1136507
van der MeerT. G. L. A.JinY. (2020). Seeking Formula for Misinformation Treatment in Public Health Crises: The Effects of Corrective Information Type and Source. Health Communication, 35(5), 560–575.
86.
VragaE. K.BodeL. (2020). Defining misinformation and understanding its bounded nature: Using expertise and evidence for describing misinformation. Political Communication, 37(1), 136–144. https://doi.org/10.1080/10584609.2020.1716500
87.
WalterN.TukachinskyR. (2020). A meta-analytic examination of the continued influence of misinformation in the face of correction: How powerful is it, why does it happen, and how to stop it?Communication Research, 47(2), 155–177. https://doi.org/10.1177/0093650219854600
88.
WaytzA.CacioppoJ.EpleyN. (2010). Who sees human?: The stability and importance of individual differences in anthropomorphism. Perspectives on Psychological Science, 5(3), 219–232. https://doi.org/10.1177/1745691610369336
89.
WeeksR.CooperL.SanghaP.SedocJ.WhiteS.ToledoA.GretzS.LahavD.MartinN.MichelA.LeeJ. H.SlonimN.Bar-ZeevN. (2022). Chatbot-delivered COVID-19 vaccine communication message preferences of young adults and public health workers in urban American communities: Qualitative study. Journal of Medical Internet Research, 24(7), Article e38418. https://doi.org/10.2196/38418
WischnewskiM.KrämerN. (2020). I reason who I am? Identity salience manipulation to reduce motivated reasoning in news consumption. In International conference on social media and society (pp. 148–155). Association for Computing Machinery. https://doi.org/10.1145/3400806.3400824
94.
XuQ.SundarS. S. (2016). Interactivity and memory: Information processing of interactive versus non-interactive content. Computers in Human Behavior, 63, 620–629. https://doi.org/10.1016/j.chb.2016.05.046
ZhangA. L.LuH. (2022). No laughing matter: Exploring the effects of scientists’ humor use on Twitter and the moderating role of superiority. Science Communication, 44(4), 418–445. https://doi.org/10.1177/10755470221114352
97.
ZhaoX.ChenL.JinY.ZhangX. (2023). Comparing button-based chatbots with webpages for presenting fact-checking results: A case study of health information. Information Processing & Management, 60(2), Article 103203. https://doi.org/10.1016/j.ipm.2022.103203
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
