Restricted accessResearch articleFirst published online 2024-12
User agency–based versus machine agency–based misinformation interventions: The effects of commenting and AI fact-checking labeling on attitudes toward the COVID-19 vaccination
This study aimed to examine the effects of commenting on a Facebook misinformation post by comparing a user agency–based intervention and machine agency–based intervention in the form of artificial intelligence (AI) fact-checking labeling on attitudes toward the COVID-19 vaccination. We found that both interventions were effective at promoting positive attitudes toward vaccination compared to the misinformation-only condition. However, the intervention effects manifested differently depending on participants’ residential locations, such that the commenting intervention emerged as a promising tool for suburban participants. The effectiveness of the AI fact-checking labeling intervention was pronounced for urban populations. Neither of the fact-checking interventions showed salient effects with the rural population. These findings suggest that although user agency- and machine agency–based interventions might have potential against misinformation, these interventions should be developed in a more sophisticated way to address the unequal effects among populations in different geographic locations.
AbhyankarPO’connorDBLawtonR (2008) The role of message framing in promoting MMR vaccination: evidence of a loss-frame advantage. Psychology, Health and Medicine13(1): 1–16.
3.
AmramOAmiriSLutzRB, et al. (2020) Development of a vulnerability index for diagnosis with the novel coronavirus, COVID-19, in Washington State, USA. Health & Place64: 102377.
BemDJ (1967) Self-perception: an alternative interpretation of cognitive dissonance phenomena. Psychological Review74(3): 183–200.
6.
BodeLVragaEK (2015) In related news, that was wrong: the correction of misinformation through related stories functionality in social media. Journal of Communication65(4): 619–638.
7.
BroniatowskiDAJamisonAMQiS, et al. (2018) Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. American Journal of Public Health108(10): 1378–1384.
8.
CarlsonM (2020) Fake news as an informational moral panic: the symbolic deviancy of social media during the 2016 US presidential election. Information, Communication & Society23(3): 374–388.
9.
CaslerKBickelLHackettE (2013) Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior29(6): 2156–2160.
10.
ChenXChenH (2020) Differences in preventive behaviors of COVID-19 between urban and rural residents: lessons learned from a cross-sectional study in China. International Journal of Environmental Research and Public Health17(12): 4437.
11.
ChenXOromHHayJL, et al. (2019) Differences in rural and urban health information access and use. The Journal of Rural Health35(3): 405–417.
12.
ChouWYSBudenzA (2020) Considering emotion in COVID-19 vaccine communication: addressing vaccine hesitancy and fostering vaccine confidence. Health Communication35(14): 1718–1722.
13.
CorreaTPavezI (2016) Digital inclusion in rural areas: a qualitative exploration of challenges faced by people from isolated communities. Journal of Computer-Mediated Communication21(3): 247–263.
14.
CoustasseAKimbleCMaxikK (2021) COVID-19 and vaccine hesitancy: a challenge the United States must overcome. The Journal of Ambulatory Care Management44(1): 71–75.
15.
CrozierJChristensenNLiP, et al. (2021) Rural, underserved, and minority populations’ perceptions of COVID-19 information, testing, and vaccination: report from a southern state. Population Health Management. Epub ahead of print 8 October. DOI: 10.1089/pop.2021.0216.
16.
DawesRMFaustDMeehlPE (1989) Clinical versus actuarial judgment. Science243(4899): 1668–1674.
GarrettRKWeeksBENeoRL (2016) Driving a wedge between evidence and beliefs: how online ideological news exposure promotes political misperceptions. Journal of Computer-Mediated Communication21(5): 331–348.
19.
Gil De ZúñigaHMolyneuxLZhengP (2014) Social media, political expression, and political participation: panel analysis of lagged and concurrent relationships. Journal of Communication64(4): 612–634.
HalpernDGibbsJ (2013) Social media as a catalyst for online deliberation? Exploring the affordances of Facebook and YouTube for political expression. Computers in Human Behavior29(3): 1159–1168.
22.
Henning-SmithC (2020) The unique impact of COVID-19 on older adults in rural areas. Journal of Aging & Social Policy 32(4–5): 396–402.
23.
HeverinTZachL (2012) Use of microblogging for collective sense-making during violent crises: a study of three campus shootings. Journal of the American Society for Information Science and Technology63(1): 34–47.
24.
HorneBDNevoDAdaliS, et al. (2020) Tailoring heuristics and timing AI interventions for supporting news veracity assessments. Computers in Human Behavior Reports2: 100043.
25.
JacksonDNTrivediNBaurC (2021) Re-prioritizing digital health and health literacy in Healthy People 2030 to affect health equity. Health Communication36(10): 1155–1162.
26.
JenningsJStroudNJ (2021) Asymmetric adjustment: partisanship and correcting misinformation on Facebook. New Media & Society. Epub ahead of print 28 June. DOI: 10.1177/14614448211021720.
27.
LeeJ (2022) The effect of web add-on correction and narrative correction on belief in misinformation depending on motivations for using social media. Behaviour & Information Technology41(3): 629–643.
28.
LeeJKimJWChockTM (2020) From risk butterflies to citizens engaged in risk prevention in the Zika virus crisis: focusing on personal, societal and global risk perceptions. Journal of Health Communication25(9): 671–680.
29.
LiuB (2021) In AI we trust? Effects of agency locus and transparency on uncertainty reduction in human–AI interaction. Journal of Computer-Mediated Communication26(6): 384–402.
30.
MenaP (2020) Cleaning up social media: the effect of warning labels on likelihood of sharing false news on Facebook. Policy & Internet12(2): 165–183.
31.
MillerTHowePSonenbergL (2017) Explainable AI: beware of inmates running the asylum. IJCAI 2017 Workshop on Explainable Artificial Intelligence (XAI). Available at: https://arxiv.org/abs/1712.00547
32.
MuellerJTMcConnellKBurowPB, et al. (2021) Impacts of the COVID-19 pandemic on rural America. Proceedings of the National Academy of Sciences118(1): 2019378118.
33.
NakovPCorneyDHasanainM, et al. (2021) Automated fact-checking for assisting human fact-checkers. arXiv preprint arXiv:2103.07769.
34.
NekmatE (2012) Message expression effects in online social communication. Journal of Broadcasting & Electronic Media56(2): 203–224.
35.
Oeldorf-HirschA (2018) The role of engagement in learning from active and incidental news exposure on social media. Mass Communication and Society21(2): 225–247.
36.
Oeldorf-HirschASchmierbachMAppelmanA, et al. (2020) The ineffectiveness of fact-checking labels on news memes and articles. Mass Communication and Society23(5): 682–704.
37.
OsatuyiBHughesJ (2018) A tale of two internet news platforms-real vs. fake: an elaboration likelihood model perspective. In: Proceedings of the 51st Hawaii international conference on system sciences, 3January, pp. 3986–3994. Available at: https://scholarspace.manoa.hawaii.edu/items/6f416a4e-349c-40f3-8c91-0d5631c717e6
38.
PaulRArifAAAdeyemiO, et al. (2020) Progression of COVID-19 from urban to rural areas in the United States: a spatiotemporal analysis of prevalence rates. The Journal of Rural Health36(4): 591–601.
39.
PennycookGMcPhetresJZhangY, et al. (2020) Fighting COVID-19 misinformation on social media: experimental evidence for a scalable accuracy-nudge intervention. Psychological Science31(7): 770–780.
40.
PingreeRJ (2007) How messages affect their senders: a more general model of message effects and implications for deliberation. Communication Theory17(4): 439–461.
41.
RaiA (2020) Explainable AI: from black box to glass box. Journal of the Academy of Marketing Science48(1): 137–141.
42.
SaundersECBudneyAJCavazos-RehgP, et al. (2021) Comparing the feasibility of four web-based recruitment strategies to evaluate the treatment preferences of rural and urban adults who misuse non-prescribed opioids. Preventive Medicine152: 106783.
43.
ShahDV (2016) Conversation is the soul of democracy: expression effects, communication mediation, and digital media. Communication and the Public1(1): 12–18.
44.
SundarS (2008) The MAIN model: a heuristic approach to understanding technology effects on credibility. In: MetzgerMJFlanaginJ (eds) Digital Media, Youth, and Credibility. Cambridge: MIT Press, pp. 72–100.
45.
SundarSSKimJ (2019) Machine heuristic: when we trust computers more than humans with our personal information. In: Proceedings of the 2019 CHI conference on human factors in computing systems, Glasgow, 4–9 May, pp. 1–9. New York: ACM.
46.
ThorsonE (2016) Belief echoes: the persistent effects of corrected misinformation. Political Communication33(3): 460–480.
VragaETullyMBodeL (2022) Assessing the relative merits of news literacy and corrections in responding to misinformation on Twitter. New Media & Society24(10): 2354–2371.
49.
WalterNCohenJHolbertRL, et al. (2020) Fact-checking: a meta-analysis of what works and for whom. Political Communication37(3): 350–375.
50.
YooSWKimJWGilDeZúñigaH (2017) Cognitive benefits for senders: antecedents and effects of political expression on social media. Journalism & Mass Communication Quarterly94(1): 17–37.
51.
Yun LeeHWangKDeavoursDF, et al. (2022) Information and communication technology use for seeking health information in Vietnamese living in rural areas. International Journal of Health Promotion and Education. Epub ahead of print 26 April. DOI: 10.1080/14635240.2022.2059694.
52.
ZhangJFeatherstoneJDCalabreseC, et al. (2021a) Effects of fact-checking social media vaccine misinformation on attitudes toward vaccines. Preventive Medicine145: 106408.
53.
ZhangJZhuLLiS, et al. (2021b) Rural–urban disparities in knowledge, behaviors, and mental health during COVID-19 pandemic: a community-based cross-sectional survey. Medicine100(13): e25207.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
