The spread of misinformation is a pressing societal challenge. Prior work shows that shifting attention to accuracy increases the quality of people’s news-sharing decisions. However, researchers disagree on whether accuracy-prompt interventions work for U.S. Republicans/conservatives and whether partisanship moderates the effect. In this preregistered adversarial collaboration, we tested this question using a multiverse meta-analysis (k = 21; N = 27,828). In all 70 models, accuracy prompts improved sharing discernment among Republicans/conservatives. We observed significant partisan moderation for single-headline “evaluation” treatments (a critical test for one research team) such that the effect was stronger among Democrats than Republicans. However, this moderation was not consistently robust across different operationalizations of ideology/partisanship, exclusion criteria, or treatment type. Overall, we observed significant partisan moderation in 50% of specifications (all of which were considered critical for the other team). We discuss the conditions under which moderation is observed and offer interpretations.
AllenJ.ArecharA. A.PennycookG.RandD. G. (2021). Scaling up fact-checking using the wisdom of crowds. Science Advances, 7(36), Article eabf4393. https://doi.org/10.1126/sciadv.abf4393
2.
AllenJ.HowlandB.MobiusM.RothschildD.WattsD. J. (2020). Evaluating the fake news problem at the scale of the information ecosystem. Science Advances, 6(14), Article eaay3539. https://doi.org/10.1126/sciadv.aay3539
3.
AllenJ. N. L.WattsD. J.RandD. (2023). Quantifying the impact of misinformation and vaccine-skeptical content on Facebook. PsyArXiv. https://doi.org/10.31234/osf.io/nwsqa
4.
ArecharA. A.AllenJ.BerinskyA. J.ColeR.EpsteinZ.GarimellaK.GullyA.LuJ. G.RossR. M.StagnaroM. N.ZhangY.PennycookG.RandD. G. (2023). Understanding and combatting misinformation across 16 countries on six continents. Nature Human Behaviour, 7(9), 1502–1513.
5.
BhardwajV.MartelC.RandD. G. (2023). Examining accuracy-prompt efficacy in combination with using colored borders to differentiate news and social content online. Harvard Kennedy School Misinformation Review, 4(1). https://doi.org/10.37016/mr-2020-113
6.
BrauerM.CurtinJ. J. (2018). Linear mixed-effects models and the analysis of nonindependent data: A unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. Psychological Methods, 23(3), 389–411.
7.
BreznauN.RinkeE. M.WuttkeA.NguyenH. H. V.AdemM.AdriaansJ.Alvarez-BenjumeaA.AndersenH. K.AuerD.AzevedoF.BahnsenO.BalzerD.BauerG.BauerP. C.BaumannM.BauteS.BenoitV.BernauerJ.BerningC.. . . ŻółtakT. (2022). Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty. Proceedings of the National Academy of Sciences, USA, 119(44), Article e2203150119. https://doi.org/10.1073/pnas.2203150119
CapraroV.CeladinT. (2023). “I think this news is accurate:” Endorsing accuracy decreases the sharing of fake news and increases the sharing of real news. Personality and Social Psychology Bulletin, 49(12), 1635–1645. https://doi.org/10.1177/01461672221117691
10.
CeylanG.AndersonI. A.WoodW. (2023). Sharing of misinformation is habitual, not just lazy or biased. Proceedings of the National Academy of Sciences, USA, 120(4), Article e2216614120. https://doi.org/10.1073/pnas.2216614120
11.
ClarkC. J.TetlockP. E. (2023). Adversarial collaboration: The next science reform. In FrisbyC. L.ReddingR. E.O’DonohueW. T.LilienfeldS. O. (Eds.), Ideological and Political Bias in Psychology: Nature, Scope, and Solutions (pp. 905–927). Springer.
12.
Clemm von HohenbergB. (2023). Truth and bias, left and right: Testing ideological asymmetries with a realistic news supply. Public Opinion Quarterly, 87(2), 267–292. https://doi.org/10.1093/poq/nfad013
13.
EpsteinZ.BerinskyA. J.ColeR.GullyA.PennycookG.RandD. G. (2021). Developing an accuracy-prompt toolkit to reduce COVID-19 misinformation online. Harvard Kennedy School Misinformation Review, 2(3). https://doi.org/10.37016/mr-2020-71
14.
EpsteinZ.SirlinN.ArecharA.PennycookG.RandD. (2023). The social media context interferes with truth discernment. Science Advances, 9(9), Article eabo6169. https://doi.org/10.1126/sciadv.abo6169
15.
GarrettR. K.BondR. M. (2021). Conservatives’ susceptibility to political misperceptions. Science Advances, 7(23), Article eabf1234. https://doi.org/10.1126/sciadv.abf1234
16.
GavinL.McChesneyJ.TongA.SherlockJ.FosterL.TomsaS. (2022). Fighting the spread of COVID-19 misinformation in Kyrgyzstan, India, and the United States: How replicable are accuracy nudge interventions?Technology, Mind, and Behavior, 3(3). https://doi.org/10.1037/tmb0000086
17.
GelmanA. (2018, November15). You need 16 times the sample size to estimate an interaction than to estimate a main effect. Statistical Modeling, Causal Inference, and Social Science. https://statmodeling.stat.columbia.edu/2018/03/15/need16
18.
GrinbergN.JosephK.FriedlandL.Swire-ThompsonB.LazerD. (2019). Fake news on Twitter during the 2016 US presidential election. Science, 363(6425), 374–378.
19.
GuayB.PennycookG.RandD. (2022). Examining partisan asymmetries in fake news sharing and the efficacy of accuracy prompt interventions. PsyArXiv. https://psyarxiv.com/y762k
20.
GuessA.NaglerJ.TuckerJ. (2019). Less than you think: Prevalence and predictors of fake news dissemination on Facebook. Science Advances, 5(1), Article eaau4586. https://doi.org/10.1126/sciadv.aau4586
21.
GwiaździńskiP.GundersenA. B.PiksaM.KrysińskaI.KunstJ. R.NoworytaK.OlejniukA.MorzyM.RygulaR.WójtowiczT. (2023). Psychological interventions countering misinformation in social media: A scoping review. Frontiers in Psychiatry, 13, Article 2872. https://doi.org/10.3389/fpsyt.2022.974782
22.
ImhoffR.ZimmerF.KleinO.AntónioJ. H.BabinskaM.BangerterA.BilewiczM.BlanušaN.BovanK.BužarovskaR. (2022). Conspiracy mentality and political orientation across 26 countries. Nature Human Behaviour, 6, 392–403.
23.
JostJ. T.van der LindenS.PanagopoulosC.HardinC. D. (2018). Ideological asymmetries in conformity, desire for shared reality, and the spread of misinformation. Current Opinion in Psychology, 23, 77–83. https://doi.org/10.1016/j.copsyc.2018.01.003
24.
KozyrevaA.Lorenz-SpreenP.HerzogS.EckerU.LewandowskyS.HertwigR.BasolM.BerinskyA. J.BetschC.CookJ.FazioL. K.GeersM.GuessA. M.MaertensR.PanizzaF.PennycookG.RandD. G.RathjeS.ReiflerJ.. . . WinebergS. (2022). Toolbox of interventions against online misinformation and manipulation. PsyArXiv. https://psyarxiv.com/x8ejt
25.
LawsonM. A.KakkarH. (2022). Of pandemics, politics, and personality: The role of conscientiousness and political ideology in the sharing of fake news. Journal of Experimental Psychology: General, 151(5), 1154–1177. https://doi.org/10.1037/xge0001120
26.
LinH.RandD. G.PennycookG. (2023). Conscientiousness does not moderate the association between political ideology and susceptibility to fake news sharing. Journal of Experimental Psychology: General, 152(11), 3277–3284. https://doi.org/10.1037/xge0001467
27.
MellersB.HertwigR.KahnemanD. (2001). Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychological Science, 12(4), 269–275.
28.
Offer-WestortM.RosenzweigL. R.AtheyS. (2022). Battling the coronavirus infodemic among social media users in Africa. arXiv. https://doi.org/10.48550/arXiv.2212.13638
29.
Organisation for Economic Co-operation and Development. (2022). Misinformation and disinformation: An international effort using behavioural science to tackle the spread of misinformation (OECD Public Governance Policy Papers No. 21). OECD. https://doi.org/10.1787/b7709d4f-en
30.
PennycookG.BearA.CollinsE. T.RandD. G. (2020). The implied truth effect: Attaching warnings to a subset of fake news headlines increases perceived accuracy of headlines without warnings. Management Science, 66(11), 4944–4957.
31.
PennycookG.BinnendykJ.NewtonC.RandD. G. (2021). A practical guide to doing behavioral research on fake news and misinformation. Collabra: Psychology, 7(1), Article 25293. https://doi.org/10.1525/collabra.25293
32.
PennycookG.EpsteinZ.MoslehM.ArecharA. A.EcklesD.RandD. G. (2021). Shifting attention to accuracy can reduce misinformation online. Nature, 592(7855), 590–595. https://doi.org/10.1038/s41586-021-03344-2
33.
PennycookG.McPhetresJ.ZhangY.LuJ. G.RandD. G. (2020). Fighting COVID-19 misinformation on social media: Experimental evidence for a scalable accuracy-nudge intervention. Psychological Science, 31(7), 770–780.
34.
PennycookG.RandD. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188, 39–50.
PennycookG.RandD. G. (2022a). Accuracy prompts are a replicable and generalizable approach for reducing the spread of misinformation. Nature Communications, 13, Article 2333. https://doi.org/10.1038/s41467-022-30073-5
37.
PennycookG.RandD. G. (2022b). Nudging social media toward accuracy. The Annals of the American Academy of Political and Social Science, 700(1), 152–164.
38.
PereiraA.HarrisE.Van BavelJ. (2018). Identity concerns drive belief: The impact of partisan identity on the belief and dissemination of true and false news. PsyArXiv. https://psyarxiv.com/7vc5d
39.
PretusC.JaveedA.HughesD. R.HackenburgK.TsakirisM.VilarroyaO.Van BavelJ. J. (2022). The Misleading count: An identity-based intervention to mitigate the spread of partisan misinformation. PsyArXiv. https://psyarxiv.com/7j26y
40.
RasmussenJ.LindekildeL.PetersenM. B. (2022). Public health communication decreases false headline sharing by boosting self-efficacy. PsyArXiv. https://psyarxiv.com/8wdfp
41.
RathjeS.HeJ. K.RoozenbeekJ.Van BavelJ. J.van der LindenS. (2022). Social media behavior is associated with vaccine hesitancy. PNAS Nexus, 1(4), Article pgac207. https://doi.org/10.1093/pnasnexus/pgac207
42.
RathjeS.RoozenbeekJ.TrabergC. S.Van BavelJ. J.van der LindenS. (2022). Letter to the Editors of Psychological Science: Meta-analysis reveals that accuracy nudges have little to no effect for U.S. conservatives: Regarding Pennycook et al. (2020). Psychological Science. https://doi.org/10.25384/Sage.12594110.v2
43.
RathjeS.Van BavelJ. J.van der LindenS. (2023). Accuracy and social motivations shape judgements of (mis)information. Nature Human Behaviour, 7, 892–903. https://doi.org/10.1038/s41562-023-01540-w
44.
RoozenbeekJ.FreemanA. L.van der LindenS. (2021). How accurate are accuracy-nudge interventions? A preregistered direct replication of Pennycook et al. (2020). Psychological Science, 32(7), 1169–1178. https://doi.org/10.1177/09567976211024535
45.
SteegenS.TuerlinckxF.GelmanA.VanpaemelW. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702–712.
Van BavelJ. J.HarrisE. A.PärnametsP.RathjeS.DoellK. C.TuckerJ. A. (2021). Political psychology in the digital (mis) information age: A model of news belief and sharing. Social Issues and Policy Review, 15(1), 84–113.
48.
van der LindenS. (2022). Misinformation: Susceptibility, spread, and interventions to immunize the public. Nature Medicine, 28(3), 460–467.
49.
van der LindenS.PanagopoulosC.AzevedoF.JostJ. T. (2021). The paranoid style in American politics revisited: An ideological asymmetry in conspiratorial thinking. Political Psychology, 42(1), 23–51.
50.
van der LindenS.RoozenbeekJ.MaertensR.BasolM.KáchaO.RathjeS.TrabergC. S. (2021). How can psychological science help counter the spread of fake news?The Spanish Journal of Psychology, 24, Article e25. https://doi.org/10.1017/SJP.2021.23
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
