The Modality-Congruent Carryover Effect: How Difficulty in Identifying (Deep)Fake News Impacts Self-Confidence in Truth Discernment and Susceptibility to Subsequent Disinformation

Abstract

With the proliferation of (deep)fake news, individuals will increasingly find it difficult to discern whether content is created by human journalists or artificial intelligence. A two-wave online experiment investigated the carryover effects of news authentication during the initial (deep)fake news exposure on responses to subsequent (deep)fake news. Participants encountered celebrity (deep)fake news in text and video formats on social media. The results showed that the difficulty induced by news authentication lowered self-confidence in truth discernment, which, in turn, decreased the perceived credibility and engagement intentions toward subsequent unrelated fake news of the same modality.

Keywords

deepfake news authentication carryover effects categorization disfluency self-confidence modality congruence

Get full access to this article

View all access options for this article.

References

Ahmed

(2021). Who inadvertently shares deepfakes? Analyzing the role of political interest, cognitive ability, and social network size. Telematics and Informatics, 57, Article 101508. https://doi.org/10.1016/j.tele.2020.101508

Ahmed

(2022). Disinformation sharing thrives with fear of missing out among low cognitive news users: A cross-national examination of intentional sharing of deep fakes. Journal of Broadcasting & Electronic Media, 66(1), 89–109. https://doi.org/10.1080/08838151.2022.2034826

Ahmed

(2023). Navigating the maze: Deepfakes, cognitive ability, and social media news skepticism. New Media & Society, 25(5), 1108–1129. https://doi.org/10.1177/14614448211019198

Ahmed

S. W. T.

Bee

A. W. T.

(2023). Understanding the role of fear of missing out and deficient self-regulation in sharing of deepfakes on social media: Evidence from eight countries. Frontiers in Psychology, 14, Article 1127507. https://doi.org/10.3389/fpsyg.2023.1127507

Ahn

Bergan

D. E.

Carnahan

(2023). Estimating the impact of immediate versus delayed corrections on belief accuracy. Communication Monographs, 90, 372–392. https://doi.org/10.1080/03637751.2023.2202728

Ajzen

(1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T

Albarracín

(2020). Conspiracy beliefs: Knowledge, ego defense, and social integration in the processing of fake news. In Greifeneder

Jaffé

Newman

E. J.

Schwarz

(Eds.), The psychology of fake news: Accepting, sharing, and correcting misinformation (pp. 196–219). Routledge.

Appel

Prietzel

(2022). The detection of political deepfakes. Journal of Computer-Mediated Communication, 27(4), zmac008. https://doi.org/10.1093/jcmc/zmac008

Bassili

J. N.

Smith

M. C.

MacLeod

C. M.

(1989). Presentation modality and type of processing effects on priming in auditory and visual word-stem completion. Quarterly Journal of Experimental Psychology: Human Experimental Psychology, 41A, 439–453.

10.

Bray

S. D.

Johnson

S. D.

Kleinberg

(2023). Testing human ability to detect ‘deepfake’images of human faces. Journal of Cybersecurity, 9(1), tyad011. http://doi.org/10.1093/cybsec/tyad011

11.

Carr

E. W.

Hofree

Sheldon

Saygin

A. P.

Winkielman

(2017). Is that a human? Categorization (dis) fluency drives evaluations of agents ambiguous on human-likeness. Journal of Experimental Psychology: Human Perception and Performance, 43(4), 651–666. https://doi.org/10.1037/xhp0000304

12.

Choi

S. Y.

(2023). When you can’t trust your own eyes: Deepfakes and metacognitive experiences in manipulative AI-mediated communication [Doctoral dissertation, University of Southern California]. https://www.proquest.com/dissertations-theses/when-you-can-t-trust-your-own-eyes-deepfakes/docview/2903100862/se-2

13.

Collier

(2023, August 29). Deepfake scams have arrived: Fake videos spread on Facebook, TikTok and Youtube. NBC News. https://www.nbcnews.com/tech/tech-news/deepfake-scams-arrived-fake-videos-spread-facebook-tiktok-youtube-rcna101415

14.

Diakopoulos

Johnson

(2021). Anticipating and addressing the ethical implications of deepfakes in the context of elections. New Media & Society, 23(7), 2072–2098. https://doi.org/10.1177/1461444820925811

15.

Dunning

(2012). Confidence considered: Assessing the quality of decisions and performance. In Briñol

DeMarree

K. G.

(Eds.), Social metacognition (pp. 63–80). Psychology Press.

16.

Faul

Erdfelder

Buchner

Lang

A. G.

(2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149

17.

Fransen

M. L.

Fennis

B. M.

Pruyn

A. T. H.

(2010). Matching communication modalities: The effects of modality congruence and processing style on brand evaluation and brand choice. Communication Research, 37(4), 576–598. https://doi.org/10.1177/0093650210368251

18.

García-Peñalvo

Vázquez-Ingelmo

(2023). What do we mean by GenAI? A systematic mapping of the evolution, trends, and techniques involved in Generative AI. International Journal of Interactive Multimedia and Artificial Intelligence, 8, 7–16. http://doi.org/10.9781/ijimai.2023.07.006

19.

Gino

Brooks

A. W.

Schweitzer

M. E.

(2012). Anxiety, advice, and the ability to discern: Feeling anxious motivates individuals to seek and use advice. Journal of Personality and Social Psychology, 102(3), 497–512. https://doi.org/10.1037/a0026413

20.

Goodfellow

I. J.

Pouget-Abadie

Mirza

Warde-Farley

Ozair

Courville

Bengio

(2014). Generative adversarial networks. Communications of the ACM, 63, 139–144.

21.

Guess

Nagler

Tucker

(2019). Less than you think: Prevalence and predictors of fake news dissemination on Facebook. Science Advances, 5(1), eaau4586. https://doi.org/10.1126/sciadv.aau4586

22.

Halberstadt

Winkielman

(2014). Easy on the eyes, or hard to categorize: Classification difficulty decreases the appeal of facial blends. Journal of Experimental Social Psychology, 50, 175–183. https://doi.org/10.1016/j.jesp.2013.08.004

23.

Hameleers

Powell

T. E.

Van Der Meer

T. G.

Bos

(2020). A picture paints a thousand lies? The effects and mechanisms of multimodal disinformation and rebuttals disseminated via social media. Political Communication, 37(2), 281–301. https://doi.org/10.1080/10584609.2019.1674979

24.

Hayes

A. F

. (2022). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford Press.

25.

Henderson

E. L.

Simons

D. J.

Barr

D. J.

(2021). The trajectory of truth: A longitudinal study of the illusory truth effect. Journal of Cognition, 4(1), 1–23. https://doi.org/10.5334/joc.161

26.

Hwang

Ryu

J. Y.

Jeong

S. H.

(2021). Effects of disinformation using deepfake: The protective effect of media literacy education. Cyberpsychology, Behavior, and Social Networking, 24(3), 188–193. https://doi.org/10.1089/cyber.2020.0174

27.

Jackson

O’keefe

D. J.

Jacobs

Brashers

D. E.

(1989). Messages as replications: Toward a message-centered design strategy. Communications Monographs, 56(4), 364–384. https://doi.org/10.1080/03637758909390270

28.

Jones-Jang

S. M.

Kim

D. H.

Kenski

(2021). Perceptions of mis-or disinformation exposure predict political cynicism: Evidence from a two-wave survey during the 2018 US midterm elections. New Media & Society, 23(10), 3105–3125. https://doi.org/10.1177/1461444820943878

29.

Kang

Valadez

(2025). Deepfakes’ cognitive, emotional, and behavioral impact: A systematic review and meta-analysis of individual responses. Journalism & Mass Communication Quarterly, 102(4), 958–992. https://doi.org/10.1177/10776990251357294

30.

Kelley

C. M.

Jacoby

L. L.

Hollingshead

(1989). Direct versus indirect tests of memory for source: Judgments of modality. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(6), 1101–1108. https://doi.org/10.1037/0278-7393.15.6.1101

31.

Kothe

E. J.

Ling

(2019, September 6). Retention of participants recruited to a multi-year longitudinal study via Prolific. https://doi.org/10.31234/osf.io/5yv2u

32.

Lecheler

de Vreese

C. H.

(2011). Getting real: The duration of framing effects. Journal of Communication, 61(5), 959–983. https://doi.org/10.1111/j.1460-2466.2011.01580.x

33.

Lee

E. J.

(2020). Authenticity model of (mass-oriented) computer-mediated communication: Conceptual explorations and testable propositions. Journal of Computer-Mediated Communication, 25(1), 60–73. https://doi.org/10.1093/jcmc/zmz025

34.

Lee

Shin

S. Y.

(2022). Something that they never said: Multimodal disinformation and source vividness in understanding the power of AI-enabled deepfake news. Media Psychology, 25(4), 531–546. https://doi.org/10.1080/15213269.2021.2007489

35.

Lee

Hameleers

Shin

S. Y.

(2023). The emotional effects of multimodal disinformation: How multimodality, issue relevance, and anxiety affect misperceptions about the flu vaccine. New Media & Society, 26, 6838–6860. https://doi.org/10.1177/14614448231153959

36.

Lewandowsky

Ecker

U. K.

Seifert

C. M.

Schwarz

Cook

(2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 12(3), 106–131. https://doi.org/10.1177/1529100612451018

37.

Yuan

(2024). “I know it’s a deepfake”: The role of AI disclaimers and comprehension in the processing of deepfake parodies. Journal of Communication, 74(5), 359–373. https://doi.org/10.1093/joc/jqae022

38.

Lyons

B. A.

Montgomery

J. M.

Guess

A. M.

Nyhan

Reifler

(2021). Overconfidence in news judgments is associated with false news susceptibility. Proceedings of the National Academy of Sciences, 118(23), e2019527118. https://doi.org/10.1073/pnas.2019527118

39.

Madigan

McDowd

Murphy

(1991). Facilitating word-fragment completion with hidden primes. Bulletin of the Psychonomic Society, 29(3), 189–191. https://doi.org/10.3758/BF03342674

40.

Maertens

Roozenbeek

Basol

van der Linden

(2021). Long-term effectiveness of inoculation against misinformation: Three longitudinal experiments. Journal of Experimental Psychology: Applied, 27(1), 1–16. https://doi.org/10.1037/xap0000315

41.

Manning

(1965, August). Hemingway in Cuba. The Atlantic. https://www.theatlantic.com/magazine/archive/1965/08/hemingway-in-cuba/399059/

42.

Mayo

(2019). The skeptical (ungullible) mindset. In Forgas

J. P.

Baumeister

R. F.

(Eds). The social psychology of gullibility (pp. 140–158). Routledge.

43.

Metz

O’Sullivan

(2019, June 11). A deepfake video of Mark Zuckerberg presents a new challenge for Facebook. CNN Business. https://www.cnn.com/2019/06/11/tech/zuckerberg-deepfake

44.

Mirsky

Lee

(2021). The creation and detection of deepfakes: A survey. ACM Computing Surveys (CSUR), 54(1), 1–41. https://doi.org/10.1145/3425780

45.

Murphy

Loftus

Grady

R. H.

Levine

L. J.

Greene

C. M.

(2020). Fool me twice: How effective is debriefing in false memory studies?. Memory, 28(7), 938–949. https://doi.org/10.1080/09658211.2020.1803917

46.

Nagler

R. H.

Vogel

R. I.

Gollust

S. E.

Yzer

M. C.

Rothman

A. J.

(2022). Effects of prior exposure to conflicting health information on responses to subsequent unrelated health messages: Results from a population-based longitudinal experiment. Annals of Behavioral Medicine, 56(5), 498–511. https://doi.org/10.1093/abm/kaab069

47.

Nagler

R. H.

Yzer

M. C.

Rothman

A. J.

(2019). Effects of media exposure to conflicting information about mammography: Results from a population-based survey experiment. Annals of Behavioral Medicine, 53(10), 896–908. https://doi.org/10.1093/abm/kay098

48.

Owen

H. E.

Halberstadt

Carr

E. W.

Winkielman

(2016). Johnny Depp, reconsidered: How category-relative processing fluency determines the appeal of gender ambiguity. PloS One, 11(2), e0146328. https://doi.org/10.1371/journal.pone.0146328

49.

Palan

Schitter

(2018). Prolific. ac—A subject pool for online experiments. Journal of Behavioral and Experimental Finance, 17, 22–27. https://doi.org/10.1016/j.jbef.2017.12.004

50.

Pennycook

Rand

D. G.

(2021). The psychology of fake news. Trends in Cognitive Sciences, 25(5), 388–402. https://doi.org/10.1016/j.tics.2021.02.007

51.

Pérez-Rosas

Kleinberg

Lefevre

Mihalcea

(2017). Automatic detection of fake news. https://web.eecs.umich.edu/~mihalcea/papers/perezrosas.coling18.pdf

52.

Pocol

Istead

Siu

Mokhtari

Kodeiri

(2024). Seeing is no longer believing: A survey on the state of deepfakes, ai-generated humans, and other nonveridical media. In Sheng

Kim

Magnenat-Thalmann

Thalmann

(Eds.), Computer graphics international conference (pp. 427–440). Springer.

53.

Rapp

D. N.

Salovich

N. A.

(2018). Can’t we just disregard fake news? The consequences of exposure to inaccurate information. Policy Insights from the Behavioral and Brain Sciences, 5(2), 232–239. https://doi.org/10.1177/2372732218785193

54.

Roediger

H. L.

McDermott (1993). Implicit memory in normal human subjects. In Spinnler

Boller

(Eds.), Handbook of neuropsychology (pp. 63–131). Elsevier.

55.

Sanchez

Dunning

(2020). Decision fluency and overconfidence among beginners. Decision, 7(3), 225–237. https://doi.org/10.1037/dec0000122

56.

Schwarz

Jalbert

(2020). When (fake) news feels true: Intuitions of truth and the acceptance and correction of misinformation. In Greifeneder

Jaffé

Newman

E. J.

Schwarz

(Eds.), The psychology of fake news: Accepting, sharing, and correcting misinformation (pp. 196–219). Routledge.

57.

Schwarz

Jalbert

Noah

Zhang

(2021). Metacognitive experiences as information: Processing fluency in consumer judgment and decision making. Consumer Psychology Review, 4, 4–25. https://doi.org/10.1002/arcp.1067

58.

Shin

(2025). Seeing through the fake: How users detect and interpret deepfakes. Information, Communication & Society. Advance online publication. https://doi.org/10.1080/1369118X.2025.2518260

59.

Shin

S. Y.

Lee

(2022). The effect of deepfake video on news credibility and corrective influence of cost-based knowledge about deepfakes. Digital Journalism, 10(3), 412–432. https://doi.org/10.1080/21670811.2022.2026797

60.

Sivathanu

Pillai

Metri

(2023). Customers’ online shopping intention by watching AI-based deepfake advertisements. International Journal of Retail & Distribution Management, 51(1), 124–145. https://doi.org/10.1108/IJRDM-12-2021-0583

61.

Snijders

Conijn

de Fouw

van Berlo

(2023). Humans and algorithms detecting fake news: Effects of individual and contextual confidence on trust in algorithmic advice. International Journal of Human–Computer Interaction, 39(7), 1483–1494. https://doi.org/10.1080/10447318.2022.2097601

62.

Somers

(2020, July 21). Deepfakes, explained. MIT Sloan School of Management. https://mitsloan.mit.edu/ideas-made-to-matter/deepfakes-explained

63.

Sundar

S. S.

(2008). The MAIN model: A heuristic approach to understanding technology effects on credibility. In Metzger

M. J.

Flanagin

A. J.

(Eds.), Digital media, youth, and credibility (pp. 72–100). The MIT Press.

64.

Sundar

S. S.

Molina

M. D.

Cho

(2021). Seeing is believing: Is video modality more powerful in spreading fake news via online messaging apps?. Journal of Computer-Mediated Communication, 26(6), 301–319. https://doi.org/10.1093/jcmc/zmab010

65.

Tandoc

E. C.

Jr. (2019). The facts of fake news: A research review. Sociology Compass, 13(9), e12724. https://doi.org/10.1111/soc4.12724

66.

Tandoc

E. C.

Jr. Ling

Westlund

Duffy

Goh

Zheng Wei

(2018). Audiences’ acts of authentication in the age of fake news: A conceptual framework. New Media & Society, 20(8), 2745–2763. https://doi.org/10.1177/1461444817731756

67.

The Yomiuri Shimbun. (2023, November 4). Fake video of Japan Prime Minister Kishida triggers fears of AI being used to spread misinformation. The Japan News. https://japannews.yomiuri.co.jp/politics/politics-government/20231104-147695/

68.

Tolosana

Vera-Rodriguez

Fierrez

Morales

Ortega-Garcia

(2020). Deepfakes and beyond: A survey of face manipulation and fake detection. Information Fusion, 64, 131–148. https://doi.org/10.1016/j.inffus.2020.06.014

69.

Vaccari

Chadwick

(2020). Deepfakes and disinformation: Exploring the impact of synthetic political video on deception, uncertainty, and trust in news. Social Media + Society. Advance online publication. https://doi.org/10.1177/2056305120903408

70.

Valkenburg

P. M.

Peter

Walther

J. B.

(2016). Media effects: Theory and research. Annual Review of Psychology, 67, 315–338. https://doi.org/10.1146/annurev-psych-122414-033608

71.

Vincent

(2018, April 17). Watch Jordan Peele use AI to make Barack Obama deliver a PSA about fake news. The Verge. https://www.theverge.com/tldr/2018/4/17/17247334/ai-fake-news-video-barack-obama-jordan-peele-buzzfeed

72.

Vogler

Rauchfleisch

de Seta

(2025). The short-term impact of an on-site literacy intervention on discerning deepfake videos based on visual features. Journalism & Mass Communication Quarterly, 102(4), 1135–1156. https://doi.org/10.1177/10776990251373088

73.

Vosoughi

Roy

Aral

(2018). The spread of true and false news online. Science, 359(6380), 1146–1151. http://doi.org/10.1126/science.aap959

74.

Wardle

(2019). Understanding information disorder. First Draft. https://firstdraftnews.org/wp-content/uploads/2019/10/Information_Disorder_Digital_AW.pdf?x32994

75.

Weathington

B. L.

Cunningham

C. J.

Pittenger

D. J.

(2017). Research methods for the behavioral and social sciences. John Wiley & Sons.

76.

Weikmann

Lecheler

(2023). Visual disinformation in a digital age: A literature synthesis and research agenda. New Media & Society, 25(12), 3696–3713. https://doi.org/10.1177/14614448221141648

77.

Williams

E. F.

Duke

K. E.

Dunning

(2020). Consistency just feels right: Procedural fluency increases confidence in performance. Journal of Experimental Psychology: General, 149(12), 2395–2405. https://doi.org/10.1037/xge0000779

78.

Winkielman

Olszanowski

Gola

(2015). Faces in-between: Evaluations reflect the interplay of facial features and task-dependent fluency. Emotion, 15(2), 232–242. https://doi.org/10.1037/emo0000036

79.

Winkielman

Schwarz

Fazendeiro

Reber

(2003). The hedonic marking of processing fluency: Implications for evaluative judgment. In Musch

Klauer

K. C.

(Eds.), The psychology of evaluation: Affective processes in cognition and emotion (pp. 189–217). Erlbaum.

80.

Yang

Davis

Hindman

(2023). Visual misinformation on Facebook. Journal of Communication, 73(4), 316–328. https://doi.org/10.1093/joc/jqac051

81.

YouGov. (2023). Misinformation susceptibility. https://d3nkl3psvxxpe9.cloudfront.net/documents/Misinformation_Susceptibility_poll_results.pdf

82.

Zajonc

R. B.

(1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, 9(2), 1–27. https://doi.org/10.1037/h0025848