The spread of misinformation has historically been attributed to emotions, thinking styles, biases, and predispositions, but only a few studies have explored the conditions influencing its prevalence. The Theory of Informative Fictions (TIF) addresses this gap by presenting propositions that predict the conditions under which misinformation is tolerated and promoted. Building on the literature on TIF and deep learning, we uncover how property messages and character messages differ in veracity and explore the relationship between visual misinformation and user engagement. By constructing a short video dataset Tikcron (N = 42,201) and a multimodal video analysis framework KILL, we classify TikTok videos as misinformation or not, and property messages or character messages. Our results indicate that character messages are more likely to convey misinformation than property messages, and character messages with misinformation are more likely to get tolerated and promoted by social media users than property messages with misinformation. This study extends the current methodological advancement of image-as-data to misinformation videos and proposes a multimodal video analysis framework to develop communication-centered theories. The broader practical implications of this study on the detection, countering, and governance of visual misinformation are also discussed.
AshleyC.TutenT. (2015). Creative strategies in social media marketing: An exploratory study of branded social content and consumer engagement. Psychology and Marketing, 32(1), 15–27. https://doi.org/10.1002/mar.20761
BrennenJ. S.SimonF. M.NielsenR. K. (2021). Beyond (mis) representation: Visuals in COVID-19 misinformation. The International Journal of Press/Politics, 26(1), 277–299. https://doi.org/10.1177/1940161220964780
4.
BurgoonJ. K.HaleJ. L. (1984). The fundamental topoi of relational communication. Communication Monographs, 51(3), 194–214. https://doi.org/10.1080/03637758409390195
5.
ChandlerM.FritzA. S.HalaS. (1989). Small-scale deceit: Deception as a marker of two-three-and four-year-olds’ early theories of mind. Child Development, 60(6), 1263–1277. https://doi.org/10.2307/1130919
6.
ChengZ.LiY. (2023). Like, comment, and share on TikTok: Exploring the effect of sentiment and second-person view on the user engagement with TikTok news videos. Social Science Computer Review, 1(1), 1–23. https://doi.org/10.1177/08944393231178603
CinelliM.QuattrociocchiW.GaleazziA.ValensiseC. M.BrugnoliE.SchmidtA. L.ScalaA.ZolloF. (2020). The COVID-19 social media infodemic. Scientific Reports, 10(1), 1–10. https://doi.org/10.1038/s41598-020-73510-5
9.
DevlinJ.ChangM. W.LeeK.ToutanovaK. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: Human language technologies, volume 1 (long and short papers) (pp. 4171–4186). https://doi.org/10.18653/v1/N19-1423
10.
EllisonN.HeinoR.GibbsJ. (2006). Managing impressions online: Self-presentation processes in the online dating environment. Journal of Computer-Mediated Communication, 11(2), 415–441. https://doi.org/10.1111/j.1083-6101.2006.00020.x
11.
FisherW. R. (1984). Narration as a human communication paradigm: The case of public moral argument. Communication Monographs, 51(1), 1–22. https://doi.org/10.1080/03637758409390180
12.
FlynnD. J.NyhanB.ReiflerJ. (2017). The nature and origins of misperceptions: Understanding false and unsupported beliefs about politics: Nature and origins of misperceptions. Political Psychology, 38(S1), 127–150. https://doi.org/10.1111/pops.12394
13.
GerodimosR. (2019). The interdisciplinary roots and digital branches of visual political communication research. In VenetiA.JacksonD.LillekerD. (Eds.), Visual political communication (pp. 53–73). Springer.
14.
GrinbergN.JosephK.FriedlandL.Swire-ThompsonB.LazerD. (2019). Fake news on Twitter during the 2016 U.S. presidential election. Science, 363(6425), 374–378. https://doi.org/10.1126/science.aau2706
15.
GuinaudeauB.MungerK.VottaF. (2022). Fifteen Seconds of fame: Tiktok and the supply side of Social Video. Computational Communication Research, 4(2), 463–485. https://doi.org/10.5117/ccr2022.2.004.guin
16.
HersheyS.ChaudhuriS.EllisD. P.GemmekeJ. F.JansenA.MooreR. C.WilsonK. (2017). CNN architectures for large-scale audio classification. In 2017 IEEE international conference on acoustics, speech and signal processing (icassp) (pp. 131–135). IEEE. https://doi.org/10.1109/icassp.2017.7952132
HolidayS.HayesJ. L.ParkH.LyuY.ZhouY. (2023). A multimodal emotion perspective on social media influencer marketing: The effectiveness of influencer emotions, network size, and branding on consumer brand engagement using facial expression and linguistic analysis. Journal of Interactive Marketing, 58(4), 1–26. https://doi.org/10.1177/10949968231171104
19.
JürgensP.StarkB. (2022). Mapping exposure diversity: The divergent effects of algorithmic curation on news consumption. Journal of Communication, 72(3), 322–344. https://doi.org/10.1093/joc/jqac009
20.
KotsiantisS. B.ZaharakisI.PintelasP. (2007). Supervised machine learning: A review of classification techniques. Emerging artificial intelligence applications in computer engineering, 160(1), 3–24. https://doi.org/10.1007/s10462-007-9052-3
LuY.PanJ. (2022). The pervasive presence of Chinese government content on Douyin Trending videos. Computational Communication Research, 4(1), 68–97. https://doi.org/10.5117/ccr2022.2.002.lu
24.
LuY.ShenC. (2023). Unpacking multimodal fact-checking: Features and engagement of fact-checking videos on Chinese Tiktok (Douyin). Social Media + Society, 9(1), Article 205630512211504. https://doi.org/10.1177/20563051221150406
25.
MargolinD. B. (2021). The theory of informative fictions: A character-based approach to false news and other misinformation. Communication Theory, 31(4), 714–736. https://doi.org/10.1093/ct/qtaa002
26.
MargolinD. B.HannakA.WeberI. (2018). Political fact-checking on Twitter: When do corrections have an effect?Political Communication, 35(2), 196–219. https://doi.org/10.1080/10584609.2017.1334018
27.
MessarisP.AbrahamL. (2001). The role of images in framing news stories. In ReeseS. D.GandyO. H.GrantA. E. (Eds.), Framing public life (pp. 215–226). Erlbaum.
28.
NyhanB.ReiflerJ. (2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32(2), 303–330. https://doi.org/10.1007/s11109-010-9112-2
29.
PapadopoulouO.ZampoglouM.PapadopoulosS.KompatsiarisI. (2019). A corpus of debunked and verified user-generated videos. Online Information Review, 43(1), 72–88. https://doi.org/10.1108/oir-03-2018-0101
30.
PearceW.ÖzkulaS. M.GreeneA. K.TeelingL.BansardJ. S.OmenaJ. J.RabelloE. T. (2020). Visual cross-platform analysis: Digital methods to research social media images. Information, Communication & Society, 23(2), 161–180. https://doi.org/10.1080/1369118x.2018.1486871
31.
PengY.LuY.ShenC. (2023). An agenda for studying credibility perceptions of visual misinformation. Political Communication, 40(2), 225–237. https://doi.org/10.1080/10584609.2023.2175398
32.
PennycookG.RandD. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition, 188(1), 39–50. https://doi.org/10.1016/j.cognition.2018.06.011
33.
PopperK. R. (2003). The logic of scientific discovery: Routledge.
34.
QiP.BuY.CaoJ.JiW.ShuiR.XiaoJ.ChuaT. S. (2023, June). FakeSV: A multimodal benchmark with rich social context for fake news detection on short video platforms. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 37, No. 12, pp.14444-14452). https://doi.org/10.1609/aaai.v37i12.26689
35.
ReissS. (2004). Multifaceted nature of intrinsic motivation: The theory of 16 basic desires. Review of General Psychology, 8(3), 179–193. https://doi.org/10.1037/1089-2680.8.3.179
36.
SchwarzN.NewmanE.LeachW. (2016). Making the truth stick & the myths fade: Lessons from cognitive psychology. Behavioral Science & Policy, 2(1), 85–95. https://doi.org/10.1177/237946151600200110
37.
SerranoJ. C. M.PapakyriakopoulosO.HegelichS. (2020). NLP-based feature extraction for the detection of COVID-19 misinformation videos on YouTube. In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020. https://aclanthology.org/2020.nlpcovid19-acl.17/
38.
ShahD. V.SunZ.BucyE. P.KimS. J.SunY.LiM.SetharesW. (2024). Building an ICCN multimodal classifier of aggressive political debate style: Towards a computational understanding of candidate performance over time. Communication Methods and Measures, 18(1), 30–47. https://doi.org/10.1080/19312458.2023.2227093
39.
ShangL.KouZ.ZhangY.WangD. (2021). A multimodal misinformation detector for covid-19 short videos on TikTok. In 2021 IEEE international conference on big data (big data) (pp. 899–908). IEEE. https://doi.org/10.1109/bigdata52589.2021.9671928
40.
SimonyanK.ZissermanA. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
41.
SundarS. S.LimperosA. M. (2013). Uses and grats 2.0: New gratifications for new media. Journal of Broadcasting & Electronic Media, 57(4), 504–525. https://doi.org/10.1080/08838151.2013.845827
42.
SundarS. S.MolinaM. D.ChoE. (2021). Seeing is believing: Is video modality more powerful in spreading fake news via online messaging apps?Journal of Computer-Mediated Communication, 26(6), 301–319. https://doi.org/10.1093/jcmc/zmab010
43.
TanA.LeeC.-J.ChaeJ. (2015). Exposure to health (mis) information: Lagged effects on young adults’ health behaviors and potential pathways. Journal of Communication, 65(4), 674–698. https://doi.org/10.1111/jcom.12163
44.
TranD.BourdevL.FergusR.TorresaniL.PaluriM. (2015). Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision (pp. 4489–4497). IEEE. https://doi.org/10.1109/iccv.2015.510
45.
VragaE. K.BodeL. (2020). Defining misinformation and understanding its bounded nature: Using expertise and evidence for de-scribing misinformation. Political Communication, 37(1), 136–144. https://doi.org/10.1080/10584609.2020.1716500
46.
WangZ.KimD.ZhuJ. (2024). To live or not to live with it? Examining the cultivation effect of state media on support for the Zero-COVID policy. International Journal of Public Opinion Research, 36(3). https://doi.org/10.1093/ijpor/edae031
47.
WangZ.ZhuJ.XuY.KimD.WilliamsD. (2023). Cross-platform information flow and multilingual text analysis: A comparative study of Weibo and Twitter through deep learning. Computational Communication Research, 5(1), 1–19. https://doi.org/10.5117/ccr2023.1.8.wang
48.
WangZ.ZhuJ.ZuoW.JiangZ.LeiJ.WangZ. (2025). Living with COVID-19 conspiracies on Chinese TikTok (Douyin): Unpacking multimodal features, national identity, and user engagement. Journal of Information Technology & Politics, 1–13. https://doi.org/10.1080/19331681.2025.2473991
49.
WeeksB. E. (2015). Emotions, partisanship, and misperceptions: How anger and anxiety moderate the effect of partisan bias on susceptibility to political misinformation. Journal of Communication, 65(4), 699–719. https://doi.org/10.1111/jcom.12164
50.
WilliamsN. W.CasasA.WilkersonJ. D. (2020). Images as data for social science research: An introduction to convolutional neural nets for image classification. Cambridge University Press.
51.
YangY.DavisT.HindmanM. (2023). Visual misinformation on Facebook. Journal of Communication, 73(4), 316–328. https://doi.org/10.1093/joc/jqac051
Yue-Hei NgJ.HausknechtM.VijayanarasimhanS.VinyalsO.MongaR.TodericiG. (2015). Beyond short snippets: Deep networks for video classification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4694–4702). IEEE.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
