This study compares the effectiveness of artificial intelligence (AI)-aided versus human-assisted fact-checking for different types of news, namely hard and soft news. To this end, an online experiment was carried out separately in Korea and the United States, employing a 2 (news type: hard news vs. soft news) × 2 (fact-checker: AI vs. human experts) between-group factorial design. Findings suggest that AI’s fact-checking, with its perceived objectivity and lack of bias, is potentially more effective for straightforward hard news, while human-assisted fact-checking appears to be more suitable for addressing misinformation within soft news, which involves more subjective aspects of information. Additionally, the study proposes that AI’s verification of hard news and human experts’ examination of soft news might trigger divergent indirect mechanisms—systematic processing driven by cognition or heuristic processing influenced by emotion—to counteract misinformation.
AdnanN.NordinS. M.bin BahruddinM. A.AliM. (2018). How trust can drive forward the user acceptance to the technology? In-vehicle technology for autonomous vehicle. Transportation Research Part A: Policy and Practice, 118, 819–836.
2.
AppelmanA.SundarS. S. (2016). Measuring message credibility: Construction and validation of an exclusive scale. Journalism & Mass Communication Quarterly, 93(1), 59–79.
3.
BanasJ. APalomaresN. A.RichardsA. S.KeatingD. M.JoyceN.RainsS. A. (2022). When machine and Bandwagon Heuristics compete: Understanding users’ response to conflicting Ai and crowdsourced fact-checking. Human Communication Research, 48(3), 430–461.
4.
BaumM. A. (2003). Soft News goes to war: Public opinion and American Foreign policy in the new media age. Princeton University Press.
5.
BaumgartnerJ. C. (2007). Humor on the next frontier: Youth, online political humor, and the JibJab effect. Social Science Computer Review, 25(3), 319–338.
6.
BoukesM.BoomgaardenH. G. (2015). Soft news with hard consequences? Introducing a nuanced measure of soft versus hard new exposure and its relationship with political cynicism. Communication Research, 42(5), 701–731.
7.
BurgoonJ. K. (1978). A communication model of personal space violations: Explication and an initial test. Human Communication Research, 4(2), 129–142.
8.
CasteloN.BosM. W.LehmannD. (2019). Let the machine decide: When consumers trust or distrust algorithms. NIM Marketing Intelligence Review, 11(2), 24–29.
9.
ChaikenS.LedgerwoodA. (2012). A theory of heuristic and systematic information processing. In Van LangeP. A.KruglanskiA. W.HigginsE. T. (Eds.), Handbook of theories of social psychology (pp. 246–266). Sage.
10.
ChungM.MoonW.-K.Jones-JangS. M. (in press). AI as an apolitical referee: Using Alternative sources to decrease partisan biases in the processing of fact-checking messages. Digital Journalism.
11.
CloudyJ.BanksJ.BowmanN. D. (2023). The Str(AI)ght scoop: Artificial intelligence cues reduce perceptions of hostile media bias. Digital Journalism, 11(9), 1577–1596.
DeVernaM. R.YanH. Y.YangK.-C.MenczerF. (in press). Fact-checking information generated by a large language model can decrease news discernment. arXiv:2308.10800.
14.
GloggerI. (2019). Soft spot for soft news? Influences of journalistic role conceptions on hard and soft news coverage. Journalism Studies, 20(16), 2293–2311.
15.
GloggerI.OttoL. P. (2019). Journalistic views on hard and soft news: Cross-validating a popular concept in a factorial survey. Journalism & Mass Communication Quarterly, 96(3), 811–829.
16.
GraefeA.BohlkenN. (2020). Automated journalism: A meta-analysis of readers’ perceptions of human-written in comparison to automated news. Media and Communication, 8(3), 50–559.
17.
GraefeA.HaimM.HaarmannB.BrosiusH.-B. (2018). Readers’ perception of computer-generated news: Credibility, expertise, and readability. Journalism, 19(5), 595–610.
18.
HayesA. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. Guilford Press.
19.
Jones-JangS. M.ParkY. J. (2023). How do people react to AI failure? Automation bias, algorithmic aversion, and perceived controllability. Journal of Computer-Mediated Communication, 28(1), 1–8.
20.
LeeH.YeonJ. (2022). The power of humor: The role of political entertainment programming in South Korea for enhancing attitudes toward the oppositional message source. Mass Communication and Society, 25(1), 77–110.
21.
LeeM. K. (2018). Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management. Big Data & Society, 5(1), 2053951718756684.
22.
LiuB.WeiL. (2019). Machine authorship in situ: Effect of news organization and news genre on news credibility. Digital Journalism, 7(5), 635–657.
23.
MarkiewiczD. (1972). The effects of humor on persuasion. The Ohio State University.
24.
MoonW. K.ChungM.Jones-JangS. M. (2023). How can we fight partisan biases in the COVID-19 pandemic? AI source labels on fact-checking messages reduce motivated reasoning. Mass Communication and Society, 26(4), 646–670.
25.
NabiR. L.Moyer-GuséE.ByrneS. (2007). All joking aside: A serious investigation into the persuasive effect of funny social issue messages. Communication Monographs, 74(1), 29–54.
26.
O’ConnorC.WeatherallJ. O. (2019). The misinformation age: How false beliefs spread. Yale University Press.
27.
OgnyanovaK. (2024). Fact-checking: Journalistic strategies and audience outcomes in diverse national contexts. Journalism & Mass Communication Quarterly, 101(2), 313–319.
28.
OttoL.GloggerI.BoukesM. (2017). The softening of journalistic political communication: A comprehensive framework model of sensationalism, soft news, infotainment, and tabloidization. Communication Theory, 27(2), 136–155.
29.
PattersonT. E. (2000). Doing well and doing good: How soft news are shrinking the news audience and weakening democracy. Harvard University Press.
30.
PennycookG.BinnendykJ.NewtonC.RandD. G. (2021). A practical guide to doing behavioral research on fake news and misinformation. Collabra: Psychology, 7(1), 25293.
31.
PettyR. E.CacioppoJ. T. (1986). The elaboration likelihood model of persuasion. In LeonardB. (Ed.), Advances in experimental social psychology (pp. 123–205). Academic Press.
32.
ReinemannC.StanyerJ.ScherrS.LegnanteG. (2012). Hard and soft news: A review of concepts, perationalizations and key findings. Journalism, 13(2), 221–239.
33.
SinclairR. C.MarkM. M.CloreG. L. (1994). Mood-related persuasion depends on (mis) attributions. Social Cognition, 12(4), 309–326.
34.
SouthwellB. G.ThorsonE. A. (2015). The prevalence, consequence, and remedy of misinformation in mass media systems. Journal of Communication, 65(4), 589–595.
35.
SundarS. S. (2008). The MAIN model: A heuristic approach to understanding technology effects on credibility. In MetzgerM. J.FlanaginA. J. (Eds.), Digital media, youth, and credibility (pp. 72–100). The MIT Press.
36.
SundarS. S. (2020). Rise of machine agency: A framework for studying the psychology of human–AI interaction (HAII). Journal of Computer-Mediated Communication, 25(1), 74–88.
37.
SundarS. S.NassC. (2001). Conceptualizing sources in online news. Journal of Communication, 51(1), 52–72.
38.
TandocE. C.YaoL. J.WuS.2020. Man vs. machine? The impact of algorithm authorship on news credibility. Digital Journalism, 8(4), 548–562.
39.
WaddellT. F. (2018). A robot wrote this?. Digital Journalism, 6(2), 236–255.
40.
WaytzA.NortonM. I. (2014). Botsourcing and outsourcing: Robot, British, Chinese, and German workers are for thinking-not feeling-jobs. Emotion, 14(2), 434–444.
41.
WojcieszakM.ThakurA.GonçalvesJ. F. F.CasasA.Menchen-TrevinoE.BoonM. (2021). Can AI enhance people’s support for online moderation and their openness to dissimilar political views?. Journal of Computer-Mediated Communication, 26(4), 223–243.
42.
WölkerA.PowellT. E. (2021). Algorithms in the newsroom? News readers’ perceived credibility and selection of automated journalism. Journalism, 22(1), 86–103.
43.
WuY. (2020). Is automated journalistic writing less biased? An experimental test of auto-written and human-written news stories. Journalism Practice, 14(8), 1008–1028.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
