Sage Journals: Discover world-class research

Abstract

As artificial intelligence (AI)-powered technology enables the efficient processing of large volumes of comments, more companies, including news publications, are experimenting with and adopting AI moderation to manage their commenting platforms. However, the resulting user experiences have been largely underexplored in relation to these technical advances in comments section management. This study used an experiment to examine the impact of AI filters on individuals’ perceptions about comments sections (i.e., bias, credibility, positive and negative affect, and use intention). The findings indicate that AI moderation had statistically significant impacts on perceived credibility and use intention. Beyond the main effects, priming with a deceptive comment issue moderated the impacts of comment filtering on user perceptions as well.

Keywords

AI moderation comment filtering comments section use intention user perception

Get full access to this article

View all access options for this article.

References

Anderson

(2017, July 11). Key takeaways on how Americans view-and experience-online harassment. Pew Research Center. https://www.pewresearch.org/fact-tank/2017/07/11/key-takeaways-online-harassment

Appelman

Sundar

S. S.

(2016). Measuring message credibility: Construction and validation of an exclusive scale. Journalism & Mass Communication Quarterly, 93(1), 59–79. https://doi.org/10.1177/1077699015606057

Berger

Milkman

K. L.

(2012). What makes online content viral? Journal of Marketing Research, 49(2), 192–205. https://doi.org/10.1509/jmr.10.0353

Broockman

D. E.

Butler

D. M.

(2017). The causal effects of elite position-taking on voter attitudes: Field experiments with elite communication. American Journal of Political Science, 61(1), 208–221. https://doi.org/10.1111/ajps.12243

Chiu

M. M.

Y. W.

(2021). How fake news differs from personal lies. American Behavioral Scientist, 65(2), 243–258. https://doi.org/10.1177/0002764220910243

Clerwall

(2014). Enter the robot journalist: Users’ perceptions of automated content. Journalism Practice, 8(5), 519–531. https://doi.org/10.1080/17512786.2014.883116

Cohen

Pham

Andrade

(2008). The nature and role of affect in consumer behavior. In Haugtvedt

C. P.

Herr

P. M.

Kardes

F. R.

(Eds.), Marketing and consumer psychology series: Handbook of consumer psychology (vol.4, pp. 297–348). Taylor & Francis.

Cohn

M. A.

Fredrickson

B. L.

(2006). Beyond the moment, beyond the self: Shared ground between selective investment theory and the broaden-and-build theory of positive emotions. Psychological Inquiry, 17(1), 39–44.

Cohn

M. A.

Fredrickson

B. L.

(2009). Positive emotions. In Snyder

C. R.

Lopez

S. J.

(Eds.), The Oxford handbook of positive psychology (2nd ed., pp.13–24). Oxford University Press.

10.

Delli Carpini

M. X

. (2000). Gen.com: Youth, civic engagement, and the new information environment. Political Communication, 17(4), 341–349. https://doi.org/10.1080/10584600050178942

11.

Fan

Liu

Zhu

Pardalos

P. M.

(2020). Investigating the impacting factors for the healthcare professionals to adopt artificial intelligence-based medical diagnosis support system (AIMDSS). Annals of Operations Research, 294, 567–592. https://doi.org/10.1007/s10479-018-2818-y

12.

Fernández-Llamas

Conde

M. A.

Rodríguez-Lera

F. J.

Rodríguez-Sedano

F. J.

García

(2018). May I teach you? Students’ behavior when lectured by robotic vs. Human teachers. Computers in Human Behavior, 80, 460–469. https://doi.org/10.1016/j.chb.2017.09.028

13.

Fredrickson

B. L.

(2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56(3), 218–226. https://doi.org/10.1037/0003-066X.56.3.218

14.

Fredrickson

B. L.

(2004). The broaden-and-build theory of positive emotions. Philosophical Transactions of the Royal Society B, 359(1449), 1367–1377. https://doi.org/10.1098/rstb.2004.1512

15.

Graefe

Haim

Haarmann

Brosius

H. B.

(2018). Readers’ perception of computer-generated news: Credibility, expertise, and readability. Journalism, 19(5), 595–610. https://doi.org/10.1177/1464884916641269

16.

Habermas

(1989). The structural transformation of the public sphere: An inquiry into a category of bourgeois society ( Burger

, Trans.). Polity Press. (Original work published 1962).

17.

Hancock

P. A.

Billings

D. R.

Schaefer

K. E.

(2011). Can you trust your robot? Ergonomics in Design: The Quarterly of Human Factors Applications, 19(3), 24–29. https://doi.org/10.1177/1064804611415045

18.

Harmon-Jones

Bastian

Harmon-Jones

(2016). The discrete emotions questionnaire: A new tool for measuring state self-reported emotions. PLoS One, 11, e0159915. https://doi.org/10.1371/journal.pone.0159915

19.

Hasell

Weeks

B. E.

(2016). Partisan provocation: The role of partisan news use and emotional responses in political information sharing in social media. Human Communication Research, 42(4), 641–661. https://doi.org/10.1111/hcre.12092

20.

S. S.

McLeod

D. M.

(2008). Social-psychological influences on opinion expression in face-to-face and computer-mediated communication. Communication Research, 35(2), 190–207. https://doi.org/10.1177/0093650207313159

21.

Houston

J. B.

Hansen

G. J.

Nisbett

G. S.

(2011). Influence of user comments on perceptions of media bias and third-person effect in online news. Electronic News, 5(2), 79–92. https://doi.org/10.1177/1931243111407618

22.

Hovland

C. I.

Janis

I. L.

Kelley

H. H.

(1953). Communication and persuasion: Psychological studies of opinion change. Yale University Press.

23.

Isen

A. M.

Daubman

K. A.

(1984). The influence of affect on categorization. Journal of Personality and Social Psychology, 47(6), 1206–1217. https://doi.org/10.1037/0022-3514.47.6.1206

24.

Iyengar

Simon

(1993). News coverage of the Gulf crisis and public opinion: A study of agenda-setting, priming, and framing. Communication Research, 20(3), 365–383. https://doi.org/10.1177/009365093020003002

25.

Kim

Sundar

S. S.

(2016). Human control or machine control—Which do we trust? The role of control and machine heuristic in online information disclosure. Paper presented at Annual Conference of Association for Education in Journalism and Mass Communication, Minneapolis, MN.

26.

Lee

E. J.

(2012). That’s not the way it is: How user-generated comments on the news affect perceived media bias. Journal of Computer-Mediated Communication, 18(1), 32–45. https://doi.org/10.1111/j.1083-6101.2012.01597.x

27.

Chen

Mukherjee

Liu

Shao

(2015). Analyzing and detecting opinion spam on a large-scale dataset via temporal and spatial patterns. Proceedings of the 9th International AAAI Conference on Web and Social Media (Vol. 9, pp. 634–637), Oxford, U.K.

28.

Liu

Wei

(2019). Machine authorship in Situ: Effect of news organization and news genre on news credibility. Digital Journalism, 7(5), 635–657.

29.

Molina

M. D.

Sundar

S. S.

Lee

(2021). “Fake news” is not simply false information: A concept explication and taxonomy of online content. American Behavioral Scientist, 65(2), 180–212. https://doi.org/10.1177/0002764219878224

30.

Mosier

Skitka

(1996). Human decision makers and automated decision aids: Made for each other? In Parasuraman

Mouloua

(Eds.), Automation and human performance: Theory and applications (pp. 201–220). Lawrence Erlbaum Associates, Inc.

31.

Neuman

W. R.

Bimber

Hindman

(2011). The Internet and four dimensions of citizenship. In Shapiro

Jacobs

(Eds.), The Oxford handbook of American public opinion and the media (pp. 22–42). Oxford University Press.

32.

OECD. (2021). 21st-century readers: Developing literacy skills in a digital world. https://www.oecd.org/publications/21st-century-readers-a83d84cb-en.htm

33.

Y. W.

Park

C. H

. (2021). Machine cleaning of online opinion spam: Developing a machine-learning algorithm for detecting deceptive comments. American Behavioral Scientist, 65(2), 389–403. https://doi.org/10.1177/0002764219878238

34.

Ott

Choi

Cardie

Hancock

J. T.

(2011). Finding deceptive opinion spam by any stretch of the imagination. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (Vol. 1, pp. 309–319), Portland, OR, United States.

35.

Papacharissi

(2004). Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6(2), 259–284.

36.

Pennycook

Epstein

Mosleh

Arechar

A. A.

Eckles

Rand

D. G.

(2021). Shifting attention to accuracy can reduce misinformation online. Nature, 592, 590–595. https://doi.org/10.1038/s41586-021-03344-2

37.

Rahman

M. M.

Lesch

M. F.

Horrey

W. J.

Strawderman

(2017). Assessing the utility of TAM, TPB, and UTAUT for advanced driver assistance systems. Accident Analysis & Prevention, 108, 361–373. https://doi.org/10.1016/j.aap.2017.09.011

38.

Rainie

Anderson

Albright

(2017, March 29). The future of free speech, trolls, anonymity and fake news online. Pew Research Center. https://www.pewresearch.org/internet/2017/03/29/the-future-of-free-speech-trolls-anonymity-and-fake-news-online/

39.

Rastogi

Mehrotra

(2017). Opinion spam detection in online reviews. Journal of Information and Knowledge Management, 16(4), 1750036. https://doi.org/10.1142/S0219649217500368

40.

Ray

Mondada

Siegwart

(2008). What do people expect from robots? Proceedings of the 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems (vol.1, pp. 3816–3821), Nice, France.

41.

Salganik

M. J.

Lee

R. C.

(2020, April 30). To apply machine learning responsibly, we use it in moderation. The New York Times. https://www.open.nytimes.com/to-apply-machine-learning-responsibly-we-use-it-in-moderation-d001f49e0644

42.

Sherrick

Hoewe

(2018). The effect of explicit online comment moderation on three spiral of silence outcomes. New Media & Society, 20(2), 453–474. https://doi.org/10.1177/1461444816662477

43.

Sohn

Kwon

(2020). Technology acceptance theories and factors influencing artificial Intelligence-based intelligent products. Telematics and Informatics, 47, 101324. https://doi.org/10.1016/j.tele.2019.101324

44.

Sundar

S. S.

(2008). The MAIN model: A heuristic approach to understanding technology effects on credibility. In Metzger

M. J.

Flanagin

A. J.

(Eds.), Digital media, youth, and credibility (pp. 73–100). The MIT Press.

45.

Takens

Kleinnijenhuis

Van Hoof

Van Atteveldt

(2015). Party leaders in the media and voting behavior: Priming rather than learning or projection. Political Communication, 32(2), 249–267. https://doi.org/10.1080/10584609.2014.944319

46.

Tandoc

E. C.

Yao

L. J.

(2020). Man vs. machine? The impact of algorithm authorship on news credibility. Digital Journalism, 8(4), 548–562. https://doi.org/10.1080/21670811.2020.1762102

47.

Thorson

Wicks

Leshner

(2012). Experimental methodology in journalism and mass communication research. Journalism & Mass Communication Quarterly, 89(1), 112–124. https://doi.org/10.1177/1077699011430066

48.

van der Linden

Panagopoulos

Roozenbeek

. (2020). You are fake news: Political bias in perceptions of fake news. Media, Culture & Society, 42(3), 460–470. https://doi.org/10.1177/0163443720906992

49.

Van Duyn

Collier

. (2019). Priming and fake news: The effects of elite discourse on evaluations of news media. Mass Communication and Society, 22(1), 29–48. https://doi.org/10.1080/15205436.2018.1511807

50.

Waddell

T. F.

(2018). A robot wrote this? How perceived machine authorship affects news credibility. Digital Journalism, 6(2), 236–255. https://doi.org/10.1080/21670811.2017.1384319

51.

Waddell

T. F.

(2019). Can an algorithm reduce the perceived bias of news? Testing the effect of machine attribution on news readers’ evaluations of bias, anthropomorphism, and credibility. Journalism & Mass Communication Quarterly, 96(1), 82–100. https://doi.org/10.1177/1077699018815891

52.

Wallace

L. E.

Wegener

D. T.

Petty

R. E.

(2020). When sources honestly provide their biased opinion: Bias as a distinct source perception with independent effects on credibility and persuasion. Personality and Social Psychology Bulletin, 46(3), 439–453. https://doi.org/10.1177/0146167219858654

53.

Wang

(2020). Moderating uncivil user comments by humans or machines? The effects of moderation agent on perceptions of bias and credibility in news content. Digital Journalism, 9(1), 64–83. https://doi.org/10.1080/21670811.2020.1851279

54.

Watson

Naragon

(2009). Positive affectivity: The disposition to experience positive emotional states. In Snyder

C. R.

Lopez

S. J.

(Eds.), The Oxford handbook of positive psychology (2nd ed., pp. 207–216). Oxford University Press.

55.

Watts

M. D.

Domke

Shah

D. V.

Fan

D. P.

(1999). Elite cues and media bias in presidential campaigns: Explaining public perceptions of a liberal press. Communication Research, 26(2), 144–175. https://doi.org/10.1177/009365099026002003

56.

Weeks

B. E.

Holbert

R. L.

(2013). Predicting dissemination of news content in social media: A focus on reception, friending, and partisanship. Journalism & Mass Communication Quarterly, 90(2), 212–232. https://doi.org/10.1177/1077699013482906

57.

Wise

Hamman

Thorson

(2006). Moderation, response rate, and message interactivity: Features of online communities and their effects on intent to participate. Journal of Computer-Mediated Communication, 12(1), 24–41. https://doi.org/10.1111/j.1083-6101.2006.00313.x

58.

Wölker

Powell

T. E.

(2018). Algorithms in the newsroom? News readers’ perceived credibility and selection of automated journalism. Journalism, 22(1), 86–103. https://doi.org/1464884918757072

59.

Wright

Street

(2007). Democracy, deliberation and design: The case of online discussion forums. New Media & Society, 9(5), 849–869. https://doi.org/10.1177/1461444807081230

60.

Yeo

S. K.

L. Y. F.

Scheufele

D. A.

Brossard

Xenos

M. A.

Corley

E. A.

(2019). The effect of comment moderation on perceived bias in science news. Information, Communication & Society, 22(1), 129–146. https://doi.org/10.1080/1369118X.2017.1356861

User Perceptions of AI-Based Comment Filtering Technology

Abstract

Keywords

Get full access to this article

References