Artificial intelligence (AI) has been increasingly integrated into content moderation to detect and remove hate speech on social media. An online experiment (N = 478) was conducted to examine how moderation agents (AI vs. human vs. human–AI collaboration) and removal explanations (with vs. without) affect users' perceptions and acceptance of removal decisions for hate speech targeting social groups with certain characteristics, such as religion or sexual orientation. The results showed that individuals exhibit consistent levels of perceived trustworthiness and acceptance of removal decisions regardless of the type of moderation agent. When explanations for the content takedown were provided, removal decisions made jointly by humans and AI were perceived as more trustworthy than the same decisions made by humans alone, which increased users' willingness to accept the verdict. However, this moderated mediation effect was only significant when Muslims, not homosexuals, were the target of hate speech.
Get full access to this article
View all access options for this article.
References
1.
WachsS, BilzL, WettsteinA, et al.The online hate speech cycle of violence: Moderating effects of moral disengagement and empathy in the victim-to-perpetrator relationship. Cyberpsychol Behav Soc Netw, 2022; 25(4):223–229; doi: 10.1089/cyber.2021.0159
2.
ObermaierM, SchmuckD, SaleemM. I'll be there for you? Effects of Islamophobic online hate speech and counter speech on Muslim in-group bystanders' intention to intervene. New Media Soc, 2021; doi: 10.1177/14614448211017527
3.
Leonard CheshireDisability. Disability Hate CrimeRising, but Few Cases Make it toCourt. 2019. Available from: https://www.leonardcheshire.org/about-us/our-news/press-releases/disability-hate-crime-rising-few-cases-make-it-court [Last accessed: May23, 2022].
4.
Anti-Defamation League. Online Hate and Harassment. The American Experience. 2021. Available from: https://www.adl.org/resources/report/online-hate-and-harassment-american-experience-2021 [Last accessed: May25, 2022].
5.
LaubZ. Hate speech on social media: Global comparisons. 2019. Available from: https://www.cfr.org/backgrounder/hate-speech-social-media-global-comparisons [Last accessed: May25, 2022].
6.
SahaK, ChandrasekharanE, De ChoudhuryM. Prevalence and psychological effects of hateful speech in online college communities. In: Proceedings of the 10th ACM Conference on Web Science. 2019; pp. 255–264; doi: 10.1145/3292522.3326032
7.
LlansóEJ.No amount of “AI” in content moderation will solve filtering's prior-restraint problem. Big Data Soc, 2020; 7(1):1–6; doi: 10.1177/2053951720920686
8.
JhaverS, BirmanI, GilbertE, et al.Human-machine collaboration for content regulation: The case of reddit automoderator. ACM Trans Comput Hum Interact, 2019; 26(5):1–35; doi: 10.1145/3338243
9.
LaiV, CartonS, BhatnagarR, et al.Human-AI collaboration via conditional delegation: A case study of content moderation. In: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 2022; pp. 1–17; doi: 10.1145/3491102.3501999
10.
RischJ, KrestelR. Delete or not delete? Semi-automatic comment moderation for the newsroom. In: Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying. (Kumar R, Ojha AK, Zampieri M, Malmasi S. eds.) Association for Computational Linguistics: New Mexico, USA; 2018; pp. 166–176.
11.
WojcieszakM, ThakurA, GonçalvesJF, et al.Can AI enhance people's support for online moderation and their openness to dissimilar political views?. J Comput Mediat Commun, 2021; 26(4):223–243; doi: 10.1093/jcmc/zmab006
12.
PanCA, YakhmiS, IyerTP, et al.Comparing the perceived legitimacy of content moderation processes: Contractors, algorithms, expert panels, and digital juries. Proc ACM Hum Comput Interact, 2022; 6:1–31; doi: 10.1145/3512929
13.
GonçalvesJ, WeberI, MasulloGM, et al.Common sense or censorship: How algorithmic moderators and message type influence perceptions of online content deletion. New Media Soc, 2021; doi: 10.1177/14614448211032310
14.
DietvorstBJ, SimmonsJP, MasseyC. Algorithm aversion: People erroneously avoid algorithms after seeing them err. J Exp Psychol Gen, 2015; 144(1):114–126; doi: 10.1037/xge0000033
15.
BigmanYE, GrayK. People are averse to machines making moral decisions. Cognition, 2018; 181:21–34; doi: 10.1016/j.cognition.2018.08.003
16.
LoggJM, MinsonJA, MooreDA. Algorithm appreciation: People prefer algorithmic to human judgment. Organ Behav Hum Decis Process, 2019; 151:90–103; doi: 10.1016/j.obhdp.2018.12.005
17.
ThurmanN, MoellerJ, HelbergerN, et al.My friends, editors, algorithms, and I: Examining audience attitudes to news selection. Digit Journal, 2019; 7(4):447–469; doi: 10.1080/21670811.2018.1493936
18.
HouYT, JungMF. Who is the expert? Reconciling algorithm aversion and algorithm appreciation in AI-supported decision making. Proc ACM Hum Comput Interact, 2021; 5:1–25; doi: 10.1145/3479864
19.
GillespieT.Content moderation, AI, and the question of scale. Big Data Soc, 2020; 7(2):1–5; doi: 10.1177/2053951720943234
20.
SuzorNP, WestSM, QuodlingA, et al.What do we mean when we talk about transparency? Toward meaningful transparency in commercial content moderation. Int J Commun, 2019; 13:1526–1543.
21.
BanasJA, PalomaresNA, RichardsAS, et al.When machine and bandwagon heuristics compete: Understanding users' response to conflicting AI and crowdsourced fact-checking. Hum Commun Res, 2022; doi: 10.1093/hcr/hqac010
22.
SilvaL, MondalM, CorreaD, et al.Analyzing the targets of hate in online social media. In: Proceedings of the International AAAI Conference on Web and Social Media, 2016; 10(1):687–690; doi: 10.1609/icwsm.v10i1.14811
23.
BrunkJ, MatternJ, RiehleDM. Effect of transparency and trust on acceptance of automatic online comment moderation systems. In: 2019 IEEE 21st Conference on Business Informatics, vol. 1. 2019; pp. 429–435; doi: 10.1109/CBI.2019.00056
24.
HöddinghausM, SondernD, HertelG. The automation of leadership functions: Would people trust decision algorithms?. Comput Hum Behav, 2021; 116:106635; doi: 10.1016/j.chb.2020.106635
25.
OhanianR.Construction and validation of a scale to measure celebrity endorsers' perceived expertise, trustworthiness, and attractiveness. J Advert, 1990:19(3):39–52; doi: 10.1080/00913367.1990.10673191
26.
TylerTR, WakslakCJ. Profiling and police legitimacy: Procedural justice, attributions of motive, and acceptance of police authority. Criminology, 2004; 42(2):253–282; doi: 10.1111/j.1745-9125.2004.tb00520.x
27.
VisschersVH, SiegristM. Fair play in energy policy decisions: Procedural fairness, outcome fairness and acceptance of the decision to rebuild nuclear power plants. Energy Policy, 2012; 46:292–300; doi: 10.1016/j.enpol.2012.03.062
EaglyAH, MladinicA, OttoS. Cognitive and affective bases of attitudes toward social groups and social policies. J Exp Soc Psychol, 1994; 30(2):113–137; doi: 10.1006/jesp.1994.1006
30.
HayesAF, (ed). Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. Guilford Press: New York; 2017.
31.
ReevesB, NassC.The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge University Press: United Kingdom; 1996.
32.
SundarSS.Rise of machine agency: A framework for studying the psychology of human–AI interaction (HAII). J Comput Mediat Commun, 2020; 25(1):74–88; doi: 10.1093/jcmc/zmz026
33.
GerrardY.Beyond the hashtag: Circumventing content moderation on social media. New Media Soc, 2018; 20(12):4492–4511; doi: 10.1177/1461444818776611
34.
ChienSY.The Influence of Cultural Factors on Trust in Automation [Dissertation]. University of Pittsburgh: Pennsylvania; 2016.
35.
RaderE, CotterK, ChoJ. Explanations as mechanisms for supporting algorithmic transparency. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 2018; pp. 1–13; doi: 10.1145/3173574.3173677
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
