Addressing Toxicity and Extremism in Games: Conversations with the Video Game Industry

Abstract

There has been an increased focus on reducing hate, harassment, and extremist exploitation in digital games through solutions from community management, trust and safety, and design perspectives. Despite this attention, the industry lacks basic standards around these issues and efforts in this area are stymied by the ubiquity of “toxic gamer culture” and fears in the industry about discussing extremism and hate within games. Today, a patchwork of strategies and tools exist. In this study, focus groups with industry professionals were conducted to increase understanding of industry professional's perceptions of these problems and solutions. Several primary themes regarding solutions emerged: game and social system design strategies, moderation and player report strategies, and wish list solutions.

Keywords

extremism radicalization community management moderation video games

Get full access to this article

View all access options for this article.

References

Anti-Defamation League (2022). Hate Is No Game Hate and Harassment in Online Games 2022 The state of hate, harassment, and extremism in online multiplayer games. https://www.adl.org/resources/report/hate-no-game-hate-and-harassment-online-games-2022

Bailey

E. N.

Miyata

Yoshida

(2021). Gender composition of teams and studios in video game development. Games and Culture, 16(1), 42–64. https://doi.org/10.1177/1555412019868381

Berger

J. M.

(2018). Extremism. MIT Press. https://mitpress.mit.edu/9780262535878/extremism/

Blodgett

Salter

(2018). Ghostbusters is for boys: Understanding geek masculinity’s role in the alt-right. Communication, Culture and Critique, 11(1), 133–146. https://doi.org/10.1093/ccc/tcx003

Boudreau

(2022). Beyond deviance: Toxic gaming culture and the potential for positive change. Critical Studies in Media Communication, 39(3), 181–190. https://doi.org/10.1080/15295036.2022.2080848

Braun

Clarke

(2021). Can I use TA? Should I use TA? Should I not use TA? Comparing reflexive thematic analysis and other pattern-based qualitative analytic approaches. Counselling and Psychotherapy Research, 21(1), 37–47. https://doi.org/10.1002/capr.12360

Brinkmann

(2015). Perils and potentials in qualitative psychology. Integrative Psychological and Behavioral Science, 49(2), 162–173. https://doi.org/10.1007/s12124-014-9293-z

Call of Duty Staff (2023, August 30). Call of Duty takes aim at voice chat toxicity, Details year to date moderation progress. Activision. https://www.callofduty.com/blog/2023/08/call-of-duty-modern-warfare-warzone-anti-toxicity-progress-report

Condis

(2020). Hateful games: Why white supremacist recruiters target gamers. In Reyman

Sparby

E. M.

(Eds.), Digital ethics rhetoric and responsibility in online aggression (pp. 143–159). Routledge.

10.

Cote

A. C.

(2020). Casual resistance: A longitudinal case study of video gaming’s gendered construction and related audience perceptions. Journal of Communication, 70(6), 819–841. https://doi.org/10.1093/joc/jqaa028

11.

Doherty

(2019, March). Play Nice, Play Fair: The Business Case for Fighting Disruptive Behaviors. Game Developers Conference. https://www.gdcvault.com/play/1026037/Play-Nice-Play-Fair-The

12.

Entertainment Software Association (2023). 2023 Essential facts about the US video game industry. https://www.theesa.com/2023-essential-facts/

13.

Fair Play Alliance, & ADL Center for Technology and Society (2020). Disruption and Harms in Online Gaming Framework: Creating and Maintaining Community Guidelines for Online Games and Platforms. https://fairplayalliance.org/wp-content/uploads/2020/12/FPA-Framework-Creating-Maintaining-Community-Guidelines.pdf

14.

FPA, & ADL (2020). Disruption and Harms in Online Gaming Framework: Building a Penalty and Reporting System. https://www.adl.org/resources/report/disruption-and-harms-online-gaming-resource-penalty-and-reporting-systems

15.

Gandolfi

Ferdig

R. E.

Krause

Copus

Ostrowski-Delahanty

Alemagno

(2025). Problematic gaming at a crossroad: Exploring the interplay between internet gaming disorder, toxic attitudes, and empathy in digital entertainment. Games and Culture, 20(5), 633–656. https://doi.org/10.1177/15554120231211991

16.

Geena Davis Institute (2023). Changing the Narrative: Why Representation in Video Games Matters I Changing the Narrative Why Representation in Video Games Matters Brothers91/E+ via Getty Images. https://seejane.org/research-informs-empowers/changing-the-narrative-why-representation-in-video-games-matters/

17.

Gergen

K. J.

Josselson

Freeman

(2015). The promises of qualitative inquiry. American Psychologist, 70(1), 1–9. https://doi.org/10.1037/a0038597

18.

Gillespie

(2021). Custodians of the internet: Platforms, content moderation, and the hidden decisions that shape social media (1st ed.). Yale University Press.

19.

Goggin

(2018). “How do those Danish bastards sleep at night?”*: Fan labor and the power of cuteness. Games and Culture, 13(7), 747–764. https://doi.org/10.1177/1555412018760918

20.

GPAHE (2024). How The Far-Right Spreads Hate Through Gaming. https://globalextremism.org/post/how-the-far-right-spreads-hate-through-gaming/

21.

Gray

K. L.

(2012a). Deviant bodies, stigmatized identities, and racist acts: Examining the experiences of African-American gamers in Xbox live. New Review of Hypermedia and Multimedia, 18(4), 261–276. https://doi.org/10.1080/13614568.2012.746740

22.

Gray

K. L.

(2012b). Intersecting oppressions and online communities: Examining the experiences of women of color in Xbox live. Information Communication and Society, 15(3), 411–428. https://doi.org/10.1080/1369118X.2011.642401

23.

Jenson

de Castell

(2018). “The entrepreneurial gamer”: Regendering the order of play. Games and Culture, 13(7), 728–746. https://doi.org/10.1177/1555412018755913

24.

Just

S. N.

Storm

Bukuru

S. L.

(2023). Onlife intersectionalities as flows of playbour: The case of women in gaming. Media, Culture and Society, 45(5), 899–915. https://doi.org/10.1177/01634437231155345

25.

Kelleher

Pausch

Kiesler

(2007). Storytelling alice motivates middle school girls to learn computer programming. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1455–1464. https://doi.org/10.1145/1240624.1240844

26.

Kilmer

E. D.

Aslan

Kowert

(2024). From avoidance to action: A call for open dialogue on hate, harassment, and extremism in the gaming industry. ACM Games.

27.

Kilmer

E. D.

Kowert

(2024). Empowering the gaming industry: Strategies for addressing hate, harassment, and extremism in online communities. Take This. https://www.takethis.org/2024/02/empowering-industry-whitepaper/

28.

Kim

(2022, February 23). 85% of Apex Legends Players Responded Better to Direct Feedback Than Outright Bans. IGN. https://www.ign.com/articles/apex-legends-positive-feedback-ban-dice-summit

29.

Kitzinger

(1995). Qualitative research: Introducing focus groups. BMJ, 311(7000), 299–302. https://doi.org/10.1136/bmj.311.7000.299

30.

Kordyaka

Laato

Weber

Niehaves

(2023). What constitutes victims of toxicity—identifying drivers of toxic victimhood in multiplayer online battle arena games. Frontiers in Psychology, 14, 1193172:1–12. https://doi.org/10.3389/fpsyg.2023.1193172

31.

Kowert

Cook

C. L.

(2022). The toxicity of our (sim) cities: Prevalence of dark participation in games and perceived effectiveness of reporting tools. Proceedings of the Annual Hawaii International Conference on System Sciences, 2022, 3180–3189. https://doi.org/10.24251/hicss.2022.390

32.

Kowert

Crevoshay

(2023). Harassment of game makers: Prevalence and impact [version 2; peer review: 1 approved, 1 approved with reservations]. F1000Research, 11, 1518:1- 21https://doi.org/10.12688/f1000research.126998.2

33.

Kowert

Kilmer

(2023a). Extremism in games: A primer. https://www.takethis.org/wp-content/uploads/2023/10/ExtremismInGames_APrimer_FINAL-1.pdf

34.

Kowert

Kilmer

(2023b). Toxic gamers are alienating your core demographic: The business case for community management. https://www.takethis.org/2023/08/research-report-toxic-gamers-are-alienating-your-core-demographic-the-business-case-for-community-management/

35.

Kowert

Kilmer

E. D.

(2023c). Resilience in the face of hate [Conference presentation]. Games for Change.

36.

Krueger

R. A.

Casey

M. A.

(2014). Focus groups: A practical guide for applied research (5th ed.). Sage Publications, Inc.

37.

Lakhani

(2021). Video Gaming and Violent Extremism: An exploration of the current landscape, trends, and threats. https://home-affairs.ec.europa.eu/system/files/2022-02/EUIF Technical Meeting on Video Gaming October 2021 RAN Policy Support paper_en.pdf.

38.

Maher

(2016, March 31). Can a Video Game Company Tame Toxic Behavior? Scientific American. https://www.scientificamerican.com/article/can-a-video-game-company-tame-toxic-behavior/

39.

Matias

J. N.

(2019). The civic labor of volunteer moderators online. Social Media + Society, 5(2), 205630511983677. https://doi.org/10.1177/2056305119836778

40.

Reid

Mandryk

R. L.

Beres

N. A.

Klarkowski

Frommel

(2022). “Bad vibrations”: Sensing toxicity from in-game audio features. IEEE Transactions on Games, 14(4), 558–568. https://doi.org/10.1109/TG.2022.3176849

41.

Rosenblat

M. O.

Barrett

P. M.

(2023). Gaming the system: How extremists exploit gaming sites and what can be done to counter them (Issue May). https://bhr.stern.nyu.edu/tech-gaming-report

42.

Rottweiler

Clemmow

Gill

(2023). A common psychology of male violence? Assessing the effects of misogyny on intentions to engage in violent extremism, interpersonal violence and support for violence against women. Terrorism and Political Violence, 1–26. https://doi.org/10.1080/09546553.2023.2292723

43.

Scull

T. M.

Malik

C. V.

Kupersmidt

J. B.

(2018). Understanding the unique role of media message processing in predicting adolescent sexual behavior intentions in the USA. Journal of Children and Media, 12(3), 258–274. https://doi.org/10.1080/17482798.2017.1403937

44.

Tomkinson

van den Ende

(2022). ‘Thank you for your compliance’: Overwatch as a disciplinary system. Games and Culture, 17(2), 198–218. https://doi.org/10.1177/15554120211026257

45.

Unity (2023). Toxicity In Multiplayer Games Report. https://create.unity.com/toxicity-report?utm_source=substack&utm_medium=email

46.

Voll

(2018, March). Honorable Intentions: Player Behavior Today at Riot Games. Game Developers Conference.

47.

Wells

Romhanyi

Reitman

J. G.

Gardner

Squire

Steinkuehler

(2023). Right-wing extremism in mainstream games: A review of the literature. Games and Culture, 19(4), 469–492. https://doi.org/10.1177/15554120231167214

48.

Wohn

D. Y.

(2019). Volunteer moderators in twitch micro communities. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–13. https://doi.org/10.1145/3290605.3300390

49.

Zsila

Shabahang

Aruguete

M. S.

Orosz

(2022). Toxic behaviors in online multiplayer games: Prevalence, perception, risk factors of victimization, and psychological consequences. Aggressive Behavior, 48(3), 356–364. https://doi.org/10.1002/ab.22023