Most content moderation approaches in the United States rely on criminal justice models that sanction offenders via content removal or user bans. However, these models write the online harassment targets out of the justice-seeking process. Via an online survey with US participants (N = 573), this research draws from justice theories to investigate approaches for supporting targets of online harassment. We uncover preferences for banning offenders, removing content, and apologies, but aversion to mediation and adjusting targets’ audiences. Preferences vary by identities (e.g. transgender participants on average find more exposure to be undesirable; American Indian or Alaska Native participants on average find payment to be unfair) and by social media behaviors (e.g. Instagram users report payment as just and fair). Our results suggest that a one-size-fits-all approach will fail some users while privileging others. We propose a broader theoretical and empirical landscape for supporting online harassment targets.
AhmedS (2012) On Being Included: Racism and Diversity in Institutional Life. Durham, NC: Duke University Press.
2.
BarocasSHardtMNarayananA (2019) Fairness and machine learning. Available at: https://fairmlbook.org/ (accessed 26 August 2019).
3.
BayerJEllisonNBSchoenebeckSY, et al. (2015) Sharing the small moments: ephemeral social interaction on Snapchat. Information, Communication & Society19(7): 956–977.
4.
BellD (2008) And We Are Not Saved: The Elusive Quest for Racial Justice. New York: Basic Books.
5.
BibasSBierschbachRA (2004) Integrating remorse and apology into criminal procedure essay. The Yale Law Journal114: 85–148.
6.
BlackwellLChenTSchoenebeckS, et al. (2018) When online harassment is perceived to be justified. In: The 12th international AAAI conference on web and social media (ICWSM 2018), Palo Alto, CA, 25–28 June. Menlo Park, CA, AAAI Press.
7.
BlackwellLDimondJSchoenebeckS, et al. (2017) Classification and its consequences for online harassment: design insights from HeartMob. Proceedings of ACM on Human-Computer Interaction1: 1–19.
8.
BoboLDThompsonV (2006) Unfair by design: the war on drugs, race, and the legitimacy of the criminal justice system. Social Research: An International Quarterly73(2): 445–472.
9.
BradfordBGriselFMearesTL, et al. (2019) Report of the Facebook Data Transparency Advisory Group. New Haven, CT: Yale Law School.
10.
BraithwaiteJ (1999) Restorative justice: assessing optimistic and pessimistic accounts. Crime and Justice25: 1–127.
11.
Brave HeartMYDeBruynLM (1998) The American Indian Holocaust: healing historical unresolved grief. American Indian and Alaska Native Mental Health Research: Journal of the National Center8(2): 56–78.
12.
CaiJWohnDY (2019) What are effective strategies of handling harassment on twitch? Users’ perspectives. In: Conference companion publication of the 2019 on computer supported cooperative work and social computing, pp. 166–170. Available at: https://dl.acm.org/doi/10.1145/3311957.3359478
13.
CaplanR (2018) Content or Context Moderation? Artisanal, Community-Reliant, and Industrial Approaches. New York: Data & Society.
14.
ChandrasekharanEGandhiCMustelierMW, et al. (2019) Crossmod: a cross-community learning-based system to assist Reddit moderators. In: Proceedings of the ACM on human-computer interaction. 3, CSCW, Article 174 (November2019), p. 30. Available at: https://dl.acm.org/doi/10.1145/3359276
15.
CitronDK (2014) Hate Crimes in Cyberspace. Cambridge, MA: Harvard University Press.
16.
ClaytonSOpotowS (2003) Justice and identity: changing perspectives on what is fair. Personality and Social Psychology Review7(4): 298–310.
17.
CoatesTN (2014) The case for reparations. The Atlantic313(5): 54–71.
FallonRHWeilerPC (1984) Firefighters v. Stotts: conflicting models of racial justice. The Supreme Court Review1984: 1–68.
23.
GillespieT (2010) The politics of “platforms.” New Media & Society12(3): 347–364.
24.
GillespieT (2018) Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions that Shape Social Media. New Haven, CT; London: Yale University Press.
25.
HaimsonOLHoffmannAL (2016) Constructing and enforcing “authentic” identity online: Facebook, real names, and non-normative identities. First Monday21(6).
26.
HoffmannAL (2019) Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse. Information, Communication & Society22(7): 900–915.
27.
HoffmannALJonasA (2016) Recasting justice for internet and online industry research ethics. In: ZimmerMKinder-KurandaK (eds) Internet Research Ethics for the Social Age: New Cases and Challenges. Available at: https://papers.ssrn.com/abstract=2836690
28.
JacksonTF (2013) From Civil Rights to Human Rights: Martin Luther King, Jr., and the Struggle for Economic Justice. Philadelphia, PA: University of Pennsylvania Press.
JhaverSApplingDSGilbertE, et al. (2019) “Did you suspect the post would be removed?” Understanding user reactions to content removals on Reddit. In: Proceedings of ACM on human-computer interaction. 3, CSCW, Article 192 (November2019), p. 33. Available at: https://dl.acm.org/doi/10.1145/3359294
31.
JhaverSGhoshalSBruckmanA, et al. (2018) Online harassment and content moderation: the case of blocklists. ACM Transactions on Computer-Human Interaction25(2): 1–33.
32.
KlonickK (2015) Re-shaming the debate: social norms, shame, and regulation in an internet age. Maryland Law Review75: 1029–1065.
MassanariA (2017) #Gamergate and The Fappening: how Reddit’s algorithm, governance, and culture support toxic technocultures. New Media & Society19(3): 329–346.
35.
MatiasJN (2019) The civic labor of volunteer moderators online. Social Media + Society5(2): 2056305119836778.
36.
MatiasJNJohnsonABoeselWE, et al. (2015) Reporting, Reviewing, and Responding to Harassment on Twitter. SSRN Scholarly Paper No. ID 2602018. Available at: https://papers.ssrn.com/abstract=2602018
37.
MeltonAP (1995) Indigenous justice systems and tribal society. Indian tribal courts and justice: a symposium. Judicature79: 126–133.
38.
MungerK (2017) Tweetment effects on the tweeted: experimentally reducing racist harassment. Political Behavior39(3): 629–649.
39.
NelsonA (2016) The Social Life of DNA: Race, Reparations, and Reconciliation After the Genome. Boston, MA: Beacon Press.
40.
NussbaumMC (2009) Hiding from Humanity: Disgust, Shame, and the Law. Princeton, NJ: Princeton University Press.
41.
PaterJAKimMKMynattED, et al. (2016) Characterizations of online harassment: comparing policies across social media platforms. In: Proceedings of the 19th international conference on supporting group work, Sanibel Island, FL, 13–16 November, pp. 369–374. New York: The Association for Computing Machinery.
ScherSJDarleyJM (1997) How effective are the things people say to apologize? Effects of the realization of the apology speech act. Journal of Psycholinguistic Research26(1): 127–140.
47.
SeeringJWangTYoonJ, et al. (2019) Moderator engagement and community development in the age of algorithms. New Media & Society21(7): 1417–1443.
48.
SunshineJTylerTR (2003) The role of procedural justice and legitimacy in shaping public support for policing. Law & Society Review37(3): 513–548.
TylerTR (1994) Psychological models of the justice motive: antecedents of distributive and procedural justice. Journal of Personality and Social Psychology67(5): 850–863.
51.
WagatsumaHRosettA (1986) The implications of apology: Law and culture in Japan and the United States. Law & Sociology Review20: 461.
52.
WenzelMOkimotoTGFeatherNT, et al. (2008) Retributive and restorative justice. Law and Human Behavior32(5): 375–389.
53.
WohnDY (2019) Volunteer moderators in Twitch micro Communities: how they get involved, the roles they play, and the emotional labor they experience. In: Proceedings of the 2019 CHI conference on human factors in computing systems (CHI’19). ACM, New York, Paper 160, p. 13. Available at: https://dl.acm.org/doi/10.1145/3290605.3300390
54.
ZehrH (2015) The Little Book of Restorative Justice: Revised and Updated. New York: Simon & Schuster.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
