Abstract
As social media use proliferates, so does hateful language on such platforms among users. How might product teams be empowered to tackle hateful speech and promote inclusive language proactively among users? We developed an intervention that targets the organizational culture of tech companies and their product teams as a driver for change. More specifically, the intervention aimed to advance inclusive language practices within product teams, while introducing product development frameworks for advancing inclusive language and mitigating hateful language among social media product users. For this study, we partnered with a large U.S.-headquartered tech firm that owns several social media products to develop and pilot the intervention. The intervention was implemented with 238 participants—representing employees of the firm across several global locations, with particular representation from the United States—in seven workshops. Forty participants were tracked semilongitudinally across three surveys—prior to the intervention, immediately after, and 4 months after. Two participants participated in in-depth interviews 6 months after the intervention. Survey data were submitted to ordinal regression models that examined difference in workshop attendees’ confidence levels across the three time periods. Findings reveal positive impacts over time among product teams in regard to enhanced agency in advancing inclusive language and mitigating hateful speech among users. More specifically, confidence levels to guide one’s team in preventing or mitigating harmful language among product users significantly increased by 13.6% (p < .0001) from before the intervention to both postworkshop surveys. Confidence levels to guide one’s team in enhancing inclusive language among users significantly increased by 18.1% (p < .0001) pre- to postworkshop. Positive correlations are present between the measures. Lastly, qualitative responses in the surveys and interviews express appreciation of learning gained in the workshops, but also the need to continue such interventions as an effort to maintain critical understandings of inclusive language practices.
Introduction
As social media has proliferated, so too have issues including harmful and hateful language on such platforms among users. Efforts to tackle hateful speech on social media products are often technical and predominantly retroactive, responding to instances of hate speech after they have occurred. Responsibility for mitigating hateful speech also tends to be relegated to teams external to product, such as those in content integrity or content moderation. We ask: How might product teams be empowered to tackle hateful speech and promote inclusive language proactively among users? In response to this question, we developed an intervention that targets the organizational culture of tech companies and their product teams as a driver for change. More specifically, it advances inclusive language practices within product teams, while introducing product development frameworks for advancing inclusive language and mitigating hateful language among social media product users.
For this study, we partnered with a large U.S.-headquartered tech firm that owns several social media products to develop and pilot the intervention. Through workshops, the intervention sought to build an understanding of what inclusive language looks like, how to advance it interpersonally and in products, and establish a product lifecycle framework for proactively advancing inclusive language and mitigating hateful language among users. The intervention was implemented with 238 participants—representing employees of the firm across several global locations, with particular representation from the United States—in seven workshops. Data to track impacts included a preworkshop survey, immediate postworkshop survey, and 4-month postworkshop survey. Findings reveal positive impacts over time among product teams in regard to enhanced agency in advancing inclusive language and mitigating hateful speech among users. More specifically, confidence levels to guide one’s team in preventing or mitigating harmful language among product users increased by 13.6% from before to 4 months following the intervention, and confidence levels to guide one’s team in enhancing inclusive language among users increased by 18.1% in the same time interval. Meanwhile, we find moderate to strong positive correlations that illustrate the connection between one’s personal understandings of language, identity, and power with confidence and abilities to mitigate harmful language and advance inclusive language in products.
In this article, we begin with a background of existing literature on organizational culture impacting product development, an overview of why and how hate speech is proliferating in social media products, and how hate speech is being addressed—or not—in tech companies and product teams (next section). We then turn to information on the particular intervention (third section) and outline the methodology and data collection (fourth section) before presenting findings (fifth section) and a discussion (final section).
Background
Organizational Culture Impacts Product Development
The culture of an organization and team impacts how products are developed. Research reveals that organizational culture is a key element in enhancing or inhibiting product innovation (Naranjo Valencia et al., 2010). Researchers find that ad hoc cultures promote innovation, whereas hierarchical cultures impede it (Naranjo Valencia et al., 2010; Naranjo-Valencia et al., 2016). While organizational culture is known to impact product development, there is little literature that explores if and how efforts to enhance an inclusive organizational culture and practices impacts inclusive product development. While this research is lacking, there are illustrative examples demonstrating how organizational culture and leadership priorities impact product development and safety considerations. Take, for instance, Twitter (recently renamed to “X”) after Elon Musk’s acquisition. The change in leadership brought reductions of established work teams, lessened restrictions on content moderation, and resulted in broad organizational culture changes including a deprioritizing of diversity and inclusion efforts—reflected by the resignation of the Chief DEI Officer and reported shutting down of employee resource groups (ERGs) for female and Black employees (Green et al., 2022). Hickey et al. (2023) examine the levels of hate speech before and after Musk’s acquisition of the company, finding that hateful language practices increased dramatically after with no indication that hate speech presence would return to prior levels. Indeed, the occurrence of hate speech toward Black Americans increased over 200% and the occurrence towards gay men and Jews increased approximately 60%. With swift reductions in content moderation and changes in personnel, the social media platform has become a high-yielding locus for hate speech (Frenkel & Conger, 2022). As hate speech continues to be a focus of empirical study, the examination of whether and how organizational culture impacts product development and management, including implications of safer environments for social media users, is vital to understand.
There is various research that delves into diversity, equity, and inclusion (DEI) within organizations and the impacts of such DEI efforts. However, existing research tends to focus on the business case for DEI, namely, as a driver for innovation (Hewlett et al., 2013; Torchia et al., 2015), enhanced reputation, teamwork and productivity (Avery et al., 2012; Hewlett et al., 2013), talent retention (Ali et al., 2015; Downey et al., 2015; Drach-Zahavy & Trogan, 2013), and enhanced reputation (Gul et al., 2011; Shimen et al., 2016). While anecdotal information exists pointing to DEI efforts as impacting product development processes, particularly as an important ingredient for and within inclusive product development (McKinsey, 2022), there is limited academic research exploring this connection.
This research fills several important research gaps. First, it examines the impacts of an intervention that targeted product teams to proactively advance inclusive language and mitigate hateful speech among social media users. Second, by focusing on organizational practices that support inclusive language as a critical precursor to tackling hateful language and promoting inclusive language among users within products, it examines the connections between inclusive organizational culture and practices for equitable product development efforts. Additionally, it adds to the growing body of literature addressing the effects of recent DEI efforts. Empirical study addressing language use and the development of metalinguistic awareness—or the ability to think about, analyze, and manipulate language as an object of thought and influence (see Mertz & Yovel, 2009)—in the workplace and tech product development is scarce, particularly as it affects organizational culture and product development.
Why Focus on Hateful Speech in Social Media Products
With technologies such as social media, which aim to connect people globally and provide users with platforms on which they can express themselves, hateful speech can proliferate quickly and easily. While hate speech lacks a universal definition and is both fluid and context dependent, hate or hateful speech generally refers to forms of expression that incite violence, instill fear, insult a person or a group’s dignity, and/or threaten the well-being of an individual or community (Anderson & Barnes, 2022). Hateful speech can range from microaggressions to calls for violence against individuals or groups, and be expressed through text, audio, visuals, or any combination of these forms of media on various platforms. It can occur with different linguistic styles and be done subtly (Fortuna & Nunes, 2018). 1
There are several factors that exacerbate the proliferation of hateful speech on social media. Importantly, online communication allows for fewer nonverbal cues, greater anonymity, more opportunity to form new social ties and bolster weak ties, and wider dissemination of information (Lieberman & Schroeder, 2020). Compared to in-person or face-to-face communication, online communication can create distance and minimize space for empathy. Researchers find that social interactions via technology can reduce feelings of social connectedness (Kushlev et al., 2017). Through reduced social and spatial distance, individuals with extreme ideologies can find others that share their perspectives (Gerstenfeld et al., 2003). Meanwhile, the anonymity afforded by social media platforms allows people to hide or disguise their identities (Matamoros-Fernández & Farkas, 2021). This can result in a lack of accountability for users creating or propagating hateful content. Social media allows people to easily broadcast their messages to millions of people across the world, making it easier to amplify hateful messages. Relatedly, algorithms on social media platforms are designed to maximize engagement. Research finds that polarizing content usually results in higher levels of engagement, which means that such algorithms can often inadvertently promote hateful speech (Nicas, 2021).
Hateful speech can have immense harms for individuals, communities, and societies. Hateful speech can cause emotional and mental distress, negatively impacting the mental health and well-being of the specific individuals or communities experiencing hate speech; amplify harmful stereotypes and bias; enhance divisiveness and polarization; be used as a tool of oppression by those in power; and in more extreme cases, support or encourage acts of violence toward the target groups (Laub, 2019). Allport (1954) revealed that individuals with negative attitudes toward groups are likely to act on those negative attitudes “somehow, somewhere” and built a scale of acts of prejudice illustrating different degrees of acting out harmful attitudes. This scale begins with antilocution, or hate speech, which he defines as verbal remarks to others that explicitly express prejudice. The next levels of the scale includes avoidance, followed by discrimination, physical attack, and a final level of extermination (or genocide) (Allport, 1954). This matters in online contexts: Large frequencies of hateful speech in online communities can create a sense of a descriptive norm and advance derogatory perceptions of those in the outgroup (Bilewicz & Soral, 2020). Addressing hateful speech, or these lower levels of prejudice, can be particularly important (Windisch et al., 2022). Organizations are not exempt from the impacts of hate speech. For the businesses or platforms on which hate speech occurs or is allowed to proliferate, this can negatively impact brand reputation (Frenkel & Conger, 2022; Nicas, 2021); enhance risk (brand, financial, and regulatory); and be at odds with their purpose, vision, and principles.
How Is “Hate Speech” Being Addressed and Mitigated on social media products?
Solutions toward addressing hateful speech on social media products tend to be reactive, with a particular focus on content moderation and automated tools for hate speech detection. While important to identify instances of hateful speech, current content-moderating capabilities (led by human moderators and/or hate speech detection algorithms) have several limitations. Content moderation approaches tend to be reactive rather than proactive (Ullmann & Tomalin, 2020), which means that even when harmful content is removed, it may have already resulted in harm. Second, algorithms built to flag harmful content may fail to grasp underlying meanings or contexts. For example, they often fail to recognize whether a term is being used as a reclaimed means of empowerment or a slur, based on who is using it; therefore, they risk filtering out voices of marginalized groups (Davidson & Bhattacharya, 2020). In general, Natural Language Processing (NLP) tools tend to underperform for demographic groups whose language varieties are not well represented in data sets. A study analyzing five widely used speech recognition tools found that they misunderstood words spoken by Black people nearly twice as often as they misunderstood words spoken by White users, which is traced back to underrepresentation of African American English in the language data sets these AI systems learn from (Koenecke et al., 2020). Additionally, AI models that are built for harmful speech detection are primarily text-based, but hate speech manifests in different types of mediums—for example, text, image, video, and sound.
Other online interventions for reducing hateful speech focus on influencing people’s behaviors such as through community standards on online platforms, training courses, anti–hate speech campaigns, adding messages to hateful online comments that countered hateful or extremist content, and redirecting people to more credible sources. A systematic review of these types of interventions on social media platforms found that evidence is insufficient to determine effectiveness of the interventions to reduce hateful content online in terms of both creation and consumption (Windisch et al., 2022). Gaps remain in the literature to evaluate online behavioral interventions.
Importantly, addressing hateful speech in social media products at large tech companies tends to be a top-down approach, often delegated to teams outside of product development. This includes, for example, a policy team that defines hate speech and provides content guidelines and community standards, as well as teams focused on “content integrity” who are responsible for enforcing these norms at scale. It often also includes teams working on content moderation, including development of algorithms for hate speech detection and monitoring of instances of hate speech. Product teams therefore may not consider implications of their products or product features related to advancing hateful speech.
The Intervention: Targeting Organizational Culture as a Lever to Impact Product Development
Recognizing the limitations of current efforts to mitigate hateful speech on social media products, we asked: How might product teams be empowered to tackle hateful speech and promote inclusive language among users? How might enhanced awareness around the power of language and strategies for enhancing inclusive language (interpersonally and in products) enhance innovation in product development processes, including proactively tackling harmful language among users? To answer these questions, we developed and piloted a workshop intervention, called the Equitable Language Certificate, with a large, U.S.-headquartered tech company that develops and manages several social media products. The program sought to have participants understand what inclusive language is and why it is important; understand how to advance inclusive language in one’s own life, team, and products; and provide a framework for the product development lifecycle to advance inclusive language and mitigating harmful language in product.
The workshop content was built by a multidisciplinary team at the UC Berkeley Haas School of Business’ Center for Equity, Gender & Leadership, drawing from research around growth and leadership mindsets, language and power, and hateful speech, and designed to ensure an interactive learning experience. To establish a base level of understanding, the workshop began by exploring the tenets of Equity Fluent Leadership 2 and grounded participants in core concepts about language and power. A series of individual and small group exercises allowed participants to put good practices around interpersonal, day-to-day communications into play in real world contexts. The workshop then honed in on the role that language plays in product development, focusing on (1) communication to users and (2) between users on social media platforms. Participants reflected on why it is important to consider how users communicate on social media platforms, acknowledging a spectrum of harm that hateful speech can cause to individuals, communities, and society, while highlighting the company’s definition and approach toward combating hateful speech. Participants learned about the root causes of hateful speech, the complexity of identifying such language, why and how hateful communication can proliferate online, and considered the negative business impacts. An important tool introduced in this session is the Product Lifecycle Inclusive Language Framework, which includes key questions throughout the product lifecycle (from discovery and design to testing and maintenance) to advance inclusive language and prevent harmful speech in products. The framework was developed through a literature review of hate speech; a review of the product lifecycle at the particular company; interviews with product managers at the particular company; and a brainstorm session with faculty, staff, and MBA students at UC Berkeley’s Haas School of Business. It was tested and iterated with employees at the particular firm. Participants read through the framework questions and highlighted those that seemed particularly important for their work. A case study scenario, which drew from real world events, prompted participants to explore harmful language against refugees and immigrants on a social media platform, leverage the framework, and identify ways to ensure harmful language would not be perpetuated through a similar platform.
The workshop concluded with an action planning session, wherein each participant built out a personal action plan, identifying goals they would like to work on related to inclusive interpersonal language as well as language on products. Participants were also assigned partners to be “peer support buddies” and encouraged to set up regular check-ins for accountability purposes.
The workshop was designed for in-person or virtual implementation, with content spanning eight hours. In total, seven workshops were held over a period of seven months, between July 2022 to January 2023 with 238 participants based across the United States and Europe, Middle East, and Africa (EMEA). Five of these workshops were virtual, and two in-person (one in New York, one in London). Participants were invited to participate through announcements of the workshop on internal communication channels. Additional recruitment was done with support of leaders at the organization who encouraged attendance through email and internal channels. Participants self-selected to participate.
Methodology and Data Collection
To understand the effectiveness of the workshop intervention, participants were asked to complete three surveys: (1) a preworkshop survey as a diagnostic tool (hereby “pre survey”); (2) a postworkshop survey issued immediately after the workshop to gauge learning (hereby “post survey”); and (3) a survey 4 months after the workshop (hereby “4-month survey”). The same prompts were included in each survey to measure mean differences among participants. Box 1 outlines the prompts.
Overview of Survey Prompts.
Each prompt was followed by a 6-point Likert scale. The left-most point is “Disagree” and movement towards the right-most point indicates less disagreement/more agreement until reaching the polar opposite and right-most point of “Agree.” These points are converted into a numerical scale of one to six, whereby “Disagree” equals one (1) and “Agree,” six (6). The ratings of each scalar item are treated as an independent variable; thus, one ordinal model (Christensen, 2018) per each prompt was fit to the raw data from evaluative scales in R. All models were fit with the predictor of survey type to see how participants’ experiences with inclusive language as related to workplace environment and product may differentiate from prior to the workshop to after. Individual participant was included as a random effect. Correlation analyses were run to determine the relationship between participants’ confidence levels enhancing inclusive language in product development and direct communication. In these analyses, r is the correlation coefficient, statistical measure that determines the strength and direction of a given relationship. This coefficient is indicated by a positive or negative number between −1 and 1 (respectively), with r = 0 indicating no correlation.
In the 4-month workshop surveys, participants were also asked to comment on the personal action plans that they completed in the last phase of the workshop. This was an open-ended question wherein they were asked to explore and reflect upon the goals they set for themselves/teams regarding equitable and precise language use in products. To augment the open-ended question and after the 4-month survey, two participants participated in follow-up interviews over Zoom to better understand the effects of the workshop on the implementation of inclusive language practices in products as motivation to reduce harmful language practices among users. The interviews lasted approximately 30 minutes. The questions posed to participants in the interviews can be found in Appendix A.
Among all the participants that completed the entire workshop, 40 participants completed the three surveys (see Table 1 for organization details of the participants and Table 2 for demographic information). Of these participants, two took part in an in-depth follow-up interview. All participants were employed at the tech company.
Organization Details of Participants (n = 40).
Job category details: Engineering individual contributors (software, security, data scientist, product strategy lead, data protection associate); Design / User experience (content designer, product designer, UX, technical writer, animator); People managers (engineering manager, content design manager, manager, marketing manager); Program managers (localization program manager, program manager, DPM, TPM, program coordinator); Human resources / Business functions (head of DEI content, communications manager, strategist, and learning architect).
Demographic details of participants (n = 40).
Multiple-choice options and fill-in option.
Open text for self-identification.
Findings
Comparable Scales Across the Three Surveys
The first two questions dealt directly with inclusive language and harmful language on social media products. The ordinal model fit to the evaluative scale in Q1 (I feel confident guiding my team in considering how to prevent or mitigate harmful language among users on our products) demonstrates a statistically significant 13.6% increase in participants’ confidence related to the reduction of harmful language practices in product development 4 months after the training compared to the baseline (p < .003) (see Figure 1). This was a slight reduction from the confidence measured immediately after the intervention. The ordinal model fit to the evaluative scale in Q2 (I feel confident guiding my team in considering how to enhance inclusive language among users on products.) revealed a statistically significant 18.1% increase 4 months after the training as compared to the diagnostic assessment (i.e., preworkshop survey). This level of confidence was also measured immediately after the workshop and was maintained 4 months after (p < .0001 for both; see Figure 2). No statistically significant drop in confidence levels occurred from the post to the 4-month period for either evaluative scale, indicating that the skill set participants gained from the workshop directly related to product development was useful in the mitigation of harmful language practices in product.
Mean ratings toward Q1 (related to mitigating harmful language among product users and advancing inclusive language), conditioned by survey type (pre, post, 4-month). p-values less than 0.01 are flagged with 2 stars (**) and (***) if less than 0.001.
Mean ratings toward Q2 (related to mitigating harmful language among product users and advancing inclusive language), conditioned by survey type (pre, post, 4-month). p-values less than 0.01 are flagged with 2 stars (**) and (***) if less than 0.001.
In addition to measuring direct impacts on confidence and agency related to mitigating harmful language and advancing inclusive language among users, we measured other aspects of inclusive product design, specifically regarding language used to communicate to users on platforms. The ordinal model fit to the evaluative scale in Q3 (I feel confident communicating precisely and inclusively to users on products [e.g., in content design.]) reveals a steady increase from the preworkshop survey to the 4-month period after completing the workshop. Both the post and 4-month surveys demonstrate that participants are increasingly more confident communicating precisely and inclusively to users than from before the workshop (p < .003 and p < .001, respectively; see Figure 3). Though not statistically significant, an increase in confidence is also evidenced from the period immediately after the workshop to 4 months later.
Mean ratings of evaluative scale in Q3 (related to content design), conditioned by survey type (pre, post, 4-month). p-values less than 0.01 are flagged with 2 stars (**) and (***) if less than 0.001.
The last set of questions includes a reflection on personal understandings of equitable and inclusive language and confidence in communicating inclusively in one’s own life. These findings help us to explore and understand correlations between the personal/interpersonal and product development. The ordinal model modeled to the evaluative scale in Q4 (I feel like I have a good understanding of what it means to communicate equitably and inclusively.) demonstrates a significant increase in participants’ comprehension of equitable and inclusive language immediately and 4 months after completing the workshop (p < .0001 for both; see Figure 4).
Mean ratings of evaluative scale in Q4 (related to generalized inclusive language practices), conditioned by survey type (pre, post, 4-month). p-values less than 0.01 are flagged with 2 stars (**) and (***) if less than 0.001.
While still an increase from the pre-survey, comprehension levels decreased from the immediate post to 4-month periods (p < .001). The ordinal model fit to the evaluative scale in Q5 (I feel confident in my knowledge and abilities to communicate in ways that are inclusive.) exhibits an increase in confidence from the preworkshop period to the postworkshop and 4-month periods (p < .0001 for both; see Figure 5). These results indicate that while workshop attendees feel confident in their learning of and increase in knowledge of inclusive and equitable language, they may feel less confident in how to put this learning into action if not given regular opportunities for practical application of equitable language practices. These results unearth the need for continuing education in this area, as language is dynamic and evolving.
Mean ratings of evaluative scale in Q5 (related to generalized inclusive language practices), conditioned by survey type (pre, post, 4-month). p-values less than 0.01 are flagged with 2 stars (**) and (***) if less than 0.001.
The last set of questions deal with talking about identity and power. The ordinal model modeled to the evaluative scale in Q6 (I feel confident talking about identity in ways that are inclusive.) demonstrates a significant increase in participants’ confidence in how to be inclusive when talking about identity immediately after the workshop and 4 months later (p < .0001 for both; see Figure 6). Similar to prior questions, their confidence levels decreased from the immediate post to 4-month periods (p < .01). The ordinal model fit to the evaluative scale in Q7 (I feel confident talking with colleagues about topics related to identity, power, and privilege.) exhibits the same pattern as Q6, as confidence increased from the pre- to postworkshop period (p < .0001; see Figure 7) and then decreased again from the postworkshop period to 4 months later (p < .0001).
Connections between participants’ confidence levels in enhancing inclusive communication and mitigating harmful language in product development
We ran correlation analyses with the quantitative data from the 4-month survey (see above) to understand the relationship between confidence levels of the workshop participants. We focus on the 4-month survey to gauge participants’ sustained confidence levels following the intervention. Being that our focus is on mitigating harmful language on social media products, we examine the potential presence of correlations between confidence levels with regard to mitigating harmful language in products (Q1 and Q2) with communicating inclusively to users (Q3), generalized inclusive language use (Q4 and Q5), and inclusive language relevant to identity and society (Q6 and Q7). These combinations resulted in 10 analyses in R (R Core Team, 2018) using the Pearson formula (see Table 3).
Mean ratings of evaluative scale in Q6 (talking about identity and power), conditioned by survey type (pre, post, and 4-month). p-values less than 0.01 are flagged with 2 stars (**) and (***) if less than 0.001.
Mean ratings of evaluative scale in Q7 (talking about identity and power), conditioned by survey type (pre, post, and 4-month). p-values less than 0.01 are flagged with 2 stars (**) and (***) if less than 0.001.
Correlation Analyses of Confidence Levels Between Enhancing Inclusive Language in Product and Direct Communication.
We found significant moderate to strong positive correlations across all confidence level comparisons. Enhanced confidence in guiding one’s team in considering how to prevent or mitigate harmful language among users on products (Q1) is positively correlated with feeling confident communicating precisely and inclusively to users on products (Q3; r = 0.49), having a good understanding of what it means to communicate equitably and inclusively (Q4; r = 0.46), feeling confident in one’s knowledge and abilities to communicate in ways that are inclusive (Q5; r = 0.59), feeling confident about talking about identity in ways that are inclusive (Q6; r = 0.68), and feeling confident talking with colleagues about topics related to identity, power, and privilege (Q7; r = 0.61). Enhanced confidence in guiding one’s team in considering how to enhance inclusive language among users on products (Q2) is similarly positively correlated with all other self-evaluations of confidence and understanding (r = 0.46; r = 0.45; r = 0.54; r = 0.67; r = 0.7, respectively). By social sciences standards (see Evans, 1996), which maintain that correlations between (–/+) 0.40 and 0.59 are moderate and (–/+) 0.60 and 0.79 are strong, the correlations in Table 3 illustrate the connection between an individual’s growing understanding of language, identity, and power with their increased confidence and abilities to mitigate harmful language and advance inclusive language in products.
Qualitative Feedback From 4-Month Survey
In the qualitative feedback regarding progress of the personal action plans, several participants expressed goals that generally fit into two categories: (1) implementing precise and inclusive language in products and (2) sharing workshop learnings and experiences with the work community. Tables 4 and 5 represent a sample of participants’ actionable goals related to these two themes, respectively. Participants expressed how they are generally more conscientious of how they are considering language use in the process of building products (nos. 1 and 3) as well as paying closer attention in particular to issues of accessibility (nos. 2 and 4). Likewise, there is an overarching theme of creating and normalizing the use of inclusive language practices as a baseline (see no. 4).
Progress on Personal Action Goals as Related to Product.
Progress on Personal Action Goals as Related to Teams/Workplace.
Table 5 offers participants’ actionable goals as related to sharing workshop experiences with others. All of the participants expressed passing information and training to their teams. One is establishing a framework for their team (no. 5) and others an accessible published work to provide to those who were unable to participate in the workshops (no. 9). Relatedly, another shared to their immediate team but also on social media (no. 8), also stating that it “felt good” to pass this information along to others outside of the organization. Lastly, one participant also expressed the awkwardness but liveliness of discussions about language inclusivity, alluding to the opportunities that such conversations can provide employees when leaders feel confident in the topics of discussion.
Follow-up Interviews
Two in-depth additional interviews were conducted with workshop participants who completed all three surveys. One of the interviewees is a Localization Program Manager (LPM) working in translation, and the other is a Technical Writer (TW). In Q1 (Prior to the ELC workshop, did your team consider how to mitigate hateful speech and advance inclusive language among users within products? Why or why not?), the LPM said that prior to workshop learning, their team was removing words like master from product language. They also expressed difficulty in translating inclusive language practices that had been implemented in English, particularly as it relates to gender identity expressions, which can be difficult in languages that have more rigid grammatical expressions of gender (e.g., male/female). The TW expressed that prior to the workshop they and their team were working on implementing more inclusive language as it related to race, but had a more difficult time doing this with language related to accessibility.
As for Q2 (Did the ELC workshop help you learn new strategies to mitigate hateful speech and/or advance inclusive language amongst users, and if so, what were these?), the TW, who previously supported teams in identifying and mitigating issues of fairness in products, highlighted the importance of going back to as early a stage as possible in topics like fairness and how asking questions earlier on can reduce the incidence of issues at a later stage. The LPM expressed that it was useful to see how older code was evolving to be more inclusive, though they noted more mitigation of gender-exclusive language than on accessibility.
Both the LPM and TW expressed that they and their teams are more aware of how to implement inclusive language in products (see Q3: Since taking the ELC workshop . . . has your team discussed how to mitigate hateful speech and advance inclusive language amongst users within products?). The LPM wrote an internal post about the workshop highlighting findings and how they are incorporating them to share with their team. However, the LPM also noted the need for Managers to take more responsibility for the normalization of inclusive language practices (as in, lead discussions on and implementation of inclusive language). They also noted that while many team members were already trying to prioritize inclusive language in their work, the workshop experience and certificate of completion legitimized their practices, giving them credibility to implement inclusive language and disseminate learned knowledge with confidence. The workshop allowed them to ground their viewpoints and priorities for inclusion and reference back to it to “back up [their perspectives], not just ‘I believe’ or ‘I think.’”
In response to Q4 (Are frameworks like the inclusive language framework that was discussed and shared in the workshop helpful in considering ways to promote inclusive language amongst users within products? Why or why not?), both interviewees maintain that this component of the workshop was helpful, but both expressed difficulty in implementing practices in the product lifecycle, partly because of the limitations of their roles but also the current tech job climate. The LPM specifically pointed at mass layoffs at the company in 2022 and 2023 that made it difficult to implement change. They expressed that employees were under pressure to perform with less resources, and if Managers do not see inclusive language as a goal, it is much harder for individuals or smaller teams to find an “excuse” to work on something that “doesn’t matter” for efficiency or profit.
The final question probed both interviewees to contemplate if equitable language workshops should continue in the future (Q5: Do you think workshops like the ELC that focus on language inclusivity amongst users and mitigating hate speech should continue? Why or why not?). Both affirmed the need to continue learning about inclusive language as necessary for product development. The LPM suggested that members of product teams attend workshops together in person, and they emphasized the value in continuing workshops like this and the potential to “normalize learning about this [inclusive language implementation].” The TW highlighted that although they were aware of some of the information shared in the workshop, it was good to have the reminder and be made aware of pieces to keep in mind when designing products. They also valued the component around language generally and the power of language, as well as how words can be used and misinterpreted.
Discussion
This study examined the effects of an equitable language workshop on the confidence and abilities of employees at a large tech firm to advance inclusive language and mitigate harmful language in social media products. Comparative quantitative measurements across the three survey instruments reveal important findings that support the need for equitable language workshops in social media product development. First, participating in the workshop resulted in a sustained increase in confidence and agency among product teams to mitigate harmful language and advance inclusive language among users on social media products. Second, participating in the workshop also resulted in sustained increases in one’s personal confidence and abilities to communicate inclusively to others. We find significant correlations between confidence in guiding one’s team to mitigate harmful language on social media products with personal understanding on topics of identity, power, and interpersonal inclusive communication. Lastly, qualitative feedback in the 4-month survey and follow-up interviews reveal that many participants shared learnings broadly illustrating what we note as a “waterfall effect,” while participants also found the workshop to be useful in both how they interact with others and making products more inclusive. This suggests that workshops and efforts to enhance knowledge as it relates to language and DEI can be levers to support workplaces that are more equipped to mitigate harmful speech and advance inclusive language in social media products. Additionally, workshops that center inclusive language practices should be integral to continuing education to make workplaces and products more accessible and equitable.
Sustained Increases in Confidence and Agency Among Product Teams to Mitigate Harmful Language and Advance Inclusive Language
Results across the three surveys (pre, post, and 4-month) indicate significant increases in confidence levels of guiding one’s team to mitigate harmful language and advance inclusive language in social media products. From baseline to 4 months after the workshop intervention, participants exhibited a 13.6% increase in their confidence levels in guiding one’s team in considering how to prevent or mitigate harmful language among users on products (Q1) and an 18.1% increase in confidence to guide one’s team in considering how to enhance inclusive language among users on products (Q2). Confidence levels measured immediately after the workshop were maintained 4 months post for Q2, and although decreasing slightly for Q1 (−4.5%), levels still remained higher (13.6%) to confidence levels prior to completing the workshop. These findings indicate the intervention is effective in the short and potentially long term, although sustained reminders and support can be valuable to maintain confidence levels.
Sustained Increases in Confidence, Knowledge, and Abilities to Communicate Inclusively to Others
Participants also demonstrated increased confidence levels, knowledge, and abilities to communicate inclusively to others. Confidence levels to communicate equitably and inclusively to users rose 18.2% from prior to the workshop to 4 months post. Reported understanding of what it means to communicate equitably and inclusively rose 17.8% from prior to the workshop to 4 months post, and feeling confident in one’s knowledge and abilities to communicate inclusively rose 18.6% in the same time period. These are lower levels than what were measured immediately after the workshop (reported increases of 26.7% and 23.5%, respectively, from baseline) representing a significant decrease from the immediate postworkshop period to the 4-month period (−8.9% and −4.9%, respectively). Participants’ confidence in talking about identity in ways that are inclusively increased 20% from baseline to 4 months post the workshop, while feeling confident talking with colleagues about topics related to identity, power, and privilege increased 16.3% in the same window. Similar to questions Q4 and Q5, there were higher increases immediately following the workshop (30% and 18.6%, respectively) indicating that while still higher than baseline, some of the confidence, knowledge, and abilities dipped several months after the workshop. The findings from Q4 to Q7 elucidate how maintaining metalinguistic skills—particularly as [inclusive] language changes and evolves (at times rapidly)—can be difficult without explicit continuing education and practice-building awareness.
Strong Relationship Between Personal Confidence to Mitigate Harmful Language on Product With Personal Inclusive Language Knowledge and Abilities
Correlation analyses examined the relationships between confidence levels with respect to mitigating harmful language in product and personal knowledge on inclusive language and interpersonal communication. That is, as confidence levels increased in one, they also did in the other. In particular, we found strong positive correlations between enhanced confidence in guiding one’s team in considering how to prevent or mitigate harmful language among users on products (Q1) with feeling confident about talking about identity in ways that are inclusive (Q6; r = 0.68); feeling confident talking with colleagues about topics related to identity, power, and privilege (Q7; r = 0.61); and feeling confident in one’s knowledge and abilities to communicate in ways that are inclusive (Q5; r = 0.59). These correlations illustrate the connection between one’s personal understandings of language, identity, and power with confidence and abilities to mitigate harmful language and advance inclusive language in products. As such, metalinguistic awareness cannot be compartmentalized to a specific task, as there may be mutual benefits to understanding how to implement inclusive language practices in daily communication as well as product development.
Waterfall Effect and Lasting Impacts for Inclusive Language in Product and in One’s Own Life
The qualitative findings from the 4-month survey and follow-up interviews reveal a waterfall effect in which participants shared learnings in the organization broadly as well as additional lasting impacts. In addition to updating their own day-to-day language use, various participants shared key takeaways with the organization and/or in their teams to spur further conversation and ongoing learnings, such as through organizing a live walk through the workshop content, writing blog posts about inclusive language practices for colleagues, developing and establishing frameworks for inclusive language use for their team, and more. Some participants noted that the workshop content informed how they crafted user-facing language, including updating demographic terminology. Some participants also reported that their ongoing meetings with their “peer support buddy” helped them stay on track with their goals. This intervention has enhanced confidence and agency, particularly through providing resources, raising awareness, and giving credibility to the internal DEI champions. Those individuals can then refer back to workshops such as this conducted with respected organizations to ground their own perspectives and normalizing inclusive language conversations and efforts within teams and products. However, many of the participants also expressed the need for team efforts, particularly coming from the top (i.e., managers) to see inclusive language prioritized in workplace and product. As expressed in a follow-up interview, DEI efforts—broadly and specifically construed—can fall at the wayside in the midst of mass layoffs and budget cutting, as “extra” efforts will continue as concerns of individuals (i.e., bottom-up), not necessarily teams or organizations.
Conclusion
This study examined the positive outcomes of an intervention within a large U.S. tech company on enhancing the confidence and agency of teams to mitigate harmful language proactively in social media products. The intervention sought to enhance understandings of inclusive language and provide participants with a framework to proactively mitigate harmful language and advance inclusive language in social media products. Although the study did not track incidence of hate speech on the particular social media platforms of the tech company, the study tracked increases in the confidence and agency of product teams to mitigate harmful language proactively on social media products. This is important as current efforts to tackle hate speech are almost exclusively focused retroactively on content moderation, as opposed to focusing on organizational and product team culture and priorities. Given the nature of hate speech as context dependent and evolving, this retroactive approach to tackling hate speech will always be flawed. Rather, this study illustrates the immense potential in reimagining interventions tackling hate speech by examining the product development process itself and increasing the agency of product teams to consider harmful language and promote inclusive language among users in the product lifecycle. Ultimately, technologies and the tools to moderate them are developed by certain people with different perspectives and priorities that are also influenced by the organization and its leadership. The study’s findings are a rare opportunity to highlight an intervention conducted with a large tech company to explore organizational approaches for more inclusive and equitable technology development.
There were several important limitations to the study. First, the number of participants that both attended the workshops and completed all of the surveys (n = 40) was only 16% of the total number of people that participated in the workshop (n = 238). During the period in which these workshops were offered, the tech industry as a whole was going through a series of major layoffs, driven by several factors including anticipation of an economic downturn, slow growth in the period following the pandemic, and pressure from investors. The intervention was carried out during two major rounds of layoffs at the particular tech company. The first layoff round occurred during a session of our Equitable Language Certificate workshop and led us to reschedule the second half of that workshop, splitting up the content for that cohort across several weeks. The impacts of the layoffs were felt across multiple cohorts of participants. These layoffs impacted the potential effects of the organization including less time for learning and development, while also resulting in greater pressure to focus on core business needs. The layoffs also affected our ability to collect 4-month post-surveys and conduct follow-up interviews. An important additional limitation regarding recruiting more participants is linked to the time commitment of the intervention. Second, participants self-selected the workshop; thus, they were already motivated to learn more about inclusive language use and generally seemed to already have an established baseline knowledge of DEI initiatives. If this workshop were required for all employees, different results may have emerged. In terms of the survey design, self-reflective responses via explicit direct questioning is only one way to examine a person’s perception of learning and environments; it is not always indicative of how the individual will behave or more implicit biases that people hold, which may differ. Lastly, an important caveat in this research is that we did not track hate speech occurrences in the social media products of the company. Rather, this research is focused on understanding levels of confidence and agency of product teams to mitigate harmful language and advance inclusive language among users.
The findings of this study highlight not only the potential of interventions to enhance confidence and agency of product teams in proactively tackling hateful speech, but also the interconnectedness between DEI efforts in the workplace and in more inclusive products. The study found that broader individual understandings of language, identity, and power are correlated with heightened individual confidence and agency to mitigate harmful language in product. This reveals an important connection between DEI-related workshops with products. Although these types of interventions do not change social media products overnight, they can plant seeds for sustained motivation, ongoing problem solving, and updated organizational practices within tech companies that enable innovation for inclusive and equitable product development. Future research can further explore interventions with product teams including trainings, resources, and tools to help them proactively consider how to mitigate hate speech on social media products and advance inclusive language. Research can also continue to explore linkages between inclusive organizational culture and inclusive products.
Footnotes
Appendix A: Follow-up In-depth Interview Questions
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This intervention was funded by the large, global, U.S.-headquartered tech company at which it was carried out.
