Sage Journals: Discover world-class research

Abstract

In the past decade, social networking sites have become central forums for public discourse and political engagement. Of particular interest is the role that Twitter plays in the facilitation of political discourse. To this end, the existing literature argues that a healthy political discussion space is key to maintaining a trusting and robust democratic society. Using Suler’s online disinhibition effect as a theoretical orientation, this study seeks to address the extent of incivility on Twitter in discourse regarding the top three 2020 Democratic primary candidates. A total corpus of 18,237,296 tweets was analyzed in an effort to assess the extent to which incivility dominated Twitter discourse surrounding these candidates. Our results reveal that tweets that mention Senator Elizabeth Warren were associated with higher levels of uncivil discourse than tweets that mentioned Senator Bernie Sanders and former Vice President Joe Biden. Interestingly, there does not appear to be a relationship with anonymity and incivility, as uncivil tweets were just as likely to originate from tweets that identified users’ names as they were to originate from anonymous or pseudonymous accounts. Finally, our findings provide evidence that certain policy issues are more closely related to uncivil discourse than others. Through the use of k-means clustering, our findings illustrate that the issue of gun control and immigration is closely related with mentions of Warren and fiscal policy with Sanders; however, we did not find any policy keywords linked to Biden.

Keywords

incivility twitter anonymity online disinhibition effect

In the past decade, social networking sites have become central forums for public discourse and political engagement. Among the leading social media platforms, Twitter has garnered particular attention within the political communication literature due to its popularity, size, and accessibility for research. Specifically, the platform has few in-network exposure limitations, which allows users to freely express and disseminate their views, enabling the widespread proliferation of opinion (Park, 2013).

As much as the open nature of Twitter has created a vibrant online space for political discourse, the platform has also fostered a culture of hostility and incivility—users insulting or attacking others with seemingly little fear of punishment or retaliation (Groshek & Cutino, 2016). Incivility, as defined in the existing literature, refers to rude or offensive speech that impedes productive, democratic dialogue (Anderson et al., 2014). Online incivility has the potential to spiral out of control and lead to severe offline consequences, including hate crimes and other forms of violence (Siegel, 2020). Understanding this phenomenon is particularly vital within the realm of political discourse and decision-making in election cycles, as it is difficult to distinguish the line between emotionally driven, personal attacks versus rationally driven, constructive debates. However, while there have been a myriad of studies dedicated to better understanding the presence of incivility in online conversations, there has yet to be a comprehensive analysis of what factors promote uncivil spaces online. As such, the goal of this study seeks to elucidate the various factors that may promulgate uncivil Twitter discourse.

A healthy political discussion space, in which ideas are able to be discussed freely without fears of repercussion, is key to maintaining a trusting and robust democratic society (Anderson et al., 2014; Papacharissi, 2004; Wang, 2020). As such, the quality of public political participation is largely dependent on the quality of collective deliberation (Page & Shapiro, 1992). To this end, it has been argued that incivility has the potential to stifle democratic discourse and cause adverse effects within the political sphere, such as altering the public’s perceptions and opinion formation (Anderson et al., 2014).

Given the negative societal implications of online incivility, specifically its ability to impede productive democratic discourse (Anderson et al., 2014; Wang, 2020), we argue that there is great need to continue studying this phenomenon. Considering the tenor of online conversations over the course of the last decade, as well as the tendency of online uncivil political discussions to bleed into offline spaces, we argue that it is important to better understand the various factors that can contribute to online incivility. Using Suler’s (2004) online disinhibition effect as a theoretical orientation, this study seeks to address the extent of incivility on Twitter in discourse regarding the top three 2020 Democratic primary candidates to illustrate the extent to which uncivil discourse was present in political discussions. Specifically addressing the role of candidate gender as well as the presence of bots in online conversations, this study seeks to build upon the existing literature pertinent to the online disinhibition effect, to better understand the myriad of factors that drive online incivility, as well as the specific policy issues that garner the greatest levels of uncivil engagement.

Literature Review

Overview of Incivility

Online incivility is a growing concern among the American public. According to a recent survey, 68% of those polled identified online incivility as a “major problem” in the United States (Weber Shandwick, 2019). Moreover, nearly 90% of survey respondents identified significant consequences from online incivility, including “cyberbullying, harassment, violence and hate crimes, intimidation and threats, intolerance, and people feeling less safe in public places” (Weber Shandwick, 2019, p. 3).

Incivility refers to rude or offensive speech that impedes productive, democratic dialogue, as defined by Anderson et al. (2014). With the introduction of social media, the concept of incivility has expanded significantly within communication research (see e.g., Borah, 2014; Groshek & Cutino, 2016; Lee et al., 2019). This strand of literature has been studied fairly rigorously to date, with a particular emphasis on the impact of incivility on online discourse (Groshek & Cutino, 2016; Megarry, 2014), civic engagement and political polarization (Hwang et al., 2014; Lee et al., 2019).

Recent research indicates several potential implications for the spread of online incivility. In one analysis, Lee et al. (2019) found that online incivility often grew as the volume of political discussions increased. The authors also found that incivility led to higher levels of polarization. Based on these findings, the authors assert that incivility may serve as a mediating force between political discourse and polarization. Regarding political polarization and incivility, Hwang et al.’s (2014) study revealed that while uncivil discourse didn’t link to attitude polarization, perceived polarization of the public and lower expectations about public deliberation were significantly affected outcomes. Similarly, Borah (2014) documented that uncivil comments associated with news articles were able to influence readers’ overall perceptions of the articles. In a similar vein, Anderson et al. (2018) identified that perceptions of bias are often greatest after audiences are exposed to incivility—further illustrating the detrimental effects of incivility in information processing.

Furthermore, Rossini (2020) adopted a more granular approach to understanding incivility and examined expressions surrounding intolerance. The study’s results indicated that there were differences between uncivil and intolerant discourse. While incivility often occurred in a heterogeneous setting of different opinions and perspectives, intolerance was more likely to come up in a homogeneous, like-minded environment (Rossini, 2020). This distinction further explains the varying level of harm that online conversations can bring to the state of democracy and to the wellbeing of a civil society.

Considering the existing literature pertaining to online incivility, our research builds off previous work by Groshek and Cutino (2016) in which they used a big data analysis of nearly 2.3 million tweets to operationalize incivility through the presence of five characteristics: (a) personal or inflammatory attacks; (b) threats; (c) vulgarities, abusive, or foul language; (d) xenophobic or other hateful language or expressions; (e) epithets or ethnic slurs, sentiments that are racist or bigoted, and/or disparaging on the basis of race/ethnicity or that assign stereotypes. Through a quantitative content analysis, the authors present interesting findings regarding the presence of incivility in online discourse. Namely, their research asserts that specific dialogic features of Twitter enable the spread of incivility, such as direct mentions and retweets (Groshek & Cutino, 2016). Considering the existing literature, this study offers the following research question in an attempt to illuminate the prevalence of incivility on Twitter pertaining to the top three candidates at the time of data collection:

RQ1: How prevalent is incivility in tweets about the 2020 Democratic primary campaign?

Online Disinhibition Effect

There has been a considerable amount of research dedicated to the impact of online settings on user behavior (Davis, 2009; Groshek & Cutino, 2016; Stromer-Galley, 2002; Suler, 2004). In the early 2000s, when the internet was growing in popularity among more mainstream audiences, Stromer-Galley (2002) observed that the absence of nonverbal cues in online settings lead to “lowered senses of social presence and the heightened sense of anonymity” (p. 35). With the introduction of social media, the existing literature demonstrates how this phenomenon has only been augmented (Rheault et al., 2019; Tromble & Koole, 2020).

This study is based on the online disinhibition theoretical framework conceptualized by Suler (2004), who posited that people tend to behave and communicate differently in cyberspace than they would in real-life. The online disinhibition effect can occur in two ways: positively (benign) or negatively (toxic). Suler’s (2004) framework identifies six key factors that contribute to disinhibition in both its forms. One of the principal factors of the framework that this study will focus on is anonymity, which occurs when individuals are able to separate their online actions from their actual personas.

To this end, there is ample evidence provided throughout the existing literature that further details the relationship between anonymity and incivility (Phillips & Milner, 2017; Suler, 2004; Wang, 2020). The literature postulates that anonymity is perhaps the most relevant affordance that can explain online incivility (Coe et al., 2014; Rossini, 2020). Wang (2020) builds upon this argument, and asserts that anonymity often motivates users to act in ways they may not otherwise, such as being more negative or uncivil in their interactions. This affordance of online communication often enables individuals to feel less vulnerable. Specifically, work from Nithyanand et al. (2017) reveals that the most offensive and uncivil political discussions on Reddit were often initiated and spread by pseudonymous accounts. Similarly, Phillips, and Milner (2017) illustrate how on Twitter, which the authors describe as “free-wheeling,” there is no policy against the creation of anonymous accounts. This often leads to a platform culture of satirical and ambivalent discourse.

Considering the anonymity of Twitter, it is likely users may feel more inclined to engage in uncivil behavior. In this article, we operationalize anonymity into two avenues. The first consists of individuals masking their identity by distorting their username. Based on the existing literature (Nithyanand et al., 2017; Phillips & Milner, 2017; Suler, 2004), we posit the following hypothesis:

H1: Tweets that originate from anonymous or pseudonymous accounts will be correlated with higher levels of incivility.

The second consists of the existence of bots, which are algorithmic actors that not only mimic, but also are known to falsify or misconstrue their identity in various ways, which will be discussed in the following section.

The Role of Bots on Twitter

As indicated by Coleman (2018), bots are automated computer programs that operate social media accounts to comment, reply, share or even create their own posts (p. 120). It is often difficult to distinguish social bots from human users, as these bots are designed to interact with human users through imitating and purporting to be real people as opposed to robots. Edwards et al. (2014) found that there were no significant differences in perceptions of credibility—including user competence and character—as well as intention to interact between human agents and Twitter bots. As a result, social bots may alter social media users’ impression on the magnitude of a given issue or argument, artificially enlarging the effects of certain opinions. Furthermore, we expand on the concept of anonymity in the online disinhibition theory by including the role of algorithmic actors—in our case, bots.

Following the 2016 U.S. presidential election, there has been growing interest in both industry and academia to better understand the role that bot accounts play in driving online discourses (Albadi et al., 2019; el Hjouji et al., 2018; Liu, 2019). As demonstrated throughout the existing literature, Twitter is inundated with bots, some of which are designed to distort reality (Liu, 2019), instigate political feuds, and spread misinformation and hateful rhetoric (Albadi et al., 2019). Along this line of research, Yuan et al. (2019) argue that social media bots can impede online discourse by posting a substantial number of automated messages and inundating online discourse. Findings from el Hjouji et al. (2018) echo this argument, as the authors illustrate that bots are able to produce a significant shift in opinion on Twitter. In an analysis of Twitter conversations on the 2016 U.S. presidential hopefuls, Hillary Clinton and Donald Trump, the authors found that bot accounts posted 100 times more frequently than human accounts—further illustrating the sheer magnitude of posts that originate from bot accounts.

In addition, there is ample empirical evidence that illustrates the deleterious impact that bots can have on social media discourse. Albadi et al. (2019) argue that bots often are able to spur political arguments, spread misinformation and propagate hateful rhetoric on Twitter. Similarly, Broniatowski et al. (2018) found that bots often are able to erode public consensus and promote harmful misinformation, such as anti-vaccination propaganda. The effects of bots pose serious implications, as work from Schuchard et al. (2019) indicates that bots are hyper-social accounts that display a disproportionately high level of structural network influence online. Thus, this implies that the online reach of bots spans far beyond the reach of an average human user (Schuchard et al., 2019).

As demonstrated, the existing literature is rife with indications that unidentifiable bots can impart deleterious effects on users, particularly those who turn to social media for news-gathering purposes. With these effects in mind, this study analyzes the content and tenor of the conversation facilitated by bot accounts and hope to offer valuable insight regarding the role that bots may have played in instigating online incivility through their algorithmically-driven anonymous identity. To this end, we offer the following research question:

RQ2: Within uncivil conversations surrounding the 2020 U.S. presidential primary candidates, what topics were being circulated by bot accounts?

Policy Issues Within Uncivil Tweets

Drawing upon the online disinhibition effect, as well as the existing literature surrounding the constructs of incivility and social media use, this study seeks to analyze the factors that stimulate online uncivil interactions, specifically within the realm of political discourse leading up to the 2020 U.S. presidential election. As offered by Rains et al. (2017), online incivility may serve as a form of identity performance. Their study analyzed the relationship between political ideology and the prevalence of incivility in newspaper discussion forums and found that audience members who had more extreme evaluations of uncivil comments were predominantly made by partisans rather than non-partisans. Our study seeks to expand upon this work by analyzing whether there is a relationship between specific policy issues and incivility. Thus, the following research question is proposed:

RQ3: Are specific public policy topics associated with uncivil communication?

Incivility and Gender

Another variable that has been shown to exacerbate online incivility is user gender. In the months leading up to the 2020 U.S. Democratic primary, politics-related incivility on Twitter often made headlines—particularly in coverage regarding Senator Bernie Sanders (Naranjo, 2020). In February 2020, Senator Elizabeth Warren criticized the online behavior of Sanders’ supporters when they “viciously attacked” women members of a culinary union who stood by Warren’s health care plan approach (Vitali & Roecker, 2020). The Sanders campaign was no stranger to this controversy; the campaign faced harsh criticism for the uncivil and aggressive behavior of its online supporters, colloquially referred to as the “Bernie Bros.” The Bernie Bros have become infamous for firing attacks and threats at other party leaders, including harassing women and minority staffers (Naranjo, 2020).

This misogyny is unfortunately not abnormal to Twitter, as various scholars have argued that there is a toxic, masculine culture on the platform where women are often targeted and harassed (Citron, 2009; Megarry, 2014; Rheault et al., 2019). Existing research demonstrates how supporters of Bernie Sanders often circulate misogynist narratives and sentiments in an effort to diminish the credibility of female political opponents (Albrecht, 2017). In addition, a 2021 report from the Wilson Center showed that over the span of 2 months, the organization identified 336,000 instances of abusive content directed toward 13 female politicians. This content was shared by over 190,000 users and relied on harsh, misogynist rhetoric (Jankowicz et al., 2021). These findings demonstrate an increased prevalence of gendered and sexualized messages online that contribute to an overall culture of toxicity targeted toward women.

As articulated by Hackworth (2018), online discourse is not immune to gender discrimination and abuse. While early feminist scholars of the internet first believed that online interactions would be able to transcend gender biases, much of the existing literature demonstrates that online spaces do in fact subvert and reinscribe gender, race, and other institutional hierarchies that were once believed could be overcome in virtual spaces (Richards, 2011). Considering how anonymity fosters uncivil discourse (Phillips & Milner, 2017; Suler, 2004; Wang, 2020), Tromble and Koole (2020) argue that the nature of social media tends to invite negativity and abuse, which typically manifests through racism and sexism. Hackworth (2018) illustrates that women tend to be the subject of more online criticism, more so than male users. These findings are reiterated by Rheault et al. (2019), whose work illustrates that female politicians tend to be more heavily targeted by uncivil messages than males. Existing work demonstrates the implications of online gendered harassment. Research shows that female users have experienced an increasing amount of misogynistic online harassment in recent years, which has subsequently impacted their freedom of expression and movement online, as well as offline—demonstrating how online misogyny can lead to deleterious offline impacts (Hackworth, 2018).

Many scholars have illustrated that there is an overwhelming amount of misogyny present online (Citron, 2009; Hackworth, 2018; Megarry, 2014; Rheault et al., 2019; Richards, 2011; Tromble & Koole, 2020). As illustrated by Megarry (2014), male voices often carry more authoritative power than female voices on Twitter. Similarly, the literature indicates that women are disproportionately targeted and harassed on Twitter (Citron, 2009). Often studied rigorously from feminist perspectives, the literature posits that social media sites often “remain firmly grounded in the material realities of women’s everyday experiences of sexism in the patriarachal society” (Megarry, 2014, p. 49). Thus, considering the existing literature pertaining to the role of candidate gender in online incivility, we propose the following hypothesis:

H2: Tweets mentioning Elizabeth Warren will be associated with higher levels of incivility than tweets that mention male candidates.

Furthermore, considering widespread interest within the American news media regarding the notorious “Bernie Bros” (Naranjo, 2020), we propose one final research question regarding the relationship between incivility and specific candidates:

RQ4: Which candidates are connected with the highest levels of incivility among uncivil tweets issued by users?

Methods

Data Collection

At the time of data collection, the top three candidates for the 2020 Democratic primary were Joe Biden, Bernie Sanders and Elizabeth Warren (FiveThirtyEight, 2020). Thus, these were the candidates used for data analyses. A total corpus of 18,237,296 tweets was gathered from Twitter’s API using the Twitter Collection and Analysis Toolkit (TCAT) between August 1 and September 30, 2019. The unit of analysis was each tweet that mentioned one or more of these specific candidates. Three separate tweet datasets pertaining to Biden, Sanders, and Warren were identified through a set of relevant keywords. Search terms included the names of each candidate and hashtags related to their campaign. These hashtags included both official and unofficial (i.e., supporter generated) campaign slogans. See Table 1 for the full list of keywords. The Biden dataset has 8,863,770 tweets; Sanders dataset has 7,729,850 tweets; and Warren dataset has 1,643,676 tweets.

Table 1.

List of Keywords.

Candidates	Keywords
Joe Biden	Biden, #Biden2020, #joe2020, #teamjoe, Biden
Bernie Sanders	Bernie, BernieSanders, #Bernie2020, #FeelTheBern, #NotmeUs, #Sanders2020
Elizabeth Warren	Elizabeth Warren, ElizabethWarren, #Allinforwarren, #Warren2020, #Yesweplan

Coding Procedure

A random sample of 1,875 tweets from the original corpus was selected using python’s random library. The selected tweets were then manually coded by the authors (Krippendorff’s α = .70, percent agreement = 89.5%) as “civil” and “uncivil” based on the operationalization provided by Groshek and Cutino (2016). The Krippendorff α of .70 was accepted based on its use as an accepted threshold in the existing literature (Mozetič et al., 2016). Based on the operational definition provided by Groshek and Cutino (2016), tweets were coded as “uncivil” if they incorporated one of the following five elements: (a) personal or inflammatory attacks; (b) threats; (c) vulgarities, abusive, or foul language; (d) xenophobic or other hateful language or expressions; (e) epithets or ethnic slurs, sentiments that are racist or bigoted, and/or disparaging on the basis of race/ethnicity or that assign stereotypes. All tweets that did not meet this criteria were categorized as “civil.”

The manually coded tweets were then passed into a supervised machine learning algorithm BERT (Google-Research, 2020), which stands for Bidirectional Encoder Representations from Transformers. This is a pre-training natural language processing processor developed by Google AI and was open-sourced in 2018. BERT is a textual algorithm that considers words in relation to the other words within a given context (Roitero et al., 2020). We utilized this neural network to extract the embedded vectors of the text of the tweets in our dataset. Compared with other machine learning algorithms, BERT generally performs highest in precision, recall and F1-score (Mozafari et al., 2019).

All machine learning codes were run using Google Colab. 70% of our 1,875 manually coded tweets were randomly selected to serve as the training set and the remaining 30% as the testing set. Following this, we used the ktrain framework in python, which is “a lightweight wrapper for the deep learning library TensorFlow Keras to help build, train, and deploy neural networks” (Amaiya, 2020). At a learning rate of 0.0001, the process was repeated 10 times and rendered a final model accuracy of 0.88. We validated the predictor on 300 tweets and the validation accuracy is 0.84, with a precision of 0.6, recall of 0.7, and F1-score of 0.65.

Since there is a 12-hr runtime limit in Google Colab, we predicted a sample of 3,000,000 tweets (1,000,000 randomly selected from each candidate’s dataset) using the trained predictor, with the algorithm categorizing each tweet as “civil” or “uncivil.” Since it is possible that one tweet can mention multiple candidates, we labeled each tweet with three dichotomous variables—Elizabeth Warren, Bernie Sanders and Joe Biden—after the 3,000,000 tweet dataset was generated. If the tweet mentions any combination of the candidates, we code the corresponding variable as 1. Otherwise, the corresponding candidate variable is coded as 0.

Finally, a small subset of the data (n = 1,000) was manually coded by the authors (Krippendorff α = .83, percent agreement = 92%) based on an abridged version of the operational definition of anonymous/pseudonymous accounts as offered by Peddinti et al. (2017). This sample was used to assess the different levels of incivility between anonymous/pseudonymous users and users whose display name on Twitter appeared to be actual full names as opposed to a pseudonym. A Twitter users’ display name differs from their username or Twitter handle, and serves as a “personal identifier” for the platform (Twitter, n.d.). As defined by Peddinti et al. (2017), pseudonymous account names are those with “no relation to individuals’ real names and effectively make users anonymous.” Furthermore, as defined by Peddinti et al. (2017), an anonymous/pseudonymous Twitter account features neither a first nor a last name. This subsample of the dataset was utilized to test H1.

Bot Analysis

To answer RQ2, bot likelihood was calculated using Botometer (formerly known as Botornot), a Python API developed by researchers at Indiana University (Davis et al., 2016). For any given account, the API extracts roughly 1,200 features to characterize the account’s profile, friends, social network structure, temporal activity patterns, language, and sentiment. While the API returns many metrics, this study uses the language-specific overall bot score (ranging from 0 to 1, where 1 represents absolute “bot-ness”) to evaluate a given account’s bot likelihood. Due to limited computational resources, for each candidate, 5,000 users were randomly selected for bot likelihood analysis. To identify Twitter accounts most likely to be bots, we utilized a 50% threshold that has been used within the existing literature (Ferrara, 2018), thus an account with a score above 0.5 was classified as a bot.

Data Analysis

We conducted chi-square tests of independence to answer H1 and H2. For RQ2 and RQ3, we performed k-means clustering (MacQueen, 1967) in R to identify and find topic clusters within an unlabeled dataset for RQ2 and RQ3. We first cleaned the corpus by removing special characters, URLs, punctuation, numbers, white spaces and stopwords. We then stemmed the corpus and removed sparse words with a sparse factor of 0.995. When performing the k-means clustering algorithm, “centers” parameter was set to 7 and “nstart” parameter was set to 10. Setting the “centers” parameter to 7 yields the most thematically diverse results for topic modeling. For RQ2, we used the sampled bots dataset for each candidate, with 5,000 tweets in each sample. For RQ3, we ran k-means clustering on each candidate’s uncivil tweets dataset. For RQ4, we used word frequency analysis using R’s tm package to find the top 10 mentioned words and most associated words for tweets mentioning each candidate. For each candidate, we created a document term matrix to calculate the number of times a word is mentioned in the tweets then selected the top 10 words mentioned. We then used “findAssoci” function to find candidate name associated words with an association rate >.20.

Results

Using supervised and unsupervised machine learning algorithms, this study yielded a variety of interesting results regarding the presence of incivility on Twitter in conversations pertaining to the top Democratic candidates.

RQ1: How Prevalent Is Incivility in Tweets About the 2020 Democratic Primary Campaign?

In terms of RQ1, 22.5% of tweets out of the 3 million used within the current subsample were categorized as uncivil by the model. Among those 3 million tweets, we then linked each tweet to the candidates they mentioned. One tweet can have more than one candidate mentioned. As a result, 22% of all tweets (n = 1,179,659) that mentioned Bernie Sanders were uncivil (n = 250,033). In comparison, 23% of all tweets (n = 1,141,701) that mentioned Joe Biden were uncivil (n = 264,356). Finally, 19% of all tweets (n = 1,153,532) that mentioned Elizabeth Warren were uncivil (n = 220,617).

H1: Tweets that originate from anonymous or pseudonymous accounts will be correlated with higher levels of incivility

A 2 (anonymity: anonymous user, non-anonymous user) × 2 (incivility in tweets: civil, uncivil) chi-square test indicated that tweets originated from anonymous accounts are not correlated with higher levels of incivility, χ²(1, N = 1,000) = 0.13426, p = .714, Φ = .017. Therefore, H1 was not supported. A crosstab analysis is provided in Table 2.

Table 2.

Crosstab Analysis for Anonymity and Incivility.

		Incivility		Total
		Civil (0)	Uncivil (1)	Total
Anonymity	Not anonymous (0)	382 (77.33%)	112 (22.67%)	494
	Anonymous/Pseudonymous (1)	384 (75.89%)	122 (24.11%)	506
Total		766	234	1,000

H2: Tweets mentioning Warren are associated with higher levels of incivility

H2 predicted that there would be a statistical relationship between candidate gender and the presence of incivility. To answer H2, we randomly selected a smaller sample of 2,000 tweets from the corpus of 3 million tweets, with 1,000 of them mentioning only Warren and 1,000 of them mentioning only the male candidates. A 2 (gender of the candidate: female, male) × 2 (incivility in tweets: civil, uncivil) chi-square test indicated that the association between candidate gender and presence of incivility in tweets mentioning the candidate is significant, χ²(1, N = 2,000) = 8.0516, p = .005, Φ = .063. A crosstab analysis is provided in Table 3. However, while the relationship between candidate gender and incivility does appear to be statistically significant, there appears to be a very weak relationship as indicated by the effect size. Thus, while the greatest proportion of uncivil tweets was directed to Joe Biden, the data indicates a statistically significant relationship between candidate gender and incivility, therefore H2 was supported. Furthermore, as demonstrated in the crosstab analysis, it appears that male candidates were more likely to be mentioned in uncivil tweets as well, echoing previous findings demonstrating that the greatest proportion of uncivil tweets were targeted toward Joe Biden.

Table 3.

Crosstab Analysis for Gender and Incivility.

		Gender		Total
		Male (0)	Female (1)	Total
Incivility	Civil (0)	760 (48.31%)	813 (51.68%)	1,573
	Uncivil (1)	240 (56.2%)	187 (43.79%)	427
Total		1,000	1,000	2,000

RQ2: Within Uncivil Conversations Surrounding the 2020 U.S. Presidential Primary Candidates, What Topics Were Being Circulated by Bot Accounts?

RQ2 sought to analyze the sorts of topics that were generated from Twitter accounts that were deemed to be bots. For all uncivil tweets, 1,830 tweets (0.93%) mentioning Biden are determined by Botometer to be posted by bots, with 715 tweets (0.31%) mentioning Bernie and 1,810 tweets (0.70%) mentioning Warren, respectively. With the help of topic modeling, we find that tweets generated by bots have a slightly different focus compared with the entire uncivil corpus. For example, “immigration” and “citizen” were not mentioned in topic modeling within the entire corpus of uncivil tweets mentioning Biden, but are present as one of the clusters within the bot tweet analysis for Biden in Table 4. These words suggest that immigration is one of the topics that bots generate the greatest proportion of tweets about in their tweets regarding Joe Biden. This is likely because immigration tends to be an incredibly polarizing and divisive partisan issue (Thompson, 2018). In addition, the “cornpop” incident takes up 1.56% of all the tweets generated by bots on Biden, referring to Biden’s claims of facing off against a gang leader named “CornPop” armed with a razor blade in 1962 (Hains, 2019).

Table 4.

Topic Modeling for Uncivil Tweets Generated by Bots Mentioning Joe Biden.

	Terms associated with the topic	Label	Proportion, %
1	trump, biden, just, ukrain, hunter, son, presid, obama, corrupt, go, investing, china, media, can famili, like, doesn, peopl, drug, vp	Joe Biden’s son	93.43
2	pop, corn, stori, name, joe, bad, dude, gangster, gang, know, fight, razor, cornpopisback, cornpopwilliam, want, guess, funni	Joe Biden’s reported attack by a gangster member in the 60s	1.56
3	teeth, screw, debat, mouth, fall, dem, don, tonight, think, elect, lap, russia, help, rais, memori, noth, runner, bullshit, front, fuck	Democratic debate	1.26
4	still, democrat, left, creepi, berni, poll, castro, high, doubt, yang, candid, longer, lead, crazi, may, kamala, made, need radic, parti	Comparison with other candidates	1.18
5	male, prison, stay, identify, femal, convict, women, hous, will, joe, secual, ident, fact, woman, sentenc, belong, jail, everi idiot, wrong	Joe Biden’s alleged sexual assault	0.98
6	brain, age, aag, clear, former, find, anyone, work, charg, better, miss, defind, attack, old, america, candid, need	Description of Joe Biden’s personal and leadership traits	0.96
7	american, lie, stroi, becom, illeg, la, immigr, wa, say, break, get, anti, citizen, increase, someone, lot	Immigration and foreign policy	0.62

The greatest proportion of tweets generated by bot accounts that mentioned Joe Biden seemed to be associated with his son, Hunter. This cluster includes words such as “ukrain,” “hunter,” “son,” “corrupt,” “famili,” and “drug.” We posit that perhaps bots focused their attention on Joe Biden’s son as he is a very polarizing figure and many considered him to be one of the greatest factors that could jeopardize the Biden campaign (Entous, 2019). Thus, it is interesting to see this reflected in the k-means cluster analysis, as the bot accounts likely targeted their uncivil tweets on this topic.

In bot tweets regarding Warren, most clusters in Table 5 are personal attacks with words such as “unabashed,” “deranged,” “liar,” and “nuts.” These suggest that bots are mostly focused on defamatory language when mentioning Warren. This is a diverge from her entire uncivil corpus, which focuses predominantly on gun violence and her self-claimed Native American heritage.

Table 5.

Topic Modeling for Uncivil Tweets Generated by Bots Mentioning Elizabeth Warren.

	Terms associated with the topic	Label	Proportion, %
1	american, warren, blame, support, trump, shooter, nativ, presid, will, elizabeth, democrat, don, want, describ, ohio, berni, law, one, dayton, harvard	References to gun violence	93.06
2	exist, white, pretend, ethnic, privileg, spend, decad, ahead, get, lie, privledg, robertm hispan, rourk, indian, really, alleg, bill, brutal, clue	Reference to Elizabeth Warren’s heritage	4.10
3	peddl, shouldn, within, mile, patholog, oval, hate, offic, liar, alleg, bill, brutal, clue, crime, demdeb, derang, exact, expect, former, held	Attacks on Elizabeth Warren	1.38
4	unabash, liar, post, new, re, alleg, bill, brutal, clue, crime, demdeb, derang, exact, expect, former, held, hit, lowest, mani, matter	Attacks on Elizabeth Warren’s character	0.65
5	elizabethwarren, indian, liar, kag, fake, trump, nut, case, god, wga, wwg, maga, call, rip, illeg, good, like, best, sick, noth	Attacks on Elizabeth Warren’s character with mention of her heritage	0.32
6	agre, offici, obama, former, say, wrong, america, call, alleg, bill, brutal, clue, crime, demdeb, deragn, exact, expect, held, hit, lowest	Crime	0.30
7	claim, brown, michael, murder, fals, defend, polic, ferguson, us, want, check, danger, obama, via, pocahonta, fact, chief, truth, use, presid	Gun violence	0.20

In bot tweets mentioning Sanders, words such as “socialist,” “bro,” “bulli” appeared in several clusters in Table 6. These words may indicate social bots’ attacks on Sanders’ supporter base, known colloquially as “Bernie Bros” (Naranjo, 2020). The mention of Sanders’ support base is actually not present in topic modeling of his entire uncivil corpus.

Table 6.

Topic Modeling for Uncivil Tweets Generated by Bots Mentioning Bernie Sanders.

	Terms associated with the topic	Label	Proportion, %
1	berni, sander, people, trump, like, warren, will, can, get, just, re, us, support, one, work, right, real, vote, racist	Comparison with other candidates	91.06
2	realli, jew, self, watch, hate, socialist, elit, notmeus, know, boss, financi, explain, capit, voter, doesn, good, said, need, semit, shit	Bernie Sander’s views	4.04
3	ass, know, 11, surpris, behind, scienc, guy, free, pay, born, bit, eat, face, pleas, hand, ignor, abort, fact, man, healthcar	Healthcare	3.70
4	accept, act, ad, ain, anyway, asham, attempt, base, beat, beyond, billion, born, boss, brain, bro, broad, capitalist, carri	Capitalist rhetoric	0.47
5	accept, act, ad, ain, anyway, asham, attempt, base, beat, beyond, billion, born, boss, brain, bro, broad, cancel, capitalist	Capitalist rhetoric	0.40
6	accept, act, ad, ain, anyway, asham, attempt, base, beat, beyond, billion, born, boss, brain, bro, broad, capitalist, carri	Capitalist rhetoric	0.27
7	accept, act, ad, ain, anyway, asham, attempt, base, beat, beyond, billion, born, boss, brain, bro, broad, capitalist, carri	Capitalist rhetoric	0.06

RQ3: Are Specific Public Policy Topics Associated With Uncivil Communication?

Similar to RQ2, RQ3 also used topic modeling to assess the sorts of policy issues that were associated with incivility on Twitter. For uncivil tweets mentioning Joe Biden, k-means clustering appeared to yield clearer topic clusters than LDA topic modeling. The largest percent (22.26%) of tweets mentioning Joe Biden was about his son Hunter Biden, which was prompted as a direct comparison to President Trump’s news regarding his conversation with the Ukraine’s president at the time. The second largest cluster of tweets was about all other democratic primary candidates at the time. These two themes seem to dominate conversations around Joe Biden, and no public policy issue was specifically mentioned in tweets mentioning Joe Biden.

In regards to Warren, there is an overwhelmingly high cluster (70.94%) around topics comparing Elizabeth Warren to other 2020 U.S. presidential candidates: “biden,” “berni,” “kamala,” and “democrat” in Table 7 (see Table 8). Within this cluster, keywords such as “nativ” and “lie” showed up as well, alluding to the accusations and uncivil conversations around her nationality and identity. The other clusters that trailed behind the topic of Warren’s comparison to other democratic candidates consisted of President Trump-related events and topics on gun violence, crime, racial tensions and immigration. Under the topic of gun control—“dayton” and “trump” often came up as a keyword to denote the specific 2019 shooting event in Dayton, Ohio and potential discussions around how Trump responded to the shooting. Some of the most glaring uncivil keywords in the Warren tweets were: “nut,” “freak,” “racist,” “evil,” and “coward.”

Table 7.

Topic Modeling for Uncivil Tweets Mentioning Joe Biden.

	Terms associated with the topic	Label	Proportion, %
1	trump, ukrain, will, biden, investing, son, get, doesn, stroi, hunter, mental, fire, media, presi, prosecutor, play, cover, one, pop	President Trump’s Ukraine scandal in comparison with Hunter Biden	22.26
2	berni, warren, sander, slizabeth, kamala, harris, still, pete, beto, booker, buttigieg, support, yang, vote, want, hate, ami, castro, klobuchar, poll	Democratic primary candidates	3.00
3	obama, impeach, cage, sell, administr, famili, countri, enrich, year, knoe, crook, didn, care, gun, lock, peopl, protect, clinton, barack, comey	National affairs in Trump era	2.80
4	corrupt deal, offici, chines, busi, cyprus, extort, energy, vp, billion, hunter, thini, china, trip, lean, wait, word, dollar, two, can	Hunter Biden’s dealings in China and Cyprus	1.04
5	kid, just, poor white, said, bright, talent, truth, fact, choos, believ, keep, smart, week, joe, time, disast, past, biden, claim	Biden’s upbringing and character	0.51
6	aint, got, shit, went, funni, kill, televis, live, guiliani, rudi, cliam, shame, nation, dumb, candid, polit	Rudy Guiliani	0.34
7	teeth, mouth, dall, demdeb, debat, jump, best, noth, right, watch, via, tonight, come, keep, straigt, folk, bullshit, night, near, almost	The Democratic debate	0.18

Table 8.

Topic Modeling for Uncivil Tweets Mentioning Elizabeth Warren.

	Terms associated with the topic	Label	Proportion
1	american, biden, berni, lie, just, warren, kamala, harri, democrat, doesn, nativ, like, joe, know, can, get, say, need, want, year	Comparison to other 2020 U.S. presidential candidates	70.94
2	blame, shooter, support, ohio, describ, self, fan, never, don, assur, squad, paso, socialist, coward, rest, anyon, trump, mile, satan	Warren vs. Trump’s responses to the Dayton Shooting	26.52
3	complicit, congress, impeach, fail, trump, start, elizabeth, via, white, continu, crime, nanci, pelosi, say, must, presidenti, disarm, condit, drop, todd	Congress’s criminal charges & impeachment of Trump	0.92
4	elizabethwarren, either, big, now, look, can, gun, like, thank, god, immigr, illeg, nut, fan, fake, one, case, dna, question, noth	Topics related to gun control and immigration	0.71
5	street, wall, presid, fear, long, far, screw, becom, fight, time, hous, white, peopl, absolut, report, freak, corrupt watch, threat, realli	Corruption in Wall Street and Trump	0.39
6	black, call, disgust, st, innoc, push, dear, yesterday, got, murder, racist, today, vote, lie, rememb, evil, none, proven, disarm, condit	Racial issues and crime	0.25
7	establish, vote, away, anti, polici, john, isn, run, senat, berni, statement, threat, singl, impeach, liz, face, tulsi, progress corpor, speech	On anti-establishment and Bernie Sanders	0.22

The topic clusters related to Sanders reflect notable tension regarding the senator and his self-described status as a Democratic socialist (see Table 9). This is reflected primarily in Topic Cluster 3, which includes words such as “money,” “capitalist,” “millionair,” and “billionair.” We posit that these are perhaps indicative of partisan jabs and snarky remarks from Sanders’ critics that he is a “millionaire socialist” (Kruse, 2019). Finally, based on the topic clusters, we can glean that the largest policy topics within the tweets that mentioned Sanders primarily focused on his fiscal policy. It is worth noting that the Sanders clusters have the greatest amount of curse words.

Table 9.

Topic Modeling for Uncivil Tweets Mentioning Bernie Sanders.

	Terms associated with the topic	Label	Proportion, %
1	biden, warren, year, support, like, democrat, will, vote, never, get, trump, million, hous, say, berni, peopl, go, joe, socialist, incom	Comparison to other 2020 U.S. presidential candidates	87.13
2	think, ain, don, presid, realli, unit, tri, describ, state, peopl, re, power, scare, noth, corpor, live, stupid, hard, elit, candid	Allusions to power	3.27
3	sander, idiot, berni, millionair, believ, money, rail, biggest, capit, capitalist, swamp, dc, fraud, donald, paid, given, billionair, govern, live, better	Corporate capitalism	2.95
4	re, scienc, one, anyth, hoax, degre, base, fact, truth, much, even, old, believ, wrong, ignor, two, chanc, hypocrit, stupid, human	Insults related to science and truth	1.68
5	fuck, yeah, oh, hell, shut, check, without, voter, yes, suck, tulsi, lol, asshol, serious, god, even, bro, insan, moron, hey	Uncivil attacks	1.49
6	doesn, want, one, see, pick, pattern, barack, doctor, fli, gun, obama, multimillionair, own, jet, harri, privat, kamala, elizabeth, warren, sander	Comparisons to prominent Democratic leaders	1.47
7	hate, make, kamala, sens, law, pass, don, shit, harri, job, joe, rhetor, biden, jew, white, racist, support, love, racism, israel	Racism and sexism	1.30

RQ4: Which Candidates Are Connected With the Highest Levels of Incivility?

RQ4 sought to analyze which candidates were associated with the highest levels of incivility. While this question could also be solved from the results of RQ1, the authors decided to deploy a further text mining analysis to analyze uncivil word frequencies associated with each candidate. This text mining analysis yielded several interesting findings regarding the most uncivil terms associated with each candidate. Regarding Warren, whose mentions received the highest proportion of incivility, these findings are particularly astounding. “Murder” was one of the most frequently associated terms that co-occurred with “Warren.” This term yielded a 0.40—indicating a 40% correlation between these terms. Furthermore, a variety of other interesting terms comprise the top 20 most associated terms with “warren”—such as “shooter,” “lie,” and “Native American.” These terms may indicate uncivil or personal attacks lodged against Warren’s claim to be a descendant of Native American heritage (Kaplan, 2019).

Within the Biden dataset, the two most frequent uncivil terms are “plagiarist” (association = .26) and “investigatethebidenfamily” (association = .42). Both these terms indicate defamatory attacks on Biden, particularly following the quid pro quo political scandal that was occuring during data collection (Re, 2020). Furthermore, the most associated uncivil terms with “sanders” are “swamp” (association = .31), “millionaire” (association = .30), and “idiot” (association = .28). These terms may indicate users criticizing Sanders’ net worth, as he has recently come under fire for what opposers’ believe to be hypocritical rhetoric (Pramuk, 2020). These word associations provide a greater lens to illustrate which terms are associated with specific candidates.

Discussion

This study offers a wide variety of findings regarding the prevalence of incivility in political Twitter discourse. As illustrated, our data reveals a statistically significant relationship between candidate gender and incivility, supporting evidence in the existing literature that online social media platforms are breeding grounds for toxicity and misogyny (Citron, 2009; Megarry, 2014). While the proportions of uncivil tweets may not be monumentally different among the male and female candidates, these findings do support prior work illustrating the relationship between gender and incivility (Citron, 2009; Megarry, 2014).

In this study, we employed a big data, computational analysis to examine the amount of uncivil Twitter conversation about Democratic presidential primary candidates in August and September 2019. Specifically, we sought to expand our understanding of (1) the extent to which candidate gender influences the amount uncivil discourse, (2) the relationship between anonymous accounts and online incivility, (3) whether automated (bot) accounts prioritize specific public policy topics, and (4) if certain public policy issues were associated with higher levels of uncivil communication. Our results show that the highest frequency of uncivil conversation surrounded Senator Elizabeth Warren, the only female candidate in our study. This finding is particularly interesting considering the widespread negative reputation earned by Sanders’ online supporters (Naranjo, 2020).

Interestingly, there does not appear to be a relationship with anonymity and incivility, as uncivil tweets were just as likely to originate from tweets that identified users’ names as they were to originate from anonymous or pseudonymous accounts. This finding is interesting for a handful of reasons. Anonymity has been well documented as a predictor of online incivility (Nithyanand et al., 2017; Phillips & Milner, 2017; Suler, 2004). Thus, it is surprising that our study was unable to replicate these findings. This also sheds light on the need for further investigation of the online disinhibition effect during a time of heightened social media usage. Thus, perhaps the anonymous characteristic of the internet is not the sole contributor to uncivil online discourse. This finding could potentially lend itself to fruitful future research regarding individuals’ intentions to engage in incivility online.

In terms of theory building, this study’s data offered interesting insights for the online disinhibition effect as proposed by Suler (2004). Perhaps this is indicative of a shift in modern society, where anonymity is not necessarily as appealing to users. To this end, Rost et al. (2016) found that in over half a million online comments, non-anonymous individuals tend to be more aggressive in their posts than anonymous individuals. Thus, perhaps future research could focus on more modern disinhibitors that drive online incivility. Similarly, our study offers a myriad of variables that facilitate uncivil online behavior, specifically the role of bots as well as user and/or candidate gender. To this end, we argue that perhaps these are new variables that can be introduced into the pre-existing model established in the online disinhibition effect. Future research would benefit from further exploration of these variables.

Our results also present interesting implications for the role of bots in online discourse. Specifically in regards to Joe Biden, it appears that bots focused the greatest proportion of their uncivil tweets on Joe Biden’s son, Hunter. As discussed, Hunter Biden was an incredibly polarizing figure, and many argued that he could jeopardize Biden’s campaign for president (Entous, 2019). Thus, it is interesting to see this reflected in the topic clusters. Perhaps bot accounts were able to focus on this one polarizing subject and infiltrate Twitter conversations with uncivil communication regarding Joe Biden’s son. These findings echo prior work that demonstrates that bot accounts often serve to instigate political feuds, spread misinformation and spread hateful rhetoric (Albadi et al., 2019).

In addition, our findings provide evidence that certain policy issues are more closely related to uncivil discourse than others. Through k-means clustering, our data reveal that the issue of gun control and immigration is closely related with mentions of Elizabeth Warren and fiscal policy with Bernie Sanders; however, we did not find any policy keywords linked to Joe Biden. Specifically in the Warren clusters, policy topics often appeared alongside mentions of opposing politicians or democratic candidates. For example, the cluster mentioning gun violence included Trump and Dayton Ohio Shooting. Our analysis of uncivil tweets regarding Sanders didn’t show any indication of his health care policies—which is one of the major policy issues that he advocated for throughout the primary. However, there was a significant focus on his democratic socialist ideology which included his economic viewpoints. In summary, our study expands the existing literature by utilizing big data analysis to analyze the specific political policy issues that are most associated with Twitter incivility during this democratic primary. In addition, in line with the existing literature (Albadi et al., 2019), our findings provide support for the argument that bot accounts were able to spread hateful and argumentative rhetoric, likely in an attempt to sow distrust and contempt regarding these three candidates.

This study is not without its limitations. Data was collected during a small window of time during 2019. Furthermore, it has been noted in the existing literature that Twitter is not necessarily widely used among the American electorate—with only 22% of Americans actively using the platform (Hughes & Wojcik, 2019). It is also worth noting that our results are not fundamentally generalizable as the data used for this research was collected during an incredibly high-profile election season, which generated substantial national interest. While the goal of the article is not necessarily to generalize our findings to other political events, we believe that this limitation does merit mention, as does it warrant future research regarding the prevalence of incivility in online discourse surrounding other topics. In addition, as with most machine learning analyses, it is difficult to glean the true tenor of discourse based on the methods we employed. Without more in-depth qualitative analysis, we are unable to pinpoint who the incivility in a tweet is specifically directed to. Finally, while we do feel quite confident in our accuracy rate of 0.88, there is the potential for error in the machine learning classification process. While topic clusters were qualitatively analyzed, we would be remiss not to acknowledge this limitation.

Despite scholarly interest in incivility within both the communication and political science disciplines, relatively little work has been done regarding the intersection of incivility, political communication, and the online disinhibition effect. Along this line of inquiry, our research has provided a number of novel insights of theoretical and practical importance, as well as potential avenues for future research. Future research could illustrate the relationship between incivility and media coverage per candidate. For example, does public incivility toward political candidates determine the valence of media coverage that they receive?

The findings offered here present a myriad of implications for theory building, future research, as well as public policy concerns. As discussed, the lack of empirical support regarding an association between incivility and anonymity is surprising, yet warrants a new perspective on what factors motivate users to act uncivilly online. This is important considering the growing rate of cyberbullying, which has increased by 35% over the last 3 years (Patchin, 2019).

Furthermore, our results present serious implications, particularly in regards to the relationship between gender and incivility. The finding that candidate gender is associated with higher levels of incivility contributes to an important body of research related to the relational dynamics between female politicians and the public (Dolan, 2014). On a practical level, there is the potential for this research to inform policy decisions made by social media platforms. From gender-based cyberbulling and sexual harassment concerns (see e.g., Twitter, 2020) to the problems related to automated accounts during elections (see e.g., Rosen et al., 2019), platforms are broadly concerned with issues examined in this study. We hope that empirical research such as ours can inform ongoing work at the nexus of digital communication, politics, and community safety. In summary, there is ample evidence demonstrating the prevalence of online incivility in Twitter discourse. As indicated, there is great potential for this incivility to bleed over into offline domains, whether in the form of radical movements, violence, harassment, hate crimes, or other forms of bullying. As such, incivility continues to be a critical topic to analyze and understand as digital communication environments grow and evolve.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Briana M. Trifiro

Author Biographies

Briana Trifiro (M.A., Bryant University) is a doctoral student at Boston University. Her research interests focus on social media use and its psychological impacts on users, as well as society as a whole. More specifically, her work analyzes the intersection of media psychology and political communication, such as the role of media in the construction of identity and individual perceptions.

Sejin Paik (M.A., Stanford University) is a doctoral student at Boston University. Her research focuses on artificial intelligence and emerging media, algorithmic and computational journalism, and news media innovation/design.

Zhixin Fang (M.A., Boston University) is a recent graduate of Boston University. Her work specializes in big data and computational analyses.

Li Zhang (M. Phil., Columbia University; M.A., University of London) is a doctoral student at Boston University. His research focuses on how emerging media technologies can be used to promote pro-social behaviors and to empower individuals.

References

Albadi

Kurdi

Mishra

(2019). Hateful people or hateful bots? Detection and characterization of bots spreading religious hatred in Arabic social media. In Proceedings of the ACM on Human-computer Interaction 3 (CSCW) (pp. 1–25).

Albrecht

M. M.

(2017). Bernie Bros and the gender schism in the 2016 US presidential election. Feminist Media Studies, 17(3), 509–513.

Amaiya. (2020). Amaiya/ktrain. https://github.com/amaiya/ktrain

Anderson

A. A.

Brossard

Scheufele

D. A.

Xenos

M. A.

Ladwig

(2014). The “nasty effect”: Online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication, 19(3), 373–387.

Anderson

A. A.

Yeo

S. K.

Brossard

Scheufele

D. A.

Xenos

M. A.

(2018). Toxic talk: How online incivility can undermine perceptions of media. International Journal of Public Opinion Research, 30(1), 156–168.

Borah

(2014). Does it matter where you read the news story? Interaction of incivility and news frames in the political blogosphere. Communication Research, 41, 809–827.

Broniatowski

D. A.

Jamison

A. M.

AlKulaib

Chen

Benton

. . . Dredze

(2018). Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. American Journal of Public Health, 108(10), 1378–1384.

Citron

D. K.

(2009). Law’s expressive value in combating cyber gender harassment. Michigan Law Review, 108, 373–415.

Coe

Kenski

Rains

S. A.

(2014). Online and uncivil? Patterns and determinants of incivility in newspaper website comments. Journal of Communication, 64(4), 658–679.

10.

Coleman

M. C.

(2018). Bots, social capital, and the need for civility. Journal of Media Ethics, 33(3), 120–132.

11.

Davis

C. A.

Varol

Ferrara

Flammini

Menczer

(2016). Botornot: A system to evaluate social bots. In Proceedings of the 25th international conference companion on world wide web (pp. 273–274).

12.

Davis

(2009). Typing politics: The role of blogs in American politics. Oxford University Press.

13.

Dolan

(2014). Gender stereotypes, candidate evaluations, and voting for women candidates what really matters? Political Research Quarterly, 67, 96–107.

14.

Edwards

Spence

P. R.

Shelton

A. K.

(2014). Is that a bot running the social media feed? Testing the differences in perceptions of communication quality for a human agent and a bot agent on Twitter. Computers in Human Behavior, 33, 372–376.

15.

el Hjouji

Hunter

D. S.

des Mesnards

N. G.

Zaman

(2018). The impact of bots on opinions in social networks. https://doi.org/1810.12398

16.

Entous

(2019). Will Hunter Biden jeopardize his father’s campaign? https://www.newyorker.com/magazine/2019/07/08/will-hunter-biden-jeopardize-his-fathers-campaign

17.

Ferrara

(2018). Measuring social spam and the effect of bots on information diffusion in social media. In Lehmann

Ahn

Y.-Y.

(Eds.), Complex spreading phenomena in social systems (pp. 229–255). Springer.

18.

FiveThirtyEight. (2020). Latest Polls. https://projects.fivethirtyeight.com/polls/president-primary-d/national/

19.

Google-Research. (2020). Google-research/bert. https://github.com/google-research/bert

20.

Groshek

Cutino

(2016). Meaner on mobile: Incivility and impoliteness in communicating contentious politics on sociotechnical networks. Social Media + Society, 2(4), 1–10. https://doi.org/2056305116677137

21.

Hackworth

(2018). Limitations of “just gender”: The need for an intersectional reframing of online harassment discourse and research. In Vickery

J. R.

Everbach

(Eds.), Mediating misogyny (pp. 51–70). Palgrave Macmillan.

22.

Hains

(2019). Joe Biden recalls terrifying 1960s public pool confrontation with razor-wielding gangster named “Corn Pop.” https://www.realclearpolitics.com/video/2019/09/15/joe_biden_recalls_terrifying_1960s_public_pool_confrontation_with_razor-weilding_gangster_named_corn_pop.html

23.

Hughes

Wojcik

(2019). 10 facts about Americans and Twitter. https://www.pewresearch.org/fact-tank/2019/08/02/10-facts-about-americans-and-twitter/

24.

Hwang

Kim

Huh

C. U.

(2014). Seeing is believing: Effects of uncivil online debate on political polarization and expectations of deliberation. Journal of Broadcasting & Electronic Media, 58(4), 621–633.

25.

Jankowicz

Hunchak

Pavliuc

Davies

Pierson

Kaufmann

(2021). Malign creativity: How gender, sex, and lies are weaponized against women online. Wilson Center Science and Technology Innovation Program. https://www.wilsoncenter.org/publication/malign-creativity-how-gender-sex-and-lies-are-weaponized-against-women-online

26.

Kaplan

(2019). Elizabeth Warren apologizes at Native American forum: “I have listened and I have learned.” https://www.nytimes.com/2019/08/19/us/politics/elizabeth-warren-native-american.html

27.

Kruse

(2019). The secret of Bernie’s millions. https://www.politico.com/magazine/story/2019/05/24/bernie-sanders-millionaires-226982

28.

Lee

F. L.

Liang

Tang

G. K.

(2019). Online incivility, cyberbalkanization, and the dynamics of opinion polarization during and after a mass protest event. International Journal of Communication, 13, 4940–4959.

29.

Liu

(2019). A big data approach to examining social bots on Twitter. Journal of Services Marketing, 33, 369–379.

30.

MacQueen

J. B.

(1967). Some methods for classification and analysis of multivariate observations. In Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability (Vol. 5.1, pp. 281–297). University of California Press.

31.

Megarry

(2014). Online incivility or sexual harassment? Conceptualizing women’s experiences in the digital age. Women’s Studies International Forum, 47, 46–55.

32.

Mozafari

Farahbakhsh

Crespi

(2019). A BERT-based transfer learning approach for hate speech detection in online social media. https://arxiv.org/abs/1910.12574

33.

Mozetič

Grčar

Smailović

(2016). Multilingual Twitter sentiment classification: The role of human annotators. PLOS ONE, 11(5), Article e0155036.

34.

Naranjo

(2020). Democrats criticize Sanders for online behavior of his supporters. https://www.politico.com/news/2020/02/19/democratic-debate-bernie-sanders-supporters-116204

35.

Nithyanand

Schaffner

Gill

(2017). Measuring offensive speech in online political discourse. In 7th USENIX Workshop on Free and Open Communications on the Internet (FOCI 17). https://www.usenix.org/conference/foci17/workshop-program/presentation/nithyanand

36.

Page

B. I.

Shapiro

R. Y.

(1992). The rational public: Fifty years of trends in Americans’ policy preferences (1st ed.). University of Chicago Press.

37.

Papacharissi

(2004). Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6(2), 259–283.

38.

Park

(2013). Does Twitter motivate involvement in politics? Tweeting, opinion leadership, and political engagement. Computers in Human Behavior, 29(4), 1641–1648.

39.

Patchin

(2019). School Bullying Rates Increase by 35% from 2016 to 2019. https://cyberbullying.org/school-bullying-rates-increase-by-35-from-2016-to-2019

40.

Peddinti

S. T.

Ross

K. W.

Cappos

(2017). User anonymity on twitter. IEEE Security & Privacy, 15(3), 84–87.

41.

Phillips

Milner

R. M.

(2017). The ambivalent Internet: Mischief, oddity, and antagonism online. John Wiley & Sons.

42.

Pramuk

(2020). Bernie Sanders is saying “millionaires” less often, one way his 2020 message has changed. https://www.cnbc.com/2020/02/13/bernie-sanders-says-millionaires-less-in-2020-democratic-primary.html

43.

Rains

S. A.

Kenski

Coe

Harwood

(2017). Incivility and political identity on the internet: Intergroup factors as predictors of incivility in discussions of news online. Journal of Computer-Mediated Communication, 22(4), 163–178.

44.

(2020). Hunter Biden scandals explained: Ukraine, China, drug history and more. https://www.foxnews.com/politics/hunter-bidens-controversies-explained

45.

Rheault

Rayment

Musulan

(2019). Politicians in the line of fire: Incivility and the treatment of women on social media. Research & Politics, 6(1), 1–7.

46.

Richards

(2011). “I could have told you that wouldn’t work”: Cyberfeminist pedagogy in action. Feminist Teacher, 22(1), 5–22.

47.

Roitero

Bozzato

Della Mea

Mizzaro

Serra

(2020). Twitter goes to the doctor: Detecting medical tweets using machine learning and BERT. In Proceedings of the International Workshop on Semantic Indexing and Information Retrieval for Health from heterogeneous content types and languages.

48.

Rosen

Harbath

Gleicher

(2019). Helping protect the 2020 U.S. Elections. Facebook. https://about.fb.com/news/2019/10/update-on-election-integrity-efforts/

49.

Rossini

(2020). Beyond incivility: Understanding patterns of uncivil and intolerant discourse in online political talk. Communication Research. https://doi.org/10.1177/0093650220921314

50.

Rost

Stahel

Frey

B. S.

(2016). Digital social norm enforcement: Online firestorms in social media. PLOS ONE, 11(6), Article e0155923.

51.

Schuchard

Crooks

A. T.

Stefanidis

Croitoru

(2019). Bot stamina: Examining the influence and staying power of bots in online social networks. Applied Network Science, 4(1), 1–23.

52.

Siegel

A. A.

(2020). Online hate speech. In Persily

Tucker

(Eds.), Social media and democracy: The state of the field, prospects for reform (SSRC Anxieties of Democracy, pp. 56–88). Cambridge University Press.

53.

Stromer-Galley

(2002). New voices in the political sphere: A comparative analysis of interpersonal and online political talk. Javnost: The Public, 9, 23–42.

54.

Suler

(2004). The online disinhibition effect. Cyberpsychology & Behavior, 7(3), 321–326.

55.

Thompson

(2018). How immigration became so controversial. https://www.theatlantic.com/politics/archive/2018/02/why-immigration-divides/552125/

56.

Tromble

Koole

(2020). She belongs in the kitchen, not in Congress? Political engagement and sexism on Twitter. Journal of Applied Journalism & Media Studies, 9(2), 191–214.

57.

Twitter. (2020). Abusive behavior. Twitter rules and policies. https://help.twitter.com/en/rules-and-policies/abusive-behavior

58.

Twitter. (n.d.). How to change your username. https://help.twitter.com/en/managing-your-account/change-twitter-handle

59.

Vitali

Roecker

(2020). Warren says Sanders “has a lot of questions to answer” about his supporters’ online attacks. https://www.nbcnews.com/politics/2020-election/warren-says-sanders-has-lot-questions-answer-about-his-supporters-n1137836

60.

Wang

(2020). The influence of anonymity and incivility on perceptions of user comments on news websites. Mass Communication and Society, 23(6), 912–936.

61.

Weber Shandwick. (2019). Civility in America 2019: Solutions for tomorrow. https://www.webershandwick.com/wp-content/uploads/2019/06/CivilityInAmerica2019SolutionsforTomorrow.pdf

62.

Yuan

Schuchard

R. J.

Crooks

A. T.

(2019). Examining emergent communities and social bots within the polarized online vaccination debate in Twitter. Social Media+ Society, 5(3), 1–12.

Politics and Politeness: Analysis of Incivility on Twitter During the 2020 Democratic Presidential Primary

Abstract

Keywords

Literature Review

Overview of Incivility

Online Disinhibition Effect

The Role of Bots on Twitter

Policy Issues Within Uncivil Tweets

Incivility and Gender

Methods

Data Collection

Coding Procedure

Bot Analysis

Data Analysis

Results

RQ1: How Prevalent Is Incivility in Tweets About the 2020 Democratic Primary Campaign?

H1: Tweets that originate from anonymous or pseudonymous accounts will be correlated with higher levels of incivility

H2: Tweets mentioning Warren are associated with higher levels of incivility

RQ2: Within Uncivil Conversations Surrounding the 2020 U.S. Presidential Primary Candidates, What Topics Were Being Circulated by Bot Accounts?

RQ3: Are Specific Public Policy Topics Associated With Uncivil Communication?

RQ4: Which Candidates Are Connected With the Highest Levels of Incivility?

Discussion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

Author Biographies

References