Curating Virality: Exploring Curated Logics Within #BlackLivesMatter on Twitter/X

Abstract

This study examines what types of messages users posted and spread about #Black/Blue/AllLivesMatter during the Black History Month of 2022. Using both qualitative and quantitative methods, about one million tweets were analyzed to test if different levels of opinion leaders tend to spread different kinds of messages related to the context. Using the curation logic of Thorson and Well and Lakoff’s semantic theory as theoretical lenses, we offer some observations about the differences in logics (incentives and norms) that opinion leaders in our dataset might face. We find that different levels of opinion leaders shared different types of messages. The implications of this study call for strategies that foster meaningful discussions on social movements amid polarized views. The study also advocates for refining platform design to encourage the dissemination of factual information over contentious arguments and reaching a societal consensus on critical social issues, such as racial inequality and police brutality. This research contributes to updating the theory of curation logics, virality, and opinion leaders, as well as provides empirical data for the discussions of the #BlackLivesMatter social movement and its related discussions of #AllLivesMatter and #BlueLivesMatter.

Keywords

BlackLivesMatter curated flows semantic theory virality opinion leaders

Introduction

Researchers have been studying the diffusion of information on the internet at least since Jurvetson (2000) coined the term viral marketing. More recently, Nahon and Hemsley (2013) suggest that a viral information spread is when many people simultaneously share a piece of information, which results in a sharp acceleration in the number of people being exposed to the information. Nahon and Hemsley (2013) used the concept of network gatekeepers to explain how some actors have more influence over the flow of information than others. However, the gatekeeping concept is limited as it implies editorial message selection by high-level actors in the network (Shoemaker & Vos, 2009; Thorson & Wells, 2015, 2016). In our socially networked world, it seems worthwhile to understand how actors with different levels of followers may influence the flow of information in different ways (Hemsley, 2019). For example, actors on social media with many followers such as politicians or journalists may influence the kinds of news we receive, but our real-life friends that we follow on social media who do not have many followers may influence how we interpret the news. Therefore, knowing the impacts of different types of actors for a given topic on social media helps us to better understand how and why users spread certain information.

Since our research focuses on how different levels of actors might influence the spread of information differently, we leverage the concept of opinion leaders, people who are followed by and have some influence over other social media users (Dalrymple et al., 2013; Katz & Lazarsfeld, 1955; Walter & Brüggemann, 2020; Walter & Hanke, 2020). In addition, we leverage the theory of curation logics (Thorson & Wells, 2016), which posits that actors’ decision to share or curate content is driven by the incentives they face and the social norms within which they are embedded. Our opinion leaders are the actors who spread Twitter messages related to #BlackLivesMatter (BLM), #BlueLivesMatter, or #AllLivesMatter (ALM) during February 2022, the Black History Month of 2022. While this study examines the technical aspects of the messages associated with the three hashtags, we acknowledge that the BLM movement is a politically charged topic in the United States, reflecting the unfair treatment such as police brutality that Black Americans have encountered for centuries (Hawkins, 2023). To provide meaning to the curation choices we see of the opinion leaders in our dataset, we also leverage George Lakoff’s (2016) semantic theory. Lakoff uses the metaphor of “strict/nurturant parent” to suggest political/moral meanings of messages.

Using content analysis, machine learning algorithms, inferential statistics, and data visualizations, this study reveals that despite there being about the same percentages of supportive and critique-related messages available for users to spread, users retweeted more critique tweets regardless of their opinion leader levels (OLLs) across the #Black/Blue/AllLivesMatter discussion spaces. We also find that different levels of opinion leaders shared different kinds of messages. Using Thorson and Wells’s (2016) curation logic and Lakoff’s (2016) semantic theory as theoretical lenses, we offer some observations about the differences in logic (incentives and norms) that opinion leaders might face.

The contribution of this study is twofold. Theoretically, it expands the understanding of opinion leaders, Lakoff’s (2016) semantic theory, virality, network gatekeepers, and curation logics. Empirically, it supplements data to the #Black/Blue/AllLivesMatter discussion space on Twitter, which has been renamed to X since this research was completed. The implications of this study underscore the necessity for strategies fostering meaningful discussions about social movements amid polarized views. The implications also advocate for refining platform design to encourage the dissemination of factual information over contentious arguments and reaching a societal consensus on critical social issues, such as racial inequality and police brutality.

Background

#BlackLivesMatter

The context for our examination is #BlackLivesMatter, a hashtag movement started in 2013 as a collective of Black radical organizers and self-identified Black/Queer feminists, Patrisse Cullors, Alicia Garza, and Opal Tometi (Edrington & Lee, 2018; Tillery, 2019). They organized the movement in response to the death of Trayvon Martin and George Zimmerman’s acquittal (Edrington & Lee, 2018; Tillery, 2019). The organizational interests are in ending police brutality and racist bias in law enforcement (Francis & Wright-Rigueur, 2021; Freelon et al., 2016; Hawkins, 2023). Although BLM started in 2013, the issues reflected by the movement are not new; the historical roots of the excessive violence perpetrated by law enforcement against Black Americans can be traced back for centuries. The BLM movement shares the historical genealogy of antilynching protests and the civil rights movement in the 1960s (Francis & Wright-Rigueur, 2021).

While the initial uptake of the hashtag #BlackLivesMatter was small, it has grown into a large movement in the last 10 years, far beyond the original organization. The hashtag gained both online (the spread of the hashtag on social media) and offline attention (mainstream media attention and nationwide protests) with unarmed Black Americans being killed by police officers, such as Michael Brown’s death in 2014, Freddie Gray’s death in 2015, or Stephon Clark’s death in 2018 (Francis & Wright-Rigueur, 2021). Numerous protests happened following the killing of Ahmaud Arbery, a Black American man, while jogging in a Georgia neighborhood by three White residents in February 2020. Two of the residents were a retired police officer and his son. Two weeks later, Breonna Taylor was shot by police in her home in Louisville on March 13, 2020, sparking nationwide protests calling for the public to “say her name.” The use of #BlackLivesMatter reached a peak after Derek Chauvin, a former police officer, killed Black American George Floyd on May 25, 2020, which resulted in global protests for police brutality and unfair treatment of Black Americans (H. H. Wu et al., 2023).

Not long after #BlackLivesMatter was formed, social media users started using #BlueLivesMatter and #AllLivesMatter to support law enforcement officers and question the BLM movement (Shanahan & Wall, 2021). #BlueLivesMatter started as a counter-movement of BLM in 2014, as a result of two New York City police officers being shot and killed by a Black American (Keyes & Keyes, 2022). Supporters of #BlueLivesMatter advocate for treating attacks on police officers as hate crimes and the attacked police officers as victims (Shanahan & Wall, 2021). Four U.S. states (Louisiana, Kentucky, Mississippi, and Texas) updated their hate crime laws to incorporate this petition, which resulted in hate crime laws in some states covering authoritarian figures as victims (Mason, 2022). Drawing on an analysis of legislative debates around the enactment of the updated hate crime laws, Gail Mason (2022) concludes that these laws escalated the tension between law enforcement officers and Black Americans. He contends that treating the police as victims reframes and minimizes the history of police brutality that Black Americans have faced.

On a similar note, ALM also started as a counter slogan toward BLM. ALM appears to be a response to the idea that BLM suggests that only Black lives matter (Atkins, 2019). ALM, then, tries to serve as a color-blind racism example that dismisses racial inequalities and police brutality (Carney, 2016). Compared to BLM and #BlueLivesMatter, ALM does not have a formal organization that organizes protests.

Supporting or opposing one of the above hashtags also reflects political ideologies. A 2018 survey shows that older conservative males in the United States are more likely to oppose BLM (Updegrove et al., 2020), and a 2023 Pew survey shows that 84% of the participants who identified as Democrats support BLM, while 82% of the participants who identified as Republicans oppose it (Pew Research Center, 2023).

BLM Studies on Twitter

Numerous studies have examined social movements on Twitter, particularly BLM, and the related Blue and All Lives hashtags. For example, Carney (2016) looked at how the debates on Twitter around “#AllLivesMatter” and “#BlackLivesMatter” reflect a power struggle over the discourse on race and racism in the United States. Carney (2016)’s study highlights strategies used by youth of color to influence the national discussion around race after the death of Michael Brown and Eric Garner in 2014. Blevins et al. (2019) seek to contribute to the literature by showing how the public and social justice groups use Twitter to involve more diverse voices in the national discussion. The authors show that members of the public frame high-profile tragedies as relatable events that have personal meaning for them.

Other research that looks at how BLM frames were constructed and contested by actors on the political left and right shows that in this context, hashtags may act as mechanisms that shape how information flows on Twitter (Stewart et al., 2017). Examining the role of content in retweeting, Keib et al. (2018) found that social movements may increase effectiveness in getting their messages spread by producing content about policy or action, or about a group, along with emotion. From the aforementioned literature, we know that Twitter is an important platform to study because it can act as a platform connecting other platforms (Agarwal et al., 2021; Bennett & Segerberg, 2013) and that viral information flows on Twitter have been shown historically to help the growth of social movements (Hemsley, 2016).

Virality and Spread of Information

The virality of social movements, as exemplified by BLM’s widespread reach on social media platforms like Twitter, is not a random or linear process. Rather, it represents a complex interplay of forces that needs careful analysis and understanding. This article adopts the theoretical framework of “negotiated diffusion” (Hemsley et al., 2017) to move beyond simplistic explanations that attribute viral occurrences to a single dominant factor or a linear combination of a few factors. Instead, it views virality as “a consequence of dialogue and tension among many forces” (Nahon & Hemsley, 2013, p. 139).

Specifically, the negotiated diffusion perspective implies that the process of information diffusion occurs through the dynamics of two processes: top-down and bottom-up (Nahon & Hemsley, 2013). On the one hand, the bottom-up force is driven by individuals’ sharing activities, where the content’s remarkability and the closeness of the relationship between the users play a role in explaining its virality. On the other hand, the top-down force is mainly explained by the role of gatekeepers who let information into their network and facilitate its flow. According to Nahon and Hemsley (2013), gatekeepers’ position in networks gives them disproportionately more influence over the flow of information. Flows from the top and bottom can harmonize (dialogue) or conflict (tension), but in most cases, they manifest as a combination of both (Hemsley et al., 2017). When they do harmonize, the likelihood of something going viral grows.

In a hypothetical example, we might find that someone posts a tweet with a video showing police brutality. If that person’s friends and family retweet it, assuming those people do not have massive followings, the post really will not go far. It is possible, but unlikely, that enough of these regular users would retweet the message such that it spreads far and wide. If it did happen, we would call this a bottom-up event. But it would be much easier for the tweet with the video to go viral if someone with a massive following retweeted the message. This is because users with, for example, a million followers, have a much wider reach in the overall network. When they retweet something, they are more likely to reach users for whom the content is novel, who are sympathetic to the content, and feel that the content should spread (Nahon & Hemsley, 2013).

The information we receive every day constitutes our social reality, and gatekeepers play a crucial role in determining what types of information we receive (Shoemaker & Vos, 2009). The concept of gatekeeping was first coined by Kurt Lewin in 1947, referring to the roles that housewives play when deciding food selections from different channels, such as gardens and markets (Lewin, 1947). While scholars have traditionally studied gatekeepers from a communication perspective (Shoemaker & Vos, 2009), Karine Barzilai-Nahon (2008) examined the concept of gatekeepers from a network perspective. Nahon and Hemsley (2013) define network gatekeepers in their conception of virality: “network gatekeepers (people, collectives, or institutions) are those with the discretion to control information as it flows in and among networks” (p. 43). From an analytic perspective, such gatekeepers have high betweenness centrality, meaning that they are the ones bridging other nodes in a network, and ultimately, bridging disparate networks (Fujita et al., 2018). Overall, gatekeepers are key to information diffusion because they can facilitate or constrain the flow of information as they select which messages pass the gates (Shoemaker & Vos, 2009). A problem with the gatekeeper view of virality is that we live in distributed networks where information may flow along many pathways. The gatekeeping view also implies control. If gatekeepers opt to spread information, people see it; otherwise it does not pass through the gate and become seen.

Other conceptual actors can be important in the flow of information. One is the concept of opinion leaders, which stems from the two-step flow of communication theory by Elihu Katz and Paul Lazarsfeld (1955). For Katz and Lazarsfeld (1955), information flows in two steps: first, from mass media (newspaper, TV, radio, and so on) to opinion leaders, and then, second, from opinion leaders to the public. They define opinion leaders as “the individuals who were likely to influence other persons in their immediate environment” (Katz & Lazarsfeld, 1955, p. 3). Just like network gatekeepers, the network position of opinion leaders also plays a role in the flow of information. Their closeness in the network to the people they influence helps them contextualize the information and make it salient. As Katz and Lazarsfeld put it, “opinion leadership is . . . an integral part of the give-and-take of everyday personal relationships . . . an opinion leader can best be thought of as a group member playing a key communications role” (p. 33).

Curation Logics and Lakoff’s Metaphors

Aside from a two-step flow framework, another way opinion leaders can be understood is within the structure of the curation logics framework. It says, “curation logics—that is, the norms and incentives that guide curation processes—will be more similar within sets of curating actors than across sets” (Thorson & Wells, 2016, p. 12). Following this line of logic, we propose that each actor in an individual’s social network is a curator, curating a selection (“flow”) of content for the individual’s consideration. The curation logics framework enables us to investigate how the norms and incentives faced by different opinion leaders within the BLM and its related online discussion spaces are reflected in the types of messages they share.

However, opinion leaders are partly restricted by the power inherent in the Twitter environment and the social networks in which they are embedded. By power we mean the design of Twitter; by tweeting, retweeting, quoting, or using the @ (at) function that Twitter provides to post or interact with other users, a connection is built. When multiple connections are linked together, a network emerges. This networking mechanism allows researchers to trace how information travels through different networks as well as enables users to discover new content that they may otherwise not have access to (Hemsley, 2016). Due to the political nature of our data, we also link the curation logics framework with George Lakoff’s (2016) cognitive semantic theory. Lakoff’s cognitive semantic theory states that what people say, or perhaps even what they curate, reflects their political ideologies.

Lakoff (2016) coined two family structure-related metaphors, “strict father” and “nurturant parent,” to explain political ideologies and describe “conservative” and “liberal” stances, respectively (pp. 65–140). Lakoff’s “strict father” metaphor means that what is curated is partly based on an opinion leader’s perceptions of absolute right and wrong; one who follows this model will appeal to the perceived patriarchal head of the conversation. The moral goal of such curation is the maintenance of authority and factuality over the well-being of their social networks. The “strict father” morality generates hierarchical semantics but also hierarchical power dynamics in the curation structure. Meanwhile, the “nurturant parent” metaphor opts for the protection and well-being of the “family” instead of focusing on absolute rights and wrongs (Lakoff, 2016). There is no appeal to a patriarchal head of the conversation, and as such, curation is less hierarchical and provides more discussion spaces.

Note that we do not assume that all messages tagged #BlackLivesMatter are “liberal” or that tweets tagged #BlueLivesMatter are “conservative.” However, we suspect that different kinds of discussion spaces may have different content makeup. For example, tweets including #BlackLivesMatter may contain almost equal amounts of support and opposition messages, while ALM may have more opposing messages than support. As such, our first research question is:

Research Question 1 (RQ1): How are the types of messages posted within the Twitter discussion space of #BlackLivesMatter, #BlueLivesMatter, and #AllLivesMatter similar and different?

Within a given discussion space (e.g., #BlackLivesMatter) on Twitter/X, we think that opinion leaders are sensitive to their audience and the norms and incentives they face. For example, we suspect that those with more followers will tend to have different curation habits than those with fewer followers. Thus, our second research question is:

Research Question 2 (RQ2): In the #BlackLivesMatter discussion space, how are the curation behaviors (sharing) of the various levels of opinion leaders different?

Finally, we are interested in knowing the possible difference between what is available to curate and what actually got curated. If curating was simply random, then we would expect that the distribution of message types would tend to be similar to what was curated. However, as noted earlier, we expect that opinion leaders are sensitive to their audience and spread different types of messages. Therefore, our third research question is:

Research Question 3 (RQ3): How are the types of messages that are curated (shared) in the #BlackLivesMatter space different from what is posted?

Data and Methods

The curation logic framework (Thorson & Wells, 2016) indicates that different groups of people face different norms and incentives to curate, which suggests that people would spread different types of messages. Our three research questions attempt to shed light on what kinds of messages exist, what kinds of messages are being curated by different levels of opinion leaders, and finally, how is the posted content (tweets) different from what opinion leaders choose to curate (the original tweets that were retweeted). To answer the three research questions, we first need to annotate the tweets, so we can see what kinds of messages are available to share (original tweets) and what were curated (the original tweets that were retweeted). We used qualitative content analysis and supervised machine learning models to achieve this step. The Qualitative Content Analysis and Supervised Machine Learning Models sections below describe these two methods in detail. The follower numbers of users help us classify the level of opinion leaders who are curating, or sharing, the messages. Fisher’s exact test helps to determine if certain levels of opinion leaders tend to share certain messages, and data visualizations provide a more straight-forward view of our data.

Figure 1.

The data pipeline from the collection, through coding, modeling, and analysis.

Data

We collected 993,320 tweets in real time that contained #Black/Blue/AllLivesMatter (case insensitive) and their variations such as BLM and ALM, from February 1, 2022, to February 28, 2022, using Twitter’s streaming application programming interface. Of all, 258,776 are original tweets, and 734,544 are retweets (73.9%). Tweets show the original content posted about the aforementioned hashtags, while retweets reflect what users chose to spread or curate.

Qualitative Content Analysis

Since we are interested in knowing the types of messages posted and curated about #Black/Blue/AllLivesMatter, we randomly selected 3,000 tweets to do qualitative content analysis with (Saldana, 2015) so that we could identify themes in the original tweets. We also applied inductive coding instead of deductive coding for our qualitative content analysis because inductive coding helps to grasp the meaning that emerged from the data while deductive coding involves a top-down coding process to fit new data into predefined codes (McKibben et al., 2020). The sample size was decided based on previous studies that explored Twitter hashtags for conducting content analysis (see Cavazos-Rehg et al., 2019; Jussila et al., 2021; Oh et al., 2021). Previous studies show that random sampling is an efficient strategy to gather a representative sample of data for tweets (Kim et al., 2018; Le et al., 2019). We chose qualitative content analysis instead of quantitative analysis because qualitative content analysis focuses on understanding the meanings of the given text, while quantitative content analysis emphasizes how information from the text is presented in frequencies, typically as percentages or actual counts within specific categories (Kleinheksel et al., 2020; White & Marsh, 2006).

To get an initial understanding of the themes that emerged from our dataset, three of the authors worked together to code a random sample of 200 tweets (retweets were not coded). We then discussed the codes and adjudicated the tweets. In the process, we merged some codes and created new ones. We repeated these steps two more times with increasingly larger sets until we reached saturation. The team then independently coded batches of 100 tweets, checked intercoder reliability (O’Connor & Joffe, 2020) using percentage agreement and Krippendorff’s alpha (Stromer-Galley & Rossini, 2023), and adjudicated tweets until two of the coding researchers reached 75% intercoder reliability using Krippendorff’s alpha, indicating good agreement.

Using Twitter’s academic API, we were able to collect tweets as soon as they are posted, so even if a user deletes their tweet right after posting it, we have a copy of the metadata (including text, userID, TweetID, posted time, and so on) in our dataset. Using TweetID from the metadata we collected, we generated the URLs (links) of the selected tweets to view the original content online and proceed with annotation. Even though some of the selected tweets for annotation became unavailable (seen with the error message “sorry this account has been suspended or this tweet has been deleted” once we clicked the URL) when we attempted to annotate them, we retained a copy of the metadata. To eliminate the unavailable tweets in our annotating sets, two coders then independently coded more random selected tweets until we had a final set of 3,000 coded tweets that were all available.

This qualitative content analysis step enabled us to develop a codebook that includes the following categories, or message types: Support, Critique, Definition, Information, and Other. We also annotated tweets as to whether they were related to BLM, #BlueLivesMatter, or ALM. We refer to this level of coding as the context.

After developing the codebook, we noticed that Support, Critique, and Information perform much like the moral agreement or disagreement that marked Lakoff’s “strict father” metaphor of communication. That is, in practice, these statements were about the simple agreement or disagreement or offer a factual moral frame without suggesting space for additional nuance. Comparatively, we noticed that Definition functions much like Lakoff’s “nurturant parent” metaphor. That is, these tweets offer spaces for disagreement, but they provide reasons or space to agree or disagree with the perceived facts. Table 1 shows the definition of each message type and provides an example of them.

Table 1.

Message Types of Our Codebook, Tweet Examples, and How They Reflect Lakoff’s Metaphors.

Message type	Definition	Tweet example	Context	Lakoff’s metaphors
Support	Tweets show a direct statement of support for the movement.	“@Fightful the blue lives matter”	Blue	“Strict father”: this tweet fits Lakoff’s idea of being in support without ambiguity.
Critique	Tweets that are negative toward the movement or a person in the context of the movement.	“Moments ago, I learned the father of her unborn child is an ‘all lives matter’ nincompoop”	All	“Strict father”: this tweet fits Lakoff’s idea of being in moral disagreement without room for disagreement.
Definition	Tweets that offer a perspective on why the movement exists or what its purpose is.	“The previous tweets were about Andy Ngo’s grift re: antifa, so why have you gone onto BLM—or more specifically the ‘Black Lives Matter Global Network Foundation’? NB. This U.S. branch of BLM does not represent the entirety of the BLM movement, which is a global/social movement.”	Black	“Nurturant parent”: this tweet fits Lakoff’s claim that there’s room for negotiating the morality of their claims, offering flexibility in stating what is “right”/“wrong” or in stating effects or feelings toward a particular attitude rather than defining moral correctness.
Information	Tweets that are neutral in tone and provide information about, for example, an event, or a new story. They often have URLs linking a source.	“Amazon suspends Black Lives Matter from its charity platform https://t.co/XhZb4jHmbI via @MailOnline.”	Black	“Strict father”: this tweet conveys “facts” without providing negotiation space of meaning or value.
Other	Tweets that are not relevant to any of the movements. Some tweets contain words like “lives” and “matter” but are not related to any of the hashtags.	“How to build aircraft for alien skies https://t.co/E0E79At190 #mikesrandomthoughts, #bluelivesmatter, #Senate‚ https://t.co/dsfMQoej6p”	Blue	N/A

Supervised Machine Learning Models

In an ideal world, we could manually annotate all the tweets and their retweets; however, this is impractical given the time needed to annotate our entire dataset. Also, one issue that arises with relying on human annotators over a long period of time is the accuracy of the annotations; annotators’ understanding of the original annotations may drift over time and therefore reduce the accuracy of annotations (Hillard et al., 2008). Using machine learning models helps resolve this issue, as the accuracy of machine learning models remains the same over time (Stromer-Galley & Rossini, 2023). It also takes less time for machine learning models to process large-scale data than human annotators. Given the aforementioned reasons, we use the 3,000 annotated tweets from the qualitative content analysis step as the gold label dataset that we train machine learning models with, so that the machine learning models can annotate the retweeted tweets, as previous studies have demonstrated (Indra et al., 2016; Stromer-Galley & Rossini, 2023).

In our study, we tried several machine learning models to find the best-performing one for annotating the data. For the traditional machine learning models, we chose Naïve-Bayes, logistic regression, random forest, gradient boosting, and support vector machine (SVM). These five models were selected because of their popularity in identifying patterns in tweets (Abdurrahman et al., 2020; Hitesh et al., 2019; Indra et al., 2016; Ragini & Anand, 2016; Saif et al., 2014). We also tried two deep learning models: Bidirectional Encoder Representations from Transformers (BERT) (Devlin et al., 2019) and Robustly Optimized BERT Pre-Training Approach (RoBERTa) (Liu et al., 2019) The former is known for its high performance in various natural language-processing tasks, while the latter is an enhanced version of BERT further trained on datasets like OpenWebText.

For the traditional machine learning models, we used regular expressions and the natural language toolkit (NLTK) python package to process the text first (e.g., removed special characters and usernames). We then made a note of the tweets that were quoted or replies, or contained a link, and used these features for the machine learning models. We also included a text string that indicated if we coded the tweet context as #Black/Blue/AllLivesMatter. We then converted the text to a Term Frequency-Inverse Document Frequency (TF-IDF) matrix for the machine learning models.

For the transformer-based deep learning models, BERT and RoBERTa, we employed specific preprocessing methods. We utilized BertTokenizer for BERT and RobertaTokenizerFast for RoBERTa to tokenize the tweets, ensuring that the text was processed in a way that aligned with the pretrained models’ requirements.

We compared the performances of these aforementioned models for both contexts (Black/Blue/All) and message types (Critique/Definition/Information/Support/Other). Table 2 shows the performance of each model. We bold-formatted the F1 scores of the two machine learning models we chose to use in Table 2.

Table 2.

The Performance of Each Machine Learning Model.

		Context		Message type
		F1 (macro)	F1 (micro)	F1 (macro)	F1 (micro)
Machine learning models	Multinomial Naive Bayes	29.89	62.83	49.55	64.08
	Logistic regression	57.35	73.41	59.94	70.73
	Support vector machine (SVM)	67.03	75.44	65.15	73.83
	Random forest	66.39	73.31	60.29	70.22
	Gradient boosting	68.60	76.26	58.53	69.12
Deep learning models	BERT	69.24	82.67	55.40	62.22
Deep learning models	RoBERTa	65.58	81.31	60.48	65.36

There were a few criteria we took into consideration in terms of selecting the models from Table 2 to run on the retweets. First, we looked at the F1 score, a measurement of the machine learning models’ performance, which combines the recall and precision scores of a model (Varoquaux & Colliot, 2023). Due to the class imbalances in both context and message type classification tasks, we considered two types of F1 scores: micro F1, which calculates precision and recall globally, giving more weight to the majority class in imbalanced situations, and macro F1, which calculates the F1 score for each class independently, ensuring that each class contributes equally to the final score, regardless of its size (Li et al., 2021).

Second, we ruled out the models that performed well for only one class. We did not include the numbers of each nuanced class in Table 2 given the limited space. The models that had the highest micro and macro F1 scores were BERT, followed by RoBERTa. However, their high performance was primarily attributed to the high F1 scores of the largest class for context, Black. In the case of BERT, the F1 score for the Black class was .9021, but for Blue, it was .6667. For RoBERTa, Black had an F1 score of .9038, but for Blue, it was .5833. On the contrary, the gradient-boosting model, which showed the third-highest Macro F1, had an F1 score of .8622 for the Black and .7500 for the Blue. Given the aforementioned explanations, we selected gradient boosting as the final model for classifying context. Again, the model’s ability to distinguish whether a tweet belongs to the Blue class or other classes, rather than the Black class, is critical for our study. This is especially important since the majority of tweets are categorized in the Black class.

For the message type classification, SVM was the model with the highest performance. This performance was not due to a particular class, which is the case when we tested with BERT and RoBERTa for context. The second and third highest-performing models, random forest and logistic regression, had lower F1 scores in the smallest two classes, Definition and Information, than SVM. Therefore, SVM was chosen as the final model for classifying message types. In our study, the performance of the more complex deep learning models was found to be lower than that of some of the traditional machine learning models. While the exact cause of this is challenging to pinpoint due to the black-box nature of deep learning models, we suggest that the short length and simple structure of the text data may have caused overly complex models to overfit.

To further contextualize our selection process based on the F1 scores, it is important to recognize that there is no universally accepted threshold for their performance in the context of supervised machine learning for social science applications (Stromer-Galley & Rossini, 2023). A previous study suggests that an F1 score below .70 or significant disparities between the F1 score and either precision or recall require cautious interpretation (Stromer-Galley & Rossini, 2023). For our study, the chosen models’ F1 micro scores surpass .7, although their F1 macro scores are slightly below .7. However, given the significant disproportion in the sizes of different classes, it is more appropriate to use the macro F1, which represents the global performance across all classes, for the final acceptance of our model. We anticipate achieving better classification performance by adopting language-specialized deep learning models in our future work.

After selecting the best-performing models (gradient boosting for context and SVM for message type), we ran the models on tweets that were retweeted at least once. We could have just run the models on all the retweets, but since a tweet can be retweeted many times, we would have been redundantly classifying the same text over and over for those cases. We found it computationally less expensive to only classify those tweets that had been retweeted and then merge the classifications back into the retweet data. Again, we are interested in the message types that get curated, or retweeted, and by comparing them to what was available to curate, we could better understand users’ curation logics (Thorson & Wells, 2016). We looked into the retweets to get the IDs of the tweets they are retweets of (retweets contain the ID of the tweet that they are a retweet of). With these IDs, we pulled those original tweets and sent them through the classifier. The retweet data are also used to identify different levels of opinion leaders (OLL). That is, the retweet metadata also contain information about the user who posted the retweet including how many followers they have. We inserted the OLL into the retweet metadata as an additional field.

Opinion Leader Levels

In this study, we view all users as being some level of opinion leader since even a user with one follower could influence someone else. We thought about using the term influencers, but according to the influencer literature (Freberg et al., 2011; Khamis et al., 2017), the concept of influencer is about users who make profits such as endorsing brands on social media, which is not generally the case in the #Black/Blue/AllLivesMatter discussion spaces. The unfit meaning of influencers leads us to choose the term opinion leaders, as mentioned in the Background section, this concept describes people who are able to shape others’ thoughts without implying the meaning of making profits through brand marketing.

Previous studies have broken users into various groups based on the number of followers they have (Djafarova & Trofimenko, 2019; Kay et al., 2020; Ouvrein et al., 2021; Starbird et al., 2023). Oftentimes, these groupings are arbitrary and follow convenient numbers. For example, Starbird et al. (2023) start with less than 1,000 followers, then go from 1 K to 5 K, 5 K to 25 K, and so on into seven different groups. Hemsley’s (2019) breakdown was more interesting to us because there were fewer categories (three), and the cut points were derived empirically. His system’s lowest level includes all users with between 0 and 1,799 followers, the middle-level users were from 1,800 to 26,000 followers, and the highest level being users with more than 26,000 followers. We use his system except that we add a fourth higher level of users with more than a million followers, which other studies also had (Alassani & Göretz, 2019; Brewster & Lyu, 2020). Our names for the different levels of opinion leaders are similar to the other researchers we cited earlier. Thus, we have the nano level as the lowest (0–1,799 followers), mini level (1,800–26,000), meso level (26,001–999,999), and mega level (1 million and above followers). Finally, Like the tweet context and message type, we add the OLL name to the retweet metadata for later analysis.

Inferential Statistics and Data Visualization

Answering RQ1 involves building a table (Table 3) for the 3,000 annotated gold-label tweets based on their message types and context, visualizing the result (Figure 2), and running Fisher’s exact test on the table. Both Chi-square test and Fisher’s test can tell if the associations or differences we see are random or not (Ragini & Anand, 2016; Upton, 1992), but we use Fisher’s test because our data are uneven, with some cells in the tables having smaller numbers than are acceptable in a Chi-square test (Ott & Longnecker, 2015). Using Fisher’s test tells us how the different discussion spaces tend to include some message types more (less) than others. Again, the 3,000 annotated tweets represent a sample of available tweets to be curated, or shared, by our opinion leaders. We expect that by understanding the different mixtures of message types, we might gain some understanding of the similarities and differences of the discussion spaces.

Table 3.

Count of Message Types for Each Hashtag.

	Critique	Definition	Information	Support
All	92	19	2	46
Black	721	247	322	653
Blue	66	9	5	31

Figure 2.

Percentage of message types by hashtags.

RQ2 and RQ3 focused only on the data from the BLM movement. For RQ2, we again built a table (Table 4), this time with the number of each message type retweeted by each of the different levels of opinion leaders. This gives us a sense of how different levels of opinion leaders post different content, providing us insights into how their norms and incentives may be different at these different levels. For RQ3, we compared, again in a table (Table 5) using Fisher’s test, the number of tweets available in the stream, with what is getting retweeted.

Table 4.

Opinion Leader Level Statistics.

	Nano level	Mini level	Meso level	Mega level
Follower range	0–1,799	1,800–26,000	26,001–999,999	1,000,000+
Count	184,118	39,050	2,336	30
Retweets	281,577	89,100	6,024	42
Retweets/per	1.53	2.29	2.58	1.4

Table 5.

Message Types Curated by Different OLLs.

	Critique	Definition	Information	Support	Total
Mega	37	0	1	5	43
Meso	3,346	40	486	1,987	5,859
Mini	49,629	3,055	7,924	28,365	88,973
Nano	150,996	15,361	39,287	84,020	289,664

Results

For our dataset, we found that 152,602 users posted 258,776 original tweets. Of those tweets, 61,964 (24%) were retweeted 734,544 times by 411,224 users. To answer RQ1, we first look at the counts of message types within the gold-label data, broken down by discussion space. As mentioned in the Data and Methods section, some categories for the gold-label data are less than 5 as Table 3 shows, so we ran Fisher’s exact test instead of a Chi-square test. The result gives us a p-value < .0001, indicating that the differences we see in Figure 2 reflect nonrandom effects.

Since Figure 2 shows the data as percentages, we can interpret the ratio of messages about the hashtags. Figure 2 shows that Critique messages are significantly higher for Blue and All than for Black, while within the discussion around BLM, the message types are more balanced. We also see that BLM has the highest percentage of Information messages. All three groups post a similar ratio of Definition messages. This brief interpretation answers RQ1.

RQ2 looks at how different levels of opinion leaders tend to share different kinds of messages within the BLM discussion space. To answer RQ2, we first offer a breakdown of some descriptive statistics about our opinion leaders in Table 4. There are only 30 opinion leaders at the mega level, and they posted 42 retweets, or 1.4 retweets per user at this level. The most active retweeters in this discussion space were at the meso level, where the average opinion leader posted 2.58 retweets. By far, and as we would expect, the nano-level opinion leaders make up the largest group.

Figure 3 shows the data as percentages of message types posted by different OLLs, so we can interpret it as the ratio of messages those OLLs tend to curate. We ran Fisher’s exact test for the data shown in Table 5 for the same reason as we did for Table 3. Figure 3 shows that as a percentage, Critique messages are curated significantly more often for all the OLLs, but for mega OLLs, it appears that Critique messages are by far the most common kind of message got curated. Also, the further down we go (from mega toward nano), the more likely OLLs are to curate messages that define the movement or send informational messages. This finding suggests a negotiation of meanings rather than a resounding moral agreement or disagreement. This interpretation shows us how OLLs practice different curation behaviors, which answers RQ2.

Figure 3.

Percentages of message types posted by different OLLs.

To answer RQ3, we zoom into the BLM section of Figure 2, which is illustrated as Figure 4. Figure 4 shows the percentage of messages available for curation across various OLLs, derived from the gold-label data, which can be compared with the information presented in Figure 3. Figure 4 shows that 39% of the messages are labeled as Critique in the gold-label data. Support is next with 35%, followed by Information with 17% and Definition with 13%.

Figure 4.

Percentage of message types posted related to #BlackLivesMatter for comparison with Figure 3.

While both Figures 3 and 4 show that the ratio of most to least used is Critique, Support, Information, and Definition in the BLM discussion space, it is clear that OLLs are selecting a disproportionately large number of Critique messages to curate. In other words, opinion leaders, especially the ones of the mega level, chose to spread more Critique information (Figure 3) about BLM even though there are about the same amount of messages from the Support and Critique categories (Figure 4). Also, while there are similar amount (17% and 13%) of Information and Definition messages available to curate, OLLs appear to select significantly fewer of these at an average across all OLLs of 8% and 2%, respectively. As both Figures 3 and 4 illustrate, we can see that while 13% of messages attempt to define the movement, this kind of message is rarely curated, and at the top, not at all.

Discussion

Contextualizing the “Nurturant Parent” vs. the “Strict Father” Metaphors

Our findings about the #Black/Blue/AllLivesMatter discussion spaces support Lakoff’s “strict father” and “nurturant parent” metaphors. While the order of message types seem to remain consistent across different discussion spaces, the conversations stand out due to the regularity of specific message types when compared to others. For example, as Figure 2 shows, the message types of BLM-related tweets tend to be more evenly distributed than #BlueLivesMatter and ALM. This echoes Lakoff’s (2016) “nurturant parent” metaphor of political discourse semantics, suggesting that opinion leaders may encounter more negotiation regarding the ideological understandings of BLM.

While we see a lot of Critique and Support tweets curated in each discussion space, ALM and #BlueLivesMatter are particularly critiquing or supportive spaces (way more Critique and Support messages than other message types) compared to BLM, as Figure 2 shows. These critiquing and supporting positions can be understood as reflective of Lakoff’s (2016) “strict father” metaphor, showing a tendency to share messages that are “right” or “wrong” for users who retweeted them. For example, the “blues lives matter” tweet example for Support in Table 1 signals that in the #BlueLivesMatter space, respecting Blue life is the right thing and is not to be contested.

However, within ALM, there is a noticeable amount of Definition happening, suggesting that there is some negotiation around what ALM implies ideologically. Much of the ALM discussion is negotiating how much focus should be on “Black lives” relative to everyone else. Instead of a Definition category in #BlueLivesMatter, we see a slightly higher amount of the Information category than in ALM, as Figure 2 presents. When we were annotating the tweets, we noticed this information consisted of semantically factual reports that imply the police are “victims” of public disputes or “heroes” standing up to wrongs. That is, like with the “strict father” metaphor, information is expressed as explaining an indisputable moral high ground, often supported by news-like or infographic-like genres of content.

Curation Behaviors of Opinion Leaders

Now, we turn to the curation behavior of our different levels of opinion leaders and try to infer the different norms and incentives they face, all in the context of the #Black/Blue/AllLivesMatter discussion spaces. First, we see that our mega-level opinion leaders overwhelmingly curated Critique messages at about 86%. In terms of Lakoff’s metaphors, we might interpret this as those with the highest number of followers are incentivized to share more critical messages and operate in a space where being critical is the norm. That is, if one perceives themselves as being at the top of the social hierarchy, they may have an incentive and operate within certain norms, to follow the “strict father” metaphor of semantics regardless of their actual political ideology. Since mega-level opinion leaders tend to have disproportionately greater influence over the flow of information (Hemsley, 2019), when they share their “strict father” messages, they reinforce doing so as a norm in the space they are in.

As we move down OLLs, we see a steady relative decline of Critique messages at the same time as we see an increase in the diversity of message types, as Figure 3 presents. We note that the most-used message type for each OLL was Critique, followed by Support. It could be that the more clearly a message is morally “for” (support) or “against” (critique), the more likely users are to share/curate it, which is possibly associated with getting more followers. While the causational direction of message content and number of followers is unclear, the incentives are overt: being critical garners more attention. That is, Twitter’s structure of conversation may tend to incentivize curating messages in the “strict father” metaphor for BLM regardless of OLL. Typically BLM is seen as a leftist conversation (Pew Research Center, 2023). Yet, the hierarchical incentives of the platform lead to more “conservative” semantic norms in this space.

Furthermore, as opinion leaders in the BLM space tend to have fewer followers, we see more of Lakoff’ “nurturant parent” metaphor. That is, despite Critique and Support being the largest two message types (in respective order), the fewer followers an opinion leader has, the more likely they are to share other message types, as shown in Figure 3. The opinion leaders may be incentivized to appeal to fact-like news and infographics (Information). Or they are likely to negotiate what these ideological spaces are about (Definition). Another way of interpreting this findng is that these users are incentivized to engage in equitable conversations with their network or appeal to ready-made narratives such as news-like or infographic-based content.

Implications

When we take our findings and look them through Lakoff’s metaphors, coupled with the curation logic framework, there are a few takeaways we would like to point out. First, zooming into the BLM discussion space alone, we can see that it has roughly the same percentage of Support and Critique messages, as Figures 2 and 4 present. This implies users have a more polarized view about BLM than about #Blue/AllLivesMatter. Such a polarized view is likely to impede users from reaching a consensus or finding compromise when discussing the BLM social movement, which raises concerns for a society as a whole, especially given what this social movement represents: advocating for racial equality and against police brutality that Black Americans have faced for centuries (Carney, 2016).

Second, Figure 2 shows that #BlueLivesMatter has the least amount of Definition messages compared to the other hashtags, suggesting that users who interacted with this hashtag had a clear view on what #BlueLivesMatter represents, while there are more disagreements for what BLM and ALM stand for. We suspect that not having the same understanding for a topic, especially a social movement, could limit the ability of users to engage in meaningful discussions, which would fill the online discussion spaces with arguments rather than factual information.

Third, our findings show that Critique has the highest percentage as a message type across #Black/Blue/AllLivesMatter, indicating that users engaged in more of a negative way regardless of the discussion spaces on Twitter. It could be that negative content gains more attention, as the study by B. Wu and Shen (2015) reports that news with negative tones exhibits certain correlations with retweet popularity, whereas news with positive tones do not. However, having Critique as the leading message type across #Black/Blue/AllLivesMatter discussion spaces also raises the concern about how to have meaningful discussions regarding these hashtags; especially coupled with the fact that mega-level opinion leaders, who have great influence given the large amount of followers they have, chose to curate Critique excessively. This finding also questions Twitter’s platform design and power; users may deliberately spread negative information about #Black/Blue/AllLivesMatter just to gain retweet popularity and followers on Twitter, ignoring what these hashtags and the social movements behind them actually stand for.

Limitations

There are a few limitations of this study that could be addressed in future research. First, we acknowledge that we are comparing message types for the different hashtags, #Black/Blue/AllLivesMatter, and that the data were collected during Black History Month. We think the ratios of message types could be different during a different month or depending on current events. We see this as an opportunity for future work comparing the effects of context on curation logics (Thorson & Wells, 2016). Second, the F-1 score of the machine learning models in this study could be further improved with more annotated tweets. Third, this study shows what users chose to curate in the #Black/Blue/AllLivesMatter discussion spaces on Twitter/X and infer the reasons behind users’ actions based on the curation logics framework (Thorson & Wells, 2016) and Lakoff’s (2016) semantic theory. However, we do not know exactly why users curated certain tweets. Future work could interview users who posted information about #Black/Blue/AllLivesMatter to probe the reasons behind their actions.

Conclusion

This study examines what types of messages users posted and spread about #Black/Blue/AllLivesMatter during the Black History Month of 2022. Using both qualitative and quantitative methods, we find that despite there being about the same percentages of supportive and critical messages in the original tweets, users retweeted more critical tweets than other message types regardless of their OLLs. We also find that different levels of opinion leaders shared different message types. Using the curation logics of Thorson and Wells (2016) and Lakoff’s (2016) semantic theory as theoretical lenses, we offer some observations about the differences in the logic (incentives and norms) that opinion leaders might face.

This study makes both theoretical and empirical contributions. Theoretically, it offers an update to the concept of virality proposed by Nahon and Hemsley (2013) by considering the concept of opinion leaders. It also suggests some developments for reconsidering how Lakoff’s (2016) semantic theory signals liberal and conservative statements. Empirically, our work supplements data to contextualize the framework of curation logics (Thorson & Wells, 2016), as well as adds to the existing data collections for #Black/Blue/AllLivesMatter discussion spaces. The work also makes a contribution to the BLM literature by contrasting the ways opinion leaders in the #Black/Blue/AllLivesMatter discussion spaces on Twitter frame messages, often for the purposes of retaining or negotiating power. We conclude by highlighting the implications of this study for social movements.

Footnotes

Acknowledgements

Although this study takes a technical approach that examines the types of messages that users posted and spread regarding #BlackLivesMatter and its related hashtags, we would like to show our gratitudes toward protesters who fight against racism and police brutality online and offline. We also want to thank the reviewers who provided constructive feedback to us in shaping this article, as well as the editoral team for their work in faciliating the publication.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Yiran Duan

Jeff Hemsley

Alexander O. Smith

Author Biographies

Yiran Duan is a PhD student at Syracuse University’s School of Information Studies. Her current research interests are at the intersection of opinion leaders and social network analysis through the lens of information flow life cycle. Specifically, she is interested in exploring how information travels across platforms as well as the life span of information flows. She approaches her research topics by applying both qualitative and quantitative methods. Her most recent qualitative work explores how may impact the design industry and education by conducting interviews with designers using this platform, while her quantitative work looks at the types of messages spread regarding #BlackLivesMatter using machine learning models.

Jeff Hemsley is an Associate Professor at the School of Information Studies at Syracuse University and an Associate Editor at the Journal of Information Technology and Politics. He is also the Director of the Center for Computational and Data Sciences at the iSchool. He earned his PhD from the University of Washington’s Information School. His research is about understanding information diffusion in social media, and he is a co-author of the book Going Viral (Polity Press, 2013, and winner of ASIS&T Best Science Books of 2014 Information award and selected by Choice magazine as an Outstanding Academic Title for 2014), which explains what virality is and how it works technologically and socially and draws out the implications of this process for social change. He works with excellent PhD students and publishes in journals such as Social Media and Society, the Journal of Information Technology and Politics, and Policy & Internet.

Alexander O. Smith is a PhD Candidate of Information Science and Technology at Syracuse University. Their work attends to the dynamics of information and culture and is particularly invested in the development of meme studies conceived across theories of library and information studies, media and culture studies, and science and technology studies. Previous work at the Journal of Documentation initiates their project of reconceptualizing what memes are for Library and Information Studies and a development of a “methodology of difference.” By extension, their contributions aim to provide a theory of information selection within and across sociotechnical environments.

Una Joh is a PhD student at the School of Information Science and Technology at Syracuse University, specializing in computational social science. With a strong background in economics from her bachelor’s and master’s studies at Sungkyunkwan University (SKKU) in Korea, her research investigates the flow of economic information and its role in shaping economic beliefs, with a particular emphasis on the influence of different media types. Her goal is to discern the impact of information flow on the macroeconomy, employing methods such as network science, agent-based modeling, simulation, and econometrics.

LaVerne Gray, PhD, is Assistant Professor at Syracuse University School of Information Studies where she teaches in the Library and Information Science program. She earned her PhD at the University of Tennessee Knoxville (2019); MLIS Dominican University (2005); BS Wilberforce University (1993). Her research focuses on the Black feminisms, informational justice, and memory work. Her publications have appeared in Library Quarterly, Journal of Critical Library and Information Studies, and Libraries: Culture, History, and Society.

Christy Khoury is a PhD student at the School of Information Studies at Syracuse University. Her primary work examines TikTok and other prominent social media platforms as spaces for crisis discourse, especially in the context of Middle Eastern crises. She is particularly interested in technologies as places where sensemaking, resilience, and cultural knowledges materialize. Her work has been in conferences such as the Association of internet Researchers (AoIR) and Association of Information Science and Technology (ASIST).

References

Abdurrahman

M. H.

Irawan

Setianingsih

(2020). A review of light gradient boosting machine method for hate speech classification on Twitter. In 2020 2nd International Conference on Electrical, Control and Instrumentation Engineering (ICECIE) (pp. 1–6). IEEE. https://doi.org/10.1109/ICECIE50279.2020.9309565

Agarwal

Harjule

Chouhan

Saraswat

Airan

Agarwal

(2021). Prediction of dogecoin price using deep learning and social media trends. EAI Endorsed Transactions on Industrial Networks and Intelligent Systems, 8(29), Article e2. https://doi.org/10.4108/eai.29-9-2021.171188

Alassani

Göretz

(2019). Product placements by micro and macro influencers on Instagram. In Meiselwitz

(Ed.), Social computing and social media. Communication and social communities (pp. 251–267). Springer. https://doi.org/10.1007/978-3-030-21905-5_20

Atkins

(2019). Black Lives Matter or All Lives Matter ? Color-blindness and epistemic injustice. Social Epistemology, 33(1), 1–22. https://doi.org/10.1080/02691728.2018.1483879

Barzilai-Nahon

(2008). Toward a theory of network gatekeeping: A framework for exploring information control. Journal of the American Society for Information Science and Technology, 59(9), 1493–1512. https://doi.org/10.1002/asi.20857

Bennett

W. L.

Segerberg

(2013). The logic of connective action: Digital media and the personalization of contentious politics. Cambridge University Press. https://doi.org/10.1017/CBO9781139198752

Blevins

J. L.

Lee

J. J.

McCabe

E. E.

Edgerton

(2019). Tweeting for social justice in #Ferguson: Affective discourse in Twitter hashtags. New Media & Society, 21(7), 1636–1653. https://doi.org/10.1177/1461444819827030

Brewster

M. L.

Lyu

(2020). Exploring the parasocial impact of nano, micro and macro influencers. International Textile and Apparel Association Annual Conference Proceedings, 77(1), Article 1. https://doi.org/10.31274/itaa.12254

Carney

(2016). All lives matter, but so does race: Black lives matter and the evolving role of social media. Humanity & Society, 40(2), 180–199. https://doi.org/10/gf3gsp

10.

Cavazos-Rehg

P. A.

Krauss

M. J.

Costello

S. J.

Kaiser

Cahn

E. S.

Fitzsimmons-Craft

E. E.

Wilfley

D. E.

(2019). “I just want to be skinny””: A content analysis of tweets expressing eating disorder symptoms. PLOS ONE, 14(1), Article e0207506. https://doi.org/10.1371/journal.pone.0207506

11.

Dalrymple

K. E.

Shaw

B. R.

Brossard

(2013). Following the leader: Using opinion leaders in environmental strategic communication. Society & Natural Resources, 26(12), 1438–1453. https://doi.org/10.1080/08941920.2013.820812

12.

Devlin

Chang

M.-W.

Lee

Toutanova

(2019). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. https://doi.org/10.48550/arxiv.1810.04805

13.

Djafarova

Trofimenko

(2019). ‘Instafamous’ – credibility and self-presentation of micro-celebrities on social media. Information, Communication & Society, 22(10), 1432–1446. https://doi.org/10.1080/1369118X.2018.1438491

14.

Edrington

C. L.

Lee

(2018). Tweeting a social movement: Black Lives Matter and its use of Twitter to share information, build community, and promote action. The Journal of Public Interest Communications, 2(2), 289–289. https://doi.org/10/gkmr32

15.

Francis

M. M.

Wright-Rigueur

(2021). Black Lives Matter in historical perspective. Annual Review of Law and Social Science, 17(1), 441–458. https://doi.org/10.1146/annurev-lawsocsci-122120-100052

16.

Freberg

Graham

McGaughey

Freberg

L. A.

(2011). Who are the social media influencers? A study of public perceptions of personality. Public Relations Review, 37(1), 90–92. https://doi.org/10.1016/j.pubrev.2010.11.001

17.

Freelon

McIlwain

C. D.

Clark

(2016). Beyond the hashtags: #Ferguson, #BlackLivesMatter, and the online struggle for offline justice. Center for Media & Social Impact, American University, Forthcoming.

18.

Fujita

Ishido

Inoue

Terano

(2018). Evaluating researchers through betweenness centrality measures of co-author networks from academic literature database: Finding gatekeeper researchers in organizational research. In 2018 IEEE International Conference on Big Data (Big Data) (pp. 4313–4320). IEEE. https://doi.org/10.1109/BigData.2018.8622311

19.

Hawkins

D. S.

(2023). “When you Search a #Hashtag, it feels like you’re searching for death”: Black Twitter and communication about police brutality within the Black community. Social Media + Society, 9(2), 20563051231179705. https://doi.org/10.1177/20563051231179705

20.

Hemsley

(2016). Studying the viral growth of a connective action network using information event signatures. First Monday, 21(8). https://doi.org/10.5210/fm.v21i8.6650

21.

Hemsley

(2019). Followers Retweet! The influence of middle-level gatekeepers on the spread of political information on Twitter. Policy & Internet, 11(3), 280–304. https://doi.org/10/gfwnpm

22.

Hemsley

Tanupabrungsun

Semaan

(2017). Call to retweet: Negotiated diffusion of strategic political messages. In Proceedings of the 8th International Conference on Social Media & Society (pp. 1–10). Association for Computing Machinery. https://doi.org/10.1145/3097286.3097295

23.

Hillard

Purpura

Wilkerson

(2008). Computer-assisted topic classification for mixed-methods social science research. Journal of Information Technology & Politics, 4(4), 31–46. https://doi.org/10.1080/19331680801975367

24.

Hitesh

Vaibhav

Kalki

Y. J. A.

Kamtam

S. H.

Kumari

(2019). Real-time sentiment analysis of 2019 election Tweets using Word2vec and Random Forest Model. In 2019 2nd International Conference on Intelligent Communication and Computational Techniques (ICCT) (pp. 146–151). IEEE. https://doi.org/10.1109/ICCT46177.2019.8969049

25.

Indra

S. T.

Wikarsa

Turang

(2016). Using logistic regression method to classify tweets into the selected topics. In 2016 International Conference on Advanced Computer Science and Information Systems (ICACSIS) (pp. 385–390). IEEE. https://doi.org/10.1109/ICACSIS.2016.7872727

26.

Jurvetson

(2000). What exactly is viral marketing? https://currypuffandtea.files.wordpress.com/2008/03/viral-marketing.pdf

27.

Jussila

Alkhammash

Alghamdi

Madhala

Khan

(2021). A netnographic-based semantic analysis of tweet contents for stress management. Computers, Materials & Continua, 70(1), 1845–1856. https://doi.org/10.32604/cmc.2022.017284

28.

Katz

Lazarsfeld

(1955). Personal influence: The part played by people in the flow of mass communications. Transaction Publishers.

29.

Kay

Mulcahy

Parkinson

(2020). When less is more: The impact of macro and micro social media influencers’ disclosure. Journal of Marketing Management, 36(3–4), 248–278. https://doi.org/10.1080/0267257X.2020.1718740

30.

Keib

Himelboim

Han

J.-Y.

(2018). Important tweets matter: Predicting retweets in the #BlackLivesMatter talk on twitter. Computers in Human Behavior, 85, 106–115. https://doi.org/10.1016/j.chb.2018.03.025

31.

Keyes

V. D.

Keyes

(2022). Dynamics of an American countermovement: Blue Lives Matter. Sociology Compass, 16(9), e13024. https://doi.org/10.1111/soc4.13024

32.

Khamis

Ang

Welling

(2017). Self-branding, “micro-celebrity” and the rise of Social Media Influencers. Celebrity Studies, 8(2), 191–208. https://doi.org/10.1080/19392397.2016.1218292

33.

Kim

Jang

S. M.

Kim

S.-H.

Wan

(2018). Evaluating sampling methods for content analysis of twitter data. Social Media + Society, 4(2), 2056305118772836. https://doi.org/10.1177/2056305118772836

34.

Kleinheksel

A. J.

Rockich-Winston

Tawfik

Wyatt

T. R.

(2020). Demystifying content analysis. American Journal of Pharmaceutical Education, 84(1), 7113. https://doi.org/10.5688/ajpe7113

35.

Lakoff

(2016). Moral politics: How liberals and conservatives think (3rd ed.). University of Chicago Press.

36.

G. M.

Radcliffe

Lyles

Lyson

H. C.

Wallace

Sawaya

Pasick

Centola

Sarkar

(2019). Perceptions of cervical cancer prevention on Twitter uncovered by different sampling strategies. PLOS ONE, 14(2), Article e0211931. https://doi.org/10.1371/journal.pone.0211931

37.

Lewin

(1947). Frontiers in group dynamics: II. Channels of group life; social planning and action research. Human Relations, 1(2), 143–153. https://doi.org/10.1177/001872674700100201

38.

Chen

Yuan

Cui

Huang

Jiang

Yao

Zhou

(2021). Establishment of noninvasive diabetes risk prediction model based on tongue features and machine learning techniques. International Journal of Medical Informatics, 149, 104429. https://doi.org/10.1016/j.ijmedinf.2021.104429

39.

Liu

Ott

Goyal

Joshi

Chen

Levy

Lewis

Zettlemoyer

Stoyanov

(2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692. https://doi.org/10.48550/arXiv.1907.11692

40.

Mason

(2022). Blue Lives Matter and Hate Crime Law. Race and Justice, 12(2), 411–430. https://doi.org/10.1177/2153368720933665

41.

McKibben

W. B.

Cade

Purgason

L. L.

Wahesh

(2020). How to conduct a deductive content analysis in counseling research. Counseling Outcome Research and Evaluation, 13, 156–168. https://www.tandfonline.com/doi/abs/10.1080/21501378.2020.1846992

42.

Nahon

Hemsley

(2013). Going viral. Polity Press. http://0-lib.myilibrary.com.emu.londonmet.ac.uk/browse/open.asp?id=586310

43.

O’Connor

Joffe

(2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 160940691989922. https://doi.org/10/ghhtrp

44.

Elayan

Sykora

Downey

(2021). Unpacking uncivil society: Incivility and intolerance in the 2018 Irish abortion referendum discussions on Twitter. Nordicom Review, 42(Suppl. 1), 103–118. https://doi.org/10.2478/nor-2021-0009

45.

Ott

R. L.

Longnecker

M. T.

(2015). An introduction to statistical methods and data analysis. Cengage Learning.

46.

Ouvrein

Pabian

Giles

Hudders

De Backer

(2021). The web of influencers. A marketing-audience classification of (potential) social media influencers. Journal of Marketing Management, 37(13–14), 1313–1342. https://doi.org/10.1080/0267257X.2021.1912142

47.

Pew Research Center. (2023, June 14). 1. Views on the Black Lives Matter movement. Pew Research Center’s Social & Demographic Trends Project. https://www.pewresearch.org/social-trends/2023/06/14/views-on-the-black-lives-matter-movement/

48.

Ragini

J. R.

Anand

P. M. R.

(2016). An empirical analysis and classification of crisis related tweets. In 2016 IEEE International Conference on Computational Intelligence and Computing Research (ICCIC) (pp. 1–4). IEEE. https://doi.org/10.1109/ICCIC.2016.7919608

49.

Saif

Fernandez

Alani

(2014). On stopwords, filtering and data sparsity for sentiment analysis of Twitter. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14) (pp. 810–817). European Language Resources Association. http://www.lrec-conf.org/proceedings/lrec2014/pdf/292_Paper.pdf

50.

Saldana

(2015). The coding manual for qualitative researchers. Sage.

51.

Shanahan

Wall

(2021). “Fight the reds, support the blue””: Blue Lives Matter and the US counter-subversive tradition. Race & Class, 63(1), 70–90. https://doi.org/10.1177/03063968211010998

52.

Shoemaker

P. J.

Vos

(2009). Gatekeeping theory. Routledge.

53.

Starbird

DiResta

DeButts

(2023). Influence and improvisation: Participatory disinformation during the 2020 US election. Social Media + Society, 9(2), 20563051231177943. https://doi.org/10.1177/20563051231177943

54.

Stewart

L. G.

Arif

Nied

A. C.

Spiro

E. S.

Starbird

(2017). Drawing the lines of contention: Networked frame contests within #BlackLivesMatter discourse. Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1–23. https://doi.org/10/gf3gsk

55.

Stromer-Galley

Rossini

(2023). Categorizing political campaign messages on social media using supervised machine learning. Journal of Information Technology & Politics, 1–14. https://doi.org/10.1080/19331681.2023.2231436

56.

Thorson

Wells

(2015). How gatekeeping still matters: Understanding media effects in an era of curated flows. In Vos

Heinderyckx

(Eds.), Gatekeeping in transition (pp. 25–44). Taylor & Francis.

57.

Thorson

Wells

(2016). Curated flows: A framework for mapping media exposure in the digital age: Curated flows. Communication Theory, 26(3), 309–328. https://doi.org/10/f878cg

58.

Tillery

A. B.

(2019). What kind of movement is Black Lives Matter? The view from Twitter. Journal of Race, Ethnicity and Politics, 4(2), 297–323. https://doi.org/10/gh7smx

59.

Updegrove

A. H.

Cooper

M. N.

Orrick

E. A.

Piquero

A. R.

(2020). Red States and Black Lives: Applying the racial threat hypothesis to the Black Lives Matter Movement. Justice Quarterly, 37(1), 85–108. https://doi.org/10.1080/07418825.2018.1516797

60.

Upton

G. J. G.

(1992). Fisher’s exact test. Journal of the Royal Statistical Society. Series A (Statistics in Society), 155(3), 395–402. https://doi.org/10.2307/2982890

61.

Varoquaux

Colliot

(2023). Evaluating machine learning models and their diagnostic value. In Colliot

(Ed.), Machine learning for brain disorders (pp. 601–630). Springer. https://doi.org/10.1007/978-1-0716-3195-9_20

62.

Walter

Brüggemann

(2020). Opportunity makes opinion leaders: Analyzing the role of first-hand information in opinion leadership in social media networks. Information, Communication & Society, 23(2), 267–287. https://doi.org/10.1080/1369118X.2018.1500622

63.

Walter

Hanke

K.-J.

(2020). Opinion leaders in the digital age—Social Network Analysis for Renewable Energies on Twitter. Mining Report, 156(6). https://mining-report.de/english/opinion-leaders-in-the-digital-age-social-network-analysis-for-renewable-energies-on-twitter/

64.

White

M. D.

Marsh

E. E.

(2006). Content analysis: A flexible methodology. Library Trends, 55(1), 22–45. https://doi.org/10.1353/lib.2006.0053

65.

Shen

(2015). Analyzing and predicting news popularity on Twitter. International Journal of Information Management, 35(6), 702–711. https://doi.org/10.1016/j.ijinfomgt.2015.07.003

66.

H. H.

Gallagher

R. J.

Alshaabi

Adams

J. L.

Minot

J. R.

Arnold

M. V.

Welles

B. F.

Harp

Dodds

P. S.

Danforth

C. M.

(2023). Say their names: Resurgence in the collective attention toward Black victims of fatal police violence following the death of George Floyd. PLOS ONE, 18(1), Article e0279225. https://doi.org/10.1371/journal.pone.0279225