Abstract
News journalism has evolved from traditional print media to social media, with a large proportion of readers consuming their news via digital means. Through an analysis of over 1.3 million posts across three social media platforms (Facebook, Twitter, Reddit) pertaining to the 2022 U.S. Midterm Elections, this analysis examines the difference in sharing patterns for four types of news sites—Real News, Local News, Low Credibility News, and Pink Slime. Through Platform-Based Analysis, this study observes that users across all platforms share Real and Local News sequentially, and Real News and Low Credibility News sequentially. Through News Type-Based Analysis, this study establishes a Relative Engagement metric, demonstrating a widely varied engagement among the news types. Real News receive the least engagement (defined as the ratio of number of likes a post has vs. the number of followers of the page), while users engage with Pink Slime news the most. Furthermore, this study finds that the sharing of automated local news reporting sites (Pink Slime sites) are divided on political lines. Finally, through a User-Based Analysis, this study finds that automated bot users share a larger proportion of Pink Slime and Low Credibility News, while human users generally share content relating to local communities.
Introduction
Journalism is a key way of disseminating information, of broadcasting “news.” The Oxford Language dictionary refers to the term “news” as newly received and noteworthy information, mostly about recent and important events.
The landscape of journalism is changing in the digital era. An increasing proportion of news are being written for and shared via social media as opposed to traditional print medium. In a 2021 survey by Pew Research Center, almost half (48%) of the adults in the United States agree that they regularly get news from social media platforms, with the largest proportion of news consumption coming from Twitter, Facebook, and Reddit (Walker & Masta, 2021).
Within journalism, there are several types of news. Distinctly, there are four main types of news that are spread: Real News which are credible news, Local News which are credible news targeted at specific geographical communities, Low Credibility News which are misleading news, and Pink Slime News which are low-credibility, often partisan news disguised as a local publication to garner readership through local trust.
Political journalism is highly scrutinized due to the advent of citizen journalism, where everyday users rather than professional journalists can post updates on social media platforms. This public participation has been observed in elections in the United States, Africa, and Asia (Moyo, 2009; Ritonga & Syahputra, 2019). The type and popularity of news shared can potentially alter election outcomes, with many past studies showing that mass media do have an effect on American politics (Dunaway & Graber, 2022), and the types of news shared can affect voting patterns (Allcott & Gentzkow, 2017). The proportion of professional news content was measured to hit its lowest point on Facebook and Twitter the day before the 2016 U.S. Presidential elections, and overpowered by junk news, many of which are shared by social media bots (Howard et al., 2017), sharing similarities with other studies of the influence of fake news on Twitter during the same event (Bovet & Makse, 2019).
The existing literature predominantly compares real and low credibility news dissemination, and more research is necessary in the analysis and comparison of other news types, such as local news and Pink Slime News. Over 1,000 of these types of news sites have appeared in the past 5 years and often contain algorithmically generated content that poses as real local news (Bengani, 2019), which has maintained a high level of trust from the American public (Gottfried & Liedke, 2021). Some existing pieces of work on pink slime news outlets observe that these outlets flood the local media ecosystem ahead of the elections and highlight contentious political issues in an inflammatory manner (Moore et al., 2023). Therefore, it is essential to include pink slime news sites and local news sites in the study of a political election. Such a study contrasts news targeting the local community (Local News and Pink Slime News) against news types targeted at broader audiences (i.e., Real News, Low-Credibility News). We build on previous work to extend examination of news type dissemination on social media toward local and pink slime news sites.
Another aspect of news dissemination on social media is the use of automated agents or bots. Bots have been observed to spread low-credibility content on social media (Shao et al., 2018), thereby influencing political outcomes (Ferrara, 2020; Ng & Carley, 2023c).The spread of low credibility news by social media bots is very much of scrutiny within the computational social science community, as these automated users are an effective tool to manipulate social media narratives, and thereby the opinions of human users (Shao et al., 2017). Much of the studies on bot users involve studies on the spread of Low Credibility News, and this study bridges this research gap to examine the bot activity in the spread of other types of news (i.e., Local News, Pink Slime News).
This article analyzes news from the 2022 U.S. Midterm Elections from a multi-platform perspective. The United States held its Midterm elections on 8 November 2022. A total of 107.7 million Americans voted 36 governors, 35 senators, and all 435 voting seats in the House of Representatives were up for election (Desilver, 2023). Since the Presidency was not up for election, the regional elections took the spotlight and forefront of news coverage. Through the analysis of URLs shared within each platform, we provide insights into the patterns of news sharing across platforms, types of news and the users that participate in the sharing activity. We do so using a combination of statistical and network-based techniques.
In our study of multi-platform news journalism, we use the following research questions to guide our analysis:
RQ1: How do the three social media platforms (Facebook, Reddit, and Twitter) differ by news type shared? We answer this question using a Platform-Based Analysis approach, where we analyze the proportions of news types in the URLs shared across each platform, and a network analysis of the sequential transition of news type sharing by users within each platform.
RQ2: How do engagement of the different news types shared differ? We answer this question using a News Type-Based Analysis approach, where we define and compare a Relative Engagement metric for each post and compare the metric across each of the news types.
RQ3: How do classes of users differ in their news type sharing behavior? We answer this question using a User-Based Analysis approach, and examine the sharing of pink slime journalism news in correlation with other types of news. Here, we segregate users into two main groups—social media bots and humans, before analyzing their news type sharing patterns using network analysis, and analyze the user sharing patterns along political lines with network and statistical analysis.
To analyze the sharing of the different types of news across social media platforms, we make use of user, content, and network analysis. Figure 1 presents a diagrammatic overview of our methodology. The analysis methodology is split into three blocks: Data Collection, Data Annotation, and Data Analysis.
Overview of analysis methodology.
We study the three most popular social media platforms for news consumption: Facebook, Twitter, and Reddit. Specifically, we collect data from Facebook Pages, Reddit posts and comments, and Twitter Tweets. We also study four different news types: Local News, Real News, Pink Slime News, and Low Credibility News. Although we are primarily interested in Pink Slime journalism, past work has shown that the consumption patterns of Pink Slime News differ considerably from other types of news, especially Low Credibility News and Local News (Moore et al., 2023). The study and discussion of Pink Slime News alongside the other three types of news provides context into the conceptualization of the similarities and distinctions of Pink Slime News.
Contributions
Our results show that across platforms, Reddit leads with the highest proportion of Real News, while Facebook leads with the highest proportion of Local News sites, and Twitter has the highest proportion of Pink Slime and Low Credibility News sites; Pink Slime sites gather the highest engagement and bots tend to share information of lower quality while humans lean toward sharing local content. Specifically, we make the following contributions:
Define news types across a scale of credibility and locality, and identify the prevalence of news types on each platform. Furthermore, we identify the likelihood of a user sharing a news type on the different platforms given their previous news types shared.
Introduce the measurement of relative engagement to Facebook Page engagement metrics and demonstrate its differences in the different news types.
Add to the understanding of the user behavior of individuals and groups sharing Pink Slime sites by comparing the news sharing habits of those users with the political ideology of the Pink Slime sites they shared.
The structure of this article is as follows: we first provide a brief literature review of past research work on journalism, news dissemination on social media. Then we define the four journalistic news type along with an overview of our methodology. Next we describe our data collection and annotation procedures, before moving on to a description of our data analysis from three perspectives: platform-based analysis, news type-based analysis, and user-based analysis. We engage in a discussion section before providing the concluding remarks.
Related Work
News Sharing on Social Media
News sharing on social media is largely focused on the differences in sharing patterns of news of different credibility. Rogers (2023) identified that through an analysis of multiple social media platforms “fake news” does not outperform mainstream news during any period of their study, but the proportion of user engagement with fake news to mainstream news stories is higher.
In a digital landscape where there is competition for user attention, Facebook emerges as the dominating platform for news sharing in a study across multiple social media platforms (Kalsnes & Larsson, 2018). Past work on the analysis of types of news shared from Facebook measured through URL shares considered the sharing of types of news that include local and Pink Slime News, discovered that the Facebook news intervention system reduced visits to low-quality news outlets (or Pink Slime News), but also affected the number of views high-quality news (or real news) received (Bandy & Diakopoulos, 2023). Whereas this work measured the impact of Pink Slime News at the platform level, we extend this analysis toward a finer granularity of Facebook Pages level.
Studying news types on social media also provides indication toward the ability of users to differentiate between real and misleading news. Previous research finds a positive relationship between age and political ideology in sharing low credibility news (Guess et al., 2019; Hopp et al., 2020). However, Real News and Low Credibility News have been observed to exhibit similar spread patterns in a multi-platform dataset (across Twitter, Instagram, YouTube, Reddit, and Gab), potentially indicating that users are unable or unwilling to distinguish between credible and non-credible news (Cinelli et al., 2020b).
Engagement of News Types
Past work has showcased poor-quality and intentionally misleading information sources in the U.S. media sphere surrounding the 2016 U.S. elections, defined as “pink slime” and “junk” information, which focuses on garnering clicks through sensationalism rather than information dissemination through journalism (Benkler et al., 2018). Other findings suggest that Pink Slime News sites are not spread on top political subreddits (Burton & Koehorst, 2020), but that smaller, neighborhood-oriented Facebook communities are more susceptible to sharing local low credibility news (Mihailidis & Foster, 2021). In particular, an analysis of user news consumption during the 2020 U.S. Presidential Election showed that Facebook disproportionally boosted Pink Slime News sites (Bandy & Diakopoulos, 2023). Concretely, the news consumption during the 2020 election measured that around 3.7% of American adults visited at least one Pink Slime site, with 17.7% of these sites coming from Facebook referrals (Moore et al., 2023). During crisis events such as the 2020 Coronavirus pandemic, Local News receive the largest engagement, and the engagement increased as the reach of the coverage decreases, highlighting the importance of catering regional news for specific neighborhoods (Le Quéré et al., 2022).
Use of Bots for News Dissemination
Another widely studied aspect of news sharing in social media is the usage of bots for news dissemination. Social bots have been identified to construct and amplify low credibility news on social media, especially on Twitter, thereby increasing the proportion and propagation of low credibility news on the platform (Shao et al., 2017). The extent of robot journalism has also created “News Bots,” where journalists and bloggers automatically post recently published articles or breaking news, or algorithmically generate news articles and post them (Lokot & Diakopoulos, 2016; Moore et al., 2023; Stieglitz et al., 2017). Bots are active news promoters, readily updating the public with valuable public health information during the 2020 Coronavirus pandemic and referencing credible news outlets (Al-Rawi & Shukla, 2020). On the contrary, news bots also disseminate specific sets and genres of news to promote specific narratives that align with the diplomatic perspectives, such as Chinese geotagged news bots focused on surveillance aspect in the 2023 U.S.–China balloon incident (Ng & Carley, 2023b). Several bot detection tools have been developed using supervised machine learning techniques to identify social bots, differentiating them from humans based on properties of their activity such as the temporal difference and activity of posts and type of posts (Feng et al., 2022; Ng & Carley, 2023a). We extend past work on the study of social media bots from studying the real news/low credibility news duality toward studying multiple news types.
Multi-Platform Studies of Social Media
With regard to multi-platform work on social media, past work looking into vaccine-related information on Facebook, Reddit, and Twitter through keyword search and textual analysis found that low credibility news and fake health information were most prevalent on Twitter and least prevalent on Facebook (Pulido et al., 2020). Work on the 2021 U.S. Capitol Riots which occurred as a protest toward the 2020 Presidential Election results showed that there are similar narratives but separate information dissemination chambers within Parler and Twitter (Ng et al., 2022a). The dissemination of Low Credibility News on Facebook, Twitter, and Reddit by Russia suggests that Reddit is used as a trial platform to identify messages that are optimal for distribution on other platforms (Lukito, 2020). A user-based study of the same event inferred how social media users interact with each other across platforms through the use of URLs, identifying the users and types of URLs that facilitate multi-platform content diffusion (Murdock et al., 2023). The identification of identical users across platforms uses friendship graph to calculate the match degree for candidate user pairs to extract the top-ranking user pairs as identical users (Zhou et al., 2015). We build on techniques developed in these past multi-platform and elections work in extracting and analysis of URLs to analyze the news sharing behavior in the 2022 U.S. elections across three social media platforms.
Types of Journalism News
Here, we define the four key types of news that we study within this work. Studying the prevalence and interactions of these four types of news provides insight into a variety of news types that exist on social media platforms. With the evolving landscape of digital journalism, the news types are no longer falling into binary categories of credible versus fake news, but rather, exist on a plane of scope and credibility. In this article, we examine the four main types of news in this digital era:
Real News, which are credible news written by journalists who adhere to the journalistic code of ethics for gathering information and writing the articles (Berkeley, 2023). Examples of Real News outlets are CNBC and WSJ.
Local News, which are news that are targeted to specific communities demarcated by geographical boundaries (Stonbely et al., 2015). Example of Local News outlets are AZ Central and The Atlanta-Journal Constitution.
Pink Slime News that masks as local news, and often publish on partisan and inflammatory articles that serve special interests (Moore et al., 2023). Examples of Pink Slime News outlets are The Arizona Sun Times and The Keystone Newsroom.
Low Credibility News, which are misleading news that can deceive readers or provide wrong information (Ng & Taeihagh, 2021; Shu et al., 2020). Examples of Low Credibility News outlets are Breitbart and LADbible.
Figure 2 represents the relationship between each type of news in terms of scope and credibility. Real News and Local News are both credible news, which is defined as news articles that are written by journalists adhering to a code of ethics for information gathering and writing the articles. The difference lies in their focus: Real News has a national focus, thus presenting articles from a broader scope, while Local News focuses on specific geographical communities which they target. Pink Slime News and Low Credibility News are both non-credible news, colloquially termed as “fake news.” In a similar fashion, Low Credibility News have a more national scope and thus present a diverse set of articles and topics, while Pink Slime News have a regional scope, typically presenting political news related to a specific geolocation.
Relationship between each news type.
Examples of each type of news headlines are presented in Table 1. Each of these news types have their unique properties, and through our analysis, we provide insight into the propagation behavior and popularity of each of these news.
Sample Headlines of the Top Shared News Stories in the Dataset by News Type.
Data Collection and Annotation
Data Collection
Data pertaining to the U.S. 2022 Midterm Elections were collected from three social media platforms: Twitter, Reddit, and Facebook. While Twitter has recently rebranded to X, within this article, we refer to the platform as “Twitter” throughout this work, because the platform-provided application programming interface (API) is still termed as “Twitter Developer API.” The data were collected relating to states with the most contentious elections. These states were selected from the most competitive districts as referred to by election analysis group FiveThirtyEight (Groskopf, 2022). These include regions in Arizona, Georgia, Pennsylvania, Nevada, Wisconsin, and North Carolina. We chose to study swing states as Pink Slime sites are more present in battleground states as they generally publish political information, and therefore influence elections (Royal & Napoli, 2022). Therefore, we chose to include only these contentious states in order to have a good number of pink slime data points. We collected the data using the corresponding platform’s API, through the use of searching terms of interest, which are terms that join both election-related keywords and state-specific keywords. The full set of keyword terms that we used to search is listed in the Appendix.
The corresponding APIs that are used are the Twitter researcher development API, 1 Reddit’s Pushshift API, 2 and Facebook’s CrowdTangle API. 3 The type of message data collected from these platforms are Twitter tweets, Reddit posts and comments, and Facebook Pages. These data type were chosen mainly because they were the returned result of the provided API. Each message was tagged to a unique user ID during this data collection phase, and this user ID will be used in the User-Based Analysis section to segregate slices of analysis by user properties. A Facebook Page is considered as a user ID, as are individual Twitter and Reddit users.
We captured the conversation on the social media platforms 1 month prior to and 1 month after the 2022 U.S. Midterm Elections. The elections took place on 8 November 2022. Therefore, we collected data from 1 October to 1 December 2022. This captures both before- and after- voting conversation, providing a more holistic view of the conversation.
We visualized the states that are studied within this article in Figure 3, which shows the key states of contention (Groskopf, 2022) where local news and Pink Slime sites would be actively promoting the elections. In total, our collection includes 1,383,896, 28,178, and 16,375 posts from Twitter, Facebook Pages, and Reddit, respectively. Of those posts, 851,828 (Twitter), 17,268 (Facebook Pages), and 7,811 (Reddit posts) linked to URLs that had a designated news type rating. These statistics are summarized in Table 2.
U.S. map with the contentious states that are studied within this study colored in red.
Statistics of Data Collected Pertaining to the 2022 U.S. Midterm Elections.
Annotating News Labels
Once the data were collected, we extracted the URLs from each post. We extracted out the site domains from the URLs, which is the identification name that refers to a website, for example, “yahoo.com.” Following which, we annotated the URLs, and thereby the corresponding posts that the URL originated from with a news type label. The news type label is one of the four news types that we study in this article (see Figure 2). We derive these labels for each URL by matching its domain with a Media Thesaurus, a precompiled list of domains and news type.
The Media Thesaurus is a news media list that contains the news domain and its news type classification. It is compiled from multiple publicly available lists: (1) from Media Bias/Fact Check 4 which lists many news sites and rates the factuality and credibility of the reporting; (2) the George Washington University Dataverse (Littman et al., 2020) which categorizes a list of over 9,600 Twitter accounts for media organizations that are derived from over 160 million Tweets between 2016 and 2020; (3) the Columbia Journalism Review as a source for hundreds of Pink Slime News outlet domains (Tow, 2020); (4) a Github Repository 5 that collages unreliable and misleading news sources from Snopes Field Guide, Wikipedia, and other domains; (5) a Github Repository 6 that consolidates a list of most frequented web domains and most frequently tweeted domains by U.S. politicians and the corresponding news type labels; and (6) a consolidation of Local News through a list of authentic local news sites owned by companies (Clemm von Hohenberg et al., 2021; Free Press, 2022). After consolidation, this Media Thesaurus is harmonized among the sources. Where there are overlaps between these sources, particularly for the less factual news outlets, to resolve any conflicts that emerge between the sources, the thesaurus errs on the side of not labeling a news source in question as low credibility news.
This Media Thesaurus also consolidates the political leaning of each site in the context of the U.S. democracy (right-leaning or left-leaning), and we annotate the URLs shared on the social media platforms with their political leanings.
Data Analysis
After the previous two stages which collect and prepare the data for analysis, we perform data analysis via a three-pronged approach to investigate our research questions. This section describes the data analysis in terms of Platform-Based Analysis, News Type-Based Analysis, and User-Based Analysis.
RQ1: Platform-Based Analysis
We answer the first research question on how the social media platforms differ by the types of news shared through performing a Platform-Based Analysis. This analysis involves analyzing the proportion of each type of news sites shared within each platform. We do this by segregating the extracted URLs and their corresponding annotated labels into the platform by which these URLs originated from, then computing the proportion of each news type against the platform.
Next, we use network analysis techniques to investigate the sequential transition of news type sharing by users within each platform. We construct directed network graphs for each platform, where the nodes represent the type of news. For each user, we parse the sequential order of their posts, noting the type of news shared in consecutive posts. A link in this network graph is thus defined as a user consecutively sharing a URL from two different news type. For example, if a user shares a Local News URL in the first post then shares a Real News URL in the second post, a link on the graph will be drawn in the direction from the Local News node to the Real News node. Only immediately sequential or consecutive shares within our dataset are included; therefore, a user could have tweeted a Local News URL and then something unrelated to the elections and then a Real News URL. The tweet that is not pertaining to the elections would not have fallen within our data collection, and it would still count as a Local News to Real News connection. Then, we quantify the proportion of each type of link, and represent them as the thickness of the link width in the network graph. These graphs are then plotted using the ORA visualization software (L. Carley et al., 2018), which facilitates graphing integration with a variety of data sources, including Pushshift, CrowdTangle, and Twitter Developer API, which we used to collect our data in this study.
RQ2: News Type-Based Analysis
The ability of social media to reach large audiences in minimal time has established it as an important source of news for voters. It is important, therefore, to determine the quality and credibility of news shared over social media and assess differences in user response and engagement by type of news media.
We do this through a News Type-Based Analysis, answering the research question on how news engagement differs by different news types. Our analysis examines the interaction ratio of individual posts relative to the overall number of users in the community. We perform this only for the Facebook data, because Facebook posts are made to Facebook Pages, therefore the metric for an individual content is the post while the measure of community is the Pages. Twitter does not have an appropriate grouping feature, and thus is excluded from this portion of the analysis. Finally, while the Reddit data provided the number of likes to individual posts and the number of subscribers for a subreddit, it does so only for the initial posts of a Reddit thread and not its comments. This thus removes a large portion of posts (only 21% of the Reddit dataset was of posts compared with 79% comments), and therefore we decided to exclude the platform Reddit from the analysis. In addition, past research shows that many Pink Slime journalism outlets rely heavily on Facebook to reach most of their users, thereby making the Facebook platform an ideal candidate for engagement analysis bandy2023facebook.
Through these interaction values, we developed a relative engagement metric to establish how well posts made to each of these groups performed. This relative engagement metric measures engagement for Facebook posts on Pages as a proportion of the total number of followers or likes to the Page. Specifically, the relative engagement is defined as per Equation 1
After calculating the Relative Engagement metric for each post, the posts were segregated into the four news types, and the Relative Engagement metrics for each news types were compared against each other.
RQ3: User-Based Analysis
To answer the research question of the differences of user classes in sharing each news type, we perform a User-Based Analysis using network analysis, in terms of understanding the sharing patterns of users that spread Pink Slime News. We focus on users that share pink slime domains in our examination of the users, because the sharing of Pink Slime News is an understudied area of research. To do so, we examine the sharing patterns of users who shared Pink Slime News in correlation with their sharing patterns of other news types. We characterized each user by the political leaning through the slant of news URLs the user shared. This results in three categories: Left, where the user shares only news from left-leaning pink slime organizations; Right, where the user shares only news from right-leaning pink slime organizations; and Both when the user shares news from both left- and right-leaning pink slime organizations. Then, for each news type, we make comparisons across the the proportion of users that share news of each category of political leaning.
We also use User-Based Analysis to study the phenomenon of Pink Slime journalism. To do so, we further filtered the dataset to exclusively include users who shared Pink Slime sites within their posts, with an aim to investigate the behavior of these individuals. We first analyze the distribution of news type by the users who shared Pink Slime News, to survey what other types of news these users share. We group these information by news type (Local News, Low Credibility News, Pink Slime, Real News, and Unknown news type) and users sharing them by the political ideology of the Pink Slime sites they shared (Left-leaning, Right-leaning, and sharing Pink Slime sites with Both political leanings). This distribution was calculated as a proportion of total links the users were posted. To better understand whether there is a significant difference in this distribution, we used a chi-squared test to compare the three distributions of news type sharing according to political ideology. The null hypothesis
where:
Next, we construct a user x domain network diagram where there are two types of nodes present: user nodes and domain nodes. The user nodes are linked to the domains they share, and the domain nodes are colored by the type of news. This analysis provides us a view of the partitions of domains shared by users.
In addition, due to the prevalence of bots on the Twitter platform (Saeed et al., 2022; Yang et al., 2019), we extend the investigation of Twitter to examine the differences in sharing patterns of bots and humans. We used a similar methodology to that used in the Platform-Based Analysis section: the network diagram visualizations were constructed using the ORA software (L. Carley et al., 2018), due to its integration with the data sources.
We then perform a deeper analysis of the users on the Twitter platform. The Twitter data consist of the most number of users and posts, and coupled with its user moderation practices, the platform is commonly studied for the activity of automated bot users (Gorwa et al., 2020). The data were treated into bot and human clusters only within this part of the analysis, which aims to examine the degree of automation used in news sharing habits within the online ecosystem.
We first annotate users into bot and human user classes. The usage of bots in the news sharing landscape represent inorganic news sharing while the usage of humans represent organic person-to-person news sharing. We do so by using the BotHunter algorithm (Beskow & Carley, 2018), which uses a tiered random forest approach to analyze the user data and provide a bot probability score in the range of [0,1]. A bot probability score that is closer to 0 indicates the user is more likely to be human, while a bot probability score closer to 1 indicates the user is more likely to be a bot. We use a threshold value of 0.7, derived from past work of statistical analyses of large datasets (Ng et al., 2022b), to determine whether a user is a bot or not. That is, if the bot probability score is above 0.7, we annotate the user with a bot label; otherwise we annotate the user with a human label. This algorithm is trained on publicly available datasets of Twitter users annotated as bot or not by expert human annotators, and have achieved an accuracy of ~90% on these datasets. Although the data that this algorithm were trained on may not fully reflect bot users in the current social media climate for users evolve, this algorithm is chosen for its ability to efficiently analyze historical data en-masse without requiring a live connection to the Twitter platform. While this article did not perform a manual verification on a sampling of data, recent studies have conducted observational verification in the elections context, manually verifying a handful of bot/human user accounts, lending weight to the algorithm (Uyheng et al., 2021).
After obtaining a bot label for each user, we split the Twitter users into bot and human user classes. From these data, we constructed directed network graphs of the two bot/human user classes for each platform, where the nodes represent the type of news, and a link between two news class types show that the user had consecutively shared news from both news classes.
Results
RQ1: Platform-Based Analysis
We present the evaluation of proportionality of news site types in Table 3. Across the three platforms, we observe that Reddit leads with the highest percentage of Real News, and the lowest percentage of Low Credibility News. This is likely due to the work of subreddit moderators, who have been observed to remove over 2.8 million comments across 10 months that include themes such as harmful speech and low credibility news (Chandrasekharan et al., 2018). These hidden warriors on the internet work behind the scenes to clean the online space. The proportion of Low Credibility News and Pink Slime is extremely low on Reddit (⩽1%).
Breakdown of News Types Shared on the Three Platforms, as Percentages of the Amount of Each News Type Site as a Total of the Number of Sites Shared Within Each Platform.
We point out the existence of Pink Slime sites on Reddit. While the proportion is low at 0.05%, it is contrary to past work by Burton and Koehorst (2020), who did not observe any Pink Slime sites linked within political subreddits. We posit that this is due to the difference in data collection: we searched the platform for posts by keywords related to the elections, while they pulled data from specific political subreddits. Within this dataset, we found 11 references to Pink Slime sites, mostly on smaller subreddits geared toward a local community.
The Facebook platform leads in the proportion of links of Local News sites, reflecting the community structure of the Facebook data, where the posts are drawn out of Facebook Pages. Out of the three social media platforms, the proportion of Real News on the Facebook platform is the lowest, suggesting the need for better content moderation on this platform.
Twitter has the highest percentage of Low Credibility News and Pink Slime sites shared as a total proportion of its overall sharing. Among the four different news types, the proportion of Low Credibility News news sites shared is 4.96%, and that of Pink Slime sites shared is 2.24%. This higher proportions are in line with past research on lists of URLs spreading political misinformation on social media during the 2016 U.S. presidential elections (Bovet & Makse, 2019; Grinberg et al., 2019), and previous findings that misinformation is most prevalent on Twitter (Pulido et al., 2020). Furthermore, the 2022 U.S. Midterm Elections coincided with a change in power and mass layoffs at Twitter, which may have altered its ability to perform content moderation.
Figure 4 presents the network diagrams constructed to visualize the URL-sharing behavior per platform. This figure illustrates the transition changes in the news type a user will post after previously posting from a designated news type. This diagram presents self-loops among all news types and among all platforms, indicating that users continue sharing content within the same news type category. In fact, across all platforms, the most likely outcome is for a user to share from the same news type category in consecutive posts. This phenomenon reflects that of political echo chambers in social media, where social media users share information that are similar to their own, because they are mainly exposed to opinions that are similar to their own terren2021echo. The formation of echo chambers and its amplification has been observed on Twitter across 10 different politically polarizing topics such as gun control and abortion (Garimella, De Francisci Morales, Gionis and Mathioudakis, 2018).
Likelihood of sharing one news type based on previous news type shared by platform. Nodes represent news type; links between two nodes represent users shared the two news types in consecutive posts; link thickness represents the likelihood of sharing two news types in sequence.
The proportion of self-loops is reflected in Table 4. Twitter and Reddit have the highest proportion of self-loops in Real News, while Facebook has the highest proportion in Low Credibility News. Twitter also has the next highest proportion of Pink Slime News as self-loops. We also observe that Reddit does not have a self-loop in Pink Slime News, because there are no users that share this category of news consecutively. These suggest the proportionality of users that remain in their same news sharing information spheres across the three platforms.
Percentage of Users Who by Platform and News Type of Continue to Post Within the Same News Type (i.e., Self-Loops).
All platforms exhibit signs of consecutive news sharing between Real News and Local News, showing that users generally consume both general and local news concurrently. Facebook and Twitter exhibit a good proportion of news sharing between Real News and Low Credibility News. This might reflect consistency with past work that humans do have difficulty differentiating Real News from Low Credibility News and therefore share both types of news (Vosoughi et al., 2018), or are also sharing Low Credibility News URLs in the context of Real News to make a statement like rebutting them. However, this transition is not present on Reddit, which is likely due to the low percentage of Low Credibility News sites shared on the platform.
No Reddit users that shared Local News also shared Pink Slime News, but the sharing between Local News and Pink Slime News exists on Facebook and Twitter. This indicates that Pink Slime News do have more credibility in the eyes of views on Facebook and Twitter.
With regard to Pink Slime News sites, the network diagrams indicates that users are likely to share these news sites in silos, visualized through self-loops and negligible proportion of links to other news types. Individuals on Twitter and groups on Facebook and Reddit are much more likely to share multiple sources of authentic local news sites than those sharing Pink Slime sites are to share more than one. This perhaps could be due to a separation of information consumption and spreading spheres for each social media platform, where news could be tailored for the user demographics of the platform (Ng et al., 2022a).
RQ2: News Type-Based Analysis
For News Type-Based Analysis, we defined communities as per Facebook Pages. The median group size of the Facebook pages is 8,928 subscribers, and the smallest 25% of groups have fewer than 1,653 subscribers. We first examine the distribution of each news type in terms of the size of Facebook Pages segregated in quartiles. This is reflected in Table 5. We observe that the two news type with the highest frequency in the smallest 25% of groups are Low Credibility News and Pink Slime. Since Pink Slime by nature targets communities in smaller geographic regions than the whole country and has some content overlap with Low Credibility News, it follows that it would be shared to smaller groups like Low Credibility News is. In addition, when Real News was shared, it was most commonly (56% of the time) shared to the 25% largest groups in the dataset. This may indicate that Real News appeals to broader, larger groups and is more widely trusted by communities with more people in them.
Distribution of News Types Shared to Facebook Across the Quartiles of the Facebook Pages Group Sizes.
We next present the logarithmic distribution of Relative Engagement measured by the number of engagements per group size for each news type in Figure 5. In this figure, we observe that Pink Slime gathers the highest relative engagement, followed by Low Credibility News, then Local News, then Real News. We break down this boxplot distribution numerically in Table 6. Besides looking at the average Relative Engagement, we also examine the standard deviation. The standard deviation of Relative Engagement for Real News is the highest, suggesting that the reception of Real News varies widely, and is likely to depend on a lot of factors not examined within this study, like catchiness of headlines (Kim et al., 2016). The standard deviation of Low Credibility News is the lowest, perhaps because Low Credibility News is generated with a templated suite of linguistic features; in fact, Low Credibility News detectors have been trained to spot Low Credibility News through eye-catchy headlines (Collins et al., 2020).
Logarithmic distribution of relative engagement measure by news type for Facebook.
Relative Engagement Metrics by News Type.
RQ3: User-Based Analysis
The distribution of news types that are also shared by users that share Pink Slime News is visualized in a bar graph in Figure 6. From this graph, we observe that users sharing left-leaning Pink Slime sites shared more local news and less disinformation than those sharing right-leaning Pink Slime sites.
Distribution of the news types shared by users who shared pink slime, grouped by whether the user shared pink slime from a right-leaning pink slime organization, a left-leaning pink slime organization, or both.
Our chi-square analysis across the users that are sharing different news type returns the following values: the chi-square statistic is 22,223.9449 with a p-value of < .00001, at 4 degrees of freedom. Using a significance level of
The network-based user analysis for Pink Slime journalism is visualized in Figure 7. A force-directed continuous graph layout algorithm is run on the graph for community segregation (Jacomy et al., 2014), where we observe two clear clusters. We demarcated the clusters within the figure itself. These clusters correspond to the political leanings of the sites: right- and left- leaning Pink Slime sites. Therefore, we observe that users do primarily share news within their own political ideology, providing yet another observation toward the existence of echo chambers in social media-based political discussions (Garimella et al., 2018). The clustering of users and news sources form clusters around political ideology, reflecting that even though Pink Slime sites are targeted at local groups, they are in reality spread along political lines.
News sources shared by users (including all platforms) who shared pink slime domains. Pink Slime sites are labeled and given a pink node coloring, local news sites are green nodes, real news sites are blue nodes, and low credibility news sites are red nodes. Gray nodes represent the users themselves. Links are formed between users and the sites they share.
The pink slime domains in the right component (demarcated with a red box) are all under the control of parent organizations pushing politically right-leaning news. This includes grandcanyontimes.com and keystonetoday.com which are controlled by Metric Media; and georgiastarnews.com, theohiostar.com, and tennesseestar.com which are owned by the Star News Network.
The pink slime domains in the left component (demarcated with a blue box) are under the control of parent organizations that push politically left-leaning news. coppercourier.com, keystonenewsroom.com, cardinalpine.com and upnorthnewswi.com are controlled by the Courier Newsroom, and americanindependent.com is under the control of The American Independent which has many more state-specific sites.
Past work on the frequency of types of news on the Reddit and YouTube platforms shows that at least a quarter of the news are divided by political leaning (Burton & Koehorst, 2020). Similarly, our network analysis yields a division of political ideology in terms of Pink Slime News sharing behavior suggests that news spread is not done along geographical lines (i.e., the six hotly contested states in the elections), but rather along political lines.
In terms of news sharing behavior by each user class (bot/human) on Twitter, we visualize that in Figure 8. This figure helps us interpret whether the news sharing behavior is similar or different across the two classes of users.
Likelihood of Twitter users sharing one news type based on previous news type shared by whether the account sharing is a bot or human. Nodes represent news type; links between two nodes represent users shared the two news types in consecutive posts; link thickness represents the likelihood of sharing two news types in sequence.
Within Twitter, the news sharing behavior between bots and humans remains consistent. Both graphs exhibit similar proportions of consecutive posts sharing of similar news types. There is a higher proportion of bot users that share post consecutively of Pink Slime to Low Credibility News sites compared with human users (12.5% for humans, 13.7% for bots). For self-loops of pink slime, 18.5% of humans continue sharing pink slime, whereas only 13.3% of bots continue sharing pink slime. Finally, we observe that for accounts sharing Pink Slime and then Local News, humans exhibited this behavior 0.24% of the time and bots did this only 0.11%. These statistics might indicate that humans do not easily differentiate between Pink Slime sites and Low Credibility News sites (Moore et al., 2023).
Discussion
In this work, we achieve a multi-platform analysis of the types of news shared among three social media platforms, and analyze the platform-based sharing patterns, news type-based sharing patterns, and user-based organic/inorganic sharing patterns.
Within the Platform-Based Analysis, we can take comfort in the observation that the proportion of Real News leads that of the other news types, especially misleading and false news like Low Credibility News and Pink Slime News.
In addition, the high proportion of Real News sharing indicates the users’ desire to portray an intelligent status in the community. A study on Facebook news sharing indicates that low-credibility news sharing can be related to a status-seeking behavior (Thompson et al., 2019), which is avoided when it has an effect on an individual’s status. The effect of gratification and prior experience post-sharing can also determine user behavior (Lee & Ma, 2012). These relate to communities like Facebook Pages and Reddit subreddit threads where the community interaction, for example, through replies, are extremely strong; and not so much to Twitter where the community structure is not as pronounced.
At the same time, the proportion of each news type on the platforms is slightly different, suggesting different content moderation policies. Twitter and Facebook adopt a more platform-based content moderation, where the platforms decide which content should be removed based on a certain set of criteria; while Reddit adopts a distributed content moderation strategy, where users take ownership of the content on the platform (Chandrasekharan et al., 2022; Ganesh & Bright, 2020).
Within the News Type-Based Analysis, we make use of the Relative Engagement metric to identify the type of news that gather the most impressions. Our observation that Pink Slime News receive the most engagement could be because the assumed locality of the news resonates with the geographical area, showcasing that geographic customization is likely a key factor to increasing post engagement. Low Credibility News received the next highest engagement, lending weight to the general adage that “fake news spreads faster than real news” (Zhao et al., 2020). This analysis slice highlights the importance of the study of Pink Slime sites on social media platforms, an area that typically flies under the radar of research. While these Pink Slime sites make up for a small minority of posts during the 2022 Midterm Elections, the posts received more likes (normalized by community size) than any other news type. These Pink Slime sites are designed to target local regions with content related to their community (Burton & Koehorst, 2020). The increased Relative Engagement indicates that users are seeing this content, and resonate with it. Therefore, promoting messaging at the local small groups level can have an outsized impact.
From a User-Based Analysis, we observe that news sharing behavior occur along ideology lines, as observed by users primarily sharing news segregated by political ideologies. When we examine the consecutive sharing habits of users, we find that more bot than human users oscillated between Pink Slime and Low Credibility posts, and more human than bot users oscillated between Pink Slime and Local News. This reflects the overlaps between Pink Slime, Low Credibility, and Local News, and the effectiveness of Pink Slime organizations in latching on to Local News geographies for information dissemination.
Pink Slime News sites are therefore an important category of news type to watch, as not only does our result show high engagement by users toward these news sites shared, media whatchdogs who are dedicated to countering low credibility news, like the co-CEO of NewsGuard, Gordon Crovitz has expressed his concern over Pink Slime sites in the 2022 U.S. Midterm Elections. He stated that the parent organizations behind these Pink Slime sites spent nearly US$4 million dollars on advertisements, most of which come from partisan donors (NewsGuard, 2023). The spread of Pink Slime sites creates an uncertain Local News environment that undercuts readership and advertising support for legitimate news sites, thereby undermining trust in news.
Past research on the measure of fake and real news spread shows that fake news spread roughly 10 times faster than true statements (Vosoughi et al., 2018). Real News received the least relative engagement. While this research does not explore the text of the posts, it may be that Real News does not have as sensational headlines to illicit reactions as Low Credibility News and Pink Slime (Shu et al., 2017).
The Relative Engagement metric can be generalized to other engagement types and platforms apart from the ones that are considered in this study. The metric relies on the number of interactions individual posts receive, which can be measured through likes or shares, against the number of users within a community, which can be defined by the analyst as a community of interacting users, or a community defined by the platform (i.e., Facebook Pages). Therefore, a possible expansion is for data collection to factor in individual and community metrics such that the Relative Engagement metric can be calculated.
Finally, with the User-Based Analysis, we observe huge similarities in news sharing behavior between bots and humans, suggesting both species act similarly to fit into the community groups, thereby reducing cognitive dissonance between themselves and other users within the community (Jeong et al., 2019; Ng & Carley, 2022). However, the observation that bots share a larger proportion of Pink Slime and Low Credibility News sites consecutively and humans share a larger proportion of Pink Slime and Local News consecutively suggests that while bots are programmed to continue to share information that is of lower quality, as supported by Lazer et al. (2018), humans are more interested in sharing content that is relevant to their local community.
In multiple analysis slices (i.e., platform-based analysis and user-based analysis), we observe that the news sharing behavior primarily circulates around the same categories of news, whether it is by news type or the political leaning of news. This is consistent with past studies that social media users tend to keep to their own information sphere (Ng et al., 2022a), which could thus result in political echo chambers (Cinelli et al., 2020a; Terren & Borge-Bravo, 2021).
Limitations and Future Work
As with all other studies, a few limitations nuance our work. First, our data collection is dependent on the social media platform’s API output, which does result in loss of data, thereby the conversation is not captured in its entirety. Furthermore, we examined the spread of types of news in contentious states within the 2022 U.S. Midterm Elections, leaving out the historically stable states. Deeper investigations should be performed to quantify the differences (or similarities) of news sharing behavior between the stable and contentious states.
Our research is also limited to those websites that have identified labels from past lists of news labels. These labels are manually identified and annotated, and need to be continually refreshed as more news sites arise on the internet.
Finally, much of our analysis of the sharing of news sites is hugely focused on the URLs, and places little emphasis of the context of the URL within the other texts in the post, or the texts within the website deriving from the URL. Future studies could expand our analysis by studying the textual content posted alongside the URLs, and similarly differentiate these analyses by platform, news type, and users. Future work also involves expanding the construction of the likelihood of news sharing graphs to consider links between non-consecutive posts rather than only news types from consecutive posts.
Our study of journalism during the2022 U.S. Midterm Elections opens up avenues for future work. This includes analysis of individual accounts sharing pink slime, and a better understanding if these accounts are sharing Pink Slime journalism in a coordinated fashion. Further work also includes assessing the distribution of bot accounts that are sharing the sites and the properties of these accounts, providing further insights toward the organic and inorganic spread of news on social media platforms.
Conclusion
Social media platforms play a critical role in facilitating the dissemination of information and the propagation of campaign narratives during events of political significance, such as the U.S. Midterm Elections. The study of news sharing behavior provides insight toward the prevalence of credible and junk news online and their dissemination patterns. This contributes to the emerging field of social cybersecurity, which deals with understanding and developing resilience against types of non-credible news and users in the online space (K. M. Carley, 2020; National Academies of Sciences, Engineering, and Medicine, 2019).
Through analyzing the 2022 U.S. Midterm Elections from a multi-platform perspective, we understand the relationship between social media platforms, the types of news shared, and the user engagement and popularity of news types. From an analysis of over 1.3 million social media posts linking to news URLs shared across three social media platforms, we find that Low Credibility News and Pink Slime News are most commonly shared on Twitter, and these two news types have the highest Relative Engagement on Facebook Pages compared with Real and Local News. We also observe that the sharing of news sites is separated by political leanings, as users tend to share news within the same ideology, as our results observed that users tend to share news by political ideologies, and users sharing Pink Slime websites share news differently depending on the political bias of the Pink Slime sites they share. Finally, in investigating the inorganicity of news sharing behavior, our results show that bots tend to share low-quality information (i.e., Pink Slime, Low Credibility News), while humans share a larger proportion of local community content.
This work contributes a theoretical illustration of the continuum of news types that exist on social media platforms and the behavior of social media users in engaging with and sharing these news types. This work further contributes empirical evidence that demonstrates the difference in news sharing behavior across and within social media platforms, and across political ideology lines. Finally, this study motivates further research into the engagement of different news types on social media platforms, in particular the differences between spaces within the continuum of real and false news.
Footnotes
Appendix
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the Office of Naval Research (ONR) Award N00014182106, the Knight Foundation, the Center for Computational Analysis of Social and Organizational Systems (CASOS), and the Center for Informed Democracy and Social-cybersecurity (IDeaS). The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the ONR, or the U.S. government.
IRB Approval
This research was conducted with IRB approval in the Fall of 2022 Federalwide Assurance No: FWA00004206, IRB Registration No: IRB00000603.
