Sage Journals: Discover world-class research

Abstract

Journalists increasingly cite and/or embed social media posts in news articles. While social media posts have been found to be of little deliberative quality, we do not know whether this also affects the deliberative quality of the news. Against the background of a hybrid media system and deliberative news media standards, we answer this research question with a content analysis of news articles including or not including posts from X (formerly Twitter) in the twelve widest-reaching German news websites prior and after the German general election 2021. We were particularly interested in the differences inside and outside election campaigns as the interdependence of the mass media and the political sphere is particularly pronounced during campaign periods. Results show that posts are more often cited and/or embedded in news articles inside than outside election campaigns. Articles including posts feature a greater number of actors but are not more diverse as mainly actors from the political center are referenced. Moreover, articles with posts are associated with a higher position responsiveness but on the other hand a decreased civility of the represented political discourse. This pattern only emerged inside but not outside campaign periods. These findings add to our understanding of contemporary hybrid media systems and the nature of political journalism during contentious political periods.

Keywords

Political journalism media quality deliberation hybrid media system content analysis

Referencing or fully embedding social media posts in political news has become an established journalistic practice (Broersma and Graham, 2018; Hine, 2020; Kapidzic et al., 2022; Metag and Rauchfleisch, 2017; Oschatz et al., 2022; Von Nordheim et al., 2018). Posts are used as a proof or example for an argument made (Broersma and Graham, 2013), underscore journalists’ correctness and accuracy (Oschatz et al., 2022), and provide verbatim quotes that are more easy to obtain via social media than press conferences and interviews. At the same time, politicians need to get their message out to the people. While social media open a direct channel to voters to communicate standpoints, the mass media remain an important intermediary to reach society at large. Hence, getting access to news coverage with an effort as little as typing a post is an attractive resource for politicians and crucial for their success (Broersma and Graham, 2018; Jacobs et al., 2020). These new routines in journalism and politics can be understood as an expression of a hybrid media system (Chadwick, 2017) characterized by interwoven logics of traditional mass media and social media. The emerging technologies do not replace old technologies but build on them in interdependent ways. That gives rise to new patterns in news production such as embedding or citing social media posts in news articles.

The question arises what consequences result from these new patterns. Noticeable changes in journalistic practice and news production have always attracted the attention of communication scholars interested in their effects on news media quality (e.g., Boumans et al., 2018; Reinemann et al., 2012). The news media are the most important information source about politics for citizens (e.g., Maurer and Oschatz, 2016). They provide information that contribute to citizens’ opinion formation and voting decisions (Asp, 2007; Jandura and Friedrich, 2014; Kovach and Rosenstiel, 2021; Van Aelst et al., 2017). A deterioration in news quality could thus have a negative impact on a functioning democracy by depriving citizens of the basis for informed decision making.

What exactly accounts as high-quality news is subject to the underlying model of democracy and its demands on citizens and news media standards (Strömbäck, 2005). As a minimum consensus, all democratic models demand the media to provide information that enables citizens to make informed choices (Jandura and Friedrich, 2014; Kovach and Rosenstiel, 2021; Van Aelst et al., 2017). In this study, we build on a deliberative model of democracy (Habermas, 2011, 2022; Wessler, 2018) which received increasing scholarly attention in the past 20 years (Jandura and Friedrich, 2014; Kiflu et al., 2022; Marcinkowski and Donk, 2012; Rinke et al., 2013; Strömbäck, 2005; Wessler, 2018). While other models concentrate on the competitive character of public debates, the deliberative model emphasizes the potential for lessons learned by an exchange of arguments (Habermas, 2022: 100–104).

To our knowledge, no study has systematically explored the deliberative effects of sourcing social media in the news. We are particularly interested in differences inside and outside election campaigns. During the intense times of political campaigning, it is normatively desirable that high-quality information on politicians’ core positions and evaluations of their competence and suitability for office are reaching citizens who must decide on their future government. However, social media have been ascribed a rather weak deliberative quality (Halpern and Gibbs, 2013; Pieper and Pieper, 2015; Wessler, 2018). It is not yet clear whether this also applies to posts used in news articles and whether this impacts the overall quality of news articles. Thus, this study aims to answer the research question: Whether and to what extent does the journalistic practice of referencing social media posts in news articles affect the deliberative quality of news inside and outside election campaigns?

We answer our research question with a manual content analysis of news articles in a news media sample collected during and after the 2021 German federal election campaign. Thus, this study contributes to the understanding of the implications of developing patterns in a hybrid media system by (1) comparing the quality of news articles with posts to news articles without posts based on deliberative criteria (Löb et al., 2022; Rinke et al., 2013; Wessler, 2018), (2) determining potential differences inside and outside election campaigns, and (3) deriving implications for news media quality.

Deliberative quality of news

In modern societies, traditional mass media are key in organizing processes of public deliberation that exceed the limitations of face-to-face encounters (Page, 1996). The mediated public sphere in news coverage needs to adhere to four deliberative standards to account as an essential component of a deliberative democratic system (Rinke et al., 2013; Wessler, 2008, 2018): Inclusiveness refers to a bottom-up approach of political communication that does not only center around opinions and positions of powerful political actors but includes voices from the civil society. The plurality of voices ideally results in a plurality of opinions, positions, and arguments in public discourse (Habermas, 2011; Jandura and Friedrich, 2014; Marcinkowski and Donk, 2012; Wessler, 2018). Furthermore, the ideal of inclusiveness also strives for diversity in terms of gender, social class, race, and ethnicity (Wessler, 2018). Thus, from a deliberative perspective, the mass media should not only focus on few actors from the powerful political center but include a multitude of different and diverse actors from the political center and the political periphery. Justification requires speakers, particularly political actors (politicians, parties, etc.), to give transparent and understandable reasons for their positions. These positions are normative as political actors explicitly articulate how, for example, an issue should be handled or what our society should be like (Löb et al., 2022; Rinke et al., 2013). Journalists need to demand and question the speakers’ justifications and hold the powerful accountable (Marcinkowski and Donk, 2012; Rinke et al., 2013; Wessler, 2018). Responsiveness means engaging with and reacting to other speakers’ positions and arguments. The idea is that the strongest arguments prevail in public discourse. The quality of public dialogue increases overtime allowing for refinement or even reversal of opinions and positions (Rinke et al., 2013; Wessler, 2008). News media can represent responsiveness in two ways. Either different speakers are allowed to have their say in talk shows and interviews, for example, or journalists relate their arguments in the news. Finally, civility refers to mutual respect for other speakers. Their justifications and positions should be met with politeness and courtesy; personal attacks and inflammatory speech should be avoided to encourage minorities with potential outsider views to engage in the public discourse (Jandura and Friedrich, 2014; Marcinkowski and Donk, 2012; Rinke et al., 2013; Wessler, 2018). Civil discourse can be considered as a precondition for the exchange of arguments and ideas (Rinke et al., 2013). Journalists should, thus, report in an unbiased and differentiated manner on political actors and issues (e.g., Schmuck et al., 2022), and refrain from over-emphasis on political scandals (e.g., Maurer et al., 2022) to avoid negative spillover effects on the electorate (e.g., Sikorski et al., 2020).

Responsiveness and justification on X are assumed to be of weak deliberative quality, inclusiveness and civility of moderative deliberative quality (Wessler, 2018). Yet, the likely quality of an individual post tells us little about the general effects on the quality of entire articles. Due to character limits on X, a single post might express the position of a politician without extensive justification or elaborated response to other positions but at the same time add to the quality on the article level if thoughtfully selected amongst other posts and arguments. A post might be highly incivil but is used to give a bad example of campaign behavior. Thus, the impact of a post on the quality of a news article largely depends on its contextualization. Since we know little about the impact of posts on the deliberative quality of news articles inside and outside election campaigns, we add a research question to explore the impact of referenced posts on news media quality:

RQ1: Does the use of posts affect the deliberative quality of news articles?

The quality of news reporting inside and outside election campaigns

Bennett (1996) argues that the representation of politics in the news is guided by three principles: (1) normative perceptions about the role of journalism in a democracy (e.g., informing the public about its representatives), (2) professional norms such as objectivity, fairness, and accuracy (also see Asp, 2007 for the ‘core capital’ of political journalism), and (3) normative constraints of economization of journalism (also see Hanitzsch et al., 2019; Seethaler, 2019). In the following, we argue how this normative base is likely to affect the use of posts inside and outside election campaigns and derive assumptions for their impact on deliberative news quality.

We expect that the use of posts from political actors is particularly valuable for journalists during election campaigns. Journalists increase their attention to political actors to fulfill their public duty of informing citizens about their future representatives. Campaigns are a condensed and intensive period for political journalists. News needs to be produced within a short period of time to cover all aspects of the campaign, but with decreasing resources due to sharpened market conditions of publishing houses (Hanitzsch et al., 2019; Seethaler, 2019). In such a situation, posts are a timesaving and reliable source of information for everyday reporting (Broersma and Graham, 2018; Lecheler and Kruikemeier, 2016; Metag and Rauchfleisch, 2017). At the same time, political actors use social media to maximize their outreach to voters. They increase their availability to journalists which provides journalists with directly accessible newsworthy material to be used in news articles. Based on this demand-and-supply rationale, we derive our first hypothesis:

H1: The proportion of news articles using posts is higher in election periods compared to non-election periods.

We can also assume that the deliberative quality of posts and consequently their impact on the quality of news articles differs between election and non-election times. On the one hand, an increase of quality compared to non-election campaign periods can be assumed with regards to justification and responsiveness. Politicians attach particular importance to clearly articulate their positions and reasoning to differentiate from their opponents. They also respond more intensively to other candidates’ positions and justifications to acclaim their greater suitability for office. This effect is likely exacerbated by the three guiding journalistic principles. Time and resource constraints increase the journalists focus on the candidates’ social media profiles. This is especially true when journalists aim to inform the public about more extreme or populist candidates who accuse journalists of producing ‘fake news’ and thus refuse to interact with them. Further, to strive for objectivity in line with professional norms, journalists balance news reports across the partisan statements and arguments and, in turn, increase their visibility in news media.

On the other hand, a decrease in deliberative quality with regards to inclusiveness and civility might occur. Heim (2021) discusses the potential of X as ‘democratizing’ or ‘normalizing’ source for journalistic practice. Democratizing in the sense of bringing a diversity of voices and perspectives to news reports. Instead, normalizing refers to adapting the use of X to existing working routines. Although a diversity of voices is desirable, journalists have a tendency to focus on elite sources (Heim, 2021; Lecheler and Kruikemeier, 2016; Oschatz et al., 2022). Moreover, journalists follow a trend of increased personalization or presidentialization and mainly focus on few top candidates (e.g., Garzia et al., 2022; Gattermann and De Vreese, 2020). This is likely to increase in intense election periods in line with the journalists’ role perceptions in a democracy. Finally, populist parties have been found successful in gaining access to news media coverage with provocations and scandalizations (Maurer et al., 2022). As no empirical evidence is currently available on potential differences of news media quality inside and outside election campaigns, we add another research question:

RQ2: Does the use of posts affect the deliberative quality of news articles to different degrees inside and outside elections campaigns?

Methods

This study used a manual content analysis to explore the use of social media posts in political news coverage inside (22 August – 26 September 2021 (=election day)) and outside (21 October – 25 November) of the German general election in September 2021. We rely on posts from X (formerly Twitter) as the (at the time) most frequently cited/embedded social platform for political journalism (Hernández-Fuentes and Monnier, 2022; Humayun and Ferrucci, 2022; Oschatz et al., 2022). Materials for replication including the codebook, data, and syntax for data analysis are available on the OSF repository: https://osf.io/aykqv/

Sample

The sample population includes news articles of the 12 online news websites with the widest reach in Germany. The websites were selected based on the market-media-study daily digital facts (2021) which summarizes information on the use of German websites. We selected the 10 widest reaching websites of original online news rooms: focus.de, bild.de, spiegel.de, welt.de, ntv.de, rtl.de, faz.net, stern.de, sueddeutsche.de, and zeit-online.de. We did not consider portals (e.g., t-online.de) that often do not produce news themselves but obtain them from news agencies or other news media channels. We further did not consider websites that are not classic news channels like ebay.de or wetter.com. As a classic market-media-study, the daily digital facts study considers only commercial websites. We complemented our sample with the websites of the two most popular public news broadcasts: heute.de and tagesschau.de.

Data collection

For a broader research project, a data base was developed to crawl all newly published articles in the RSS feeds of the news websites in regular intervals and to parse them in a structured SQL database. We created subscriptions where necessary to also capture articles behind paywalls. Fully embedded posts were identified based on their unique post ID in the source code of the news articles. The corresponding posts and associated metadata (e.g., screen name) were downloaded via the X REST API and merged with the corresponding media articles. Referenced posts in the main text (i.e., cited, paraphrased, or just mentioned) were identified with the search string “tweet OR twitt”. Overall, N = 113.271 articles were collected during the investigation period. In a first data cleaning step, we eliminated empty observations, entries consisting only of a video as well as duplicates based on the article link and the time stamp (n = 80.038). Next, we excluded live- and news tickers (n = 7.580). In a final step, we eliminated rubrics such as ‘sports’, ‘travel’, ‘feuilleton’ and ‘foreign news’ (n = 6.875) as well as news feeds that mainly included such rubrics and thus created a lot of noise in the data such as BILD_news, RTL_news (n = 7.989). The cleaned news article sample consisted of n = 10.789 news articles of which n = 1.276 news articles cited, embedded, or cited and embedded posts (12%).

A random subsample of n = 500 news articles was selected from the news database including a balanced number of articles with or without posts in both data collection periods Drawing a random subsample was necessary to reduce the news article data base to a manageable amount for in-depth manual coding.

Manual content analysis

A codebook was developed based on openly shared codebooks analyzing the use of posts in German news coverage (Oschatz et al., 2022) as well as deliberative news media quality (Löb et al., 2022; Rinke et al., 2019). The central categories on deliberative news media quality– Inclusiveness, Responsiveness, Justification, and Civility – are described below. Please see the OSF repository for the entire codebook in German and English.

Inclusiveness

All actors from the political center and the political periphery present in a news article (in text and post) were coded. Actors are individuals (e.g., candidate for chancellor Olaf Scholz) and/or organizations (e.g., the government) whose actions are described or whose statements are indirectly or directly quoted (Brennan & Prediger’s kappa = .82). Mere mentions of an actor were not sufficient for coding. We further coded the gender for each actor (Brennan & Prediger’s kappa = .96).

Justification

First, we listed all positions stated in a news article. Positions are communicative utterances in which normative opinions, evaluations, goals, or actions are presented. Positions are normative as actors express that something should be done (or not) or is desirable (or not) (Brennan & Prediger’s kappa = .71). For example, “I am in favor of a compulsory vaccination against Covid 19” or “We are against a coalition with the Left Party”. Next, we coded all justifications which are explicit arguments used to justify a position. For example, justifications can be expressed by referring to (1) consequences of actions (e.g., “Strategy X has already been successful in the past, so we should pursue it again”), (2) certain values (e.g., “to preserve human dignity”, “to live up to moral standards”), and (3) interests (e.g., “to secure our prosperity in the long term”) (Brennan & Prediger’s kappa = .60). This reliability coefficient is at the low end of acceptable values. The coding of the justifications is a follow-up category, hence, if a position is overlooked, it also decreases the justification reliability score. Given the complexity of the codebook and the news material, we consider it sufficient to produce credible results.

Responsiveness

We coded explicit references to other actors (actor-responsiveness, Brennan & Prediger’s kappa = .83) and/or other positions (position-responsiveness, Brennan & Prediger’s kappa = .83) expressing engagement with other actors’ arguments. An explicit reference can be expressed in different ways such as direct and indirect quotes (e.g., “The Young Union of Germany (= youth organization of the conservative party) welcomes Friedrich Merz’ renewed candidacy for the federal chairmanship and expresses its support.”), in the form of actions directed at another actor (e.g., pats on the back, assaults, demonstrations, etc.); and by commenting and evaluating another actor’s actions (e.g., “Annalena Baerbock made mistakes in her election campaign”).

Civility

To assess the civility of the public discourse, i.e., to what extent responses to other actors and their positions are met with politeness and respects, we coded the tone of the responsiveness. Following Löb et al. (2022), we coded whether the reference to another actor (Brennan & Prediger’s kappa = .79) or reference to another position (Brennan & Prediger’s kappa = .83) included a positive or negative evaluations. A positive evaluation is for example the appraisal of an action (e.g., “The consequent Corona policy of the government saved millions of lives”). A negative evaluation is critique (e.g., “Resistance is forming in the conservative base in Saxony. The chancellor and her refuge policy are to blame”). Such an evaluation must have been explicitly present in the text, but did not have to be strongly pronounced (e.g., expressing dislike but not necessarily incivil or impolite).

Procedure

The coding was conducted by three student assistants experienced in content analysis after intense training. Irrelevant instances such as references to X as a company as well as articles that did not include any thematic reference to politics in Germany (access criterion) were filtered out manually. Articles related to celebrities, the weather, as well as foreign news were still included in the general “news” feed of the media in our sample. The final sample brought forward to analysis are N = 338 news articles inside (n = 191) and outside (n = 147) the election period.

Strategy of analysis

RQ1 was tested using Welch t-tests for unequal sample sizes. For H1, we performed a 2x2 chi-squared test. RQ2 was tested using a 2-way ANOVA using the inclusion of posts in news stories and campaign period as independent binary factors. Whenever interactions were significant, post-hoc analyses were further run using Tukey multiple mean comparisons.

Results

RQ1 explored whether the use of posts affects the deliberative quality of the news articles. Results for inclusiveness show that news articles including posts featured a greater number of voices, both from the political center (M = 3.86, SD = 2.87) and the periphery (M = 1.37, SD = 1.65), than news articles without posts (center, M = 2.83, SD = 2.67, Welch t(282.78) = -3.35, p < .001; periphery, M=.97, SD = 1.66, Welch t(298.6) = -2.17, p < .05). However, the representation of peripheral voices relative to the total number of actors featured in news articles did not significantly improve when the articles included posts (on average, 27% of all voices included) versus articles not including them (25%, Welch t(321.55) = -.43, p = .667).

Similar differences apply in the case of party front runners versus other party actors. Our data show a greater number of front runners in articles using posts (M = 3.44, SD = 2.34) than those without (M = 2.43, SD = 2.27, Welch t(291.17) = -3.94, p < .001), but no significant differences across groups for other party actors. This means that it was mainly front runners who gained statistically significant representation. However, the relative prominence of one type of actor over the total number of voices remained stable, with front runners representing around the 69% of the total party actors featured in news, independent of the presence of posts in stories (Welch t(311.45) = −.15, p = .882).

Finally, in terms of gender inclusiveness, news articles featuring posts included an overall greater number of actors, both male (M = 3.11, SD = 2.05) and female (M = 1.11, SD = 1.19) than articles without (male, M = 2.12, SD = 1.93, Welch t(284.93) = −4.46, p < .001; female, M=.65, SD=.96, Welch t(254.13 = −3.79, p < .001). Female actors remained overall less prominent than male actors, and such relative prominence did not change across groups: they represented around 22% of the total number of actors included in articles without posts and the 25% in articles with at least one post (Welch t(310.27) = −1.02, p = .308).

Analyzing differences on justification, we found that the sets of news articles with or without posts in our data showed highly similar distributions for the total number of both positions (about 2.3, on average) and justifications (about 1.2) presented per article, without significant differences across groups (see Table 1).

Table 1.

Total number of justifications presented in news articles, by number of positions.

Total number of positions presented in story	Total number of justifications presented in story												Total
Total number of positions presented in story	0	1	2	3	4	5	6	7	8	9	10	12	Total
0	115	—	—	—	—	—	—	—	—	—	—	—		115
1	22	8	7	3	0	0	0	0	0	0	0	0		40
2	18	15	16	5	4	3	0	0	0	0	2	0		63
3	9	11	7	6	3	1	2	0	0	0	0	0		39
4	13	3	4	6	4	1	1	0	0	1	0	0		33
5	4	2	4	2	0	2	1	0	0	0	0	0		15
6	2	2	2	1	0	1	0	0	0	0	0	0		8
7	2	2	0	1	1	1	0	1	1	0	0	0		9
8	1	0	0	1	3	0	0	0	0	1	0	0		6
9	1	0	0	0	0	0	0	0	0	0	0	0		1
10	1	0	0	0	0	0	0	0	0	0	0	1		2
11	1	0	0	1	0	0	0	0	0	0	0	0		2
12	1	0	0	0	0	0	0	0	0	0	0	0		1
13	2	0	1	0	0	0	0	0	0	0	0	0		3
19	0	0	0	0	0	0	0	0	0	0	1	0		1
Total	192	43	41	26	15	9	4	1	1	2	3	1		338

Since a sizeable portion of the news articles did not present any position (n = 115, 34%), we decided to further test the number of justifications looking only at those news articles which included at least one position, since, by definition, news articles including zero positions would also include zero justifications. News articles including posts did not present, on average, a significantly greater ratio of arguments per position presented (M = .66, SD=.90) than news articles without posts (M = .65, SD = .72; Welch t(170.37) = −.14, p = .889). In all, the findings confirmed that news quality was not impacted by the inclusion of posts in terms of the justification criterion.

To assess responsiveness, the number of responses presented in the article from one actor towards other actors and/or their positions were compared. We found that more actor responses were represented in news articles with posts (M = 1.9, SD = 3.31) than in articles without posts (M = 1.3, SD = 2.84), but these differences were not statistically significant (Welch t(266.93) = −1.86, p = .06). The same occurred with position responses (M = .34, SD = .92 and M = .15, SD = .50 for articles with and without posts, respectively); this time with significant differences across both groups (Welch t(194.07) = −2.24, p < .05).

Finally, for civility, the number of negative actor responses was greater in those news articles that included at least one post (M = 1.06, SD = 2.21) than in those without posts (M = .49, SD = 1.1; Welch t(185.68) = −2.81, p < .01). The same pattern was also observed for negative position responses (with posts, M = .15, SD = .50; without posts, M = .06, SD = .29), except without the established statistical significance of .05 (Welch t(206.27) = −1.93, p = .054). Nevertheless, these differences were not large enough to impact the relative predominance of negative responses over the total number of responses provided per news article. Differences in negative-to-total response ratios were not significant across groups in the case of actor responses (.50 and .56 in articles with and without posts, respectively; Welch t(147)=.86, p = .393), nor for position responses (.56 and .59 in articles with and without posts, respectively; Welch t(39.84)=.22, p = .830).

H1 proposed that the proportion of news articles in political news coverage using posts is higher in election periods compared to non-election periods. We answered this hypothesis with a Chi square test based on the entire sample collected in both investigation periods and stored in our SQL database (N = 10.789). Table 2 summarizes the results. As one could expect, numbers of news articles on German politics significantly increased during the election period by 12% (χ²(1, N = 10.789) = 63.70, p = .000). In line with our assumptions, the number of posts embedded and/or cited in news coverage substantially increased during the election (15%) compared to the non-election period (10%), (χ²(1, N = 1,279) = 28.95, p < .000). The increase spans across all types of used posts (embedding, citing, embedding & citing) in news articles (see Table 2). Thus, H1 was supported by the data.

Table 2.

Use of posts in news articles inside and outside election campaigns.

	Campaign N (%)	No campaign N (%)	Chi square
Embedded	80 (1%)	46 (1%)	9.17 (p = .002)
Embedded & cited	197 (3%)	115 (2%)	21.55 (p < .000)
Cited	500 (9%)	338 (7%)	31.32 (p < .000)
No posts	5032 (87%)	4481 (90%)	31.91 (p < .000)
Sum	5809	4980	63.70 (p < .000)

Finally, RQ2 asked whether the use of posts affected the deliberative quality of news articles to different degrees inside and outside elections campaigns. We analyzed news quality separately for each criterion:

The main effects for inclusiveness observed in RQ1, namely the greater number of actors featured in news stories with posts than in those without posts, did not significantly vary from the campaign period to the period outside elections (see means and standard deviations by period in Table 3; center, F(1, 334)=.13, p = .72, periphery, F(1, 334)=.11, p = .74). The campaign period did also not affect the degree of inclusiveness of neither front runners (F(1, 334)=.02, p = .89) and other party members (F(1, 334)=.81, p = .37), nor gender inclusiveness (male, F(1, 334)=.29, p = .59; female, F(1, 334)=.25, p = .61). In sum, all interaction terms tested were not statistically significant.

Table 3.

Means and standard deviations of all independent Variables across subgroups.

	Campaign (22 Aug – 26 Sept)		No campaign (22 Oct – 25 Nov)
	Article without posts (N = 109)^a	Article with posts (N = 82)^a	Article without posts (N = 90)^a	Article with posts (N = 57)^a
Inclusiveness, center vs periphery	M SD	M SD	M SD	M SD
Center actors	2.83 (2.85)	3.95 (3.03)	2.83 (2.44)	3.74 (2.65)
Periphery actors	1.10 (1.93)	1.54 (1.57)	0.81 (1.25)	1.12 (1.73)
Ratio periphery-to-total-actors	n = 104; 0.27 (0.38)	0.30 (0.28)	n = 86; 0.23 (0.34)	n = 56; 0.22 (0.30)
Inclusiveness, front runner vs secondary party actors
Front Runners	2.65 (2.45)	3.61 (2.20)	2.17 (2.00)	3.19 (2.52)
Secondary Party actors	1.28 (2.01)	1.88 (2.17)	1.48 (2.07)	1.67 (1.97)
Ratio Front-to-total-party-actors	n = 104; 0.72 (0.33)	0.71 (0.29)	n = 86; 0.64 (0.37)	n = 56; 0.66 (0.35)
Inclusiveness, gender
male actors	2.08 (1.98)	3.17 (2.16)	2.17 (1.87)	3.02 (1.89)
female actors	0.66 (0.92)	1.17 (1.16)	0.64 (1.01)	1.04 (1.24)
ratio Female-to-total-actors	n = 98; 0.23 (0.31)	n = 80; 0.27 (0.25)	n = 80; 0.21 (0.29)	n = 55; 0.23 (0.23)
Justification
positions	1.92 (2.24)	2.29 (2.69)	2.72 (3.33)	2.25 (2.30)
justifications	0.95 (1.49)	1.12 (2.25)	1.43 (2.10)	1.44 (1.99)
average Num. Justifications per position	n = 66; 0.56 (0.65)	n = 53; 0.59 (1.03)	n = 64; 0.73 (0.78)	n = 40; 0.75 (0.68)
Responsiveness
actor Responses	1.68 (3.51)	2.56 (3.92)	0.74 (1.59)	0.95 (1.83)
position Responses	0.14 (0.50)	0.51 (1.15)	0.16 (0.50)	0.09 (0.29)
total Responses	1.82 (3.56)	3.07 (4.21)	0.90 (1.74)	1.04 (1.95)
Civility
positive Or neutral actor responses	1.06 (3.12)	1.11 (1.98)	0.39 (0.88)	0.44 (0.95)
negative actor responses	0.62 (1.23)	1.45 (2.61)	0.32 (0.88)	0.49 (1.27)
positive Or neutral position responses	0.09 (0.35)	0.28 (0.82)	0.07 (0.29)	0.05 (0.23)
negative position responses	0.05 (0.25)	0.23 (0.61)	0.08 (0.34)	0.04 (0.19)
total Positive or neutral responses	1.15 (3.13)	1.39 (2.22)	0.46 (0.93)	0.49 (0.95)
total Negative responses	0.67 (1.28)	1.68 (2.75)	0.40 (1.06)	0.53 (1.38)
civil-To-total response ratio	n = 48; 0.57 (0.40)	n = 60; 0.53 (0.38)	n = 33; 0.60 (0.43)	n = 24; 0.50 (0.44)

^aUnless stated otherwise.

Justification was not affected by whether news included posts (i.e., no significant main effects in RQ1). The interaction terms between post inclusion and whether a news article was published inside or outside the election period were also not significant. That is, not only did the number of positions and justifications provided in news stories remain highly similar across articles with and without posts, but they also remained stable inside and outside the election campaign (interaction effects post inclusion x campaign period: positions, F(1, 334) = 2.01, p = .16; justifications, F(1, 334)=.14, p = .71).

Our analyses for responsiveness revealed that the number of actor responses included in news stories overall increased during the campaign period (M = 2.06, SD = 3.70) compared to outside the campaign period (M=.82, SD = 1.68; i.e., significant main effect of campaign period, F(1, 334) = 13.586, p < .001). That is, news quality increased in terms of actor responsiveness during election periods compared to outside the election. In terms of position responses, there was a significant interaction between campaign period and the inclusion of posts (F(1, 334) = 8.178, p < .01). Such interaction indicated that a greater number of position responses was provided in news articles including posts (M=.51, SD = 1.15) rather than in news articles not including them (M=.14, SD=.50), but only during the campaign period. Outside elections, differences in the number of position responses provided in news stores were not significantly larger between news articles with (M=.16, SD=.50) and without posts (M=.09, SD=.29).

The campaign period had a significant and positive main effect of civility measured as the number of both positive or neutral and negative responses. For negative responses, an interaction term was significant between campaign period and the use of posts (F(1, 334) = 5.275, p < .05). As Figure 1 illustrates, the number of uncivil responses was significantly greater in news articles including posts (M = 1.68, SD = 2.75) than in those not including any (M=.67, SD = 1.28), but once again these differences were significant only during the election campaign period, during which the tone is overall more negative. Outside the campaign, articles used a similar number of negative responses independently of whether they included posts (M=.53, SD = 1.38) or not (M=.40, SD = 1.06).

Figure 1.

Boxplots of the number of negative responses included in articles, by whether articles include posts and campaign period.

Discussion

The goal of this study was to examine whether the journalistic practice of embedding and/or citing social media posts in news articles affected the deliberative quality of news media coverage and whether this potential impact differed between election and non-election election campaign periods. Our study examined how posts from X (formerly Twitter) influence deliberative news quality, as X was the most frequently used social media platform by journalists and politicians at the time. Although the standing of X might have diminished somewhat since Elon Musk’s takeover in October 2022, the platform still remains the main arena where politicians, opinion leaders and journalists intermingle. The insights gained in this article are relevant beyond X, as the platform might change or relevant activity get more dispersed across platforms. Nonetheless, arenas where journalists and politicians can connect and communicate with each other remains crucial components of modern journalism in the hybrid media system.

Our study uncovers three key findings. First, we show an overall increase in the use of posts in political news coverage, based on the average percentage of posts appearing in news articles outside election campaigns (10%), compared to a previous study (5%, Oschatz et al., 2022). This increase is likely driven by mainly two factors underscoring the need of access and benefits for journalists and politicians alike. Politicians are increasingly applying social media in their professional communication (Broersma and Graham, 2018; Haman and Školník, 2021), providing journalists with an increasing amount of statements, opinions, and visual footage to be used in news articles. Most importantly, citing or embedding these posts is beneficial for journalists. Posts of political leaders and top candidates are available free of charge, at any time, and beyond borders which makes them a valuable resource for everyday reporting. However, it can be assumed that the use of posts in news coverage is underestimated as journalists might not always mention social media as a source for a simple information or idea (Broersma and Graham, 2018: 11).

Second, when deliberative quality measures are applied, articles including posts differ from articles that do not include posts in the dimensions of inclusiveness and civility. News articles with posts are more inclusive as they feature a greater number of voices. These voices are however not more diverse. For a large part, the increase in voices represents actors from the political center, front runners as well as male politicians. In accordance with theories of a “normalizing” role of social media, journalists do not use a broader range of sources, but fit social media to existing working routines that amplify elite voices from the political center which already play the dominant role in news reports. Moreover, the results provide evidence that when posts are integrated in news articles, political discourse among politicians is represented in a substantially more negative tone. Studies examining the use and spread of negative utterances on X show that posts including attacks or negative emotions spread further as they create more likes and shares than positive posts (Mueller and Saeltzer, 2022; Rozado et al., 2022; Schöne et al., 2021). The negativity on X apparently spills over to news coverage. There are compelling interpretations for this observation. One, if negative posts are more salient and visible than positive posts, journalists are more likely to come across these posts and therefore embed/cite them to a greater extent. In addition, negativity is an important news value (Galtung and Ruge, 1965) and journalists have been found to pay disproportionally greater attention to negative news (Van der Meer et al., 2019) to attract a greater audience. This strategy is rewarded as audiences are particularly attentive to negative compared to positive news (Lengauer et al., 2012; Van der Meer et al., 2020).

Finally, the results of the present study confirm that the practice of using posts in news coverage differs between election and non-election campaign periods. Posts are used in news coverage to a greater extent during the campaign, which amplifies effects for responsiveness and civility. Only during the campaign period, a greater number of responses to other (parties’ or candidates’) positions was presented in news articles when they included posts than when they did not. This finding can be explained, on the one hand by the greater availability of such responses during election campaigns as candidates aim to differentiate themselves from the political opponent and sell their unique suitability for office to voters. On the other hand, journalists are more likely to appeal to party and candidate positions and the responses made to fulfill their normative role in democracy to inform the public (Bennett, 1996). Moreover, in line with the professional norm of objectivity (e.g., Asp, 2007; Bennett, 1996), equal attention is given to all candidates across the party spectrum further increasing the positions and responses. In news articles including posts, the presented campaign style was substantially more negative compared to articles without posts. This aligns with findings from studies examining the use of negative campaigning in elections. This strategy has substantially increased over the past decade – especially on social media campaigns (e.g., Fowler et al., 2022; Geer, 2012). The pattern disappears outside election campaigns. The number of negative posts presumably decreases on social media because coalitions need to be formed and day-to-day business takes over in politics, and thus, the number of negative utterances does not differ between articles with and without posts outside the election campaign.

Certain limitations of this study could be addressed in future research. We focus on deliberation as a benchmark for news media quality. We chose this concept as a demanding model for democracy that increasingly received attention in the last decade. Furthermore, no study has so far explored the consequences of a newly established journalistic routine, although the deliberative quality of social media had been debated (Halpern and Gibbs, 2013; Pieper and Pieper, 2015; Wessler, 2018). This leaves out other quality indicators such as the accuracy of news that are related to concerns, e.g. about so-called fake news (e.g., Tandoc et al., 2019; Waisbord, 2018). Future studies should thus take additional quality indicators suggested by other democracy models and media standards into account. Moreover, our study examined the impact of citations and embeddings of posts from X. Given the decline in reputation of X as a reliable and trustworthy information source, the use of X by politicians and consequently journalists is likely to change. Further research will be needed to determine where the journalistic and political public sphere is going to re-connect and how the popular journalistic practice of sourcing social media in the news might change. In addition, our study focuses on only one country and one election and requires replication in different contexts to gain confidence in the empirical findings.

In summary, this study provides the first systematic examination of the consequences of sourcing posts in news articles on deliberative news media quality. It contributes to our understanding of the extent and impact of this journalistic practice. Sourcing posts was particularly intense during elections. We find that this practice has positive and negative effects on deliberative news media quality–particularly pronounced during campaign periods in the representation of the responsiveness and the civility of the political discourse.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Correction (March 2024):

University of Mannheim, School of Social Sciences, Germany has been added to the affiliation of Sebastian Stier and country of the affiliation of Dylan Paltra has been updated since the original publication of the article.

ORCID iDs

Corinna Oschatz

Teresa Gil-Lopez

Author biographies

Corinna Oschatz is an Assistant Professor of Political Communication & Journalism at the Amsterdam School of Communication Research (ASCoR), University of Amsterdam. She holds a PhD in communication science from the University of Mainz (2016). Her research focuses on political communication, political journalism, narrative persuasion, and methods of empirical social science.

Teresa Gil-Lopez (PhD Communication, University of California, Davis, 2019) is a Postdoctoral researcher in Political Communication and Media Effects at the University Carlos III of Madrid. She investigates the ways in which digital technologies may have altered the relationships between social movements, the media, and the citizen discourse.

Dylan Paltra is PhD student at the University of Vienna, Department of Government. His research interests are political online behavior, political psychology, polarization, and methods of computational social science.

Sebastian Stier is a Full Professor at the Department Computational Social Science of GESIS – Leibniz Institute for the Social Sciences in Cologne and the School of Social Sciences, University of Mannheim. His research is situated in the fields of computational social science, digital media, political communication and comparative politics.

Tanjev Schultz is Professor of Journalism at the Johannes Gutenberg University of Mainz. His research focuses on journalism research and media ethics. Before coming back to academia he worked as a journalist for Germany’s national daily newspaper “Süddeutsche Zeitung.”

References

Asp

(2007) Fairness, informativeness and scrutiny. The role of news media in democracy. Nordicom Review 28: 31–49.

Bennett

(1996) An introduction to journalism norms and representations of politics. Political Communication 13(4): 373–384.

Boumans

Trilling

Vliegenthart

, et al. (2018) The agency makes the (online) news world go round: the impact of news agency content on print and online news. International Journal of Communication 12: 1768–1789.

Broersma

Graham

(2013) Twitter as news source: how Dutch and British newspapers used tweets in their news coverage, 2007–2011. Journalism Practice 7(4): 446–464.

Broersma

Graham

(2018) Tipping the balance of power: social media and the transformation of political journalism. In: Enli

Bruns

Skogerbø

, et al. (eds) The Routledge Companion to Social Media and Politics. Routledge Companions. New York: Routledge, 89–103.

Chadwick

(2017) The hybrid media system: politics and power. In: 2nd edition Oxford Studies in Digital Politics. New York: Oxford University Press.

Daily Digital Facts (2021) February. Frankfurt: Arbeitsgemeinschaft Onlineforschung e.V. (agof).

Fowler

Franz

Ridout

(2022) Political Advertising in the United States. 2nd edition. New York, NY: Routledge, Taylor & Francis Group.

Galtung

Ruge

(1965) The structure of foreign news: the presentation of the Congo, Cuba and Cyprus crises in four Norwegian newspapers. Journal of Peace Research 2(1): 64–90.

10.

Garzia

Ferreira da Silva

De Angelis

(2022) Partisan dealignment and the personalisation of politics in West European parliamentary democracies, 1961–2018. West European Politics 45(2): 311–334.

11.

Gattermann

de Vreese

(2020) Awareness of Spitzenkandidaten in the 2019 European elections: the effects of news exposure in domestic campaign contexts. Research & Politics 7(2): 205316802091533.

12.

Geer

(2012) The news media and the rise of negativity in presidential campaigns. PS: Political Science & Politics 45(03): 422–427.

13.

Habermas

(2011) The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society. 1. Publ. Cambridge: Polity. paperback, repr.

14.

Habermas

(2022) Ein neuer Strukturwandel der Öffentlichkeit und die deliberative Politik [A new structural Change of the Public Sphere and Deliberative Politics]. Wiederabgedruckt, 1962 Erschienen. Berlin: Suhrkamp.

15.

Halpern

Gibbs

(2013) Social media as a catalyst for online deliberation? Exploring the affordances of Facebook and YouTube for political expression. Computers in Human Behavior 29(3): 1159–1168.

16.

Haman

Školník

(2021) Politicians on social media. In The Online Database of Members of National Parliaments on Twitter. El profesional de la información. e300217

17.

Hanitzsch

Seethaler

Wyss

(2019) Journalismus in schwierigen Zeiten [Journalism in difficult times]. In: Hanitzsch

Seethaler

Wyss

(eds) Journalismus in Deutschland, Österreich Und Der Schweiz Journalism in Transition [Journalism in Germany, Austria, and Switzerland]. Studies in International, Transnational and Global Communications. Wiesbaden: VS Verlag für Sozialwissenschaften, 1–23.

18.

Heim

(2021) Tweets and source diversity: newspapers’ sourcing of Twitter posts from 2009 to 2016. Mass Communication and Society 24(3): 394–417.

19.

Hernández-Fuentes

Monnier

(2022) Twitter as a source of information? Practices of journalists working for the French national press. Journalism Practice 16(5): 920–937.

20.

Hine

(2020) The evolution and diversification of Twitter as a cultural artefact in the British press 2007–2014. Journalism Studies 13(2): 1–19.

21.

Humayun

Ferrucci

(2022) Understanding social media in journalism practice: a typology. Digital Journalism 10: 1502–1525.

22.

Jacobs

Sandberg

Spierings

(2020) Twitter and facebook: populists’ double-barreled gun? New Media & Society 22(4): 611–633.

23.

Jandura

Friedrich

(2014) The quality of political media coverage. In: Reinemann

(ed) Political Communication. DE GRUYTER, 351–374. Available at: https://www.degruyter.com/document/doi/10.1515/9783110238174.351/html (accessed 24 January 2022).

24.

Kapidzic

Neuberger

Frey

, et al. (2022) How news websites refer to Twitter: a content analysis of Twitter sources in journalism. Journalism Studies 23: 1247–1268.

25.

Kiflu

Ali

Nigussie

(2022) Media and public sphere in Ethiopia: mediated deliberations in public and commercial television programs. Journalism 24: 146488492110482.

26.

Kovach

Rosenstiel

(2021) The Elements of Journalism. Revised and. 4th edition. New York: Crown. updated

27.

Lecheler

Kruikemeier

(2016) Re-evaluating journalistic routines in a digital age: a review of research on the use of online sources. New Media & Society 18(1): 156–171.

28.

Lengauer

Esser

Berganza

(2012) Negativity in political news: a review of concepts, operationalizations and key findings. Journalism 13(2): 179–202.

29.

Löb

Rinke

Weinmann

, et al. (2022) Unpacking the determinants of outrage and recognition in public discourse: insights across socio-cultural divides, political systems, and media types. The International Journal of Press/Politics 29: 194016122210842

30.

Marcinkowski

Donk

(2012) The deliberative quality of referendum coverage in direct democracy: findings from a longitudinal analysis of Swiss media. Javnost - The Public 19(4): 93–109.

31.

Maurer

Oschatz

(2016) The influence of online media on political knowledge. In: Political Communication in the Online World. Theoretical Approaches and Research Designs. New York, NY: Routledge, 73–86.

32.

Maurer

Jost

Schaaf

, et al. (2022) How right-wing populists instrumentalize news media: deliberate provocations, scandalizing media coverage, and public awareness for the Alternative for Germany (AfD). The International Journal of Press/Politics 28: 194016122110726

33.

Metag

Rauchfleisch

(2017) Journalists’ use of political tweets. Digital Journalism 5(9): 1155–1172.

34.

Mueller

Saeltzer

(2022) Twitter made me do it! Twitter’s tonal platform incentive and its effect on online campaigning. Information, Communication & Society 25(9): 1247–1272.

35.

Oschatz

Stier

Maier

(2022) Twitter in the news: an analysis of embedded tweets in political news coverage. Digital Journalism: 1526–1545.

36.

Page

(1996) Who Deliberates? Mass Media in Modern Democracy. American Politics and Political Economy. Chicago: University of Chicago Press.

37.

Pieper

(2015) Political participation via social media: a case study of deliberative quality in the public online budgeting process of Frankfurt/Main, Germany 2013. Universal Access in the Information Society 14(4): 487–503.

38.

Reinemann

Stanyer

Scherr

, et al. (2012) Hard and soft news: a review of concepts, operationalizations and key findings. Journalism 13(2): 221–239.

39.

Rinke

Wessler

Löb

, et al. (2013) Deliberative qualities of generic news frames: assessing the democratic value of strategic game and contestation framing in election campaign coverage. Political Communication 30(3): 474–494.

40.

Rinke

Dobbrik

Löb

(2019) Mediated Contestation in Comparative Perspective (MedCon): Online Project Documentation. Mannheim Centre for European Social Research. University of Mannheim. Available at: https://medcon-doc.github.io

41.

Rozado

Hughes

Halberstadt

(2022) Longitudinal analysis of sentiment and emotion in news media headlines using automated labelling with Transformer language models. PLOS ONE Consoli S 17(10): e0276367.

42.

Schmuck

Matthes

von Sikorski

, et al. (2022) Tearing us apart? Muslims’ attitudes toward the majority population in response to differentiated versus undifferentiated news about terror. International Journal of Communication International Journal of Communication 16: 1420–1440.

43.

Schöne

Parkinson

Goldenberg

(2021) Negativity spreads more than positivity on Twitter after both positive and negative political situations. Affective Science 2(4): 379–390.

44.

Seethaler

(2019) Journalismus im Wandel [Journalism in Transition]. In: Hanitzsch

Seethaler

Wyss

(eds) Journalismus in Deutschland, Österreich Und Der Schweiz [Journalism in Germany, Austria, and Switzerland]. Studies in International, Transnational and Global Communications. Wiesbaden: VS Verlag für Sozialwissenschaften, 213–236.

45.

Sikorski

Heiss

Matthes

(2020) How political scandals affect the electorate. Tracing the eroding and spillover effects of scandals with a panel study. Political Psychology 41(3): 549–568.

46.

Strömbäck

(2005) In search of a standard: four models of democracy and their normative implications for journalism. Journalism Studies 6(3): 331–345.

47.

Tandoc

Jenkins

Craft

(2019) Fake news as a critical incident in journalism. Journalism Practice 13(6): 673–689.

48.

van Aelst

Strömbäck

Aalberg

, et al. (2017) Political communication in a high-choice media environment: a challenge for democracy? Annals of the International Communication Association 41(1): 3–27.

49.

van der Meer

TGLA

Kroon

Verhoeven

, et al. (2019) Mediatization and the disproportionate attention to negative news: the case of airplane crashes. Journalism Studies 20(6): 783–803.

50.

van der Meer

TGLA

Hameleers

Kroon

(2020) Crafting our own biased media diets: the effects of confirmation, source, and negativity bias on selective attendance to online news. Mass Communication and Society 23(6): 937–967.

51.

von Nordheim

Boczek

Koppers

(2018) Sourcing the sources. Digital Journalism 6(7): 807–828.

52.

Waisbord

(2018) Truth is what Happens to News: on journalism, fake news, and post-truth. Journalism Studies 19(13): 1866–1878.

53.

Wessler

(2008) Investigating deliberativeness comparatively. Political Communication 25(1): 1–22.

54.

Wessler

(2018) Habermas and the media. Theory and Media. Medford, MA: Polity.

Social media posts as source for political news coverage inside and outside election campaigns: Examining effects on deliberative news media quality

Abstract

Keywords

Deliberative quality of news

The quality of news reporting inside and outside election campaigns

Methods

Sample

Data collection

Manual content analysis

Inclusiveness

Justification

Responsiveness

Civility

Procedure

Strategy of analysis

Results

Discussion

Footnotes

Declaration of conflicting interests

Funding

Correction (March 2024):

ORCID iDs

Author biographies

References