Sage Journals: Discover world-class research

Abstract

This study investigates to what extent specific features of news articles about election campaigns impact reader engagement and civility in news comments. Using content analysis of articles (N = 830) and comments (N = 29,421) published during the 2015 Portuguese Legislative elections, we test the impact of negative coverage, issue coverage and game coverage (politics as a game) on the number of comments that an article receives and the level of civility thereof. Additionally, we explore how affective polarisation of a commenter may moderate the effects on incivility. Findings show that negativity towards political actors in an article is tied to both an increase in the number of comments and their level of incivility. Game coverage only led to a significant increase in the number of comments, while actor-related positivity was also related to an increase in incivility. Issue coverage had neither positive nor negative effects. The results inform newsrooms and academics about the implications of different types of election reporting, while accounting for features of news articles that are typically not integrated in a single study.

Keywords

Affective polarisation audience engagement content analysis elections incivility political journalism user comments

Over recent years there have been many trends which may be considered to be troublesome by those who subscribe to a deliberative view of democracy. Citizens are more polarised along partisan identities (Iyengar and Westwood, 2015), and place less trust in political institutions (Ceron, 2015). While the internet has the potential to mitigate these issues (Papacharissi, 2002), research reveals that opinions shared online tend to be more extreme and express a greater sense of outrage than in the past (Antoci et al., 2016; Sobieraj and Berry, 2011). When searching for solutions to online incivility and polarisation, it may be relevant to explore the influence of political reporting. It is arguably easier to modify the reporting practices of a newsroom than to address other potential causes of incivility and polarisation, such as individuals’ media diets (Stroud, 2011).

Through a content analysis of articles and comments from three Portuguese news outlets during the 2015 Legislative elections, this paper explores the relationship between reporting strategies that emphasise partisan features, and online participation and incivility. In this framework, affective polarisation, the distrust and dislike of other viewpoints, and its strong connection to partisan social identity (Iyengar et al., 2019), is explored as the theoretical mechanism that explains an increase in incivility, as previous studies have revealed that the affective component is a predictor of incivility in comments (Ziegele et al., 2017). In particular, we see game coverage and negativity as features of news articles that evoke partisan identities and animosity.

The impact of game coverage and negative reporting has been researched by other scholars, but the findings are not always conclusive. While some studies show negative effects of reported political incivility on trust (Mutz and Reeves, 2005), others claim that incivility does not lead to detrimental effects and may increase engagement (Brooks and Geer, 2007). Likewise, while studies have found negative effects of game coverage (see Aalberg et al., 2012 for a review), such coverage may also increase attention to politics and revenue for news organisations (Iyengar et al., 2004), a particularly relevant caveat for newsroooms facing financial troubles. One reason for these mixed findings may be that most of the previous works are limited by not differentiating and accounting for the impacts of both game coverage and negative news (see Trussler and Soroka, 2014 for an exception). Despite efforts to conceptualise and distinguish these concepts (Aalberg et al., 2012; Lengauer et al., 2012), studies that simultaneously theorise and operationalise them are scarce. Furthermore, most studies come from the bipartisan context of the United States of America (USA), and may not generalise adequately to other contexts.

This study aims to address both of these gaps by focusing on the understudied Portuguese context and by making an operational distinction between game coverage and negativity in news and analysing their effects within the same models. Portugal presents a particularly interesting context of study since, contrary to its classification within the polarised pluralist model by Hallin and Mancini (2004), it represents one of the media contexts that has the lowest level of audience political polarisation (Newman et al., 2017), in stark contrast to the USA. Despite the small scale of its news organisations by international standards, they show a remarkable resistance to social and economic pressures (Quintanilha et al., 2019). Reporting practices in Portugal are characterised by a lack of negativity towards political protagonists and parties (Salgado, 2011). In this context, the 2015 Legislative Elections present a unique context, since they signalled a break from traditional non-partisan reporting practices in at least three meaningful ways: (1) a greater emphasis on game coverage (e.g. through daily tracking polls and reporting on mutual political attacks), (2) the emergence of Observador, a news organisation that has a slight ideological leaning to the right (Newman et al., 2017), (3) a higher degree of polarisation in relation to the austerity policies of the previous government during the 2009–2012 financial crises, creating a clear binary divide and outgroups in a multi-party system. This shifting context provides additional variation in news reporting styles, which in turn makes it an ideal setting to research how audiences respond to these styles. Although this is not a longitudinal study, by investigating this shifting context, we may better understand the underlying causes of the high degrees of polarisation and press distrust that currently exist in countries such as the USA and Spain.

This paper starts by looking at online incivility and engagement, interpreting these outcomes in light of deliberative democracy theories and concerns for newsrooms. We then suggest social identity and affective polarisation as the key theoretical explanations that tie news characteristics, such as negativity and game coverage, to engagement and incivility. We finally explore results and discuss some implications our findings for theory and practice, while listing the limitations of this study and suggestions for further research.

The role of polarisation, incivility and engagement for democracy

Following previous work on reader comments (e.g. Ruiz et al., 2011; Stroud et al., 2015), this paper tackles the relationship between reporting practices and reader participation from a deliberative perspective. According to deliberative democracy, political debate is organised around alternative conceptions of the public good and, therefore, a more pluralistic scheme – where citizens, parties and groups operate in a narrow standpoint in pursuit of a sectional interest – is not suitable (Cohen, 1997). Political deliberation requires citizens to orient themselves to public interests and deliberation aims to improve political decision making, especially with regard to achieving common ends (Bohman and Rehg, 1997). This means that practices such as incivility undermine the ideal of deliberative debates, which require a rational and civil exchange of positions (Ziegele and Jost, 2016: 3). Conversely, engagement may be seen as desirable since it aligns with the requirements of discursive inclusion and equality that underlie the deliberative conception of democracy.

While it is sometimes argued that deliberative democracy presents an unrealistic and even undesirable (Mouffe, 1999) ideal for democracy, we select this perspective not because we consider it to be superior to other theories of democracy, but because we believe it to be useful for an empirical normative assessment (Althaus, 2012) from a journalistic point of view. For instance, while incivility may be seen as a necessary part of democracy from an agonistic (Mouffe, 1999) or counter-publics (Dahlberg, 2011) perspective, Portuguese newsrooms are accountable for the reader comments in their websites (see Portuguese Media Regulator deliberation 2/CONT-NET/2011) and therefore face legal repercussions for violations of norms in those spaces. Furthermore, news editors themselves consider partisanship and incivility to be undesirable traits of comment sections (Goodman and Cherubini, 2013). These features, along with the fact that previous work has been developed under a deliberative perspective, provide a reasonable argument for selecting a deliberative approach to news and reader content.

Incivility has been a matter of interest for both scholars of political communication (Brooks and Geer, 2007; Mutz and Reeves, 2005) and of journalism (Prochazka et al., 2016). It is often studied in the context of news comment sections (Muddiman and Stroud, 2017; Santana, 2015) and evidence demonstrates that news content (Coe et al., 2014; Ziegele et al., 2018) and actions by journalists (Stroud et al., 2015) may change the characteristics of messages left by readers in such discussion spaces. Beyond the scope of news comment sections, it is noteworthy that exposure to uncivil comments leads to higher levels of issue polarisation (Anderson et al., 2014). In contrast, exposure to uncivil content may also have positive effects from a deliberative perspective, such as increasing an individual’s willingness to participate (Borah, 2014) or the credibility of a news article (Borah, 2013). The latter finding is relevant for this study, because the study by Borah (2013) shows that effects of uncivil reader comments on news credibility and political trust are moderated by the type of news frame employed.

Incivility is a notoriously difficult concept to define, since what is perceived as incivility can vary greatly (Stryker et al., 2016). As such, definitional efforts have sought to distinguish it from related concepts such as politeness (Papacharissi, 2004). For this study, our definition draws close to that provided by Coe et al. (2014: 3): ‘features of discussion that convey an unnecessarily disrespectful tone towards the discussion forum, its participants, or its topics.’ However, recognising that some form of critique is necessary for democratic discussion (Schudson, 1997), we draw a line by considering that an attack on someone’s competence is not uncivil, provided that it does not constitute an attack on the person’s identity or honour. Additional details are provided in the codebook in Appendix A.

In contrast, the concept of reader engagement seems to be more straightforward to define. Engaging with, and participating in, news can take many forms – from the most basic act of reading or clicking on an article to writing a letter to the editor or leaving a news comment, the latter being the form of engagement used in our study. For this study, we operationalise engagement through reader comments. Engagement is usually considered to be desirable, both by newsrooms and theorists of democracy, developing a public sphere (Habermas, 1989) that keeps political power in check and does not exclude participants.

A key concept that may have an impact on reader engagement through news comments and the civility of these comments is affective polarisation. Iyengar and Westwood (2015: 691) define affective polarisation as ‘the tendency of people identifying as Republicans or Democrats to view opposing parties positively and copartisans negatively’. While polarisation itself may not be a detrimental feature in deliberative democracy, that is, a stronger identification with a political party is not negative in itself, affective polarisation is seen as negative since it implies the rejection of opposing views because of the affective component, and not their validity according to critical-rational criteria. Although the Portuguese contexts lack the bipolarity that drives Iyengar and Westwood’s definition, citizens still identify with a particular political party or pre-electoral coalition, while all remaining parties, even if they form coalitions after the elections, are in effect opposing that party. While it is true that the concept of affective polarisation may be more nuanced in a multi-party system, this does not mean that it is not applicable. Since news coverage often primes partisan identities, affective polarisation is a useful mechanism to explain the connection between article content and reader participation. In particular, affective polarisation appears as a key basis for our hypotheses since it may lead to higher levels of political participation (Iyengar and Krupenkin, 2018), but negative affect is also related to incivility in news comments (Ziegele et al., 2017).

Types of political reporting

Most definitions of strategic, game, or strategic game types of coverage share the competitive component that underlies the horse race metaphor of election coverage (Broh, 1980). We follow previous research (Trussler and Soroka, 2014) that sometimes used the term ‘coverage’ instead of ‘frame’, to distance ourselves from the conceptual inconsistency that has characterised framing research (Scheufele and Iyengar, 2014). Our definition of game coverage is adapted from the review of Aalberg et al. (2012): ‘Game coverage refers to news stories that focus on game-like aspects of politics and are centred around: who is winning or losing elections and in the battle for public opinion; expressions of public opinion (polls, vox pops); speculate about electoral or policy outcomes or potential coalitions; or use the language of war and sports to describe politics.’ While this definition takes many aspects from the one by Aalberg et al. (2012), it differs slightly from it since it mentions language aspects and does not refer to frames.

The presence of game-like political coverage may increase the salience of political identities and therefore reinforce the boundaries between the ingroup and the outgroup. When political campaigns are portrayed as a game, this emphasises the multiple sides in conflict, in what is often a zero-sum game. By highlighting this competition, journalists prompt readers to think about their own political affiliation and choose sides, which in turn may resonate with their affective polarisation. Considering the effects of affective polarisation on political participation and incivility, we hypothesise the following:

H1. The presence of game coverage in election articles is related to higher levels of incivility in news comments.

H2. The presence of game coverage in election articles is related to higher levels of engagement.

These hypotheses are supported by findings from previous research, since strategy coverage and game coverage are tied to incivility in elite discourse (Rinke et al., 2013) and generate more interest, especially among politically interested readers (Trussler and Soroka, 2014) who may be more likely to participate.

Issue coverage is often studied alongside and in relation to game coverage and strategy coverage (e.g. Dekavalla, 2016; Rhee, 1997; Shehata, 2013). Issue coverage relates to the policy proposals and stances of each candidate or party. In general terms, ‘the “issue frame” is defined as stories about policy problems and solutions, descriptions of politicians’ policy stands, and implications of different proposals or legislations’ (Aalberg et al., 2012: 168). This type of coverage may be regarded as normatively more desirable within the framework of deliberative democratic theory, and has been found to have positive mobilisation effects (Shehata, 2013).

Although game coverage makes partisan identities salient, issue coverage may be more closely related to citizens’ attitudes towards policy topics. While the effects of issue identity might not be as strong as partisan identity (Huddy et al., 2015), readers might still be affectively polarised on policy positions, leading to higher incivility. However, from a normative perspective, a focus on issues may steer the debate away from a strength-based zero-sum game towards a deliberative argument of finding solutions. This would lead discussion in the comments section to be less conflict-oriented and, consequently, more civil. Due to the lack of clear directionality, we address issue coverage through a research question:

RQ1. How does issue coverage in news articles relate to incivility in news comments?

Strategy coverage, although abundantly mentioned in the literature, is less relevant for this study. This is not due to theoretical reasons but primarily to contextual factors that will be further explored in the methods section, since national context is a relevant factor in political reporting (Aalberg et al., 2012: 165).

Similarly to game coverage, we looked at a previous work (Lengauer et al., 2012) to establish our definitions of negativity and positivity in news articles. Following Lengauer et al. (2012: 195–197), we distinguish between two types of negativity (and positivity): non-directional and actor-related. Non-directional negativity concerns the overall tone and outlook of a news article, and refers to a negative impression of politics, events or prospects. These may be articles that lament the current state of politics due to high abstention rates or offer grim economic forecasts for a country, but do not refer to specific political actors, such as candidates or parties. Overall, negativity has been shown to attract news consumers’ attention (Soroka and McAdams, 2015; Trussler and Soroka, 2014), so one might expect non-directional negativity to increase engagement.

H3. Non-directional negativity will lead to higher levels of engagement.

By contrast, actor-related negativity conveys a negative impression of a politician or political institution, being by definition more closely related to the concept of affective polarisation. This kind of news articles may suggest that a candidate is incompetent, has failed in a certain endeavour, or that a party is underperforming in the campaign. It is important to note that such forms of negativity do not have to come from the journalists themselves, but may be opinions or political speeches that are cited in a news article. Actor-related negativity is often a manifestation of affective polarisation in the political elites, and may lead readers to become more polarised as well. This sort of actor-related negativity also often crosses the boundaries of uncivil content, which has been shown to increase willingness to participate (Borah, 2014). Considering the relationship between affective polarisation, incivility, and engagement, the following hypotheses are put forward:

H4. Actor-related negativity is related to higher levels of incivility.

H5. Actor-related negativity is related to higher levels of engagement.

While negativity has often been the focus of previous literature, the opposing concept of positivity may also be relevant to analyse, by reversing the definitions. Whenever we refer to both positivity and negativity in this paper, the word valence will be used. Regarding positivity, if readers perceive the election as a zero-sum game, affective polarisation may drive them to be uncivil towards an opponent who is praised in a news story in order to mitigate possible gains. However, from a social norms perspective, in which the commenters mimic the tone set by the article, positivity leads to less aggressive forms of engagement, thus reducing incivility. Since the directionality of effects is not clear, we add the following research question:

RQ2. To what extent does actor-related positivity relate to incivility in the comments?

Although game coverage and negativity overlap at times, these are distinct concepts. When an article about a poll claims a party is winning or losing, it uses the language of games as well as displaying actor-related positivity and negativity, respectively. However, an article may cite a candidate criticising an opponent’s economic policies – which would be an instance of actor-related negativity and issue coverage, but not of game coverage. As Trussler and Soroka (2014) argue, the two concepts may be related, but are clearly different. Given that one of the main goals of this paper is to disentangle the effects of game coverage and negativity, we ask the following overarching question:

RQ3. To what extent does accounting for both negativity and game coverage differ from considering their effects separately?

Finally, affective polarisation has been presented throughout this paper as a key mechanism to explain the relationship between news content and comments. Although an experimental setting would be ideal to test if this is indeed the case, if affective polarisation is driving our effects, we may expect them to be stronger for highly polarised individuals. We test this assumption through the following research question:

RQ4. To what extent does commenter polarisation moderate the relationship between coverage type and commenter engagement and civility?

Methods

To test how different kinds of election coverage relate to reader engagement and civility, we look at articles covering the 2015 Portuguese Legislative elections and the comments posted under those articles. We focus on the campaign period since this is the timeframe when election coverage and reader commenting are more intense. While an informal pre-campaign period with debates and political statements exists in Portugal, commencing 4 to 6 weeks before the elections, it is in the 12-day official campaign period that parties are allowed to use public spaces for organising rallies and events, have additional TV time devoted to campaigns ads, and are given spaces to display campaign posters and other forms of print material. In practice, it means that most party resources, as well as press coverage, are condensed in this two-week period before the elections take place. Selecting Portugal as a context of study provides a shift from US-based studies (e.g. Coe et al., 2014; Santana, 2014; Scacco et al., 2017) not only because of the multi-party parliamentary system, but also because Portugal and the USA are on opposite sides of the spectrum in terms of polarisation of online news audiences, wherein the USA is the most polarised and Portugal the least polarised of the countries in the Reuters Institute Digital News Report 2017¹ (Newman et al., 2017: 39).

In order to assess if effects hold for newsrooms with different reporting styles and comment moderation policies, three major news outlets were selected for the analysis (Público, Expresso and Observador). Their websites were selected because they are leading Portuguese newspapers, have diverse editorial origins and commenting systems, and have active commenter communities. All news and opinion articles published on their websites during the official campaign period (September 20 to October 2, 2015) were collected for the analysis, using a data scraping programme that was specifically coded for the purpose (N = 830). Relevant articles were identified by having the 2015 election tag assigned to them by the respective newsrooms. The 29,421 comments posted under those articles on the news outlets’ websites were also collected, but 806 were excluded because they were duplicate comments or SPAM/advertising content. Details of each news outlet are provided in Table 1.

Table 1.

Newspaper characteristics.

	Público	Expresso	Observador
Founded	1990	1973	2014
Medium	Paper (daily) and online	Paper (weekly) and online	Online
Moderation	Pre-moderation	Post-moderation based on user signalling/complaints	Post-moderation
Moderators	Commenter community supervised by journalists	Newsroom	Newsroom
Reply levels	2	4	2
Character limit	800	No limit	No limit
Time limit for commenting	30 days after article was published	No limit	No limit
Registration	Mandatory with email, Facebook or Twitter account	Mandatory with email address	Mandatory with email address or social media account

All comments and news articles were labelled by the researchers using the categories in the codebook provided in Appendix A. Categories and operational definitions were initially drawn from previous works (Aalberg et al., 2012; Coe et al., 2014; Lengauer et al., 2012) and subsequently adjusted after reliability tests with two graduate student coders to establish pre-coding reliability. Once acceptable (>.67, Krippendorff’s alpha [KA]) levels were achieved (Krippendorff, 2004), one of the researchers coded the entire corpus². Given the size of the coding task, intracoder reliability tests were also conducted (i.e. the coder recodes content that was coded in the early stages of the process to ensure criteria did not change over time) wherein the results were always above .80 KA. Final reliability values were obtained by asking a graduate student to code a random sub-sample of 218 comments and 93 news articles after 3 hours of training and comparing the results with the actual coding. Disagreements were resolved by discussing the specific instances until a consensus was reached. Reliability values were calculated using the KALPHA SPSS macro (Hayes and Krippendorff, 2007) and 95% confidence intervals for 5000 bootstrapped samples for each variable are provided in Table 2. Although some lower limits of the CI are below .67, especially in terms of actor-related positivity, the values for alpha are all well above the minimum acceptable threshold.

Table 2.

Inter-coder reliability for comments (N = 218) and articles (N = 93).

Variable (N)	Percent agreement	Krippendorff’s alpha	α 95% CI [Low]	α 95% CI [High]	Prob α < .67
Incivility	89.9%	.76	.66	.85	.029
Game coverage	88.2%	.76	.63	.89	.079
Issue coverage	91.4%	.83	.69	.93	.007
Non- directional neg./pos.	90.3%	.81	.69	.92	.012
Actor-related neg.	91.4%	.83	.70	.94	.007
Actor-related pos.	87.1%	.74	.58	.87	.146
Party position	91.3%	.86	.81	.90	.000

A polarisation score for a user was calculated based on the party positions (in favour vs. against) expressed in that user’s comments. Since inferring positions from a limited number of comments would be unreliable, only users with five or more comments were assigned a polarisation score. The formula for calculating this score is detailed on Appendix B, accounting for the difference in the score of a user’s preferred party and their least preferred party, an approach that has been used by other scholars in multi-party systems (e.g. Hansen and Kosiara-Pedersen, 2015). Only the four main political forces are included, since the remaining ones are only mentioned in 0.4% of comments and could artificially inflate polarization scores.

Although we initially aimed to label strategy coverage as well as game coverage, the former failed to reach adequate reliability values in pre-coding, even after extensive coder training. This can be attributed to two main reasons: (1) instances coded as strategy coverage were very infrequent (<5.0%); (2) Portuguese political coverage, especially during the official campaign period, does not seem to fit within the usual definitions of strategy coverage. Substantial cross-national variation exists in how journalists approach elections (Aalberg et al., 2012: 165). In Portugal, the official campaign period only lasts 2 weeks, which means that journalists and pundits focus on reporting and interpreting developments in this fast-paced environment, rather than drawing considerations about party performance or strategy. While being substantially different from the long electoral race that defines US politics, this is actually similar to other European political contexts, such as Spain or the United Kingdom.

Results

Incivility was present in 30.9% of the analysed comments. However, due to a stricter moderation policy, comments on Público’s site displayed lower levels of incivility (19.4%). There is also a substantial difference between the average number of comments per article on Expresso (M = 102.36), Observador (M = 16.3) and Público (M = 16.9). Taken together, these differences illustrate the importance of controlling for newspaper, in both incivility and engagement models.

In terms of news article characteristics, 36.7% of the articles displayed issue coverage while 44.6% had some form of game coverage. These categories were coded independently and are not mutually exclusive. This shows that game coverage played a substantial role in campaign reporting during the 2015 elections, and that we do find substantial variability in this regard in our dataset. Regarding valence, 48.3% of articles displayed actor-related negativity, while 41.0% had some form of actor-related positivity. Concerning non-directional valence, 70.0% of articles have a neutral tone, while 23.0% were overall negative and only 6.6% were positive. The categories for the overall non-directional valence are mutually exclusive, but the actor-related ones are not, that is, an article may include praise towards one politician while criticising another.

To test the hypotheses related to incivility, we ran cross-classified multilevel logistic regression models with random intercepts. The binary variable for incivility is the dependent variable, while news article characteristics (valence and coverage type), newspaper, and whether the article was blocked by a paywall are level 1 independent variables. The individual comments are the level 1 units in the model, while the article and the author/username are level 2 variables. Since there are two level 2 variables (i.e. a comment belongs to a specific author and is posted under a specific article) that are not nested within each other, the model is considered to be cross-classified. Since part of the goal of this study is to disentangle the effects of valence and issue/game coverage (RQ3), we also ran models (Models 1 and 2) that account for these factors separately.

Looking at all three models on Table 3, it is clear that both actor-related negativity and positivity are significant positive predictors of incivility, which therefore supports H4 (actor-related negativity leads to higher incivility) and answers RQ2 (To what extent does actor-related positivity relate to incivility in the comments?). In practical terms, this means that a comment on an article with actor-related negativity is 11.6% more likely to be uncivil, while a comment on an article with actor-related positivity is 12.7% more likely to be uncivil. However, game coverage and issue coverage were not significant predictors of incivility in models 1 to 3, therefore H1 (game coverage leads to higher incivility) can be rejected and the answer for RQ1 (How does issue coverage in news articles relate to incivility in news comments?) is that issue coverage has no impact on incivility. Furthermore, answering RQ3 (To what extent does accounting for both negativity and game coverage differ from considering their effects separately?), we see that the significance and directionality of valence and coverage types does not change when these are considered separately.

Table 3.

Multi-level models for incivility.

	Model 1	hModel 2	Model 3	Model 4
	Valence only	Coverage type only	Coverage type and valence	Polarisation as moderator
	Coefficient (SE)	Coefficient (SE)	Coefficient (SE)	Coefficient (SE)
		Individual-Level variables (Level-1)
Intercept	−0.91 (0.06)***	−0.75 (0.06)***	−0.86 (0.07)***	−1.49 (0.11)***
Newspaper (Público = 1)	−0.57 (0.10)***	−0.57 (0.10)***	−0.57 (0.10)***	−0.61 (0.12)***
Newspaper (Observador = 1)	−0.38 (0.10)***	−0.44 (0.10)***	−0.38 (0.10)***	−0.31 (0.12)*
Paywall	−0.11 (0.08)	−0.15 (0.08)	−0.11 (0.08)	−0.09 (0.08)
Non-directional positivity	0.00 (0.07)	–	0.02 (0.07)	0.01 (0.06)
Non-directional negativity	−0.02 (0.06)	–	−0.02 (0.06)	−0.02 (0.06)
Actor-related positivity	0.10 (0.04)*	–	0.11 (0.05)*	0.17 (0.08)*
Actor-related negativity	0.11 (0.05)*	–	0.12 (0.05)*	0.26 (0.08)**
Issue coverage	–	−0.06 (0.05)	−0.06 (0.05)	−0.04 (0.06)
Game coverage	–	−0.03 (0.05)	−0.08 (0.05)	−0.25 (0.08)**
Polarisation score	–	–	–	3.12 (0.40)***
Polarisation*Game coverage	–	–	–	0.66 (0.32)*
Polarisation*Actor-related pos.	–	–	–	−0.27 (0.31)
Polarisation*Actor-related neg.	–	–	–	−0.75 (0.32)*
	Variance component (SD)	Random effects
	Variance component (SD)	Variance component (SD)	Variance component (SD)	Variance component (SD)
Article Variance (Level-2)	0.09 (0.30)	0.09 (0.30)	0.08 (0.29)	0.07 (0.27)
User Variance (Level-2)	0.67 (0.82)	0.67 (0.82)	0.67 (0.82)	0.55 (0.74)
Deviance	32360.1	30759.0	32347.2	29442.1
AIC	32376.1	32376.1	32371.2	29474.1
N of level one units (comments)	27753	27753	27753	25444
N of level two units (news articles)	652	652	652	624
N of level two units (unique users)	2057	2057	2057	702

p < 0.05; ^**p < 0.01; ^***p < 0.001.

When considering the role of polarisation for incivility in Model 4 (RQ4. To what extent does commenter polarisation moderate the relationship between coverage type and commenter engagement and civility?), we see, in line with previous studies, that users with higher polarisation values are more likely to leave uncivil comments, somewhat supporting the convergent validity of our polarisation score. The positive interaction between game frames and polarisation shows that highly polarised individuals tend to be more uncivil in the presence of game frames, in line with our expectations. However, the opposite seems to be the case for actor-related negativity, where the negative coefficient indicates that highly polarised individuals are less likely to be uncivil when faced with actor-related negativity than their less polarised counterparts.

To test the hypotheses related to engagement we used negative binomial regression models with a log-link, since the dependent variable takes the form of overdisperse count data. As with the analysis of incivility, separate models were run to assess the effects separately. The independent variables correspond to the level 1 variables used in the previous models.

The models on Table 4 show that both game coverage and actor-related negativity predict a significantly higher number of comments on an article, supporting H2 (game coverage leads to higher engagement) and H5 (actor-related negativity leads to higher engagement). If we consider model 7, articles with game coverage have a 64.9% higher predicted number of comments than those without it, while those with actor-related negativity have 41.9% more comments. Non-directional negativity, however, is only a significant predictor of the number of comments in Model 5. This means that when issue coverage and game coverage are accounted for, the negative tone of a news story is no longer a significant indicator of how many comments that story will have. In light of this, H3 (non-directional negativity leads to higher engagement) is rejected and an answer to RQ3 (To what extent does accounting for both negativity and game coverage differ from considering their effects separately?) can be put forward. The effects of valence and coverage type on incivility do not differ substantially if accounted for separately or together. However, actor-related positivity and non-directional negativity are only significant predictors of engagement when issue coverage and game coverage are not accounted for. To assess the performance of our model, we used the Akaike Information Criterion (AIC), which measures the model performance by rewarding its predictive power while penalising the use of additional variables (lower values are better). Thus, the fact that Models 3 and 7 have a lower AIC also reinforces the case for integrating both types of variables in a single model or study.

Table 4.

Negative binomial models for number of comments.

	Model 5	Model 6	Model 7
	Valence only	Coverage type only	Coverage type and valence
	Coefficient (SE)	Coefficient (SE)	Coefficient (SE)
Intercept	3.95 (0.09)***	4.00 (0.08)***	3.77 (0.10)***
Newspaper (Público = 1)	−2.34 (0.09)***	−2.37 (0.09)***	−2.33 (0.09)***
Newspaper (Observador = 1)	−1.71 (0.11)***	−1.91 (0.10)***	−1.79 (0.11)***
Paywall	−0.76 (0.14)***	−1.00 (0.14)***	−0.89 (0.14)***
Non-directional positivity	0.14 (0.13)	–	0.15 (0.13)
Non-directional negativity	−0.20 (0.09)*	–	−0.10 (0.13)
Actor-related positivity	0.27 (0.08)***	–	0.13 (0.08)
Actor-related negativity	0.39 (0.08)***	–	0.35 (0.08)***
Issue coverage	–	0.01 (0.09)	−0.01 (0.09)
Game coverage	–	0.62 (0.08)***	0.50 (0.09)***
Deviance	1897.58	30759.0	1857.44
AIC	6665.13	32376.1	6628.98
N (articles)	819	819	819

Discussion

The findings suggest that actor-related negativity and positivity are the main consistent predictors of incivility in news comments. The fact that both forms of actor-related valence are positive predictors of incivility hints that party identity and intergroup conflict may be adequate to explain commenter behaviour. The finding that affective involvement is one of the main factors driving uncivil comments (Ziegele et al., 2017) is also aligned with this explanation, since attacks on one’s party may trigger this sort of affective response. However, interaction effects with polarisation scores suggest a more complex picture. Actor-related negativity actually had a weaker effect on civility for highly polarised individuals. While this finding may seem counter-intuitive at first, it could be that actor-related negativity is already naturalised by individuals who are themselves often negative towards out-parties. If that is the case, encountering this sort of negativity in the news would be less likely to trigger the use of uncivil language. Furthermore, recent studies have shown that the presence of incivility in the media, a similar construct to actor-related negativity, may decrease affective polarisation for in-party media outlets (Druckman et al., 2019).

In contrast, and diverging from previous studies (Rinke et al., 2013), game coverage by itself does not lead to higher levels of incivility in posted comments. We believe that, although game coverage does establish the conditions for a zero-sum game context that lead to the effects of actor-related valence, it does increase not incivility under all circumstances. However, a positive interaction term hints that game coverage may actually increase the likelihood of uncivil posts for highly polarised individuals, when compared to less polarised readers. This suggests that game coverage may be detrimental to the civility of discussions, but only in settings where the audience itself if already polarised.

The results regarding engagement are more clearly aligned with findings from previous studies (Trussler and Soroka, 2014). Both actor-related negativity and game coverage led to a higher number of comments, thus justifying their prevalence in election coverage. This highlights the dilemma facing newsrooms, especially regarding the use of negativity. While reporting on politicians attacking each other may decrease the civility of the discussion, it does generate more engagement from readers.

It is relevant that issue frames had no significant effect on either civility or engagement, contrary to the findings of previous studies (Stroud & Muddiman, 2018). While no improvements regarding civility were observed when the articles focused on policies and issues, it is also noteworthy that this sort of coverage did not depress reader engagement in Portugal, something that was observed in the USA when looking at shares and clicks (Scacco et al., 2017). This suggests that newsrooms do not have to sacrifice reader participation in the comments section if they aim to address some of the issues and policies that emerge during election campaigns. This would be desirable from a normative point of view according to the principles of deliberative democracy, even if it does not have any practical consequences in terms of civility and engagement in discussions.

Overall, these findings indicate that game coverage and actor-related negativity are effective business strategies for newsrooms even in non-US contexts. The fact that this is applicable in a media and political system that is quite different from the American system suggests a wider trend, where newsrooms that face economic pressures may naturally shift their political reporting styles to strategies that generate higher engagement from their readers.

However, our results also suggest that this may have negative consequences for political debate, at least from a deliberative perspective. Actor-related negativity increased incivility in news comments, highlighting the tension between democratic and business outcomes of newsrooms. Additionally, the role of polarisation as a relevant moderator highlights the importance of accounting for this factor in future studies. Theoretically, we have shown that polarisation is a relevant factor in the civility of discussions even outside an experimental setting and in a multi-party context.

From a practical point of view, we have shown newsrooms effective ways of increasing engagement through their editorial choices over political coverage. However, these choices may also have negative consequences from the perspective of deliberative democracy, often aligned with the ideals of the newsrooms themselves.

Finally, the findings regarding engagement highlight why accounting for both negativity and coverage types in a single study is a relevant decision. If game coverage and issue coverage are not accounted for, the effects of valence on engagement are overestimated. Furthermore, the interactions of actor-related negativity and game coverage with polarisation had effects in opposite direction. This finding is in line with studies (Rains et al., 2017) that highlight that the relationship between political identities and incivility is a complex one, showing that effects vary according to the side of the political spectrum. Our study contributes to scholarship by illustrating the importance of distinguishing these two related, but clearly different concepts. The interactions suggest not only different effects, but also different theoretical mechanisms at work.

Limitations and future research

This study looks at the connection between how journalists cover election campaigns and how readers discuss politics. However, such a study is not without limitations. While the unobtrusive nature of content analysis boosts the external validity of these findings, potential confounding variables also limit their generalisability. One main shortcoming was our inability to control for the salience of an article in the news outlet’s homepage and social media, since this may have obvious implications for the number of comments that a story receives. It may be that news outlets assign higher prominence to stories containing game coverage and actor-related negativity, and that in turn justifies a higher number of comments resulting in significant effects. While that would be significant by itself, the fact that salience cannot be accounted for in this study limits the causal interpretation of our findings, due to potential confounds. However, even if this is the case and causality is not proven, this would still reveal that newsrooms privilege negative and game coverage articles, which is a revealing fact in its own right. Furthermore, salience should not have an impact on civility metrics (i.e. even paywalls, a metric for accessibility, did not have a significant influence on those outcomes). Additionally, analysing three different news outlets and a large volume of comments contributes to the robustness of the results and may somewhat compensate for these shortcomings. Finally, the use of multilevel models with random intercepts for incivility allows us to control for article level and author level variance, ensuring that a single article or commenter does not excessively condition the results and limiting confounding factors.

Additional experimental studies that manipulate both coverage types and different forms of valence in news stories may be useful not only to address these causal elements, but also to clarify the underlying mechanisms that account for the effects. While the partisan identity and affective polarisation frameworks seem to provide an adequate explanation for our findings, survey experiments (e.g. Ziegele et al., 2017) allow further exploration of these mechanisms and testing, for instance, potential mediators. Furthermore, we urge other scholars to continue the effort of introducing data from political contexts that are understudied, ideally from a comparative perspective.

Another avenue for future research may be to replicate this study on platforms such as Facebook and Twitter, since these may reach other types of audiences and therefore reveal additional nuances on the effects of different types of political reporting on civility and engagement. However, it should be considered that studying reader comments within a news organisation’s website is a stronger guarantee that the commenters were indeed exposed to the content of the article, while social media users may comment on a link to a news story without actually opening it. As such, it may be that any effects found in our study may appear weaker if the study is replicated in social media platforms. Nonetheless, the relative absence of studies comparing comments on news sites to social media platforms (see Rowe, 2015 for an exception) would make this a worthy endeavour.

Supplemental Material

sj-pdf-1-jou-10.1177_1464884920958367 – Supplemental material for How to report on elections? The effects of game, issue and negative coverage on reader engagement and incivility

Supplemental material, sj-pdf-1-jou-10.1177_1464884920958367 for How to report on elections? The effects of game, issue and negative coverage on reader engagement and incivility by João Gonçalves, Sara Pereira and Marisa Torres da Silva in Journalism

Supplemental Material

sj-pdf-2-jou-10.1177_1464884920958367 – Supplemental material for How to report on elections? The effects of game, issue and negative coverage on reader engagement and incivility

Supplemental material, sj-pdf-2-jou-10.1177_1464884920958367 for How to report on elections? The effects of game, issue and negative coverage on reader engagement and incivility by João Gonçalves, Sara Pereira and Marisa Torres da Silva in Journalism

Footnotes

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by POCH Programa Operacional Capital Humano, participated by the European Social Fund and by national funds by the MCTES by an individual doctoral scholarship granted by Fundação para a Ciência e Tecnologia (SFRH/BD/103335/2014). Text revision funded by Fundação para a Ciência e Tecnologia through project UIDB/00736/2020.

ORCID iD

João Gonçalves

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

João Gonçalves is a Lecturer at the Department of Media and Communication of Erasmus University Rotterdam. He has a PhD in Communication Sciences from the University of Minho, Portugal, and his research interests are online political discussions, audience engagement and artificial intelligence.

Sara Pereira is an Associate Professor at the Communication Sciences Department and researcher at the Communication and Society Research Centre at the University of Minho, having obtained a PhD in Child Studies from the same university. Her research focuses on audiences, communication and citizenship, and media literacy. She has been coordinating several research projects on these areas. She co-coordinates the Media and Information Literacy Observatory (MILObs) and she is co-author of ‘Ouvido Crítico’, a weekly Media Literacy radio programme. Currently, she is chair of the Media Education Research section at IAMCR – International Association for Media and Communication Research.

Marisa Torres da Silva is an Assistant Professor at the Communication Sciences Department of the Nova University of Lisbon, having obtained a PhD in Communication Sciences from the same university. Her research focuses on hate speech, letters to the editor and online news comments.

References

Aalberg

Strömbäck

Vreese

CH de

(2012) The framing of politics as strategy and game: A review of concepts, operationalizations and key findings. Journalism 13(2): 162–178.

Althaus

(2012) What’s good and bad in political communication research? normative standards for evaluating media and citizen performance. In: Semetko

Scammell

(eds) The SAGE Handbook of Political Communication. London: SAGE Publications Ltd, pp. 97–112.

Anderson

Brossard

Scheufele

, et al. (2014) The “Nasty Effect:” online incivility and risk perceptions of emerging technologies. Journal of Computer-Mediated Communication 19(3): 373–387.

Antoci

Delfino

Paglieri

, et al. (2016) Civility vs. incivility in online social interactions: An evolutionary approach. PLoS One 11(11): e0164286.

Bohman

Rehg

(1997) Introduction. In: Bohman

Rehg

(eds) Deliberative Democracy. Essays of Reason and Politics. Cambrigde, MA: The MIT Press, p. 477.

Borah

(2013) Interactions of news frames and incivility in the political blogosphere: Examining perceptual outcomes. Political Communication 30(3): 456–473.

Borah

(2014) Does it matter where you read the news story? interaction of incivility and news frames in the political blogosphere. Communication Research 41(6): 809–827.

Broh

(1980) Horse-race journalism: Reporting the polls in the 1976 presidential election. The Public Opinion Quarterly 44(4): 514–529.

Brooks

Geer

(2007) Beyond negativity: The effects of incivility on the electorate. American Journal of Political Science 51(1): 1–16.

10.

Ceron

(2015) Internet, news, and political trust: The difference between social media and online media outlets. Journal of Computer-Mediated Communication 20(5): 487–503.

11.

Chen

(2017) Online political discourse: Exploring differences in effects of civil and uncivil disagreement in news website comments. Journal of Broadcasting & Electronic Media 61(1): 108–125.

12.

Coe

Kenski

Rains

(2014) Online and uncivil? patterns and determinants of incivility in newspaper website comments. Journal of Communication 64(4): 658–679.

13.

Cohen

(1997) Deliberation and democratic legitimacy. In: Bohman

Rehg

(eds) Deliberative Democracy. Essays of Reason and Politics. Cambrigde, MA: The MIT Press.

14.

Dahlberg

(2011) Re-constructing digital democracy: An outline of four ‘positions’. New Media & Society 13(6): 855–872.

15.

Dekavalla

(2016) Issue and game frames in the news: Frame-building factors in television coverage of the 2014 Scottish independence referendum. Journalism 19(11): 1588–1607.

16.

Druckman

Gubitz

Levendusky

, et al. (2019) How Incivility on partisan media (de)polarizes the electorate. The Journal of Politics 81(1): 291–295.

17.

Goodman

Cherubini

(2013) Online Comment Moderation: Emerging Best Practices. Paris: WAN-IFRA.

18.

Habermas

(1989) The Structural Transformation of the Public Sphere: An Inquiry into a Category of Bourgeois Society. Cambridge: Polity Press, p. 301.

19.

Hallin

Mancini

(2004) Comparing Media Systems: Three Models of Media and Politics. Cambridge: Cambridge University Press.

20.

Hansen

Kosiara-Pedersen

(2015) How campaigns polarize the electorate: Political polarization as an effect of the minimal effect theory within a multi-party system. Party Politics 23(3): 181–192.

21.

Hayes

Krippendorff

(2007) Answering the call for a standard reliability measure for coding data. Communication Methods and Measures 1(1): 77–89.

22.

Huddy

Mason

Aarøe

(2015) Expressive partisanship: Campaign involvement, political emotion, and partisan identity. American Political Science Review 109(1): 1–17.

23.

Iyengar

Krupenkin

(2018) The strengthening of partisan affect. Political Psychology 39(S1): 201–218.

24.

Iyengar

Westwood

(2015) Fear and loathing across party lines: New evidence on group polarization. American Journal of Political Science 59(3): 690–707.

25.

Iyengar

Lelkes

Levendusky

, et al. (2019) The origins and consequences of affective polarization in the United States. Annual Review of Political Science 22(1): 129–146.

26.

Iyengar

Norpoth

Hahn

(2004) Consumer demand for election news: The horserace sells. Journal of Politics 66(1): 157–175.

27.

Krippendorff

(2004) Content Analysis: An Introduction to Its Methodology. 2a ed. Thousand Oaks: Sage Publications.

28.

Lengauer

Esser

Berganza

(2012) Negativity in political news: A review of concepts, operationalizations and key findings. Journalism 13(2): 179–202.

29.

Mouffe

(1999) Deliberative democracy or agonistic pluralism? Social Research 66(3): 745–758.

30.

Muddiman

Stroud

(2017) News values, cognitive biases, and partisan incivility in comment sections. Journal of Communication 67(4): 586–609.

31.

Mutz

Reeves

(2005) The new videomalaise: Effects of televised incivility on political trust. American Political Science Review 99(1): 1–15.

32.

Newman

Fletcher

Kalogeropoulos

, et al. (2017) Reuters Institute Digital News Report 2017. Reuters Institute for the Study of Journalism. Available at: https://reutersinstitute.politics.ox.ac.uk/sites/default/files/DigitalNewsReport2017web_0.pdf.

33.

Papacharissi

(2002) The virtual sphere: The internet as a public sphere. New Media & Society 4(1): 9–27.

34.

Papacharissi

(2004) Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society 6(2): 259–283.

35.

Prochazka

Weber

Schweiger

(2016) Effects of civility and reasoning in user comments on perceived journalistic quality. Journalism Studies 19(1): 1–17.

36.

Quintanilha

Silva

Lapa

(2019) Fake news and its impact on trust in the news. Using the Portuguese case to establish lines of differentiation. Communication & Society 32(3): 17–33.

37.

Rains

Kenski

Coe

, et al. (2017) Incivility and political identity on the internet: Intergroup factors as predictors of incivility in discussions of news online. Journal of Computer-Mediated Communication 22(4): 163–178.

38.

Rhee

(1997) Strategy and issue frames in election campaign coverage: A social cognitive account of framing effects. Journal of Communication 47(3): 26–48.

39.

Rinke

Wessler

Löb

, et al. (2013) Deliberative qualities of generic news frames: Assessing the democratic value of strategic game and contestation framing in election campaign coverage. Political Communication 30(3): 474–494.

40.

Rowe

(2015) Deliberation 2.0: Comparing the deliberative quality of online news user comments across platforms. Journal of Broadcasting & Electronic Media 59(4): 539–555.

41.

Ruiz

Domingo

Micó

, et al. (2011) Public sphere 2.0? the democratic qualities of citizen debates in online newspapers. The International Journal of Press/Politics 16(4): 463–487.

42.

Salgado

(2011) Os Candidatos Presidenciais: Construção de Imagens e Discursos Nos Media. Coimbra: Minerva.

43.

Santana

(2014) Virtuous or vitriolic. Journalism Practice 8(1): 18–33.

44.

Santana

(2015) Incivility dominates online comments on immigration. Newspaper Research Journal 36(1): 92–107.

45.

Scacco

Potts

Hearit

, et al. (2017) General Election News Coverage: What Engages Audiences Down the Ballot. Center for Media Engagement. Available at: https://mediaengagement.org/wp-content/uploads/2017/04/General-Election-News-Coverage-What-Engages-Audiences-Down-the-Ballot.pdf.

46.

Schudson

(1997) Why conversation is not the soul of democracy. Critical Studies in Mass Communication 14(4): 297–309.

47.

Scheufele

Iyengar

(2014) The state of framing research: A call for new directions. In: Jamieson

Kenski

(eds) The Oxford Handbook of Political Communication. New York: Oxford University Press.

48.

Shehata

(2013) Game frames, issue frames, and mobilization: Disentangling the effects of frame exposure and motivated news attention on political cynicism and engagement. International Journal of Public Opinion Research 26(2): 157–177.

49.

Sobieraj

Berry

(2011) From incivility to outrage: Political discourse in blogs, talk radio, and cable news. Political Communication 28(1): 19–41.

50.

Soroka

McAdams

(2015) News, politics, and negativity. Political Communication 32(1): 1–22.

51.

Stroud

(2011) Niche News: The Politics of News Choice. New York: Oxford University Press.

52.

Stroud

Muddiman

(2018) Political Coverage on Facebook. Available at: https://mediaengagement.org/wp-content/uploads/2018/06/Political-Coverage-on-Facebook-Report.pdf

53.

Stroud

Scacco

Muddiman

, et al. (2015) Changing deliberative norms on news organizations’ facebook sites. Journal of Computer-Mediated Communication 20(2): 188–203.

54.

Stryker

Conway

Danielson

(2016) What is political incivility? Communication Monographs 83(4): 535–556.

55.

Trussler

Soroka

(2014) Consumer demand for cynical and negative news frames. The International Journal of Press/Politics 19(3): 360–379.

56.

Ziegele

Jost

(2016) Not funny? the effects of factual versus sarcastic journalistic responses to uncivil user comments. Communication Research 47(6): 1–30.

57.

Ziegele

Weber

Quiring

, et al. (2017) The dynamics of online news discussions: Effects of news articles and reader comments on users’ involvement, willingness to participate, and the civility of their contributions. Information, Communication & Society 21(10): 1–17. DOI: 10.1080/1369118X.2017.1324505.

58.

Ziegele

Quiring

Esau

, et al. (2018) Linking news value theory with online deliberation: How news factors and illustration factors in news articles affect the deliberative quality of user discussions in SNS’ comment sections. Communication Research 0: 0093650218797884. DOI: 10.1177/0093650218797884.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB

0.14 MB

0.16 MB

0.14 MB

How to report on elections? The effects of game,issue and negative coverage on reader engagement and incivility

Abstract

Keywords

The role of polarisation, incivility and engagement for democracy

Types of political reporting

Methods

Results

Discussion

Limitations and future research

Supplemental Material

sj-pdf-1-jou-10.1177_1464884920958367 – Supplemental material for How to report on elections? The effects of game, issue and negative coverage on reader engagement and incivility

Supplemental Material

sj-pdf-2-jou-10.1177_1464884920958367 – Supplemental material for How to report on elections? The effects of game, issue and negative coverage on reader engagement and incivility

Footnotes

Funding

ORCID iD

Supplemental material

Notes

Author biographies

References

Supplementary Material