Abstract
Many blame partisan news media for polarization in America. This paper examines the effects of liberal, conservative, and centrist news on affective and attitude polarization. To this end, we rely on two studies that combine two-wave panel surveys (N1 = 303, N2 = 904) with twelve months worth of web browsing data submitted by the same participants comprising roughly thirty-eight million visits. We identify news exposure using an extensive list of news domains and develop a machine learning classifier to identify exposure to political news within these domains. The results offer a robust pattern of null findings. Exposure to partisan and centrist news websites—no matter if it is congenial or crosscutting—does not enhance polarization. These null effects also emerge among strong and weak partisans as well as Democrats and Republicans alike. We argue that these null results accurately portray the reality of limited effects of news in the “real world.” Politics and partisan news account for a small fraction of citizens’ online activities, less than 2 percent in our trace data, and are nearly unnoticeable in the overall information and communication ecology of most individuals.
Keywords
Over the past fifty years, the American public has grown increasingly polarized along party lines (Iyengar et al. 2019). Some observers blame the emergence of partisan media, in part in response to polarizing elites, for the growing divides. Exposure to partisan outlets (Garrett et al. 2014, 2019; Stroud 2010) and their messages (Levendusky 2013b) is shown to reinforce prior attitudes and out-party hostility. Evidence for these effects largely comes from surveys or experiments, which face various challenges in measuring exposure to partisan media and in ascertaining their polarizing effects in the real world. It is crucial to offer over-time evidence on these effects in naturalistic settings, where people can select content from an unrestricted set of available alternatives.
Given the potential implications of partisan media and the methodological gaps in extant work, this project examines whether actual exposure to partisan websites leads to polarization, causally and over time. We make three additional contributions. Heeding the call regarding ideological asymmetries in exposure and its effects (Tucker et al. 2018), we compare polarization among strong and weak partisans and also among Democrats and Republicans. Second, whereas most scholars focus on partisan media as the key culprit, there are reasons to believe that centrist news outlets can exert polarizing effects (Arceneaux and Johnson 2015), especially among strong partisans, a notion we test. Lastly, some work measures partisan exposure on the level of news domains, which also comprise articles on sports or lifestyle (Cardenal et al. 2019), whereas others focus on hard news within these domains (Flaxman et al. 2016; Guess 2021). Yet, reading about celebrities or sports on a partisan site also exposes readers to political content in the surrounding headlines and source cues can trigger partisan identity and influence people's views. We thus test polarizing effects from both, exposure to news domains and solely to articles classified as political using machine learning.
We rely on two studies, which combine two-wave panel survey data with internet browsing data. Study 1 comprises a sample of American Facebook users (N = 303) and study 2 a quota sample of US adults (N = 904). Prior to taking the surveys, respondents in both studies shared their browsing history data stored on their computers, resulting in data for over thirty-seven million visits. We use those behavioral traces to predict over-time changes in people's self-reported polarization from exposure to partisan and centrist domains and also to explicitly political articles within those domains, as determined by a neural binary classifier, built on top of a large transformer-based language model, BERT. In addition, we examine heterogeneous effects by partisanship and its strength and attend to these effects on two facets of polarization: attitude polarization on several salient policies and affective polarization.
Our findings are best characterized as robust null findings. Actual online exposure to liberal and conservative news sites, whether congenial or dissimilar, did not make respondents’ policy attitudes more extreme (i.e., attitude polarization), did not make people more hostile toward out-party supporters (i.e., affective polarization), and exerted no significant polarizing effects among Democrats or Republicans or even strong partisans. Parallel null effects emerged when considering exposure to political articles within partisan and centrist sites. Taken together, these null results run counter to the popular narrative that partisan news is to blame for the ills of contemporary US politics. Although not aligned with past evidence, we argue that our null findings portray the reality of (very limited) effects of partisan news in the real world more accurately, as we outline in the discussion. Many studies rely on forced or selected exposure to isolated (partisan) content. In contrast, politics in general and partisan news, in particular, account for a small fraction of citizens’ online consumption. Because partisan content is nearly unnoticeable in the context of overall information ecology of most individuals, as we show, its polarizing effects are also very limited.
(Partisan) News Media and Polarization
One only needs to turn on the TV, open a newspaper, or go online to realize that the American public is sharply divided. Some citizens, especially strong partisans, hold more extreme policy positions (Abramowitz and Saunders 2008), leading to a growing gap between the left and the right (Gallup Aug. 2017). Whereas the extent of policy-based attitude polarization is subject to some scholarly debate, there is a largely undisputed agreement (but Druckman et al. 2020) that affective polarization is on the rise. American partisans dislike out-party supporters, see them as stupid or dishonest, and do not want to interact with them in various hypothetical situations (Iyengar et al. 2019). Attitude and affective polarization have important consequences. The former may thwart consensual governance and leads to disproportionate representation of extreme voices in the political arena (see Mason 2018), while the latter leads partisans to distrust the government run by the opposing party (Hetherington and Rudolph 2015) and influences citizens’ nonpolitical behaviors, biasing decisions in the labor market, for instance (Iyengar et al. 2019).
Various factors account for these divides (see, e.g., Boxell, 2020; Iyengar et al. 2019), but “[t]he prevailing consensus in political science is that elite behavior, rather than communication, is driving political polarization” (Vaccari 2018: 40). That said, media are the only avenue from which most citizens learn about elite behavior in the first place. As the current media environment facilitates opportunities for citizens to tune in to partisan outlets and access hyperpartisan content, scholars worry that partisan news fuels polarization. This would happen because partisan media, by definition, favor one side (Baum and Groeling 2008), while covering the opposition in a negative light (Jamieson and Cappella 2008; Pew Oct. 2017), focusing on its scandals or wrongdoings (Puglisi and Snyder 2011). Relatedly, partisan news often contains incivility, such as ad hominem attacks or references to the opposition as Nazis or Communists (Berry and Sobieraj 2013; de Leeuw et al. 2020), and features “in your face” debates, which lead the audience to see the opposition as not legitimate (Mutz 2007).
Evidence on these effects—although suggestive and important—is rather limited given the difficulties of measuring exposure to (partisan) news. (Longitudinal) surveys show that the use of partisan media reinforces citizens’ candidate preferences (Stroud 2010) and is associated with negative feelings toward opposing parties (Garrett et al. 2014), partly through negative affective reactions it elicits (Lu and Lee 2019). Capturing exposure using self-reports, however, presents challenges, such as selection, recall, and social desirability biases (Prior 2009). Experiments address some of these issues. For instance, Levendusky (2013b) finds that people become more extreme and more affectively polarized after watching clips from congenial outlets (e.g., Fox for conservatives) and from crosscutting clips (e.g., Fox for liberals). These forced-exposure experiments, however, tell us little about polarizing effects in the real world.
To offer more externally valid evidence, some work augments forced-exposure treatments with a choice condition, in which subjects select from congenial, crosscutting, and nonpolitical content. In their seminal work, Arceneaux et al. (2013) show that forced partisan news polarizes attitudes among some viewers, but this effect decreases when viewers are given the option to tune out (see also Stroud et al. 2019). Additionally accounting for subjects’ pretest media preferences, de Benedictis-Kessner et al. (2019) find that one-shot exposure to a partisan outlet polarizes its regular audience and those who prefer entertainment and that crosscutting partisan news can attenuate polarization. Even such improved experiments cannot possibly capture the multiplicity of content online and approximate actual exposure contexts, where users can tune in to a nearly unlimited number of sources and where many do not use partisan news (Flaxman et al. 2016; Prior 2013).
We take advantage of the developments in behavioral tracking tools, which collect real-world data on online exposure unobtrusively in the naturalistic environment, and which can be combined with survey measures from the same participants. If done over time, this allows scholars to assess change in individual attitudes and behaviors as causally attributed to changes in news media consumption. Earlier work has used such data cross-sectionally to validate self-reports (e.g., Guess 2015), describe the prevalence of partisan news exposure online (Guess 2021), or assess how individual characteristics shape exposure (Cardenal et al. 2019; Möller et al. 2020; Scharkow et al. 2020). Two recent field experiments combine online trace data with survey responses in encouragement designs that test partisan news effects. Guess et al. (2021) incentivized participants to change the homepage on their browsers to Fox News or HuffPost, follow the source's Facebook page, and subscribe to affiliated newsletters for one month. In turn, Casas et al. (forthcoming) incentivized liberals to read news on extreme conservative outlets (Breitbart, The American Spectator, and The Blaze) and conservatives to read extreme left sites (Mother Jones, Democracy Now, and The Nation) for twelve days. In both projects, induced exposure to (congenial or dissimilar) partisan media had limited polarizing effects. To our knowledge, no similar work test over-time effects of actual exposure as observed (not experimentally manipulated) on polarization.
We do precisely that by relying on two primary datasets combining self-reports with logs from participants’ web browsing histories. We expect that exposure to partisan news will increase attitude (H1a) and affective (H1b) polarization in the aggregate. After all, features of partisan media, such as incivility or depictions of conflict, are polarizing in and of themselves (Levendusky 2013a). Also, as motivated reasoning theory posits, most citizens are driven by directional goals when processing political information (Kunda 1990; Taber and Lodge 2012). That is, “most citizens most of the time will be biased reasoners, finding it difficult to evaluate new, attitude-relevant information in an evenhanded way” (i.e., the hot cognition hypothesis; Lodge and Taber 2005: 456).
That said, exposure to partisan media from one's own side should lead to greater attitude (H2a) and affective (H2b) polarization than exposure to media across the ideological isle. Congenial news validates and strengthens one's priors by offering arguments and evidence for one side of an issue or a policy. Also, inasmuch as congenial outlets are seen as one's political in-group (Stroud et al. 2014), they may strengthen partisan identities. As Garrett et al. (2019: 494) note, “[t]o the extent that partisan media are engaging in tribal politics—building up their in-group while denigrating the outgroup—audiences that share an outlet's political orientation—who belong to the in-group—are more likely to be polarized by its messages.”
Furthermore, even though directional goals are said to be the norm among the public, research on motivated reasoning shows that it is the strongly opinionated individuals who have the greatest motivation to protect their beliefs (Lodge and Taber 2005). Those with strong attitudes most readily accept congenial arguments and dismiss or counter-argue crosscutting information, processes that lead to polarization (Leeper and Slothuus 2014; Taber and Lodge 2006). In fact, those with strong priors polarize following exposure to congenial content (Levendusky 2013a) and to information from across the political aisle (Levendusky 2013a; Taber and Lodge 2006). Thus, we expect that among strong partisans, the levels of attitude (H3a) and affective (H3b) polarization will increase following exposure to partisan news, irrespective of whether it comes from crosscutting or congenial sites.
Centrist News Media
Because partisan media may exacerbate polarization, it seems sensible to encourage exposure to centrist outlets, which ideally engage in fair, balanced, and objective reporting, covering all sides and relaying news without biased commentary. Encountering balanced viewpoints and dissimilar perspectives is hoped to enhance tolerance and moderate individual positions (Mutz 2006). We expect, however, that exposure to centrist nonpartisan sources will increase attitude (H4a) and affective (H4b) polarization among strong partisans.
This expectation draws on several theoretical frameworks (see also Arceneaux and Johnson 2015). As noted, strongly opinionated citizens are most likely to engage in motivated reasoning (Taber and Lodge 2006). Also, the work on party cues (Druckman et al. 2013; Mullinix 2016; Nicholson 2012) suggests that exposure to elite viewpoints—in and of itself—can distort citizens’ preferences, enhancing reliance on party attachments in forming policy attitudes. This is especially the case as parties are polarized. News media, partisan or centrist, reflect the polarized stances of party elites, cover partisan disagreement and highlight the existence of intergroup conflicts. Such news may activate viewers’ partisan identity and encourage partisans to perceive politics through the us-versus-them lens, ultimately polarizing the public (Druckman et al. 2013; Levendusky and Malhotra 2016). Because those with strong priors are most susceptible to party cues and have most salient partisan identities, they should polarize most as the result of exposure to centrist news media (Leeper and Slothuus 2014). In fact, Wojcieszak et al. (2018) show that mere news exposure leads to attitude polarization among strongly opinionated citizens, and Leeper (2014) finds that strong attitudes polarize even when there is a balance of perspectives in an information environment, such as the one characteristic of centrist news media reporting that gives equal voice to both sides.
In addition to these directional expectations, we explore whether exposure to strictly political news within news domains exerts different effects than domain-level exposure, which contains hard news about political issues as well as nonpolitical content about sports, celebrities, or weather (RQ1). Lastly, to address the gap identified by Tucker et al. (2018), who call for research not only on whether extremists react to media differently than moderates (which we do) but also whether there are ideological asymmetries between the left and the right in partisan news consumption and polarization (Bail et al. 2018; Eady et al. 2019), we test whether the polarizing effects of exposure to congenial, crosscutting and centrist news domains and political content wherein differ among Democrats and Republicans (RQ2).
Data
This project relies on two primary studies that combine two-wave panel surveys, each with distinct samples of American adults, with web traffic data from the same respondents’ browsing histories, collected with their consent, at two-time points in time (roughly six months’ worth of logs of online behavior per study, as detailed below). These data enable mapping changes in actual exposure to centrist and partisan news on the domain level and to political content within these domains, and examining their over-time effects on polarization, both attitude and affective. We first describe Web Historian, an open-source tool developed for trace data collection. We then discuss the recruitment and sampling procedure for each study, the measures used, and our analytical choices. We note that the dataset and the replication code are made public on GitHub at https://github.com/ercexpo/EXPO-partisan-news.
Ultimately, 636 participants completed wave 1 and uploaded their data, of which 339 completed wave 2 (53 percent retention rate), and 303 successfully submitted at least seven days of trace data and are included in the analyses (48 percent retention rate). The final sample was diverse: participant ages ranged from eighteen to over sixty-five (median thirty-five to thirty-nine), education levels ranged from less than high school to a graduate degree (median four-year college degree), 75.50 percent were women, 80.20 percent were white, 62.38 percent identified as Democrat, 15.18 percent as Independent (leaning to neither party), and 22.44 percent as Republican (see Table B1 in the Supplementary Information file). The sample was geographically diverse and included respondents from forty-eight states and the District of Columbia. Attrition did not affect the overall composition of the sample on eight out of nine demographic and key political variables, the level of education being the sole exception (Table B1 in the Supplementary Information file). Furthermore, we did not detect any significant differences in composition in terms of browsing behavior, such that it is not the case that those who did not return to wave 2 were lighter or heavier consumers of (partisan) news (Table B2 in the Supplementary Information file). Power analysis revealed a sufficient statistical power to detect small effect sizes for both main and interaction effects (see Figure B1 in the Supplementary Information file).
As could be expected, given that individuals who are in online panels and comfortable providing access to their online behavior are not likely to represent an average American citizen, the samples do not reflect the general population (they are younger and better educated, and S1 sample is more female, see Table B1 in the Supplementary Information file for comparison of our samples to the US population). Yet, we do not find evidence that our samples differ in news browsing behavior, in that there is correspondence between our news site rankings and those of Amazon's Alexa rankings (see Table B4 in the Supplementary Information file). We address the issue of sample representativeness in the discussion.
Behavioral Measures
Web Historian collects up to ninety days of one's web browsing history and so we have data that span the ninety-day time period preceding wave 1 and the ninety days in between the two waves in each study, up to six months of continuous data per study or a year in total. In study 1, we report survey and browsing data from 303 participants, which contained over six million visits. The median participant provided browsing data from 174 days, actively browsed on 136 of these days, and visited 875 different domains. In study 2, we report survey and browsing data from 904 respondents, containing over thirty million visits. Here, the median participant provided browsing data from 180 days, actively browsed on 126 days, and visited 826 different domains. In total, we use 36.8 million visits to websites during twelve months. The descriptive information about the exposure measures is presented in Figure 1.
News consumption in perspective. Note. This figure shows some key statistics of the trace data of studies 1 and 2 combined.
Web Historian records data at the visit level, that is, each visit to a page is a record in the data and includes a timestamp, the full URL of the site visited, and the title of the page. Having data at the visit level allows us to calculate how often participants visited news over the 180-day period per study by matching the visited domain (e.g., nytimes.com and foxnews.com) to identifiable news website domains that have the corresponding ideology scores from Robertson et al. (2018), as detailed below, for a total of 976 web domains (S1: 507, S2: 969), which span international, national, and local news.
To classify the ideological leaning of the news domains, we use scores based on the Twitter linking patterns of partisans from Robertson et al. (2018). These scores were cross-validated with self-reported data and other methods of measuring a domain's political leaning (see Robertson et al. 2018 for details and robustness checks) and highly correlate with classifications from other work (r = .98; Eady et al. 2019). Lower scores indicate the outlet has a more liberal audience and higher scores indicate a more conservative audience. Using these scores, we categorized the domains as either liberal, centrist, or conservative, such that liberal news sites were those with an ideological score of –.20 or lower, conservative sites included those with scores of .09 or higher, and news sites with scores between –.19 and .08 were categorized as a centrist. These categorizations were based on natural cut points in the data that made intuitive sense and had face validity. Because our dataset is public (https://doi.org/), these categories can be reassigned. Figure B2 in the Supplementary Information file visualizes the ideology ranking of all sites that were visited at least ten times by the respondents in our samples and had ideology scores. The full table with raw scores is available in the GitHub replication repository as “domain_frequencies.csv.”
Domain-level news exposure. After classifying all visits, we use the visit-level data to first create three behavioral measures of domain-level exposure to partisan news on the left, partisan news on the right, and centrist news websites per study. We use trace data from the ninety days before the first wave to construct wave 1 measures and trace data from the ninety days between the waves to construct wave 2 measures. We assign to each participant a score for the exposure variables based on the mean number of unique visits to liberal, centrist, and conservative sites per active day. If, say, during six months, a participant visited eight unique pages (at the day level) on the Fox News website and seven pages on Breitbart that individual would be assigned a score of 15 for the total number of visits to conservative websites. These totals would then be divided by the total number of days the individual logged onto the computer. The day-level unique URLs measure allows us to account for regular home page visitors accurately. If we only measured unique URLs over the 180-day period, someone who visited the page of the New York Times once would have the same score as someone who visited every day, whereas with this process the daily visitor would have a score of 180.
Descriptive statistics based on these six months of trace data indicate that 6.18 percent and 6.16 percent in studies 1 and 2, respectively, did not visit liberal sites, 15.28 percent and 10.89 percent did not visit any conservative sites, and 3.96 percent and 6.08 percent did not visit any centrist sites. In both studies, the mean number of visits was low and highly skewed, indicating that a few users visited many news sites and the majority visited very few (see Table B3 in the Supplementary Information file for the raw summary statistics of the behavioral data prior to transformation and visit totals). To normalize the distribution, we transformed the three variables using a natural log transformation. To facilitate an easy reading of the results, we rescaled these variables to range between 0 and 100, so that the coefficients depict the predicted percentual change in the outcomes in the function of a 1 percent increase of the predictors.
Second, we use the same information to construct measures tapping whether the news websites visited were congenial versus crosscutting with respect to each participant by first relying on wave 1 self-reported partisanship to categorize participants as Democrats/Left (i.e., from self-reported strong Democrats to leaning Democrat) or Republicans/Right (i.e., from self-reported strong Republicans to leaning Republican). To minimize the exclusion of independents for whom congenial or crosscutting exposure could not be determined, we used their ideological self-placement (e.g., conservatives categorized as Republicans/Right; liberals as Democrats/Left). 3 If they were also an ideological moderate (middle point on the 11-point scale), we relied on their job approval rating of President Trump (i.e., those approving of Trump categorized as Republicans/Right; those disapproving as Democrats/Left). For each participant, we calculate the mean number of visits to news sites of the same ideological leaning and of the opposite leaning per day respondents’ logged onto the computer (see Table B3 in the Supplementary Information file for the raw summary statistics and visit totals). In view of their highly skewed distributions, we also log-transformed and rescaled these measures to range between 0 and 100.
Political news exposure. Because the above measures are on the level of news domains, they comprise exposure to articles on entertainment, sports, or lifestyle, in addition to political news. We thus additionally separate visits to political content from nonpolitical content on news sites. Two trained annotators manually labeled 2,887 news article titles for whether they were political or nonpolitical (Cohen's κ = 0.98). We used these annotated data to train a neural binary classifier, built on top of a large transformer-based language model, namely BERT (Devlin et al. 2018; precision = 0.92; recall = 0.94; accuracy = 0.91; see Appendix A2 in the Supplementary Information file for details on the coding procedure and the classifier). After identifying titles that were about political issues in news sites in our trace data, we construct measures of exposure to political content in (a) liberal, (b) conservative, and (c) centrist news domains, and (d) congenial and (e) crosscutting content as detailed above, but this time focusing on visits to political news (see Table B2 in the Supplementary Information file for the raw means and totals). We also log-transformed these measures, after which we rescaled them to range between 0 and 100.
Outcome Measures
We estimate the effects of these exposure indicators on polarization. Although both studies contain the same outcomes, their measurement differs slightly, and so any detected effects are not due to any idiosyncrasies in measurement. To facilitate the reading of the results across models, we rescale the scores to range between 0 and 100, and so the value of the point estimates indicates the predicted percentual change in the outcome following a 1 percent increase in the predictor.
Table A3.1 in the Supplementary Information file presents question-wording and descriptive statistics for all scales. We measure attitude polarization by averaging responses to items gauging whether participants agreed more with a liberal (minimum) or a conservative (maximum) solution to several issues or policies that were salient at the time of each study. Study 1 included 7-point validated batteries on immigration, gun control, sexual assault, and Muslims and Islam (Cronbach's α = 0.89). Study 2 asked respondents about the economy, the environment, gun control, and immigration on 13-point scales (α = 0.89). To construct the outcome variables, that is, increases in attitude extremity as an indicator of attitude polarization, we fold all items so that 0 represents a moderate position and the maximum ‘3’ the most extreme position on these policies, after which we average the scores. We reestimated all the models testing attitude polarization on each of the issues separately, finding nearly identical effects (results shown in Figures C1 and C2 in the Supplementary Information file). To measure affective polarization, both studies used a feeling thermometer toward out-partisans (0 ‘cold and unfavorable’ and 100 ‘warm and favorable’). We reverse the scale so that higher values indicate higher polarization. Besides this measure, study 1 contained two other standard indicators of affective polarization, that is, negative trait ratings and social distance, and study 2 included social distance and a novel measure gauging participants’ understanding of out-partisans. For parsimony and to allow for direct comparison between the studies, we present the results for the feeling thermometers in the main text and report the remaining—and nearly identical—results in Figures C3 and C4 in the Supplementary Information file.
Analytical Approach
We primarily rely on fixed effects regression analysis, which is advantageous over other approaches, as it leverages variation within rather than between respondents. In other words, a fixed-effect analysis study over-time changes of individuals, mapped in Figures B3 and B4 in the Supplementary Information file. 4 It does so by estimating a random intercept for each respondent, thereby factoring out the potentially confounding influence of all time-invariant differences between respondents, whether observed or unobserved. Subsequently, it regresses the over-time change in the outcome on the over-time change in the predictor. It is thus a particularly potent way to handle the omitted variable problem, offering the most stringent causal test outside experiments in situations when the data do not permit creating a synthetic control, such as a “no exposure” group (Allison 2009). 5 Because fixed effects models are unsuitable to estimate interactions with a time-invariant variable, in our case partisanship strength, we rely on so-called within-between models that lift the constraint on the variability in the strength of the within-subject effects. Allowing the coefficient of within-subject effects to vary (i.e., random slopes) enables us to assess whether the average value of within-subject effects varies across partisanship strength. As we test the same hypotheses in multiple ways, we adjust the significance levels using a Bonferroni correction for multiple comparisons. The summary statistics for all variables are displayed in Table B5 in the Supplementary Information file, and the variable distributions in Figures B5 and B6 in the Supplementary Information file.
Results
We first put partisan news exposure in perspective. Do our respondents consume news? If so, how much and how much of it is partisan news? We describe the nearly thirty-seven million visits in our trace data from both studies. Figure 1 summarizes this important descriptive evidence. As shown, 1.69 percent of these data comprised visits to news domains. Among these, centrist sites were most popular, comprising 55.93 percent of the news browsing, with liberal and conservative websites accounting for 25.73 percent and 18.34 percent of all news browsing, respectively. This means that less than half of news visits were to partisan news domains. These statistics reveal that, on average, participants only visited less than one partisan news domain for every 100 pages they visited. An even smaller fraction of the trace data comprised visits to political news. In general, 55.12 percent of overall news browsing was to political news articles. This percentage was yet lower for partisan domains: More than half (52.04 percent) of what people consumed on these domains was nonpolitical content, such as recipes or sports. Ultimately, an average participant encountered only one partisan political news article for every 200 sites they visited! In short, less than 2 percent of what people saw online was news. Most news was not partisan, and most partisan news was not political. We return to this key finding in the discussion. 6
To test our first theoretical predictions, we estimate sixteen models in total, one for each possible combination of main-effects hypothesis, measurement of news exposure (domain vs. political news), and study. We describe the high-level results below, making their more detailed description, including the coefficients and confidence intervals and fit statistics available on GitHub, together with replication data and the code (https://github.com/ercexpo/EXPO-partisan-news.). Furthermore, all additional analyses and robustness checks are in online Appendix in the Supplementary Information file as indicated throughout the article.
First, two sets of fixed effects analyses with exposure to liberal, conservative, and centrist news domains (i), and to articles classified as political (ii) as the core predictors tested H1, according to which the consumption of partisan news in the aggregate leads to increases in (a) attitude and (b) affective polarization. Model 1 in Figure 2 visualizes the results (S1 depicted in grey, S2 in blue).
Coefficient plot results.
The results reveal a consistent pattern of null findings. The use of liberal and conservative news sites has no causal effects on attitude or affective polarization. These insignificant coefficients for domain-level partisan exposure are also not consistent in direction. In fact, all exposure effects, whether they are linked to liberal, centrist, conservative, congenial, or crosscutting news exposure, are statistically indistinguishable from zero. Although exposure to explicitly political news in partisan sites could exert stronger polarizing effects than domain-level exposure, we find a parallel null pattern for both attitude and affective polarization from political news within both liberal as well as conservative domains. In short, across both studies, we do not find evidence in favor of H1. Exposure to partisan domains—whether liberal or conservative—and to hard news within these domains has remarkably little impact on attitude and affective polarization.
To examine whether the use of partisan websites from one's own side has stronger polarizing effects than crosscutting or nonpartisan news (H2), we estimate a series of fixed effects models with congenial, crosscutting, and centrist news exposure as the key predictors. Model 2 in Figure 2 visualizes the results. Again, we find no significant effects. These null findings emerge across both studies when examining domain-level exposure and also for explicitly political exposure within congenial, crosscutting, and centrist news domains. A pairwise comparison with the estimates for crosscutting and centrist exposure suggests that people do not respond more strongly to congenial news than other news. In effect, there is no empirical evidence for H2, namely that congenial exposure exacerbates attitude or affective polarization.
H3 predicted that strong partisans polarize following exposure to both like-minded and dissimilar news more than weak partisans. We estimate a series of within-between models, with an interaction between congenial, crosscutting, and centrist exposure and a folded party identification scale, which we use to measure the strength of partisanship (such that 0 indicates a true independent and 3 a strong partisan, Democrat or Republican). The results are visualized in model 3. The insignificant and near-zero main effects of the news exposure variables indicate that congenial and crosscutting news does not polarize weak partisans. The interaction terms are also consistently null and near-zero, suggesting that strong partisans do not become more extreme or more hostile toward out-partisans any more than weak partisans following congenial or crosscutting exposure and also—as parallel models find—following exposure to explicitly political news in congenial and crosscutting domains. We reject H3.
To test our last hypothesis, which predicted that strong partisans should also polarize from centrist news use, the within-between models interacted centrist media exposure and partisanship strength. Model 3 visualizes the results showing no evidence that centrist news increases attitude or affective polarization among weak partisans, let alone among strong partisans. All models report an insignificant and near-zero difference between stronger and weaker partisans for exposure to both centrist domains and to political news wherein. The findings of neither study support H4.
Lastly, we asked whether Democrats and Republicans differ in their responses to partisan news. To this end, we disaggregate all analyses according to individuals’ partisanship. Figures C6 and C7 in the Supplementary Information file report all results, and here we mention the fixed effects of exposure to liberal, centrist, and conservative news domains. Figure 3 shows the predicted change in polarization in the function of individuals’ change in news consumption. The solid line denotes the predictions for Democrats and the dashed line for Republicans. The nearly identical and near-zero predictions for both groups reaffirm our findings: partisan exposure does not bolster attitude or affective polarization, among Democrats and Republicans alike. 7
Exposure effects by partisanship.
Discussion
We leveraged two projects that combine behavioral indicators of online exposure that took place on people's computers over—in total—a twelve-month period with two two-wave panel surveys, one using a diverse sample of Facebook users, the other a cross-section of the US population. Aiming to offer evidence on the effects of partisan news, we examined the contribution of liberal, conservative, and centrist news websites to polarization in contemporary America.
Our models tested within-individual changes in attitude and affective polarization, both as aggregate constructs as well as attitudes toward various policies and individual indicators of affective polarization toward out-party supporters. Extending past work (Flaxman et al. 2016; Guess 2021), we tested exposure to news domains and to political content within these domains, and also shed light on the potentially differential effects among different subgroups (i.e., strong and weak partisans, Democrats, and Republicans). Despite the stringency of the analyses and various robustness checks, the results offer a consistent pattern of null effects. We find no evidence that exposure to partisan news, whether conservative or liberal, congenial or crosscutting, to news domains or hard news wherein, fosters polarization, among independents and strong partisans alike.
Among the tested samples, partisan media, whether conservative or liberal, did not exacerbate attitude polarization in the aggregate (despite the fact that their coverage contains features that should be polarizing regardless of whether or not the audience agrees with its slant; Arceneaux and Johnson 2013, 2015), nor did they polarize the partisan audience to which they cater. This last null effect is of note because it counters some experimental evidence suggesting that partisan news from one's own side leads to more extreme positions (Levendusky 2013a), a point we discuss below. Further, although those with established political identities are most likely to process information—whether pro- or counter-attitudinal—in biased ways, we do not find that strong partisans polarized from exposure to partisan news from their own side or from across the aisle or from exposure to centrist news sources (a finding that held when we looked at those with strong political ideology, results not shown). That is, although centrist news is hoped to be depolarizing (inasmuch as centrist outlets engage in balanced and objective reporting free of partisan commentary) and although we expected that their use should polarize strong partisans, we find—again—no effects of online exposure to centrist sites and to hard news within. Lastly, speaking to extant literature on ideological asymmetries in partisan exposure and polarization (Tucker et al. 2018), we find robust null effects among Democrats and Republicans in our two samples.
Again, these null effects are not due to any specific operationalization of the outcomes, insufficient statistical power, or the characteristics of the samples and the exogenous events during data collection (the two studies were more than a year apart). We argue that these null effects are due to the reality of partisan news use online and—in fact—offer a more realistic portrayal of partisan news effects in the “real world” than most past work. Surveys testing polarization from partisan news are subject to various limitations (e.g., recall or social desirability biases, among others; Prior 2009). In turn, experiments show subjects a specific partisan video (e.g., Levendusky 2013a) or like-minded content (e.g., Knobloch-Westerwick and Meng 2011) or allow them to choose from among several slanted articles (Wojcieszak et al. 2020) or among partisan news and entertainment (e.g., Arceneaux et al. 2013). Such designs, which also ask subjects to report their attitudes immediately after exposure, maximize the chances of detecting polarization. In addition, these designs, with their focus on slanted political content, reflect the appealing narratives explaining polarization, namely that the online environment fosters echo chambers where citizens encounter hyperpartisan information that reinforces their priors (Sunstein 2018).
Needless to say, these narratives show only a small fraction of reality. This is not only because many citizens use centrist outlets (Flaxman et al. 2016; Fletcher and Nielsen 2018) and only a minority of partisans consume primarily partisan media (Guess 2021). The “powerful partisan news” narrative, we argue, is inaccurate primarily because politics is a small drop in the overall ocean of what citizens do online. Theoretically, people use media that satisfy their needs and desires (Katz et al. 1973). Because politics is perceived as complex, boring, or overly divisive (Klar et al. 2018), people may avoid it altogether, especially as they have nearly unlimited entertainment and nonpolitical content at their disposal (Feldman et al. 2013; Prior 2007). In our data, spanning a total of a year of individual web browsing, visits to news websites comprised less than 2 percent of the browsing. This is normatively problematic (inasmuch as citizens should stay informed about current events), but—crucially—puts into perspective concerns about the polarizing effects of news altogether.
Partisan news is even a smaller drop in this ocean of content. Across two studies, partisan news browsing in total accounted for less than 1 percent of all the URLs accessed by our participants. The consumption of explicitly political content within these domains, as determined by our machine learning methods, was negligible. Given the reality of an overall dearth of news exposure—and particularly of partisan and hard news exposure—to begin with, it is of no surprise that we find consistently null effects.
We deem these insignificant findings theoretically, substantively, and practically important. Slanted political content, isolated in most experiments, accounts for a small fraction of people's exposures. Yes, some partisan TV outlets are popular, with Fox News attracting an average of 2.57 million viewers and MSNBC 1.80 million, followed by CNN with 1 million (Live + 7 Nielsen ratings, 2019). Yet their viewership pales in comparison to the top 100 most popular entertainment shows. Sunday Night Football (NBC) had 20.1 million viewers, followed by NCIS (CBS) with 15.3 million, and Thursday Night Football (Fox) with 15.0 million. 8 Even the shows in the last five of the top hundred averaged about 3.5 million viewers, more than double the television audience of Fox News (see Prior 2013, for evidence that the average primetime audience for Fox News, CNN, and MSNBC combined was about 1.1 percent of the population).
As any project, ours is not free from limitations. Most importantly, we acknowledge that—as other similar studies—we do not account for the overarching communication and information ecology of our participants, such as their news exposure on multiple computers or mobile devices, or their offline use of partisan news (e.g., listening to conservative talk radio in a car or watching MSNBC over dinner). Our lack of mobile data is particularly problematic as more Americans report getting news through mobile devices than through a desktop or laptop computer (57 percent vs. 30 percent; Pew Research Center Nov. 19, 2019). To shed light on the extent to which this lack biases our estimates, we take advantage of survey items gauging how individuals distribute their time on the web across different devices (e.g., desktop or laptop vs. mobile phones). Figure C3 in the Supplementary Information file replicates our descriptive findings for those participants in both studies who said they used their cellphones more often than their computers (N = 267). This figure reveals that those who use their computers remarkably little are very similar when it comes to consuming news on their computers to other respondents, suggesting that our findings are not due to “displacement effects,” whereby people consume less news on their computers because they are getting it elsewhere.
We also note that we cannot get at partisan pages people follow on social media or partisan content shared by those they follow. Inasmuch as social media users see news in their feed (e.g., headlines or embedded news videos), we are underestimating mere exposure and instead capture a more meaningful engagement with news (i.e., accessing it by clicking on the URL). Although surveys aim to examine this passive exposure using self-reports of incidentally coming across news (e.g., Fletcher and Nielsen 2018; see Thorson 2020 for a critique), our study—and those that rely on online traces in general—speaks to the domains and articles people actually visited, not the ones they potentially saw. Although this may underestimate the volume of (partisan and/or political) news encountered through social media, other evidence suggests this may not be the case. News makes up roughly 4 percent of News Feed on Facebook (Zuckerberg 2018) and public affairs news comprises 1.8 percent of the average Facebook feed of college students, with the median participant liking zero pages from journalists or news organizations (Wells and Thorson 2017). These low estimates, largely consistent with the descriptive evidence we present, suggest that the lack of social media data may not bias news exposure in any dramatic ways in our project.
That said, we note that overall exposure to partisan news, across multiple media, devices, platforms, or carried through interpersonal discussions (Druckman et al. 2018) could exert stronger polarizing effects than those detected for browser-based online use alone. Ascertaining whether this is the case may not be possible, as researchers cannot access all the information about all the (partisan news) outlets that people see offline and online, and likely never will. If scholars had access to Facebook newsfeed combined with mobile and desktop data, we would still be missing Facebook Messenger and WhatsApp, where people also share news (Waterson 2018). Even if we could access all of the digital traces a person produces, offline exposure and interpersonal communication are still crucial, and no trace data can account for these sources. To the extent that an overwhelming majority of Americans (89 percent) get at least some of their news online and the share of Americans who prefer to do so is growing (Pew Research Center Sept. 11, 2019), our data offer important insight into the (limited) polarizing effects of the online news environment.
In addition, one might be concerned about the generalizability of these results. Study 1 relied on a small sample of Facebook users, and even though study 2 comprised a much larger cross-section of the population, no study that requires respondents to install online trackers or share their browsing data (ours included) can make point estimates about the population. Individuals who are in online panels and comfortable providing access to their online behavior may systematically differ from those who do not. As aforementioned, although our samples do not reflect the general US population, we find strong correlations between our news site rankings and those of Amazon's Alexa rankings (Table B4 in the Supplementary Information file). Also, we compared the demographics of our final study 1 sample to those who completed the consent and the demographics survey, but did not complete wave 1 (N = 1,422), those who installed Web Historian but declined to upload browsing data, and those who uploaded the data but did not complete W1. As shown in Table B2 in the Supplementary Information file, W1 only participants and the final sample are demographically similar, with slightly higher education level for the final sample. Although the number of those who declined to upload is quite small, we see a tendency toward a more politically conservative profile among this group.
Lastly, we account for exposure to news domains and to explicitly political news simultaneously and side by side, which—as we see in our data—are far from synonymous. Yet, because we examined news outlets only, we do not consider political exposure that happens outside domains categorized as news. Our focus—and extant worries—pertain to partisan media, yet we acknowledge that detailed attention to the content seen by individuals outside news domains is needed to offer more nuanced evidence on the effects of partisan content.
Despite these limitations, our findings have crucial implications for understanding the effects of online partisan news in today's media environment. Scholars and public observers often blame partisan media for enhancing extremity and hostility toward political opponents. Despite the fact that this explanation is broadly accepted, our studies suggest very limited causal effects of actual exposure to partisan media online. It could be the case that a shock in partisan news consumption's quantity or content (as in experiments) is needed to find any effects on attitudes, cognitions, and behaviors (polarization included). Yet, this consumption was relatively stable among our samples despite several major events during data collection for study 1 (e.g., Trump's family separation policy, Mueller investigation, Kavanaugh's hearings, and the focus on sexual misconduct) and study 2 (e.g., start of Trump's impeachment process, the protests in Hong Kong, or the US women's soccer team win in the 2019 FIFA Women's World Cup and the debate about gender discrimination in soccer). Online partisan exposure did not spike or did it lead to an extremity or out-party animosity.
There are various reasons for rising polarization, such as more ideologically consistent elites, the growing alignment of partisanship with other social identities (Mason 2018), or mere demographic changes in the American electorate (Boxell 2020). This is not to say that partisan media are not a problematic feature in contemporary America. They may promote misperceptions (Weeks et al. 2021), generate anger and outrage (Hasell and Weeks 2016), or decrease trust in mainstream media (Guess et al. 2021). (Partisan) news may still play a paramount role in the development of political attitudes during socialization (Möller and de Vreese 2019) and have cumulative effects on people's perceptions of (political) reality (Gerbner 1998). These effects, which can set the “foundation” for one's propensity to consume and be influenced by (partisan) news, cannot be easily captured empirically after six months. Yet, we do not find that these six months’ worth of online partisan exposure radicalize people's attitudes toward such policies as immigration or gun control or make them more hostile toward the other political side, or that they disproportionately influence partisan extremists or Democrats or Republicans.
We hope that these results will spark more theorizing and research on partisan news, research that triangulates people's responses with over-time data on their actual unconstrained news media use in naturalistic settings and—ideally—across different devices. Only when we account for the abundance of content available online and offline and situate partisan news as part of a larger information and communication ecosystem of (many political disinterested) Americans, will we be able to understand the unique effects (or lack thereof) of partisan news.
Supplemental Material
sj-pdf-1-ijpp-10.1177_19401612211047194 - Supplemental material for No Polarization From Partisan News: Over-Time Evidence From Trace Data
Supplemental material, sj-pdf-1-ijpp-10.1177_19401612211047194 for No Polarization From Partisan News: Over-Time Evidence From Trace Data by Magdalena Wojcieszak and Sjifra de Leeuw, Ericka Menchen-Trevino, Seungsu Lee, Ke M. Huang-Isherwood, Brian Weeks in The International Journal of Press/Politics
Footnotes
Acknowledgments
The authors gratefully acknowledge the support of the European Research Council, “Europeans exposed to dissimilar views in the media: investigating backfire effects,” Proposal EXPO-756301 (ERC Starting Grant, Magdalena Wojcieszak—PI). The authors are also grateful to Sam Davidson a PhD candidate in computational linguistic for the development of the political classifier and also the Editor and the three anonymous reviewers for their feedback. Any opinions, findings, and conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the European Research Council.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This work was supported by the European Research Council (ERC) Starting Grant, “Europeans exposed to dissimilar views in the media: investigating backfire effects,” Proposal EXPO- 756301 (PI Magdalena Wojcieszak). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the European Research Council.
ORCID iDs
Supplemental Material
Supplemental material for this article is available online.
Notes
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
