Testing Causal Inference Between Social Media News Reliance and (Dis)trust of EU Institutions with an Instrumental Variable Approach: Lessons From a Null-Hypothesis Case

Abstract

Given the well-documented negativity bias and attitudinal entrenchment associated with sharing and debating news in social media, a reasonable and already substantially investigated assumption is that those getting news about the European Union (EU) mostly from social media would be more sceptical of its institutions than others. Empirical research on this topic has thus far largely deployed experimental and observational methods to investigate this assumption. We contribute to the existing literature with an instrumental variable approach well-suited to establishing causal relationships in non-experimental data. However, we find no blanket causal relationship between relying on social media for news about the EU polity and becoming less trustful of its institutions. EU policies aiming to tackle negative effects of social media news consumption, therefore, need to be tailored to different demographic groups.

Keywords

social media public opinion trust European Union media effects instrumental variable approach

Introduction

Recent government and policy debates have drawn a clear link between social media, post-truth politics and disinformation, underlining how digital communications can damage trust in democratic institutions, particularly during periods of crises (Bayer et al., 2019; European Commission, 2018; Ireton and Posetti, 2018; Lewandowsky et al., 2020). This is particularly relevant to the European Union (EU), which is still grappling with the aftermath of the 2010 sovereign debt crisis, Brexit and the COVID-19 pandemic, as well as multiple refugee waves, democratic backsliding in certain member states and, most recently, the Russian attack on Ukraine. Similarly, academic research has highlighted the many challenges that arise for democracy from the near monopolistic ownership of online platforms and the communication structure that they promote – which favours virulent spread of information and amplification of biases (Caplanova et al., 2021; Engesser et al., 2017; Hameleers, 2020; Jennings et al., 2021). In this context, the EU has taken on a leading role – not without shortcomings (Bechmann, 2020; European Court of Auditors, 2021) – in combating misinformation and preserving democratic values in the digital era at both the European and global level. However, where policy documents identify a causal relationship between the use of social media as news sources and decrease in trust in democratic institutions, academic researchers have typically remained more sceptical of this relationship’s causal direction (Bail et al., 2018; Bode, 2016: 37; Gil de Zúñiga et al., 2017; Hameleers, 2020; Iosifidis and Nicoli, 2021; Kiratli, pre-print, 2023; Strömbäck et al., 2020).

We set out to make a methodological contribution to this literature and the approaches traditionally used to investigate social media effects on support for the EU by implementing an instrumental variable (IV) methodology. IV allows us to answer the research question ‘Does reliance on social media as a news source affect citizens’ trust towards European Union institutions?’ We, thus, take a different tack from previous contributions that seek to cover some of the same ground (Kiratli, pre-print, 2023), in two ways. First, we focus specifically on how the active choice to rely on social media for one’s news about the EU alters trust in the EU as a polity. Second, our methodology allows us to establish causal, rather than correlational, relationships between social media use and trust in the EU.

Our motivation for using this methodology is two-fold. On the one hand, the existing literature provides sound reason to expect that we can corroborate the negative effects of news consumption via social media observed on individuals’ attitudes towards the EU at aggregate level (Lorenz-Spreen et al., 2023). By exploiting variations in causally important variables only attributable to an instrument uncorrelated to the outcome, an IV approach lets us come closer to measuring the causal effect of social media use on trust in EU institutions – a key measure of the public legitimacy of the EU (Hobolt and Tilley, 2014) – than what is possible using traditional regression modelling. On the other hand, we wanted to use a method that minimises common biases, such as endogeneity and omitted variable bias, to investigate the aggregate effects of social media use on trust in the EU. Previous studies deploying IVs with the aim of measuring causal and correlational evidence on the link between digital media and a decline in trust in democratic or state institutions, or a decline in democracy more broadly, have either not distinguished between Internet use/Internet access in general and the use of social media platforms (such as Twitter, Facebook, Instagram, TikTok) as news sources in particular; or have not focused on democratic institutions but have measured trust towards government institutions or societal groups in non-democratic states or flawed democracies (Lorenz-Spreen et al., 2023).

Working with repurposed Eurobarometer data from 2015 to 2021 (European Commission, 2015, 2019a, 2019b, 2020, 2022), we deploy the frequency of use of traditional TV sets as an IV for social media news reliance. Because such use is correlated with social media news consumption but unlikely to be directly correlated with trust in EU institutions, we can investigate the correlation between social media news reliance and trust in the EU using only the variation in social media news reliance driven by TV use rather than relevant control variables. Taking only this variation into account, which is highly likely to be unrelated to trust in EU institutions, lets us come closer to a causal effect of social media news reliance (Morgan and Winship, 2014: 291; Sovey and Green, 2011). Our analysis has produced null-hypothesis outcomes, which, albeit unexpected, raises important methodological and empirical questions regarding the ways in which we investigate social media effects and the true role it plays in shaping social trust.

Media Effects and EU Contestation

Media effects research shows that the mode and structure of communication of digital legacy media (newspapers, television) can influence audiences both through shaping the communicative process and by favouring certain narratives about the EU, either positive or negative, particularly during period of elections (e.g. De Vreese et al., 2016; Ernst et al., 2019; Mosca and Quaranta, 2017; Van Spanje and De Vreese, 2014). However, empirical evidence thus far shows no unconditional ‘dose–response’ relationship between media discourses and changing political attitudes (De Vreese and Boomgaarden, 2006; Van Klingeren et al., 2017). For instance, Van Spanje and De Vreese (2014: 336–337) have shown that the probability of voting for Eurosceptic parties depends on media exposure to Eurosceptic parties and beliefs. Furthermore, they have shown that exposure to frames emphasising the benefits of the EU decreases the probability of voting for Eurosceptic parties. At the same time, exposure to media content focusing on the conflict between two positions or showcasing opposing sides to an argument also appears to reinforce existing beliefs. Consistent empirical observations of this process have led to the thesis positing social media as an arena of further polarisation and entrenchment (Dubois and Blank, 2018; Karlsen et al., 2017). The entrenchment effect is compounded by the diminishing-knowledge effect that Gil de Zúñiga et al. (2017) have found, whereby those who rely on social media as a staple of their news diets end up less knowledgeable about politics and thus potentially more susceptible to polarisation.

The ‘polarisation arena’ thesis is essential to our conceptualisation of social media use as an enhancing mechanism of distrust towards EU institutions or of the EU polity, particularly in the context of the last decade’s multiple crises. These have triggered increasingly negative media coverage of the EU (Joris et al., 2018; Marquart et al., 2019). Increasingly polarised views about the EU may not necessarily stem from individuals selectively choosing content that confirms their own beliefs. Instead, the brief, emotive and symbol-based communicative style that social media facilitates may allow users to process this content through familiar cognitive frames and refine arguments for their previously held beliefs (Karlsen et al., 2017). Thus, our theorised causal mechanism does not rely on citizens using social media as their primary source of news about European politics getting access to frames that are distinctly different from those seen by users of other media, even though there is evidence to suggest that social media users are more likely to share Eurosceptic content (De Wilde et al., 2014; Galpin and Trenz, 2017; Quinlan et al., 2015). Indeed, Europeans getting news from social media are more likely to encounter diverse views than other news consumers (Mitchell et al., 2018), even though micro-targeting and algorithms that tailor content to each user have been shown to polarise debate at the societal level (Bayer et al., 2019: 58, 134). Moreover, it is likely that many of the same news items will be seen by users of both social and non-social media: Many journalists are active – and often influential – on social media in their professional capacity, while news media outlets maintain official profiles and rely on social media channels to disseminate their content (Lecheler and Kruikemeier, 2016; Neuberger et al., 2019; Paulussen and Harder, 2014; Von Nordheim et al., 2018).

However, the strong presence of professional journalists on some social media platforms does not translate to misinformation correction mechanisms as rigorous as those found in professional news media outlets. The absence of such correction on social media, found by previous literature to mitigate misinformation effects (Maertens et al., 2020), could be particularly important in the case of the EU, whose citizens, regardless of their preferred news source, are likely to have less knowledge of the EU than their own national polities (Wilson and Hobolt, 2015). In combination with the more aggressive promotion style and negative framing used by right-wing Eurosceptic actors on social media (Lappas et al., 2019; Nai et al., 2022) and the tendency of social media users to share more negative news and narratives (Hansen et al., 2011; Heidenreich et al., 2022), this absence of correction mechanisms increases the probability that those who are primarily getting their news about the EU from social media will be exposed to both more negative and less accurate coverage of the EU than other news consumers. Crucially, those choosing to consume news through social media are more likely to already trust neither institutionalised media nor national and European institutions (Ceron, 2015; Hameleers et al., 2022). Kiratli (pre-print, 2023) also finds results suggesting a strong negative correlation between using social media for information about the EU and trust in it. This body of literature on the profile of news-via-social-media consumers, type of content one is exposed to on social media and effects of said content on political attitudes leads us to our first hypothesis:

Hypothesis 1 (H1): Trust in EU institutions will be lower among those who get their news about European politics primarily from social media.

To get a more nuanced understanding of the interrelation between social media news consumption and trust (or lack thereof) in EU institutions, we factor in the role that ideology plays as trust moderator, particularly for social media users. News consumers are likely to seek out sources that conform to their own views of the EU, take cues from political leaders they trust and be more likely to remember negative news stories (Leruth et al., 2017; Soroka and McAdams, 2015; Steenbergen et al., 2007). They are also likely to reject opposing viewpoints they encounter through this process. Karlsen et al. (2017: 270) define this process as ‘the double set of echo chamber and trench warfare effects’. Confirmation and disconfirmation bias mechanisms may, therefore, be pulling towards the same outcome, namely, polarised opinions. We expect those who consume EU news primarily through social media to also form their opinions through this same mechanism.

Existing research shows that both far-right and far-left voters tend to distrust traditional media sources and be sceptical of the EU (Bartlett, 2014; Van Elsas and Van der Brug, 2015), meaning that polarisation is unlikely to be restricted to the right. There is also evidence suggesting that right-wing populist communication on social media about EU issues frequently targets EU institutions or ‘supranational elites’ (Engesser et al., 2017: 118). We, thus, hypothesise the following:

Hypothesis 2 (H2): Trust in EU institutions will be lower for non-centrist EU citizens that rely primarily on social media for news about European politics than for other similar citizens.

Another important correlate of support in the EU and trust in its institutions is specific knowledge of the EU (Harteveld et al., 2013; Karp et al., 2003). As Wilson and Hobolt (2015: 107–108) show, the average EU citizen is more likely than subject-matter experts to believe that the EU has a more substantial influence over national policies than what is actually the case. Thus, knowledge about the EU could also moderate the extent of social media effects, as social media users with less knowledge of the EU would be less likely to detect misinformation about the EU and may be more strongly influenced by frames that blame the EU for national challenges. EU knowledge level has not been explicitly considered a likely moderator of social media news reliance in previous treatments of this question (Mosca and Quaranta, 2017; Nguyen and Vu, 2019). The same can be said for how negative views of the EU may lead to a differentiated social media effect. We, thus, hypothesise the following:

Hypothesis 3 (H3): Trust in EU institutions will be lower among social media news consumers with low levels of knowledge of the EU.

Hypothesis 4 (H4): Trust in EU institutions will be lower among social media news consumers with a negative image of the EU.

Data and Methods

Our analysis proceeds by using repurposed data from several waves of Standard Eurobarometer surveys in 2014–2018 and in 2021 (European Commission, 2015, 2019a 2019b, 2020, 2022). There is a gap in the coverage between 2018 and 2021 because the relevant question about social media news reliance was not asked in surveys in 2019 and 2020. Nevertheless, we have opted to use EB surveys, because they give us the opportunity to measure support for a broad range of EU institutions across a representative sample of all EU countries across several years. Because of the large samples found in all surveys, the lack of data from 2019 and 2020 do not pose a substantial challenge to the statistical power of our analysis.

To investigate how social media use impacts support for the EU we implement a two-stage least squares IVs’ regression known to provide robust causal inference, whether this leads to a rejection of the hypothesis or not, where its assumptions are met (Sovey and Green, 2011). This IV approach potentially minimises endogeneity between the treatment, social media news consumption, and trust in EU institutions. Minimising such endogeneity is important because those with low levels of trust in EU institutions may in fact select social media as a news source due to a broad distrust in institutions (Hameleers et al., 2022). IVs, thus, also reduce the potential for omitted variable bias. Our instrument is a variable asking people how often they watch TV on a traditional TV set. As appendix item A6 suggests, the use of traditional TV sets is still widespread, both in the number of respondents who use them and the frequency with which they do so.

Our method has two stages: We first regress reliance on social media as a primary source of news about European politics (D) on use of traditional TV sets (Z) and a full set of control variables. In the second stage, the fitted values of the social media news reliance coefficient from the first regression are inserted into a second ordinary least squares regression as our treatment variable, together with the same control variables. Given a valid instrument, the estimated effect of social media news reliance from the second stage mimics the estimates from a randomly assigned treatment. This method relies on two assumptions: First, the values of the instrument must correlate with the values of the treatment, in this case social media news reliance. Thus, a person’s propensity to use a traditional TV set must be correlated with their reliance on social media for news about the EU. Second, there must be no direct path through which the instrument is likely to influence the value of the dependent variable after we control for relevant variables.

We expect any links between use of traditional TV sets and trust in EU institutions to stem from either (a) correlations between this use and age, gender, education, media trust or media use, which also correlates with trust in EU institutions (Harteveld et al., 2013); or (b) the fact that lower levels of use of traditional TV sets may lead users to choose non-traditional media sources, such as social media, instead of digital versions of professional news media. We control for the relevant variables mentioned in our discussion of pathway (a) and believe we have sufficiently severed direct links between TV use and trust in the EU institutions that do not go through pathway (b). This is supported by a robustness test (A5), which shows that the independent relationship between the use of a traditional TV and trust in the EU is zero. Thus, our instrument fulfils both conditions needed for it to be valid: It is robustly correlated with our treatment, social media news reliance, and has no independent relationship with our outcome variable.

While some research suggests a link between television use and political attitudes, both at the national and European level (Aarts and Semetko, 2003; De Vreese, 2007; De Vreese and Boomgaarden, 2006; Ejaz, 2018; Marcinkowski and Starke, 2018; Peter, 2003), the literature is clear that the mechanism through which this manifests itself is through exposure to certain frames. Our instrument, on the other hand, focuses solely on the frequency of use of TV sets, rather than exposure to a specific broadcast or sets of programmes or genres. It is unlikely that the choice to watch a programme on a traditional TV set, as opposed to webcasted TV, itself will shift trust in EU institutions, as we do not know of a theoretical framework positing a causal chain between the use of traditional TV sets and increased Euroscepticism.

We also run a weak instruments F test (Stock et al., 2002) to test the strength of our instrument. Our results suggest that the instrument is strong enough and usable for causal identification in all samples that we analyse, as the F statistics are consistently >10. These statistics are reported in A4. The strength of the instrument may also reduce bias in cases where weak correlations between the instrument and the dependent variable exist (Sovey and Green, 2011: 190).

We repeat this procedure on data from the full sample, and subsets of respondents with low levels of political sophistication regarding the EU, respondents belonging to the left and right and those with a negative image of the EU. We use fixed effects for country-years to account for invariant sources of trust specific to each country and year and cluster the standard errors at the same level to account for clustered correlations of the residuals (Zeileis et al., 2020).

Dependent Variable

Using multiple indicators to capture the many dimensions of trust in EU institutions is well-established in recent literature on this issue (Torcal and Christmann, 2019; Van Elsas et al., 2020). As we are primarily interested in trust in the EU institutions as a group, rather than the differentiated impact of social media use on each of them, we create an equally weighted index of trust in the European Parliament, Commission and Central Bank. We prefer this approach over using single-indicator indicators of trust, as combining multiple indicators into one measure of an underlying dimension lets us measure the overall trust that a citizen has in the EU’s institutional framework. Our measure also allows us to measure the theoretical concept ‘Trust in EU institutions’ more comprehensively than single-indicator operationalisations of the same concept do. As shown by a factor analysis (Brown, 2015), all three variables measure one underlying dimension, which may be termed ‘Trust in EU institutions’. Appendix items A11–A13 show how the correlations between social media news reliance and our preferred operationalisation substantially overlap with those found when using indicators of trust in each institution as dependent variables. This yields a continuous variable scaled between 0 and 1, where 0.33 corresponds to expressing trust in only one of the institutions, 0.66 corresponds to expressing trust in two of them and 1 corresponds to trust in all three institutions. The factor loadings are found in appendix item A2.

Independent Variable of Key Interest and Instrument

Our key independent variable is whether respondents primarily use what the Eurobarometer question refers to as online social networks (which we will otherwise refer to as social media) to gain access to news about European politics. This is a dummy variable, where those who mention social media as their primary source of news about European politics (A7) are coded with a 1, and everyone else with a 0. The question is phrased in the following manner: ‘Where do you get most of your news on European political matters? First?’

This variable has a key benefit compared to variables asking about general social media use: As news consumption is just one of many reasons that people may choose to use social media, a more general question would likely capture a lot of social media use that would be highly unlikely to lead to exposure about news related to the EU and European politics. Using this question, we are, thus, more likely to capture the effect of exposure to what we assume to be predominantly Eurosceptic discourses regarding the EU. It is, however, important to be aware that content recall on the part of news consumers is known to be an imprecise measure (Prior, 2013). There are also two other limitations of this variable: First, those who get their news about the EU primarily from other sources may still get a lot of exposure to news about the EU from social media. This may understate the effects of social media use. This limitation is difficult to mitigate through our chosen methodology. Second, our strategy risks comparing heavy users of news media who nevertheless get their news primarily from somewhere other than social media, with social media users who do not engage as strongly with news. We partly mitigate this limitation by including measures of both knowledge of the EU, general media use and trust in the media. This helps ensure that we compare news consumers that are as similar as possible.

Our instrument, described above, is a variable that asks ‘Could you tell me to what extent you. . .? Watch TV on a TV set’. The variable features an ordinal scale of responses, from ‘Everyday’ to ‘No access to this medium’. We code ‘Don’t know’ responses as missing. This reduces the N of the data but is necessary as valid responses on the instrument are how we leverage the methodological benefits that the IV approach offers.

Control Variables

We rely on a broad range of control variables common in the literature on EU media effects (Brosius et al., 2020; De Vreese et al., 2016; De Vreese and Boomgaarden, 2006; Mosca and Quaranta, 2017). We first include a measure of political sophistication regarding the EU, which measures a respondent’s objective knowledge of the EU through three questions. Those with three ‘Do not know’ or wrong answers to the survey’s factual questions are coded as having low EU sophistication. Those with three correct answers are coded as having high sophistication, whereas those with two are coded as having an average level of knowledge. This is identical to the Eurobarometer operationalisation of the same measure. We include low political sophistication regarding the EU and measures of general media use and media trust as control variables. The latter is likely to correlate with social media use, as news consumption among such individuals is often driven by a distrust of traditional media sources (Hameleers et al., 2022).

In addition, we include a wide range of control variables that are, to our knowledge, not previously used in the study of the effect of social media on support for the EU. These are more common, instead, in the literature on public support for the EU. The first is trust in national institutions. We create an equally weighted index composed of variables measuring trust in the national government, parliament, and political parties. This has previously been found to predict support for the EU (Harteveld et al., 2013; Hooghe and Marks, 2005). The index features an equally weighted average of three variables, trust in national government, trust in parties and trust in parliament, found by a factor analysis (Brown, 2015) to measure the same latent variable. The factor loadings are found in appendix item A2. We also include a control variable that asks the respondents about their image of the EU.

We include similar measures measuring each respondent’s evaluations of the national and personal financial and job market security. Previous literature has consistently shown that positive evaluations of the economy are associated with higher levels of support for the EU (Harteveld et al., 2013: 554; Hooghe and Marks, 2005). We assume that support for the EU flows from positive evaluations of the national economic situation, confounding relationships between social media and support for integration. Appendix item A1 shows the factor loadings of all variables on both factors.

We also use demographic variables used in the literature on public support for the EU (Gabel, 1998; Harteveld et al., 2013; Hooghe and Marks, 2005). The first is political orientation on a left-right scale, with 0 being very left-wing and 10 very right-wing. We also include gender, age, level of education and manual worker status as a proxy for low income.

To operationalise pre-existing political polarisation, we add variables for left- and right-wing ideology. Those who have reported values between 1 and 2 on a 10-point scale, where 1 signifies far-left ideology, are coded as being left-wing. Those who self-report values between 9 and 10 are coded as right-wing. Such identification has, as previously stated, been found to correlate with Eurosceptic beliefs. A dummy for manual worker status acts as a proxy for lower income, which has previously been found to predict lower support for integration (Gabel, 1998: 346; Hooghe and Marks, 2005: 434). Appendix table A10 shows descriptive statistics for all variables.

Regression Model

The model of trust in EU institutions (Y) for each individual i in country-year t can be expressed as a function of a constant B₀, social media news reliance (D_i), the IV television use (Z_i), and a vector of control variables (knowledge of the EU, trust in national institutions, image of the EU, media use, perception of the economy, left-wing politics, right-wing politics, gender, age, education and a dummy signifying manual worker status) Χ_it. The two-stage IV approach estimates two regressions. The first uses social media news reliance as its dependent variable D and uses OLS to regress this on our instrument and our control variables. This can be formalised as:

D_{i t} = B_{0} + Ζ_{i t} + Χ_{i t} β + ε

(1)

In the second stage, the fitted values of D_it, the variation in social media news reliance that can be attributed only to use of traditional TV sets, are used as a treatment variable in an almost identical OLS regression. This regression can be formalised as:

Y_{i t} = B_{0} + D_{i ι} + Χ_{i t} β + ε

(2)

Results and Discussion

Our results show inconclusive effects of social media news reliance on trust in EU institutions: Whereas the overall effects are weak, we find signs that the interaction between social media news reliance and ideology leads to a differentiated response. We, thus, fail to reject our null hypotheses, as none of the coefficients of social media use is significantly correlated with a decreased trust in the EU’s institutions. An identical analysis of trust in the three institutions covered by our aggregate measure (see appendix items A11–A13) confirms the same broad picture of a non-significant relationship. While the relationship between social media use and trust in the European Parliament in the full sample is significant, its positive sign directly contradicts our hypothesis. The only significant effects consistent with our hypotheses relate to the European Central Bank. Here, we find negative correlation between social media news consumption and trust towards this institution among the full set of respondents. This result is affected by the significant negative correlation between those who identify with the political left and trust in the European Central Bank. This suggests that while social media news reliance may in very specific contexts lead to less trust in individual EU institutions, this does not necessarily lead to generalised distrust towards the EU polity. The polaris ed nature of social media may, thus, not inevitably lead to a decline in trust towards supranational institutions. Indeed, we find evidence of an opposite effect.

There are several correlations that go in the expected direction, such as the negative correlation between social media use and trust in EU institutions among left-wing citizens. The overall non-significant results are surprising, given previous research findings discussed earlier in this article. This shows the need for further research to further establish whether media source effects exist in the case of social media use as news source, as distinct from exposure to particular frames regarding the EU that one may encounter on any format of news media sources (digital, print, broadcast; professional or citizens’ journalism). This nuance is crucial to verify, as it has direct implication for the EU’s regulatory actions that aim to safeguard media freedom, quality journalism and democracy in the EU.

Another implication of our (non)findings is that, as A9 clearly shows, missing data presents an analytical challenge: Using multiple imputation (Rubin, 1996) to impute realistic missing values, we find that the effect of social media use in the full sample becomes both strongly negative as well as significant at conventional levels. However, the non-significance of the regressions performed on the imputed subsets, for instance of right- and left-wing partisans, suggests that our primary results may be robust.

Table 1.

Trust in EU Institutions, Second Stage of Instrumental Variable 2SLS Regression, SEs Clustered at Country-Years.

	Full sample	EU sophistication: low	Right-wing	Left-wing	Negative EU image
Social media news consumer (2. stage)	0.015 (0.114)	0.022 (0.326)	0.595 (0.466)	−0.069 (0.429)	−0.002 (0.209)
EU sophistication: Low	−0.065*** (0.012)		−0.065 (0.042)	−0.070** (0.025)	−0.010 (0.023)
Trust in nationalInstitutions	0.252*** (0.012)	0.312*** (0.043)	0.214*** (0.024)	0.298*** (0.018)	0.130*** (0.033)
Media use	0.017*** (0.004)	0.033* (0.013)	0.023* (0.011)	0.025*** (0.007)	−0.003 (0.008)
Image of the EU	0.185*** (0.004)	0.134*** (0.014)	0.190*** (0.009)	0.189*** (0.006)
Perception of nationalEconomy	0.011*** (0.003)	0.022⁺ (0.012)	0.009 (0.008)	0.018* (0.008)	0.010 (0.008)
Perception of personalFinances	0.019*** (0.002)	−0.001 (0.011)	0.007 (0.006)	0.019*** (0.005)	0.003 (0.004)
Left-wing	−0.025** (0.008)	0.010 (0.029)			−0.010 (0.016)
Right-wing	−0.032** (0.010)	−0.017 (0.034)			−0.007 (0.015)
Gender	0.019*** (0.004)	0.035⁺ (0.021)	0.009 (0.016)	0.021* (0.010)	0.004 (0.009)
Age	0.000 (0.000)	−0.001 (0.001)	0.002* (0.001)	−0.001 (0.001)	0.000 (0.001)
Education	0.002* (0.001)	−0.003 (0.004)	0.001 (0.003)	0.001 (0.002)	0.001 (0.002)
Manual worker	−0.032*** (0.005)	−0.034 (0.021)	−0.041** (0.015)	−0.012 (0.011)	−0.008 (0.010)
Media trust	0.082*** (0.004)	0.061*** (0.015)	0.098*** (0.009)		0.028** (0.009)
Number of observations	58,791	2098	4514	8350	2454
R² adj.	0.416	0.386	0.399	0.426	0.100
Standard errors	Clustered by: country-year	Clustered by: country-year	Clustered by: country-year	Clustered by: country-year	Clustered by: country-year
Fixed effects (FE): country-years	X	X	X	X	X

Source: The authors. Data sources: European Commission, 2015, 2019a, 2019b, 2020, 2022, The authors’ analysis of Eurobarometer data

Adj R2: Adjusted R2

p < 0.1; *p < 0.05, **p < 0.01, ***p < 0.001.

The robustness of our results is further supported by figure A10: The figure shows that there are small differences in the patterns of missing data between those who use social media and those relying on other news media. However, because the same figure shows that there are differences between those who have not responded to questions about their preferred news source and everyone else, future research should replicate our research in new contexts to further probe whether the non-significance of our results is driven by systematic missingness or a genuine attitudinal overlap between these two groups of news consumers.

Concluding Remarks

While our study does not explicitly challenge experimental results showing a clear connection between narratives found in the news and attitudes towards European integration, it does show the necessity of complementary approaches. While experiments seek to precisely mimic the contexts that social media news consumers experience, the data and quasi-experimental methods used in this study serve as an essential test of these experiments’ external validity. It also offers an important test of whether the effects of exposure to negatively framed media messages are driven by the communicative context surrounding the message or the message itself.

Our analysis opens two avenues for further research: One important question for future research is how prolonged exposure to social media content with a uni-directional bias shapes views of the EU. Another question is whether the impact of social media content on trust is platform-specific. Platform-specific characteristics have for instance been found to impact the extent to which they successfully increase beliefs in COVID-19 conspiracies (Theocharis et al., 2021). The IV method employed in this article is particularly relevant for investigating both of these questions, either as a stand-alone tool of enquiry or in combination with experimental and observational methods.

Supplemental Material

sj-docx-1-psw-10.1177_14789299231183574 – Supplemental material for Testing Causal Inference Between Social Media News Reliance and (Dis)trust of EU Institutions With an Instrumental Variable Approach: Lessons From a Null-Hypothesis Case

Supplemental material, sj-docx-1-psw-10.1177_14789299231183574 for Testing Causal Inference Between Social Media News Reliance and (Dis)trust of EU Institutions With an Instrumental Variable Approach: Lessons From a Null-Hypothesis Case by Martin Moland and Asimina Michailidou in Political Studies Review

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We gratefully acknowledge the funding support received by the following projects and funding bodies, which allowed for the cross-project research presented in this article: Democracy and Expert Rule: The Quest for Reflexive Legitimacy (REFLEX) 2016–2021, funded by the Research Council of Norway; Coordinator: ARENA Centre for European Studies; EU Differentiation, Dominance and Democracy (EU3D) 2019–2023, funded by the European Commission, Horizon 2020 Grant Agreement number 822419, Coordinator: ARENA Centre for European Studies; The Jean Monnet Network on Post-truth politics, nationalism and the (de-)legitimation of European integration (PTP) 2019–2023, funded by the Erasmus + Programme of the European Union; Coordinator: the University of Iceland.

ORCID iDs

Martin Moland

Asimina Michailidou

Supplemental Material

Additional supplementary information may be found with the online version of this article.

Testing causal inference between social media news reliance and (dis)trust of EU institutions with an instrumental variable approach: Lessons from a null-hypothesis case

Appendix

A1. Factor loadings for indices of sociotropic and egotropic economic evaluations. Source: the authors.

A2. Factor loadings for trust indices. Source: the authors.

A3. Robustness test excluding instrumental variables. Source: the authors.

A4. Strength and endogeneity tests of instrumental variable ‘TV use’ for all models reported in Table 1. Source: the authors.

A5. Regression using ‘TV use’ as treatment variable. No instrument. Country-year FEs. SEs clustered at the same level. Source: the authors.

A6. Distribution of the use of traditional TV sets among Eurobarometer respondents. N = 165,973. Source: Eurobarometer 2013–2017. Source: the authors.

A7. Distribution of respondents relying on social or other media (% of respondents). Source: Eurobarometer 2013–2017. Source: the authors.

A8. Multiply imputed instrumental variable regression in all samples from A2. Source: the Authors.

A9. Multiple imputed instrumental variable regression in Schengen frontier and destination countries, Source: the authors.

A10. Descriptive data with missingness information, Source: the authors.

A11. IV regression of social media news reliance on trust in European Parliament. 95% SEs clustered at the country-year level.

A12. IV regression of social media news reliance on trust in European Commission. 95% SEs clustered at the country-year level.

A13. IV regression of social media news reliance on trust in European Central Bank. 95% SEs clustered at the country-year level.

Author Biographies

Martin Moland is a doctoral researcher at ARENA, Centre for European Studies studying the determinants of popular Euroscepticism. His work has been previously featured in West European Politics and Journal of Common Market Studies.

Asimina Michailidou is a senior researcher at ARENA Centre for European Studies, University of Oslo, studying the effects of digitalisation on the European public sphere and politics. She is the editor of Europe in the era of Post-truth politics (with M. Conrad, G. Hálfdanarson, C. Galpin and N. Pyrhönen) and her work has been previously featured in, among others, the Journal of European Public Policy, the International Journal of Press/Politics and Media, Culture & Society.

References

Aarts

Semetko

(2003) The Divided Electorate: Media Use and Political Involvement. Journal of Politics 65 (3): 759–784.

Bail

Argyle

Brown

, et al. (2018) Exposure to Opposing Views on Social Media Can Increase Political Polarization. Proceedings of the National Academy of Sciences 115 (37): 9216–9221.

Bartlett

(2014) Populism, Social Media and Democratic Strain. In: Sandelind

(ed.) European Populism and Winning the Immigration Debate. Stockholm: Fores, pp.99–116.

Bayer

Bitiukova

Bard

, et al. (2019) Disinformation and Propaganda – Impact on the Functioning of the Rule of Law in the EU and Its Member States. Brussels: European Parliament, LIBE Committee, Policy Department for Citizen’s Rights and Constitutional Affairs. Available at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3409279 (accessed 26 April 2023).

Bechmann

(2020) Tackling Disinformation and Infodemics Demands Media Policy Changes. Digital Journalism 8 (6): 855–863.

Bode

(2016) Political News in the News Feed: Learning Politics from Social Media. Mass Communication and Society 19 (1): 24–48.

Brosius

Van Elsas

De Vreese

(2020) Bad News, Declining Trust? Effects of Exposure to Economic News on Trust in the European Union. International Journal of Public Opinion Research 32 (2): 223–242.

Brown

(2015) Confirmatory Factor Analysis for Applied Research, 2nd edn. New York and London: Guilford Press.

Caplanova

Sivak

Szakadatova

(2021) Institutional Trust and Compliance with Measures to Fight COVID-19. International Advances in Economic Research 27 (1): 47–60.

10.

Ceron

(2015) Internet, News and Political Trust: The Difference between Social Media and Online Media Outlets. Journal of Computer-Mediated Communication 20 (5): 487–503.

11.

De Vreese

Azrout

Moeller

(2016) Cross Road Elections: Change in EU Performance Evaluations during the European Parliament Elections 2014. Politics and Governance 4 (1): 69–82.

12.

De Vreese

(2007) A Spiral of Euroscepticism: The Media’s Fault? Acta Politica 42 (2–3): 271–286.

13.

De Vreese

Boomgaarden

(2006) Media Effects on Public Opinion about the Enlargement of the European Union. JCMS: Journal of Common Market Studies 44 (2): 419–436.

14.

De Vreese

Boomgaarden

(2006) Media Message Flows and Interpersonal Communication: The Conditional Nature of Effects on Public Opinion. Communication Research 33 (1): 19–37.

15.

De Wilde

Michailidou

Trenz

H-J

(2014) Converging on Euroscepticism: Online Polity Contestation during European Parliament Elections. European Journal of Political Research 53 (4): 766–783.

16.

Dubois

Blank

(2018) The Echo Chamber Is Overstated: The Moderating Effect of Political Interest and Diverse Media. Information, Communication & Society 21 (5): 729–745.

17.

Ejaz

(2018) Investigating Malaise and Mobilization Effects of Media Use on European Identity before and after the Eurozone Crisis. Global Media Journal 8 (1): 1–23.

18.

Engesser

Ernst

Esser

, et al. (2017) Populism and Social Media: How Politicians Spread a Fragmented Ideology. Information, Communication & Society 20 (8): 1109–1126.

19.

Ernst

Esser

Blassnig

, et al. (2019) Favorable Opportunity Structures for Populist Communication: Comparing Different Types of Politicians and Issues in Social Media, Television and the Press. The International Journal of Press/Politics 24 (2): 165–188.

20.

European Commission (2015) Eurobarometer 82.3 (2014) Eurobarometer 82.3 (2014). Brussels: Directorate-General for Communication. Available at: https://data.europa.eu/data/datasets/s2041_82_3_std82?locale=en (accessed 26 April 2023).

21.

European Commission (2018) Joint Communication to the European Parliament, The European Council, The Council, The European Economic and Social Committee and the Committee of the Regions: Action Plan against Disinformation. Brussels: European Commission.

22.

European Commission (2019a) Eurobarometer 84.3 (2015) Eurobarometer 84.3 (2015): Priorities of the European Union, European Economy, European Citizenship, and Information on European Political Matters. Brussels: Directorate-General for Communication. Available at: https://europa.eu/eurobarometer/surveys/detail/2098 (accessed 26 April 2023).

23.

European Commission (2019b) Eurobarometer 90.3 (2018) Eurobarometer 90.3 (2018). Brussels: Directorate-General for Communication. Available at: https://data.europa.eu/data/datasets/s2215_90_3_std90_eng?locale=en (accessed 26 April 2023).

24.

European Commission (2020) Eurobarometer 86.2 (2016) Eurobarometer 86.2 (2016): Priorities of the European Union, European Economy, European Citizenship, and Information on European Political Matters. Brussels: Directorate-General for Communication. Available at: https://data.europa.eu/data/datasets/s2137_86_2_std86_eng?locale=en (accessed 26 April 2023).

25.

European Commission (2022) Eurobarometer 88.3 (2017) Eurobarometer 88.3 (2017). Brussels: Directorate-General for Communication. Available at: https://data.europa.eu/data/datasets/s2143_88_3_std88_eng?locale=en (accessed 26 April 2023).

26.

European Court of Auditors (2021) Special Report 09/2021: Disinformation Affecting the EU: Tackled but Not Tamed. Luxembourg: European Court of Auditors.

27.

Gabel

(1998) Public Support for European Integration: An Empirical Test of Five Theories. The Journal of Politics 60 (2): 333–354.

28.

Galpin

Trenz

H-J

(2017) The Spiral of Euroscepticism: Media Negativity, Framing and Opposition to the EU. In: Caiani

Guerra

(eds) Euroscepticism, Democracy and the Media. London: Palgrave Macmillan, pp.49–72.

29.

Gil de Zúñiga

Weeks

Ardèvol-Abreu

(2017) Effects of the News-Finds Perception in Communication: Social Media Use Implications for News Seeking and Learning About Politics. Journal of Computer-Mediated Communication 22 (3): 105–123.

30.

Hameleers

(2020) Augmenting Polarization via Social Media? A Comparative Analysis of Trump’s and Wilders’ Online Populist Communication and the Electorate’s Interpretations Surrounding the Elections. Acta Politica 55 (3): 331–350.

31.

Hameleers

Brosius

De Vreese

(2022) Whom to Trust? Media Exposure Patterns of Citizens with Perceptions of Misinformation and Disinformation Related to the News Media. European Journal of Communication 37 (3): 237–268.

32.

Hansen

Arvidsson

Nielsen

FÅ

, et al. (2011) Good Friends, Bad News – Affect and Virality in Twitter. In: Future Information Technology: 6th International Conference Futuretech 2011, Loutraki, 28–30 June, pp.34–43. Heidelberg and Berlin: Springer.

33.

Harteveld

Meer

Van der Vries

CED

(2013) In Europe We Trust? Exploring Three Logics of Trust in the European Union. European Union Politics 14 (4): 542–565.

34.

Heidenreich

Eisele

Watanabe

, et al. (2022) Exploring Engagement With EU News on Facebook: The Influence of Content Characteristics. Politics and Governance 10 (1): 121–132.

35.

Hobolt

Tilley

(2014) Blaming Europe? Responsibility without Accountability in the European Union, 1st edn. Oxford: Oxford University Press.

36.

Hooghe

Marks

(2005) Calculation, Community and Cues: Public Opinion on European Integration. European Union Politics 6 (4): 419–443.

37.

Iosifidis

Nicoli

(2021) Digital Democracy, Social Media and Disinformation. New York: Routledge.

38.

Ireton

Posetti

(2018) Journalism, Fake News & Disinformation: Handbook for Journalism Education and Training. UNESCO Publishing. Available at: https://unesdoc.unesco.org/ark:/48223/pf0000265552

39.

Jennings

Stoker

Bunting

, et al. (2021) Lack of Trust, Conspiracy Beliefs and Social Media Use Predict COVID-19 Vaccine Hesitancy. Vaccines 9 (6): 593.

40.

Joris

Puustinen

d’Haenens

(2018) More News from the Euro Front: How the Press Has Been Framing the Euro Crisis in Five EU Countries. International Communication Gazette 80 (6): 532–550.

41.

Karlsen

Steen-Johnsen

Wollebæk

, et al. (2017) Echo Chamber and Trench Warfare Dynamics in Online Debates. European Journal of Communication 32 (3): 257–273.

42.

Karp

Banducci

Bowler

(2003) To Know It Is to Love It? Satisfaction with Democracy in the European Union. Comparative Political Studies 36 (3): 271–292.

43.

Kiratli

(pre-print, 2023) Social Media Effects on Public Trust in the EU. Forthcoming in Public Opinion Quarterly 87 (3).

44.

Lappas

Triantafillidou

Yannas

(2019) Members of European Parliament (MEPs) on Social Media: Understanding the Underlying Mechanisms of Social Media Adoption and Popularity. The Review of Socionetwork Strategies 13 (1): 55–77.

45.

Lecheler

Kruikemeier

(2016) Re-Evaluating Journalistic Routines in a Digital Age: A Review of Research on the Use of Online Sources. New Media & Society 18 (1): 156–171.

46.

Leruth

Kutiyski

Krouwel

, et al. (2017) Does the Information Source Matter? Newspaper Readership, Political Preferences and Attitudes Towards the EU in the UK, France and the Netherlands. In: Caiani

Guerra

(eds) Euroscepticism, Democracy and the Media. London: Palgrave Macmillan, pp.95–120.

47.

Lewandowsky

Smillie

Garcia

, et al. (2020) Technology and Democracy: Understanding the Influence of Online Technologies on Political Behaviour and Decision-Making. Brussels: European Commission: Joint Research Centre.

48.

Lorenz-Spreen

Oswald

Lewandowsky

, et al. (2023) A Systematic Review of Worldwide Causal and Correlational Evidence on Digital Media and Democracy. Nature Human Behaviour 7: 74–110.

49.

Maertens

Anseel

Van der Linden

(2020) Combatting Climate Change Misinformation: Evidence for Longevity of Inoculation and Consensus Messaging Effects. Journal of Environmental Psychology 70: 2–11.

50.

Marcinkowski

Starke

(2018) Trust in Government: What’s News Media Got to Do with It? Studies in Communication Sciences 18 (1): 87–102.

51.

Marquart

Goldberg

Van Elsas

, et al. (2019) Knowing Is Not Loving: Media Effects on Knowledge about and Attitudes toward the EU. Journal of European Integration 41 (6): 641–666.

52.

Mitchell

Simmons

Matsa

, et al. (2018) News Media in Western Europe: Populist Views Divide Public Opinion. Washington, DC: Pew Research Center. Available at: https://www.pewresearch.org/journalism/2018/05/14/in-western-europe-public-attitudes-toward-news-media-more-divided-by-populist-views-than-left-right-ideology/ (accessed 26 April 2023).

53.

Morgan

Winship

(2014) Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge: Cambridge University Press.

54.

Mosca

Quaranta

(2017) Comparing News Diets, Electoral Choices and EU Attitudes in Germany, Italy and the UK in the 2014 European Parliament Election. In: Caiani

Guerra

(eds) Euroscepticism, Democracy and the Media. London: Palgrave Macmillan, pp.141–168.

55.

Nai

Medeiros

Maier

, et al. (2022) Euroscepticism and the Use of Negative, Uncivil and Emotional Campaigns in the 2019 European Parliament Election: A Winning Combination. European Union Politics 23 (1): 21–42.

56.

Neuberger

Nuernbergk

Langenohl

(2019) Journalism as Multichannel Communication: A Newsroom Survey on the Multiple Uses of Social Media. Journalism Studies 20 (9): 1260–1280.

57.

Nguyen

(2019) Testing Popular News Discourse on the ‘echo Chamber’ Effect: Does Political Polarisation Occur among Those Relying on Social Media as Their Primary Politics News Source? First Monday 24 (5). https://doi.org/10.5210/fm.v24i6.9632

58.

Paulussen

Harder

(2014) Social Media References in Newspapers: Facebook, Twitter and YouTube as Sources in Newspaper Journalism. Journalism Practice 8 (5): 542–551.

59.

Peter

(2003) Why European Tv News Matters: A Cross-Nationally Comparative Analysis of Tv News about the European Union and Its Effects. Amsterdam: Universiteit van Amsterdam, pp.1–220.

60.

Prior

(2013) Media and Political Polarization. Annual Review of Political Science 16 (1): 101–127.

61.

Quinlan

Shepard

Paterson

(2015) Online Discussion and the 2014 Scottish Independence Referendum: Flaming Keyboards or Forums for Deliberation? Electoral Studies 38: 192–205.

62.

Rubin

(1996) Multiple Imputation After 18+ Years. Journal of the American Statistical Association 91 (434): 473–489.

63.

Soroka

McAdams

(2015) News, Politics and Negativity. Political Communication 32 (1): 1–22.

64.

Sovey

Green

(2011) Instrumental Variables Estimation in Political Science: A Readers’ Guide. American Journal of Political Science 55 (1): 188–200.

65.

Steenbergen

Edwards

De Vries

(2007) Who’s Cueing Whom? Mass-Elite Linkages and the Future of European Integration. European Union Politics 8 (1): 13–35.

66.

Stock

Wright

Yogo

(2002) A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments. Journal of Business & Economic Statistics 20 (4): 518–529.

67.

Strömbäck

Tsfati

Boomgaarden

, et al. (2020) News Media Trust and Its Impact on Media Use: Toward a Framework for Future Research. Annals of the International Communication Association 44 (2): 139–156.

68.

Theocharis

Cardenal

Jin

, et al. (2021) Does the Platform Matter? Social Media and COVID-19 Conspiracy Theory Beliefs in 17 Countries. New Media & Society. Epub ahead of print 9 October. DOI: 10.1177/14614448211045666.

69.

Torcal

Christmann

(2019) Congruence, National Context and Trust in European Institutions. Journal of European Public Policy 26 (12): 1779–1798.

70.

Van Elsas

Van der Brug

(2015) The Changing Relationship between Left–Right Ideology and Euroscepticism, 1973–2010. European Union Politics 16 (2): 194–215.

71.

Van Elsas

Brosius

Marquart

, et al. (2020) How Political Malpractice Affects Trust in EU Institutions. West European Politics 43 (4): 944–968.

72.

Van Klingeren

Boomgaarden

De Vreese

(2017) Will Conflict Tear Us Apart? The Effects of Conflict and Valenced Media Messages on Polarizing Attitudes toward EU Immigration and Border Control. Public Opinion Quarterly 81 (2): 543–563.

73.

Van Spanje

De Vreese

(2014) Europhile Media and Eurosceptic Voting: Effects of News Media Coverage on Eurosceptic Voting in the 2009 European Parliamentary Elections. Political Communication 31 (2): 325–354.

74.

Von Nordheim

Boczek

Koppers

(2018) Sourcing the Sources: An Analysis of the Use of Twitter and Facebook as a Journalistic Source Over 10 Years in The New York Times, the Guardian and Süddeutsche Zeitung. Digital Journalism 6 (7): 807–828.

75.

Wilson

Hobolt

(2015) Allocating Responsibility in Multilevel Government Systems: Voter and Expert Attributions in the European Union. The Journal of Politics 77 (1): 102–113.

76.

Zeileis

Köll

Graham

(2020) Various Versatile Variances: An Object-Oriented Implementation of Clustered Covariances in R. Journal of Statistical Software 95 (1): 1–36.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.22 MB