Swaying citizen support for EU membership: Evidence from a survey experiment of German voters

Abstract

The United Kingdom’s 2016 ‘Brexit’ referendum vote to leave the European Union (EU) raised concerns that other countries would follow suit. This article examines how arguments about EU membership related to economic, cultural, political, and security and peace issues could influence how citizens would vote in EU membership referendums. Our two-wave survey experiment on a random sample of the German population and difference-in-differences analysis revealed that only fears of being outvoted in EU decision-making swayed German voters’ attitudes about EU membership, particularly voters with weaker EU support, little EU knowledge and low levels of political engagement. We therefore conclude that concerns about sovereignty loss can be drivers of Euroscepticism even in a country that has vast influence over EU decisions.

Introduction

The United Kingdom’s (UK’s) ‘Brexit’ referendum of 23 June 2016 was the first time an electoral majority voted to leave the European Union (EU). This raised concerns that other member states might put their EU membership to a public vote and ultimately leave the union. Populist and right-wing parties have gained momentum in many member states; these parties have questioned the integration process and in some cases advocate EU membership referendums (see e.g. Alternative for Germany, AfD, 2019: 12). It is therefore important to understand what types of arguments in EU exit discussions can sway voters’ opinions about EU membership and which voters are most susceptible to specific arguments in the remaining member states.

Previous studies have shown how economic (e.g. Eichenberg and Dalton, 1993; Gabel, 1998a, 1998b; Gabel and Whitten, 1997) and cultural (e.g. Abbarno and Zapryanova, 2013; Bruter, 2005; Carey, 2002; Kuhn, 2015; McLaren, 2002) factors influence public support towards the EU. Furthermore, research on how country-specific factors shape citizens’ views about European integration (e.g. De Vries, 2018; Sánchez-Cuenca, 2000) has shown that citizens’ support for the EU is negatively related to their evaluation of the quality of their national government and economic conditions. However, prior studies have largely overlooked the stability of EU attitudes (Hobolt and De Vries, 2016: 426). Existing research shows that generally pro-EU arguments increase (and anti-EU arguments decrease) EU support (Abbarno and Zapryanova, 2013; Gabel and Scheve, 2007; Karstens, 2019; Maier et al., 2012; Medrano, 2003; Schuck and De Vreese, 2006; Vliegenhart et al., 2008). Yet, we know little about the conditions under which given arguments can change EU public opinion, or whether specific arguments are more persuasive than others.

Recently, Goodwin et al. (2020) proposed that the magnitude of framing effects depends on their level of novelty in a given context. They analysed how pro-Remain and pro-Leave arguments derived from the competing Brexit campaigns influenced the stability of public opinion in the UK. As predicted, they found that a combination of positive arguments about EU membership had a greater potential to influence voters than a combination of negative arguments. They explained this finding with the long tradition of Euroscepticism in the UK (Daddow, 2012; Goodwin et al., 2020; Schmitt et al., 2008). According to this argument, pro-Leave arguments had already been ‘priced in’ in people’s minds and therefore were too commonplace to cause a change in opinion. By contrast, pro-EU arguments were more novel and had a greater potential to sway voters’ opinions.

These findings suggest that the political context – and particularly the predominant rhetoric in it – affects which arguments can change EU membership attitudes. If this rationale is correct, it should hold in other contexts, too.

We examine this claim by investigating the effect of key pro- and anti-EU membership arguments on vote intention in a hypothetical EU exit referendum in Germany. This case presents a strong contrast to the UK, as Germany has historically had one of the continent’s lowest levels of Euroscepticism (De Vries, 2018; Schmitt et al., 2008). Building on Goodwin et al. (2020), we expect that negative arguments about the economic, cultural, political, and peace and security aspects of European integration should have a greater potential to sway public opinion on EU membership in Germany since they have been less common in the discussion. This should hold especially for negative arguments about the political or peace and security aspects of the country’s EU membership, which have typically been described in rather positive terms in the German discourse.

To further highlight the channels through which these arguments sway public opinion, we investigate the potential of individual-level moderators to influence the effectiveness of novel campaign messages. In particular, we look at how individual biases and political predispositions interact with the treatment to sway voters’ opinions.

To test our hypotheses, we designed a two-wave survey experiment within the probability-based German Internet Panel (GIP). In both waves, we asked respondents to indicate how they would vote if a referendum on Germany exiting the EU (‘Dexit’) were held next Sunday. Before posing this question in the second wave, we split respondents into nine groups. One group served as a control and received no additional information, while the other eight groups received a single argument for or against EU membership related to a central economic, cultural, political or security and peace issue.

Our difference-in-differences analysis shows that, overall, negative arguments do not have a stronger effect than positive arguments on respondents’ attitudes. Yet, the analysis reveals that negative political arguments have a statistically significant effect on vote intentions. We find that a concern about sovereignty loss – i.e. the possibility that Germany could be outvoted at the EU level, and that other countries’ interests could be imposed on Germany – has a statistically significant negative effect on voters’ support for EU membership and can sway their attitudes towards leaving the union. ¹ The negative political argument decreased the share of respondents in favour of remaining in the EU from 85% to 81%, despite the observed, general trend towards increasing support for remaining in the EU over time. While this is a significant and substantial effect, an overwhelming majority of respondents exposed to the negative political argument continued to support remaining in the EU.

Additionally, as expected, we show that citizens with weak priors – i.e. those with low levels of EU knowledge and low political engagement – are more likely to be persuaded or framed by the negative political argument. Moreover, in line with previous studies on perceptual biases, we find that voters tend to give greater weight to information that confirms their prior beliefs and predispositions (Ditto et al., 1998; McDonald and Hirt, 1997; Pyszczynski and Greenberg, 1987). Thus, voters who already have negative predispositions towards the EU are more susceptible to the negative political framing than those who are positively predisposed towards the union.

Our findings suggest that the EU’s majoritarian decision-making procedures have political costs within member states. These costs could potentially call into question the sustainability and future of European integration. In particular, the goal of an ever-closer union often necessitates compromises and a loss of sovereignty. This process creates winners and losers, and may further increase Euroscepticism.

Theoretical expectations

Media and politicians can choose what information to present – and how to present it. They can highlight different aspects, or reframe the discussion. Subsequently, an individual’s exposure to a given argument can influence his or her attitudes through two mechanisms: (1) it may make new information (an unknown argument) available to them (known as the persuasion effect) or (2) it may make information that was already available (a known argument) more easily accessible in their mind (known as the framing effect) (Lenz, 2009). We build upon prior research on persuasion and framing to investigate whether (and how) various arguments about the EU can change German voters’ attitudes about staying in or leaving the union. We are interested in the overall impact of a given argument on voters’ opinion, whether this influence is due to persuasion or framing.

The literature on voters’ attitudes towards European integration has generally shown that positive (negative) information and arguments increase (decrease) voters’ support for integration (Abbarno and Zapryanova, 2013; Gabel and Scheve, 2007; Maier et al., 2012; Medrano, 2003; Schuck and De Vreese, 2006; Vliegenhart et al., 2008). More recent research highlights that the (political) context affects the magnitude of the framing and persuasion effects (Goodwin et al., 2020). Arguments that have been more dominant in a given context are likely to have been ‘priced in’ to voters’ opinions. Therefore, exposure to these arguments should have a weaker effect than exposure to new arguments. Indeed, Goodwin et al. (2020) show that positive frames about EU integration had a greater potential to affect voting in the Brexit referendum. They proposed that this difference was caused by the fact that negative arguments have dominated public discussions in the UK for decades.

We extend the literature by exploring the proposition that unknown or less familiar arguments should have a stronger potential to sway public opinion, using the case of Germany. Given Germany’s long tradition of EU approval and support for integration (De Vries, 2018; Schmitt et al., 2008), positive arguments about membership should have been more easily accessible to voters than negative ones. We therefore expect to find that:

H1a: Negative arguments about the EU affect EU-exit vote intentions more than positive arguments.

Moreover, German voters are likely to be more familiar with some negative arguments about the EU than with others. While the economic and cultural aspects of the union have arguably been more divisive, historically, political and peace/security aspects of the EU have been described in positive terms in Germany. On the one hand, the political aspect has been perceived favourably because Germans have had little reason to fear a loss of sovereignty or self-determination. As the largest EU member state, Germany is often perceived to be taking a leading role in the union, rather than being unable to assert its interests in EU decisions (De Vries, 2020). On the other hand, the preservation of peace in Europe was a key reason for the creation of the European Community, and for Germany to join as a founding member (Dinan, 2004). Indeed, the half-century of peace on the European continent following European integration is remarkable in light of the world wars. The EU’s Nobel Peace Prize in 2012 reinforced its image as a peace-keeper despite its rather limited powers in foreign and security policies. Thus, we expect negative considerations about a loss of sovereignty or peace/security to be new to German voters – and thus to have a greater potential to sway their attitudes towards the EU than other, more familiar negative considerations.

H1b: Negative arguments about political and peace/security aspects of European integration affect EU-exit vote intentions more than negative arguments about economic and cultural aspects.

Notably, negative arguments related to political and peace/security considerations are not purely hypothetical. They are attracting an increasing amount of attention, and are likely to become more and more important in the near future. These considerations directly affect recent initiatives to strengthen EU foreign policy – e.g. EU army plans, common economic sanctions against Russia after its annexation of Crimea, tightening EU border controls in the aftermath of the refugee influx and terrorist attacks in EU cities – and have heightened disagreements between and within member states over such collective political decisions. This makes our study of such considerations timely. This article will also add to the broader literature on EU public opinion, which has extensively studied how citizens’ trust in (and support for) national and EU institutions influences their attitudes towards the union (e.g. Armingeon and Ceka, 2013; De Vries, 2018; Rohrschneider, 2002; Sánchez-Cuenca, 2000), but so far has paid limited attention to the role of political and security concerns in shaping EU attitudes. ²

Furthermore, we expect that some individuals will be more susceptible to framing or persuasion with novel, less common or forgotten arguments than others, either due to rational mechanisms or because of individual biases (De Vries and Edwards, 2009; Hobolt, 2007; Maier et al., 2012). According to a Bayesian rational perspective, strong priors will limit the effect of belief or attitude updates after receiving new information, while weak priors should increase the probability that new information will impact preferences (e.g. Bullock, 2009). Therefore, individuals who do not already have a clear stance or pronounced opinion should be more receptive to persuasion and framing. We expect that individuals with moderate predispositions towards the EU (i.e. those with neither extremely positive nor extremely negative EU attitudes) are more likely to be affected by our arguments. The same holds for people who are generally less politically engaged and those with low political knowledge, since they are less likely to hold crystallized beliefs or to have strong priors (Chong and Druckman, 2007; Schuck and De Vreese, 2006). Accordingly, we hypothesise that:

H2: Arguments about the EU have a stronger impact on EU-exit vote intentions of individuals with moderate predispositions towards the EU (a), individuals with little political engagement (b) and individuals with low EU knowledge (c).

Prior beliefs may not weaken the framing and persuasion effects of less familiar arguments, but instead serve as a filter when individuals are processing new information (Bartels, 2002; Evans and Andersen, 2006; Goodwin et al., 2020; Tesler, 2015; Tilley et al., 2008). Arguably, people do not simply absorb all available information, but seek to reconcile or interpret it in a way that is consistent with their prior beliefs and predispositions (Nickerson, 1998; Tilley and Hobolt, 2011). In particular, people tend to avoid, disregard or downplay information that conflicts with their existing beliefs (‘disconfirmation bias’) (Edwards and Smith, 1996). At the same time, they easily accept and give greater weight to information that confirms or is consistent with their prior opinions, beliefs and directional motivations (‘confirmation bias’) (Ditto et al., 1998; Kunda, 1990; McDonald and Hirt, 1997; Nickerson, 1998; Pyszczynski and Greenberg, 1987). Consequently, they consider information that is consistent with their prior beliefs to be more convincing than anti-attitudinal information (Taber and Lodge, 2006).

People’s prior beliefs are often associated with their ideological preferences and partisan affiliations (Nyhan and Reifler, 2010). One’s party affinity produces a ‘perceptual lens’. This in-group bias leads voters to discount or disregard new information that conflicts with their party’s stance (Bartels, 2002; Bolsen et al., 2014; Bullock, 2011; Druckman et al., 2013; Jerit and Barabas, 2012; Slothuus and De Vreese, 2010; Tilley and Hobolt, 2011). We therefore expect that:

H3: Negative (positive) arguments about the EU have a stronger impact on EU-exit vote intentions of individuals predisposed against (in favour of) the EU (a) and individuals supporting anti-EU (pro-EU) parties (b).

Survey experiment set-up

To test our hypotheses, we conducted a two-wave survey experiment within the GIP waves of July 2016 (Blom et al., 2017a) and March 2017 (Blom et al., 2017b). The GIP maintains a probability-based sample of the German population aged between 16 and 75 years that was recruited offline using face-to-face interviews (Blom et al., 2015). The GIP provided computers with internet access for participants if needed, thus ensuring that their sample is representative of both the online and offline German population (Blom et al., 2015). Our original sample consisted of 2800 respondents, 80% of whom completed both waves.

In both waves, we asked respondents how they would vote in a hypothetical referendum on Germany’s EU membership. In the second wave, we randomly divided the respondents into one control and eight treatment groups of approximately 300 respondents each. ³ Immediately before the EU membership question in the second wave, respondents in the treatment groups were shown a single argument either in favour of or against EU membership. The control group received the EU membership question with no additional information.

Unlike Goodwin et al. (2020), we do not use arguments drawn from an ongoing political campaign, as the possibility of a referendum on EU membership in Germany is purely hypothetical. Therefore, we focused on key aspects of the European integration project and develop general arguments for and against EU membership based on the history and current state of the union. The EU has its roots in organisations created in the aftermath of World War II with the explicit goal to foster economic cooperation and interdependence in order to maintain peace and security on the continent. European integration has introduced the free movement of goods, capital and services throughout the union. It also ensures the free movement of people to travel, study, work and live in any other EU country. Member states have yielded significant power to the union, but all of them have a say in its decisions.

Some may thus associate EU membership with economic benefits, cultural diversity and exchanges, peace and security, and/or political influence within EU bodies and over EU decisions. Conversely, others may view EU membership as entailing economic costs, cultural threats, security threats and/or national sovereignty loss. We therefore presented our survey respondents with a single economic, cultural, political or peace/security-related argument either for or against EU membership, as summarised in Table 1 (the full text of each argument is available in the Online appendix).

Table 1.

Summary of the eight positive and negative treatments.

Aspect	Framing	Treatment
Economy	Positive:	The German economy benefits from the EU’s common market.
	Negative:	EU exit would allow Germany to support its own, rather than other states’, poor citizens/regions.
Culture	Positive:	EU citizens have the unlimited right to travel, study, work and live anywhere in the EU.
	Negative:	EU exit would allow Germany to preserve its cultural values and norms.
Political	Positive:	EU membership allows Germany to shape the EU’s decisions.
decisions	Negative:	In the EU, Germany can be outvoted; EU exit would allow Germany to make decisions autonomously.
Peace/	Positive:	EU membership fosters peace and security in Europe.
Security	Negative:	EU exit would allow better protection of the German border.

Note: Full treatment texts in German and English can be found in the Online appendix.

Our research design has two important features that could be related to lower effect sizes. First, to establish which arguments can change EU exit attitudes, we exposed our respondents to one, rather than a combination of positive or negative arguments, as in Goodwin et al. (2020). This makes it more difficult to observe significant changes in EU exit attitudes, as the amount of information in each treatment is reduced. Second, we study vote choice in a hypothetical EU exit referendum. This setting makes the choice less salient, which again could reduce the effect of a given treatment. However, we have no reason to expect that the hypothetical nature of the referendum would affect the impact of one treatment differently than any of the other treatments. Even if it did, our difference-in-differences design should correct for this effect.

Our two-wave panel design allows us to compare EU membership support before (wave 1) and after (wave 2) exposure to arguments within subjects. We compare the average change in EU membership support between the two waves in the control group (which did not receive any arguments) with the average change in EU membership support in (each of) the eight treatments groups (which received either a positive or a negative argument). This difference-in-differences design allows us to isolate the effect of a treatment (a positive or a negative argument) from any general trends between the two waves.

Our dependent variable measures the change in support for Germany remaining in the EU from wave 1 (GIP wave 24) to wave 2 (GIP wave 28). Respondents in both waves were asked ‘If a vote on Germany’s EU membership took place next Sunday, how would you vote?’ and were presented with the following answer categories: ‘Remain’, ‘Leave’, ‘Don’t know’, ‘I won’t vote’, ‘I’m not eligible to vote (not of age or no German citizenship)’ and ‘I don’t want to say’. ⁴ We used the former three answer categories to construct our dependent variables, and coded the latter three as missing cases. The dependent variable ΔRemain is the difference between a dummy variable capturing the intention to vote Remain in wave 2 and a dummy variable capturing the intention to vote Remain in wave 1. It equals 1 if participants changed their vote intention from either ‘Leave’ or ‘Don’t know’ to ‘Remain’, 0 if there was no change to or away from ‘Remain’, ⁵ and –1 if the participants changed their vote intention from ‘Remain’ to either ‘Don’t know’ or ‘Leave’ between the two waves. Thus, our dependent variable is identical to the one used by Goodwin et al. (2020) and captures whether a treatment makes respondents more or less likely to vote Remain. ⁶ The mean of the dependent variable captures the percentage point change in the Remain vote between the two survey waves, as it corresponds to the difference in the percentage of voters supporting Remain in wave 2 and the percentage of voters supporting Remain in wave 1 (see the Online appendix for further explanation). This feature of the dependent variable facilitates the interpretation of the substantive regression results in the next section.

Note that the treatments cannot make voters who already wanted Germany to leave/remain in the EU in wave 1 want that even more in wave 2. This makes it more difficult to observe significant effects, particularly towards the dominant choice of voting Remain.

Analysis

This section presents the analysis and results. We use regression analyses and F-tests to assess our hypotheses in turn.

Positive versus negative arguments

We first analyse hypothesis 1a. To establish whether treatment with negative arguments about the EU has a stronger effect on EU exit attitudes than treatment with positive arguments, as expected due to the relative novelty of the former, we run an ordinary least squares (OLS) regression based on the following equation:

$$\Delta \mathit{Remai}{n}_i=\alpha +{\beta }_1\times \mathit{Positiv}{e}_i+{\beta }_2\times \mathit{Negativ}{e}_i+{\mathrm{ϵ}}_i$$

(1)

where i indexes individual respondents. Δ $\mathit{Remai}{n}_i$ is the change in support for Germany remaining in the EU from wave 1 to wave 2 (see above); Positive is a binary variable that equals 1 if the respondent received any positive argument in wave 2 and 0 if he or she was in the control group. Negative equals 1 if the respondent received any negative argument in wave 2 and 0 if he or she was in the control group. The coefficient estimates β thus capture how the average change in support for remaining in the EU between the two waves is affected by exposure to the treatment (i.e. a positive or a negative argument) compared to the control group (i.e. no argument). Negative (positive) values indicate that respondents who received a treatment (positive or negative) decreased (increased) their EU membership support by β units more than those who did not (control group). We use OLS to maintain comparability to Goodwin et al. (2020) and for ease of interpretation. Furthermore, given the setup of the experiment, we expect the substantive differences between OLS estimates and ordered logit regression estimates to be small (Angrist and Pischke, 2008: 94–99). In fact, in the Online appendix, we show that our results remain substantively the same in both statistical models.

These difference-in-differences estimates, alongside their 95% confidence intervals (CIs), are displayed in the top two bars in Figure 1(a) (see the Online appendix for the full regression table). ⁷ The results indicate no statistically significant effect of exposure to either positive or negative arguments. Furthermore, the difference between the two effects, displayed in the bottom bar of Figure 1(a), is not significant. ⁸ Hence, we do not find evidence to support hypothesis 1a, i.e. we do not find that negative arguments about the EU have a stronger potential to sway attitudes about EU exit than positive arguments.

Figure 1.

Coefficients and 95% CIs of the OLS regressions on the change in support for remaining in the EU between waves 1 and 2 in the treated groups versus the control group. (a) All positive versus all negative treatments. (b) Negative political and peace arguments versus negative cultural and economic arguments.Note: The bottom row shows the difference between the two coefficients in a sample of 1000 draws from the regression’s sampling distribution, and the corresponding 95% CI. The corresponding regression tables are presented in the Online appendix.

Stronger effects of less familiar negative arguments?

Hypothesis 1b predicts that exposure to less familiar negative arguments about the EU (related to negative political or peace/security aspects of European integration) will have a stronger effect on EU exit attitudes than more familiar negative arguments (related to negative economic or cultural aspects). To test this hypothesis, we run the following regression:

$$\begin{array}{l}\Delta \mathit{Remai}{n}_i=\alpha +{\beta }_1\times \mathit{Positive}\hspace{0.17em}\_\hspace{0.17em}\mathit{Political}\hspace{0.17em}\_\hspace{0.17em}\mathit{Peac}{e}_i\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\hspace{0.17em}{\beta }_2\times \mathit{Negative}\hspace{0.17em}\_\hspace{0.17em}\mathit{Political}\hspace{0.17em}\_\hspace{0.17em}\mathit{Peac}{e}_i\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\hspace{0.17em}{\beta }_3\times \mathit{Positive}\hspace{0.17em}\_\hspace{0.17em}\mathit{Cultural}\hspace{0.17em}\_\hspace{0.17em}{\mathit{Economic}}_i\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\hspace{0.17em}{\beta }_4\times \mathit{Negative}\hspace{0.17em}\_\hspace{0.17em}\mathit{Cultural}\hspace{0.17em}\_\hspace{0.17em}{\mathit{Economic}}_i+{\mathrm{ϵ}}_i\hfill \end{array}$$

(2)

We incorporate four binary variables to measure the causal effects of receiving (1 if yes, 0 otherwise): (a) a positive political or peace/security argument; (b) a negative political or peace/security argument; (c) a positive cultural or economic argument; (d) a negative cultural or economic argument. The coefficients are interpreted the same way as in the previous subsection. Finding a stronger effect of treatment with a negative political or peace/security argument than treatment with a negative cultural or economic argument would offer support for hypothesis 1b.

The results, displayed in Figure 1(b), indicate exactly that (see also the Online appendix). Of the four effects tested in the regression, only exposure to a negative political or peace/security argument significantly changed preferences for remaining in the EU. Relative to the change in support for Remain in the control group, exposure to a negative political or peace/security argument produced on average a 5.1 percentage point decrease in support for remaining in the EU. Moreover, the F-test (p = 0.021) confirms that exposure to a negative political or peace/security-related argument has a significantly stronger negative effect on the Remain vote than exposure to a negative cultural or economic argument.

To examine whether this result is driven by any particular negative argument, we use a pairwise approach to compare the effects of single negative arguments (see the Online appendix). Figure 2(a) shows that only exposure to the negative political argument significantly decreases support for remaining in the EU. This effect is significantly different from the effects of the other three negative treatments, as our F-tests (summarised in Figure 2(b)) show.

Figure 2.

Comparison of negative treatments’ effect sizes. (a) Coefficients and 95% CIs of the OLS regressions on change in support for remaining in the EU between waves in a treatment versus the control group (see the Online appendix). (b) Summary of statistical tests of differences between coefficients of different negative treatments.Note: White cells show tests with p-values > 0.05, and grey cells indicate p-values < 0.05.

We find partial support for hypothesis 1b: treatment with the negative peace and security argument has no independent effect compared to treatment with the negative political argument. This may be explained by events that unfolded around the time of the survey. Historically, negative peace and security arguments have been less pronounced in Germany. However, the influx of refugees in 2015, mass sexual harassments during 2015/2016 New Year’s Eve celebrations in multiple German cities, and the terrorist attack in December 2016 at the Berlin Christmas market raised public concerns about security issues (Frankfurter Allgemeine Zeitung, 2016a, 2016b; The Economist, 2015, 2016a, 2016b). These events also drew public attention to the security challenges within the EU, which are complicated by the Schengen policy of open borders between member states (The Economist, 2015).

In contrast, those who were exposed to the argument that Germany can be outvoted at the EU level and other states’ interests can be imposed on Germany significantly lowered their support for remaining in the EU compared to the control group. Relative to the change in support for Remain in the control group, exposure to this argument produced on average an 8.1 percentage point decrease in support for remaining in the EU. This effect is also clearly visible in simple descriptive statistics. For example, consider the changes in the share of remainers, leavers and unsure voters across the treatment and control groups between the two waves presented in Figure 3. The effect of our negative political frame is in line with our expectations: the share of remainers decreased from 85.4% to 81.0% after receiving the negative political argument (indicated by the bold line in the left plot in Figure 3). In contrast, the share of remainers in the control group increased from 81.7% to 85.4%. Similarly, the share of leavers and unsure voters who received the negative political treatment increased compared to the control group (see the bold line in the middle and right plots in Figure 3). Overall, the effect of exposure to the negative political treatment runs counter to the general trend of increased share of remainers and a decreased share of leavers between the waves (see e.g. the lines for the control group in Figure 3).

Figure 3.

Changes in the share of remainers, leavers and unsure voters between the two survey waves across treatment and control groups.

We rerun our analyses using two alternative dependent variables – ΔLeave and ΔDon’t know – defined analogous to $\Delta \mathit{Remai}{n}_i$ (see the Online appendix). We find that the negative political argument has a statistically significant positive effect on ΔLeave, which is consistent with our main findings. This argument also has a positive impact on ΔDon’t know, although it fails to reach conventional levels of significance. Our results are also robust to different model specifications, such as using ordered logistic regressions instead of OLS regressions or an alternative difference-in-differences specification using the raw survey responses (Leave = –1, Don’t know = 0, Remain = 1) from both waves to measure the dependent variable and interactions of each treatment with a wave dummy as the main covariate (see the Online appendix).

Conditioning effects of individual characteristics and political predispositions

This section investigates whether the persuasion and framing effects of novel arguments depend on individual characteristics as well as prior political predispositions – i.e. whether respondents hold an extreme stance towards the EU, whether they are politically engaged and whether they are knowledgeable about the EU (hypotheses 2a to 2c) as well as whether they hold an anti-EU attitude or are supporters of a Eurosceptic party (hypotheses 3a and 3b). Descriptive statistics on sub-group sizes are reported in the Online appendix.

To test our conditional hypotheses 2 and 3, we ran regression models for each treatment group and hypothesised respondent characteristic. We ran 40 models in total (8 treatment groups × 5 characteristics) on samples of one treatment group plus the control group. The regressions are specified as follows:

$$\begin{array}{l}\Delta \mathit{Remai}{n}_i=\alpha +{\beta }_1\times \mathit{Treatmen}{t}_i\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\hspace{0.17em}{\beta }_2\times \mathit{Respondent}\hspace{0.17em}\_\hspace{0.17em}\mathit{Characteristi}{c}_i\hfill \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}+\hspace{0.17em}{\beta }_3\times \mathit{Treatmen}{t}_i\times \mathit{Respondent}\hspace{0.17em}\_\hspace{0.17em}\mathit{Characteristi}{c}_i+{\mathrm{ϵ}}_i\hfill \end{array}$$

(3)

where Treatment is an indicator variable for one of the eight treatment groups (which equals 1 if the respondent received that treatment and 0 otherwise) and Respondent_Characteristic (which equals 1 if the respondent has a given characteristic and 0 otherwise) is an indicator variable for one of the five characteristics with an expected conditioning effect based on hypotheses 2a to 2c and 3a and 3b, respectively.

We operationalised respondents’ characteristics as follows. To capture the strength of their predisposition towards the EU (see hypothesis 2a), we asked respondents how they generally felt about the EU on a five-point scale. The middle answer categories ‘mostly negatively’, ‘neither negatively nor positively’ and ‘mostly positively’ capture EU Moderate views, while the categories ‘very negatively’ and ‘very positively’ capture more extreme views (EU Extremist).

We used answers to the classical vote intention question to create an indicator variable for political engagement (hypothesis 2b). Respondents were asked to indicate which party, if any, they would vote for if general elections were held in Germany this Sunday. We consider those who indicated they would vote for a particular party to be politically Engaged, and the rest (who do not intend to vote, refused an answer or do not know) as politically Disengaged. We treated respondents who are not eligible to vote as missing cases.

We created a variable on low EU knowledge (hypothesis 2c) based on a question asking respondents to indicate how well they understand the EU decision-making process on a scale from 1 (very well) to 10 (not at all). We coded respondents who chose a number above 5 as Not EU Knowledgeable, and the rest as EU Knowledgeable. ‘Don’t know’ answers were treated as missing cases.

To capture voters’ predispositions (see hypotheses 3a and 3b), we relied on information from the EU attitudes and voting intention questions. We coded respondents as against the EU (or as having No Pro-EU Attitude) when they generally felt ‘mostly negatively’ or ‘very negatively’ towards the EU. The reference category includes respondents who view the EU ‘mostly positively’ or ‘very positively’, i.e. those with a Pro-EU Attitude. Furthermore, respondents who indicated that they would vote for Die Linke, AfD or the National Democratic Party of Germany (NPD) were coded as supporters of an anti-EU party (Eurosceptic Party Supporters) (following the classifications by Treib (2015) and Hobolt (2015)). They are compared to those who indicated they would vote for a pro-EU party (Grüne, Social Democratic Party (SPD), Christian Democratic Union/ Christian Social Union (CDU/CSU), Free Democratic Party (FDP) or Pirate Party), i.e. Not Eurosceptic Party Supporters.

Figure 4 summarises the results of the 40 regression models (see also the Online appendix). As an example, consider the model in the top left corner of Figure 4. The analyses are based on all respondents from the group who received the positive political argument (+Political) and all respondents from the control group. In the first three lines of the left column of Figure 4, the dots represent (in order) (a) the average change in support for remaining in the EU after treatment with the positive political statement for respondents with extreme EU stances (EU Extremist), (b) the average change in support for remaining in the EU after treatment with the positive political statement for respondents with moderate EU stances (EU Moderate) and (c) the difference in the two effects, i.e. the size of the interaction effect (β₃ in equation (3)). The figure also displays the 95% CIs around each estimate: if this interval includes zero, the respective effect is not significant. A significant effect of any of the characteristics’ coefficients (when black lines in Figure 4 do not cover zero) implies that the treatment has an effect in a given group that is statistically distinguishable from zero. A statistically significant interaction effect (when the grey lines in Figure 4 do not cover zero) indicates a statistically significant difference in effect sizes between the two groups.

Figure 4.

Coefficients and 95% CIs of the OLS regressions on the change in support for remaining in the EU between waves 1 and 2. Note: The coefficient shows the difference-in-differences estimate in support for Remain in each treatment group relative to the control group. The corresponding regression tables are presented in the Online appendix.

We find significant conditional effects of individual characteristics only for the negative political treatment (–Political). In other words, even in these sub-samples, only the negative political treatment has a statistically significant effect on respondents’ vote choices (in some subgroups). ⁹ Hypotheses 2a to 2c expect stronger persuasion and framing effects of novel arguments for people without strong priors. Contrary to hypothesis 2a, we find no significant differences between the effect sizes of the negative political treatment in the EU Extremist and EU Moderate subgroups. Yet, in line with hypothesis 2b, we find that the negative political treatment affects Disengaged voters significantly more than Engaged voters, as shown by the statistically significant negative coefficient of the interaction effect (grey dot and line in Figure 4’s second block in the second column). Further, we demonstrate that voters with little EU knowledge (Not EU Knowledgeable) are significantly more affected by the negative political treatment than EU Knowledgeable voters (results in the third block of the second column of Figure 4). This finding supports hypothesis 2c. Overall, we find evidence that past biases decrease susceptibility to novel frames, but it is not consistent.

The results are also inconclusive with respect to the (dis)confirmation bias hypotheses (hypotheses 3a and 3b). Exposure to the negative political treatment significantly decreased support for remaining in the EU for those who were less supportive of the EU (No Pro-EU Attitude), but it had no significant impact on respondents with a Pro-EU Attitude (black lines in the fourth block of the second column). The difference between these two effects is statistically significant and corroborates hypothesis 3a. At the same time, contrary to hypothesis 3b, the bottom block of the second column in Figure 4 shows that there is no significant difference in how the negative political treatment affected those who were Eurosceptic Party Supporters and those who were not. Overall, we find mixed evidence for the hypotheses that strong priors and (dis)confirmation bias condition the persuasion and framing effects of novel arguments in a consistent or predictable fashion.

Discussion

This article analysed whether (and when) eight arguments for or against the EU can shift vote intentions on a hypothetical German EU exit (‘Dexit’) referendum. We conducted a two-wave survey experiment on a random representative sample of the German population. Our difference-in-differences analysis of respondents’ vote intentions reveals that, in a context with strong public support for the EU, negative arguments, in general, do not have a stronger impact than positive ones on voters’ attitudes towards Germany’s EU membership. However, we find that a negative political argument that highlights the possibility that Germany’s interests can be overruled in the EU has a statistically significant negative effect on voters’ support for EU membership. One in 20 remainers who were exposed to that argument became so much more sceptical as to support exit of their country from the EU if a membership referendum were held next Sunday – the opposite was true for the control group, where the Remain vote increased between wave 1 and wave 2. This is a remarkable treatment effect given that we prompted respondents with just a single short argument. It is even more remarkable given that we asked them to choose between their country remaining in the EU and leaving it. This reflects a fundamental attitude, and we expect that the effect would have been stronger if we had asked respondents about their general evaluation of their country’s EU membership or support for (further) European integration. Notably, despite the significant effect of the negative political frame, the vast majority of citizens exposed to it remained supportive of continued EU membership.

As predicted, the negative political argument had the strongest effect on citizens with weaker prior biases (politically disengaged and less knowledgeable about the EU) and those with congruent pre-existing attitudes against the EU (those with strong anti-EU attitudes). This is in line with theories of voters’ information processing. Contrary to expectations, we find no individual effects of the remaining negative and positive cultural, economic or peace/security-related arguments on support for EU membership.

Our results have important implications for the current debate about Euroscepticism and the future of European integration. While voters might support European integration in general, the resulting loss in sovereignty and the possibility that undesired policies may be imposed can sway the opinion of some citizens towards leaving the EU, even in Germany. While the trade-off between EU integration and sovereignty loss has been a concern in the media, political debates and the scholarly literature (e.g. Hix and Hagemann, 2015; Marks et al., 2002; Thomson et al., 2006), our study offers empirical evidence that sovereignty loss considerations can decrease support for European integration. Our findings therefore highlight the political cost of majoritarian decision-making procedures at the EU level and suggest that policy-makers need to address concerns about sovereignty loss if they seek to maintain support for European integration.

A promising avenue for future research is to study the effect of sovereignty loss frames on voters in different institutional and political contexts. Smaller member states are a particularly interesting case. While sovereignty loss concerns could be more pronounced in these countries because they have lower voting weights in the Council, sovereignty loss frames could still be less important, as smaller countries benefit more from access to the common market and combined political power on the world stage. Furthermore, future studies could explore how arguments related to sovereignty loss or power-sharing influence public opinion in other contexts such as international treaties, trade and investment, or even coalition governments (e.g. Elkins et al., 2006; Fortunato, 2019; Hahm et al., 2019; Harbridge and Malhotra, 2011). We hope our findings will encourage future research on public opinion and support for cooperation at the international and national levels.

Supplemental Material

sj-pdf-1-eup-10.1177_1465116520923735 - Supplemental material for Swaying citizen support for EU membership: Evidence from a survey experiment of German voters

Supplemental material, sj-pdf-1-eup-10.1177_1465116520923735 for Swaying citizen support for EU membership: Evidence from a survey experiment of German voters by Nikoleta Yordanova, Mariyana Angelova, Roni Lehrer, Moritz Osnabrügge and Sander Renes in European Union Politics

Supplemental Material

sj-zip-2-eup-10.1177_1465116520923735 - Supplemental material for Swaying citizen support for EU membership: Evidence from a survey experiment of German voters

Supplemental material, sj-zip-2-eup-10.1177_1465116520923735 for Swaying citizen support for EU membership: Evidence from a survey experiment of German voters by Nikoleta Yordanova, Mariyana Angelova, Roni Lehrer, Moritz Osnabrügge and Sander Renes in European Union Politics