Beyond the Ballot: The Impact of Voting Margin and Turnout on the Legitimacy of Referendum Outcomes in Europe

Abstract

This study delves into the criteria under which referendums can legitimise political choices. It employs survey experiments regarding EU membership across seven European nations, focusing on variations in referendum outcomes, majority margins and voter participation. The empirical results reveal a consistent pattern of legitimacy attributed to referendums, emphasising the influence of majority margin and voter turnout. It also uncovers the critical role of status quo bias on outcome favorability and the apprehension regarding false majorities in shaping public acceptance of referendums. This research contributes to understanding the mechanisms by which democratic procedures legitimise political decisions, revealing the nuanced role of referendums in democratic governance.

Keywords

referendum democratic legitimacy experiment turnout majority rule

Introduction

Under what conditions do referendums legitimise political decisions? Being responsive to the will of the people is an important aspect of legitimate democratic governance. In a politically turbulent time characterised by the gradual erosion of political trust and widespread success of populist parties and leaders, redelegation of political decision-making power from elected representatives to direct democratic procedures has sometimes been proposed as an instrument for strengthening democratic legitimacy.

Referendums between two alternative decision outcomes constitute the most typical manifestation of majority rule that also is used in most contemporary democratic societies around the world (cf. De Vreese and Boomgaarden, 2005; Morel and Qvortrup, 2017). It represents a simple form of decision-making, where all eligible voters have an equal opportunity to influence the outcome, and ultimately, the alternative with the most votes wins. Globally, these governance approaches have gained significant traction (Donovan and Karp, 2006; Scarrow, 2001), with issues related to European integration, including matters of membership, major policy decisions and the approval of treaties and constitutional texts, ranking as the most frequently subjected to a vote worldwide. But are referendum outcomes always accepted by the people?

This study presents a series of survey experiments on the topic of EU membership referendums fielded in five EU member states – France, Germany, the Netherlands, Sweden and the United Kingdom¹ – and two non-member states – Iceland and Norway. Survey participants are confronted with a fictitious situation where their nation is embroiled in a debate over its status in the EU, and a non-binding referendum has been conducted on this matter. Three referendum characteristics – which option won (outcome), the margin of victory (majority size), and the voter participation rate (turnout) – are varied randomly. Participants are then questioned on whether they think the government should adhere to the referendum’s result, irrespective of the preference of the parliamentary majority. Fluctuations in the percentage of participants who believe the referendum’s decision should be respected can be seen as shifts in the perceived legitimacy of the referendum. In this context, legitimacy is defined as the acceptance of the referendum as a necessary and appropriate means to act on the will of the people.

Arnesen et al. (2019) find that how the referendum fared in terms of majority size and turnout also substantially affects the degree to which it legitimises political decisions, along with outcome favourability, in the case of Norway. However, we do not know whether the effects would be the same in other contexts. We also know little about what mechanisms are at play.

Our study makes several contributions that increase our understanding of when and how referendums legitimise political decisions. First, we show that central referendum attributes affect its ability to legitimise political decisions virtually the equally across countries in our sample. In other words, we show that the conditional legitimacy of referendums observed in Norway is part of a universal pattern, travelling to a broader set of countries, each with different relations to the EU and varying experiences with referendums. This suggests that the underlying drivers of legitimacy in this case to a large degree are universal.

Second, when comparing effects in different contexts, we find that the presence of outcome favorability bias is linked to a status quo bias: Respondents who want to keep the status quo display far more outcome favorability bias than citizens in favour of change. This asymmetry potentially alters how we should view outcome favorability bias.

Third, we find evidence of a new mechanism behind the effect of turnout on a referendum’s ability to legitimise political decisions, which could be labelled ‘false majority concerns’. When the outcome is unfavourable, citizens claim the outcome would have been different if everyone voted. This claim gets stronger as turnout decreases.

Taken together, this study contributes to research devoted to disentangling the micro-level mechanisms concerning whether, how, and why democratic procedures legitimise decision outcomes among citizens (Christensen et al., 2015; Esaiasson et al., 2019; Marien and Kern, 2018; Persson et al., 2013).

Theory and Hypotheses

Growing public discontent and the populist surge coincide with a renewed interest in empirical studies of the relationship between democratic decision-making procedures and democratic legitimacy (e.g. Arnesen, 2017; Bengtsson and Mattila, 2009; Esaiasson et al., 2019; Starke and Lunich, 2020; Strebel et al., 2019). This literature connects with a long-standing scholarly debate about the factors that influence citizens’ legitimacy perceptions of authorities, and to what degree these are mainly driven by input considerations (citizens’ abilities to influence policy decisions), and output considerations (what decisions they end up with) (Easton, 1965; Scharpf, 1999). We will not engage deeply in this debate here other than noting that legitimacy perceptions matter because it is regarded as a reservoir of loyalty on which leaders can draw, giving them the discretionary authority they require to govern effectively. Weber (2009) defines legitimacy as a conviction on the part of persons subject to authority that it is right and proper and that they have some obligation to obey, regardless of the basis on which this belief rests. This focus on compliance emphasises the voluntary aspects of political power, imposing considerable influence on the effectiveness of authorities in the hands of those they lead, that is, the citizens (Tyler, 2006).

In the search for measures that may enhance involvement, participation and decision-making influence among citizens, the use of referendums and other forms of direct democracy are often viewed as procedures that meet the criteria. Referendums represent the most well-known and popular direct democratic supplement to the representative system. It is a simple form of decision-making, where all eligible voters have an equal opportunity to influence the outcome, and ultimately, the alternative gaining the most votes wins. Referendums offering a choice between two distinct policy options represent the most common expression of majority rule and are utilised in the majority of modern democratic societies globally (De Vreese and Boomgaarden, 2005; Morel and Qvortrup, 2017), and with increasing popularity (Donovan and Karp, 2006; Scarrow, 2001), not least on European integration issues. EU memberships, key policies, and the ratification of treaties constitute the most voted-on issues globally, and since 1972, no fewer than 59 referendums on EU-related issues have been held (See Online Appendix A for more details). A strong majority of citizens across Europe view direct democratic involvement in political issues as important for democracy (cf. Ferrin and Kriesi, 2016; Schuck and De Vreese, 2015).

However, whereas the majority of European citizens support the idea of more involvement in direct democratic instruments such as referendums and agenda initiatives, the results are mixed in terms of whether political support actually increases among citizens who participate in these arrangements (e.g. Christensen, 2019; Esaiasson et al., 2012; Persson et al., 2013). We argue that the degree to which referendums actually increase the legitimacy of political decisions depends upon several factors. These include factors related to majority size and turnout, as well as citizens’ instrumental considerations about whether the result aligns with their own policy preferences.

Two Dimensions of Majority

Why do people who lose a vote comply with the winning majority? Majority rule as a preference aggregation mechanism is a deeply rooted decision procedure used among human societies since ancient times, and for a considerable time, academics have noted that individuals tend to align their views with those of the majority (Lazarsfeld et al., 1944; Sherif, 1936). The social cost of maintaining and voicing opinions that deviate from the norm is something people generally prefer to avoid, all things being equal, and knowing the majority’s stance exerts social pressure to agree with that consensus (Cialdini and Goldstein, 2004; Kuran, 1997; Noelle-Neumann, 1974). In some situations, it may even be logical for a person to adopt the majority view, especially if they consider the collective to be more informed than themselves (Lupia and McCubbins, 1998). Individuals often regard the opinions of others as either informational input affecting their own views or as social cues indicating which opinions they are expected to share (Cialdini and Goldstein, 2004). Therefore, people may align with the majority believing that the collective is likely to make decisions that are superior to those of an individual (Landemore and Elster, 2012; Mutz, 1998; Surowiecki, 2004). Moreover, citizens differentiate between their personal interests and the collective interests of their political community (Gilley, 2009). When faced with outcomes that are personally displeasing or detrimental, our reaction is shaped by whether we deem these outcomes to align with the common interests of our political community. Recent research shows that being informed about the majority opinion on EU-related issues serves as cue for citizens’ legitimacy beliefs about political processes (Wratil and Wackerle, 2023).

Consequently, the perceptions citizens have about how well a particular decision process truly expresses these shared interests is a crucial factor in understanding what authority it exerts on the decision outcome. What these shared interests are can be signalled in various forms, and with a varying degree of certainty. A referendum is arguably the most accurate mechanism available in the democratic toolbox in terms of identifying the majority opinion of the target population. Compared to elections, referendums are narrower in scope, so voters do not need to balance a range of policy considerations into one single vote cast. Compared to opinion polls or mini-publics, referendums avoid challenges of selecting representative samples since everyone formally has an equal opportunity to cast their vote.

However, in practice, even referendums fall short of the ideal of achieving a clear and unambiguous manifestation of what the people want. The Brexit referendum demonstrated to the full that a referendum result does not necessarily constitute the end of a political discussion, as there was no consensus among the public concerning what policy outcomes resulting from the referendum were seen to respect the outcome of the vote (Hobolt et al., 2020). Furthermore, although all eligible voters have the right to cast their vote, not everyone does so. Low turnout means that the sample size of the eligible voting population is small, which in turn has the potential to lead to skewed representativeness (Bechtel et al., 2016; Lijphart, 1997; Qvortrup, 2002). Indeed, it is rare that turnout levels and majority sizes in a referendum are large enough to confirm with certainty that it corresponds to the majority of the people. Among other things, this may create doubt about whether the majority in the referendum reflects the true majority in the population. This mechanism – let us call it false majority concerns – is one potential reason why citizens should regard referendums with low turnout and/or slim majorities as less legitimate. The popular conversation following the Brexit referendum hints at false majority concerns being a reason why the ‘Leave’ camp won the referendum: had only younger voters turned out in the same proportion as the older voters did, remainers would have strengthened and maybe won (see e.g. Burn-Murdoch, 2024; Dunford and Kirk, 2016).

The internal variation among referendums is large when it comes to the degree of certainty they display about the majority opinion. Exploring how differences in voter turnout and the scale of the majority impact the perceived legitimacy of referendum results stands as one of the most urgent issues requiring academic scrutiny (Hobolt, 2006; Jung and Tavits, 2021). Empirical research has lagged behind in meeting this demand, but it has recently been shown that citizens put much emphasis on these referendum attributes when expressing their view about whether or not the referendum results ought to be followed by the political representatives. Citizens are significantly more reluctant to accept an EU membership referendum result if turnout was on the lower scale of observed turnout in real-world referendums on EU issues than if turnout was on the high end (Arnesen et al., 2019).

Hence, earlier theoretical and empirical research together leads us to propose the following hypotheses:²

H₁: The smaller the majority, the less legitimate is a referendum result likely to be considered for making a political decision.

H₂: The lower the turnout, the less legitimate is a referendum result likely to be considered for making a political decision.

The Role of Outcome Favourability

It is well established that support for democratic procedures is conditional on the outcomes they produce. Support for democratic procedures are coloured by self-serving, instrumental considerations about whether the process is likely to be beneficial in achieving the outcome citizens themselves prefer (Werner, 2020). The willingness to accept decisions are more affected by what the outcome is than how the decision came about (Esaiasson et al., 2019), but that is not to say that procedures are irrelevant. Indeed, it is precisely when decisions are unfavourable that procedures become a concern to citizens (Lind and Tyler, 1988; Tyler, 2006). Since they are so important for the legitimacy of democratic governance, the ‘losers’ in the democratic game thus constitute a subgroup of citizens which have received considerable attention in democracy research over the years.

Known as the outcome favourability effect, or the winner-loser gap, citizens’ assessments of democratic decision procedures are heavily influenced by instrumental considerations (Esaiasson et al., 2019). Numerous studies have shown that those voting for a winning party express higher levels of political support than those voting for losing parties (cf. Anderson et al., 2005; Blais et al., 2017). Moreover, winning and losing seem to be lasting experiences as the gap has been shown to be stable over electoral cycles (Dahlberg and Linde, 2017; Loveless, 2020). Also in referendums, the gap in fairness perceptions between winners and losers are strong and long lasting (Van der Eijk and Rose, 2021), and after holding a referendum, political support increases only among winners of the referendum (Marien and Kern, 2018). Experiencing several losses in a row in referendums creates incentives for the individual to externalise responsibility for the losses, which then in turn may erode legitimacy perceptions (Kern et al., 2024). Thus, based on the vast literature on outcome favourability bias, we hypothesise that:

H₃: An outcome that is unfavourable to a voter, makes the voter less likely to consider the referendum result as legitimate for making a political decision.

We also expect that losers will be more sceptical of a referendum with a small majority or low turnout than winners will be:

H₄: When the majority size is small, voters who receive an unfavourable outcome are less likely to consider the referendum as legitimate for making a political decision than those who receive a favourable outcome.

H₅: When turnout is low, the voters who receive an unfavourable outcome are less likely to consider the referendum as legitimate for making a political decision than those who receive a favourable outcome.

Experimental Design

Our method of investigation is a factorial experimental design, which is well suited to manipulate turnout levels, majority size and outcome while holding other factors constant. Survey participants are initially queried about the intensity of their support or opposition to their nation’s EU membership, or the absence of such membership. Subsequently, each participant is provided with a hypothetical scenario of a referendum concerning EU membership. The exact wording can be seen in Figure 1(a). The wording underlines that the situation includes contexts in which the national parliament disagreed with the outcome, priming the respondents about situations where the representative body is incongruent with the will of the majority. The constructed referendum scenarios differ in terms of voter participation, the margin by which the winning option prevails, and the nature of the winning choice (outcome). The treatment values depicted in Figure 1(b) are selected to encompass the entire spectrum of potential referendums. To be able to identify the effect of the referendum’s outcome, we include control conditions where the outcome characteristics (turnout, majority size, and winning side) are not shown. The idea of implementing a ‘blank’ control condition is that we consider this ‘referendum by itself’ – without knowing anything about level of turnout, majority size and what side won – to be the baseline to which people assess the legitimacy of the referendum result when knowing about the referendum characteristics.

Figure 1.

The Experimental Design: (a) Mock-Up in English of the Experiment As Seen by the Subjects. (b) The Treatments and Possible Values.

In order to assess the favorability of the outcome, we align the respondents’ initial stance on EU membership (expressed ex ante as opposition or support on an eleven-point scale) with the given result of the referendum. This is done to make outcomes comparable between respondents who either favour or oppose EU membership. Thus, a favourable outcome means that their preferences align with the outcome of the referendum, and an unfavourable outcome means that they do not align.

Identification

In the discussion referenced previously, the null hypothesis asserts that the willingness to implement the referendum result is not influenced by the contextual variations mentioned earlier. Our theory suggests that the level of voter turnout, the margin of victory and the direction of the result will have an impact on how legitimacy is judged. Drawing on Hainmueller et al. (2014) we evaluate these factors by calculating the average marginal component effect (AMCE) for each distinct treatment level, using the scenario where no information was provided as the baseline for comparison. To accurately determine the AMCEs, we make certain assumptions (see Bansak et al., 2017; Hainmueller et al., 2014), including the orthogonality of the different treatments.³ For example, we assume that the order in which we present the treatments (which we do not randomise since they are presented in a sentence format) does not affect the estimates. We use cluster robust standard errors for the pooled analysis, clustering by country. This is because we wish to make inferences about a wider European population but only have sampled from a handful of countries (see Robinson, 2020: 20).

We anticipate a direct relationship between the perceived legitimacy of the referendum (as reflected by the AMCE) and both the size of voter turnout and the majority margin. In mathematical terms, we expect that $\frac{β^{t u r n o u t}}{a t 35 %} < \frac{β^{t u r n o u t}}{a t 85 %} a n d \frac{β^{m a j o r i t y}}{a t 51 %} < \frac{β^{m a j o r i t y}}{a t 70 %}$ β^turnout < β^turnout and β^majority < β^majority. We also hypothesise that a favourable outcome will create a more positive evaluation of legitimacy than an unfavourable outcome. Formally, we expect that $β_{u n f a v o u r a b l e}^{o u t c o m e} < β_{f a v o u r a b l e}^{o u t c o m e}$

Case Selection

The experiment was administered to 17,924 respondents as part of the 2017 European Internet Panel Study – EIPS. EIPS was a coordinated survey fielded by six probability-based online survey panels (see Supplementary Material B and Blom et al, 2015; Blom et al., 2018; Ivarsflaten and NCP Team, 2017; Martinsson et al., 2018 for more information about the panels), resulting in 17,405 responses.⁴

As seen in Table 1, the countries included in this study display substantial variation in terms of majority size and turnout, as well as in experience of referendums on European integration. Among the EU member states, Germany has never used the referendum in any decisions about European integration. France, the Netherlands and Sweden have held referendums concerning different issues of European integration. These demonstrate quite substantial variation in terms of turnout, size of the winning majority, and the result in terms of pro- or anti-integration. The Norwegian people have twice rejected EU membership in referendums with high levels of turnout and small winning majorities. In Iceland, membership in the EU has for quite some time been a debated issue. In July 2009 – in the wake of the severe financial crisis – Iceland applied to join the EU. Since then, however, the negotiations have been stalled, and currently, the position on behalf of the government is that Iceland should not be considered a candidate country.⁵

Table 1.

An Overview of European Integration Referendums for the Sample Countries.

Country	Membership status	Turnout	Majority sizes	Type of ref.	Integration outcome
France	Member	60-70-69	68-51-55	NR & B	2 for, 1 against
Germany	Member	N/A	N/A	N/A	N/A
Iceland	Non-member	N/A	N/A	N/A	N/A
Netherlands	Member	63-32	62-62	NR & NB	2 against
Norway	Non-member	79-89	53-52	NR & NB	2 against
Sweden	Member	83-83	52-58	NR & NB	1 for, 1 against

NR: Non-Required; R: Required; NB: Non-Binding; B: Binding.

The referendums that have been held in the sample countries have all been non-required, that is, they have been optionally initiated by the government rather than by constitutional requirements. In all countries, except for France, the outcome of the referendums has been non-binding, although in all cases, the government chose to follow the decision made in the vote. Our cases thus constitute a relatively heterogeneous group with variation in terms of EU membership status, the frequency of EU-related referendums, levels of turnout and majority size in EU-related referendums, as well as with respect to the outcomes of referendums held (pro-integration vs anti-integration).

Results

Figure 2 shows the marginal effects of the treatments – outcome favourability, majority size and turnout – on whether respondents believe that the referendum should be followed. It is divided into three sub-figures, with the one at the top-left corner (2a) showing the main estimates based on the full sample. The sub-figures at the top-right (2b) and at the bottom (2c) show the individual country-level estimates for the non-EU and EU member countries, respectively. Each sub-figure is divided into three rows, one for each treatment, with the possible treatment values on the y-axis. The dots on the vertical centre line shows the baseline not shown condition; it represents the proportion of respondents who believe that the referendum should be followed when not explicitly knowing anything about that referendum attribute. The relative position of the dots with bars on the x-axis shows the estimated change in that proportion (AMCE) when instead treated with a specific attribute. If they are to the left of the centre line, this means that fewer respondents.

Figure 2.

The Effect of Outcome Favourability, Majority Size and Turnout on the Probability of Thinking That the Government Should Follow an EU referendum.

Believe that the referendum should be followed, and vice versa if they are to the right. The country-level results (2b, 2c) are indistinguishable – with few exceptions – so we will focus on the pooled results in Figure 2(a).⁶

Two Dimensions of Majority

Figure 2 reveals strong support for H₁ and H₂. The likelihood that citizens will think that the referendum result should be followed drops on average by 19 percentage points when the turnout level is 35% compared to when the turnout level is not known. From Figure 2(a), it can be deduced that there is a notably lower inclination among citizens to regard the referendum as decisive when the victory margin is slim (with only 51% or 55% of the vote) in comparison to scenarios where the majority size is either large (70%) or unspecified. At a 51% majority, citizens have a four and a half percentage points lower likelihood of agreeing that the referendum outcome should be followed compared to an unknown majority size. The corresponding effect when the majority size is 55% is a four percentage point difference. Citizens do not distinguish between these two typical majority sizes in the history of EU-related referendums.

The effect sizes here are also quite stable across the six countries. A majority size of 70% tends to slightly increase the likelihood of accepting the referendum in a range of from zero to five percentage points, while small majorities tend to decrease the likelihood by the same numbers. It is important to note that while the effect sizes are comparable, the same coefficients in each country are not all statistically significant against the ‘Not shown’ baseline. However, when comparing the highest and the lowest values for both turnout (85% vs 35%) and majority size (70% vs 51%), respectively, the difference is statistically significant in all cases.⁷ Thus, lowering turnout and/or majority size reduces the share of citizens thinking that the government should follow an EU referendum in all countries.

Robustness of the Results

To validate the robustness of the results, we applied the same experimental design to three other issues, using the Norwegian Citizen Panel. Here, we keep the treatments identical to the ones presented in Figure 2 while asking about respondents’ preferences on issues that arguably are, compared to EU membership, more closely related to people’s everyday life and have recently been on the national political agenda. The results are shown in Figure 3. Interestingly, the patterns we observe are surprisingly similar, although with somewhat smaller effect sizes compared to the EU referendum case.

Figure 3.

Other Referendum Issues.

In addition, we fielded an experimental design where the subjects were treated with all possible values and combinations of turnout and majority size. In the experiment, respondents were described a hypothetical referendum and asked whether they in that case think Norway should join the EU. Figure 4 shows the predicted probability of thinking that Norway should join across different combinations of turnout and vote share size, estimated using a machine learning model. The results provide further support for our findings, suggesting that two dimensions of majority – the size of majority and turnout – matter for people when assessing the legitimacy of a referendum result. More details on the design and analysis of these alternative experiments can be found in Section E in the Online Appendix.

Figure 4.

Two Dimensions of Majority Rule.

Outcome Favourability and Status Quo Bias

Coming back to Figure 2, it further shows that when a respondent is provided information on the result of a referendum, an unfavourable outcome decreases the likelihood of accepting the referendum outcome by nine percentage points, while a favourable outcome increases it by eleven points (H₃). In other words, the difference between a favourable and an unfavourable outcome is no less than 20 percentage points. There is however no interaction between outcome favourably and respectively turnout (H₄) or majority size (H₅).⁸

While it is unsurprising and in line with previous research that having the desired outcome in a referendum makes people perceive that outcome as more legitimate than how those on the losing side perceive the outcome, the results here reveal that this strong outcome favourably affects is not equally distributed among citizens. Rather, it appears to be conditional on whether the respondents’ preferences for EU membership represent the status quo. For ease of interpretation, we present the estimates graphically in Figure 5. The figure demonstrates a strong and consistent outcome favourably effect among those who are in favour of the current membership status, while this effect is absent among those who prefer a change from the status quo. In the non-member states (Iceland and Norway), citizens who are opposed to EU membership are more strongly affected by knowing the outcome than those who are in favour of joining the EU. In the countries that are already members of the EU, the effect goes in the opposite direction. Here, those who are opposed to leaving the EU are accordingly less likely to accept such an outcome compared to EU opponents when the result is to stay in the EU. The pattern also holds for the out-of-sample countries Germany and France, as hypothesised in the preregistration.

Figure 5.

Status Quo Bias in the Effect of Outcome Favourability.

Although it is beyond the scope of this study to delve further into the mechanisms behind the finding that the outcome favorability effect interacts with the status quo in the form of a status quo bias, it does make sense when taking stock of earlier research. Referendums on national sovereignty entail decisions that can take a polity in fundamentally new directions and, as such, draw clear lines between the status quo and the new alternative. Faced with new decisions, individuals often stick with the status quo alternative (Samuelson and Zeckhauser, 1988). Especially in high-stake political decisions such as referendums on sovereignty, risk-averse voters will be more likely to want to keep the status quo alternative (Verge et al., 2015; Morisi, 2018).

Status quo bias surfaces in many ways within the political domain, among voters and politicians alike (Alesina and Passarelli, 2019; Sheffer et al., 2018). It is, among other things, connected to political candidates’ incumbency advantage (Quattrone and Tversky, 1988), political inaction (Anderson, 2003), and voters’ support for referendum proposals (Kriesi, 2005). Consequently, in referendums on sovereignty, campaign strategies playing on these sentiments are beneficial for defendants of the status quo alternative (Brie and Dufresne, 2020; Nadeau et al., 1999). The status quo bias thus has important implications for the politics of preference formation (Druckman and Lupia, 2000). In future research, an important avenue would be to further experimentally investigate if the interaction between outcome favorability and status quo found here also is present in other types of decision-making processes.

Follow-Up Study of Mechanism. The Case of Brexit

While the results from the cross-country study provide solid experimental evidence that turnout and majority size matter for the legitimacy of referendums, we know less about what mechanisms are at play. As discussed in Section 2.1, reasons why people tend to conform to the majority opinion are that they believe the majority knows better or because it is socially costly to go against the majority. In the case of referendums, low turnout and/or a small majority (of those participating) could arguably be seen less as a result of the general will than a referendum with high turnout and a strong majority.

The Brexit referendum in 2016 was the last real referendum conducted on EU membership and the only case where an existing EU member state held a referendum on membership and the majority voted to leave the union. Taking advantage of the opportunity to tap into British citizens’ perceptions about how turnout and majority sizes could have affected the outcome of an actual EU membership referendum, we fielded an experiment in the British Election Study online panel. The fact that Britain recently had carried out a real EU referendum and that the discussion about whether the result should be followed was ongoing at the time allows us to investigate a potential mechanism as to why citizens care about turnout levels and majority size. Following our earlier theoretical arguments, we hypothesise that majority size and turnout signal the certainty with which one can conclude that the referendum result indeed represents the majority opinion in the population:⁹

H₇: The lower the turnout and smaller the size of majority, the more likely respondents are to believe that the outcome could have been different from the proposed scenario. Subject to outcome favourability bias, we expect the losers to drive this effect:

H₈: This effect (H₇) is moderated by outcome favourability, that is respondents who get a scenario with an unfavourable outcome are more likely to believe that the outcome could have been different if all eligible voters would have turned out to vote.

The sample (N = 3, 179) has been weighted to be representative of all adults in Britain, and the experiment was fielded 1-7 February, 2019.¹⁰ Before we turn to the experimental results, we first present Figure 6, which demonstrates how British citizens’ preferences for EU membership were split and strongly polarised, where more than a quarter of the population was completely opposed to EU membership and an equal proportion completely in favour. In total, more than a third do not think that the government should follow the result of the 2016 referendum. Of these, the vast majority is ‘Remainers’, giving observational support to the experimental scenarios that support for referendums are conditional on outcome favourability bias. It is worth noticing, though, that a sizable proportion of ‘Remainers’ differentiates between their own personal preference for membership and what the government should do now that a referendum has been held. Among the agnostic, the clear majority believes the referendum result should be honoured.

Figure 6.

Support for British EU Membership and Willingness to Follow 2016 Referendum.

The experimental design is largely similar to the EIPS study presented earlier, with some tweaks to accommodate the particular context and the added hypotheses. The respondents are first asked to imagine a different scenario for the 2016 referendum, with different turnout and shares of Leave/Remain votes. The treatment values are randomly and uniformly drawn from integers between 35–100 (turnout) and 1–99 (majority size) in order to provide a more nuanced assessment of potential turnout and majority size thresholds. In a second task, respondents were asked to imagine that in the future, there would be a second referendum on EU membership. An extra post measure is introduced in addition to the original measure about their willingness to follow the referendum result: ‘Which outcome do you think would have been more likely in this scenario if all eligible voters had turned out to vote?’, with the two response options ‘Leave’ and ‘Remain’. The results from the first post-measure (‘should the government follow the result’) are in line with the EIPS results, so we focus here on the results from the extra post-measure. If we find that the share of respondents who believe the result would have been different is negatively correlated with turnout levels, we identify this as evidence of false majority concerns. If we observe that this association is stronger when the outcome is not in line with respondent’s own preference, we identify a moderating effect from outcome favorability bias.

As for the results, we start with the latter – outcome favorability bias. Table 2 shows how respondents who are given a scenario where they see their camp winning majority are not affected at all by the turnout level. Both among ‘Leavers’ and ‘Remainers’, nine out of ten think the result would have been the same regardless of turnout if the outcome aligned favourably with their own preferences. However, if they are presented with an outcome where they are on the losing side, turnout matters quite a bit. When turnout is under 50%, 74% of Leavers believe that a ‘Remain’ result would have been flipped to ‘Leave’ if the whole population turned out to vote. This number decreases to 59% when turnout is above 75%. Likewise, for ‘Remainers’, 65% believe a ‘Leave’ result would have become ‘Remain’ if they were presented with a turnout lower than 50%. When turnout increases to above 75%, the share number drops to 52%.

Table 2.

Share Who Believed ‘Remain’ Would Have Been More Likely in Given Scenario If All Eligible Voters Had Turned Out to Vote.

Referendum winner	Referendum turnout (%)	Among leavers	Among remainers
	35–50	8%	65%
Majority votes Leave	51–75	7%	64%
	75–100	6%	52%
	35–50	26%	89%
Majority votes Remain	51–75	33%	89%
	76–100	41%	85%

The first column shows the values of the first treatment condition, that is, whether the respondent is presented with an outcome whether the majority votes ‘Leave’ or ‘Remain’. The second column presents the values of the second treatment condition, that is the turnout levels. The third and fourth columns, respectively, present the results for respondents who favoured ‘Leave’ and ‘Remain’.

These results yield support for both H₇ and H₈: Low turnout creates doubt about the results, and the losers of a referendum point to it as a reason to believe that the referendum is not a true representation of the preferences of the entire population. That is not to say that false majority concerns are the only concerns triggered by low turnout, as it may also signal to citizens a limited degree of popular consent. Also, we do not find the same pattern for majority size, nor for the scenario where the respondents are asked to imagine a second, future referendum (see Online Appendix Section D.4). Overall, though, the results are indicative that a false majority mechanism is part of the reason why referendums with low turnout are perceived as less legitimate among losers than referendums with high turnout.

The results offer a perspective on the heated debate that followed the 2016 Brexit referendum. We can reasonably speculate that an even higher turnout than the 72.2% observed in the referendum would have given the political representatives a stronger mandate among the citizens to carry out Brexit, everything else being equal.

Conclusion

Referendums – the redelegation of decision-making power from representatives to the people – are sometimes regarded as a complementary democratic mechanism that might improve the legitimacy of representative democracy. Indeed, the use of referendums has been common in decision-making concerning European integration, often based on the argument that direct voting by the citizens is a more legitimate form of democratic decision-making. However, the results of our experiments demonstrate that the legitimacy of a referendum is strongly contingent on attributes related to the referendum itself. This is a perception shared by citizens across countries with widely different historical experiences with referendums.

Referendums are not necessarily a quick fix that ends the discussion on a contentious topic once and for all. While all citizens theoretically have an equal opportunity to take part, express their views and influence the end result of a referendum, it is not prudent to take for granted that they will see any justly carried out referendum as a definitive directive for enacting the decision. The interpretation we draw from the findings suggests that this conception of majority rule is overly reductive. Our results show that citizens are reluctant to allow a far-reaching decision such as being an EU member or not be decided by a referendum with no more than a simple majority if it implies a shift from the status quo – regardless of whether it entails joining or leaving the EU. As such, this study helps us understand how people perceive the role of referendums in political decision-making processes and what conditions may influence their legitimacy perceptions of a referendum. It invites a review of how we interpret the legitimacy of referendums in the political decision-making process. In decisions that have far-reaching implications, such as EU membership decisions, there might be a case for adopting conservatory mechanisms that favour the status quo, such as quorum rules that require a certain turnout threshold and/or more than a simple majority. Several countries already have quorum rules for turnout levels (for an overview, see Morel, 2017), and the results in this study show that there are good reasons to have such rules in place.

A potential limitation of this study is that the experiments were conducted in the period 2017-2019. European politics has changed since then, meaning that newer experiments may yield different results. Regarding the Brexit case study, one may argue this is not the best case to get a grip at the false majority mechanism. The time when the study was fielded was indeed a period of turmoil British politics. The Brexit issue had become extremely politicised, forming new ‘Leaver’ and ‘Remainer’ identities that have been found to be associated with policy norms that go beyond views on the EU membership issue (Tilley and Hobolt, 2023). It could be argued that this makes the Brexit case study not a particularly strong test or a ‘most likely case’ for our argument. That said, it is important to note that our study deals with attitudes towards referendums and not policy issues as such. Also, the results from the comparative experiment travel well across political cultures and country contexts, which indicate that they may be robust to contextual changes over time within countries as well.

Another potential limitation concerns the fact that our experiment only asks about one type of referendum, that is a non-binding referendum about the country’s membership in the EU. As there are several types of referendums, and countries vary with regard to legal provisions for the use of referendums (Morel, 2017), we are not in a position to argue that our findings are generalisable to all types of referendums.

Handing over power to the people serves as a core motivation behind the increased use of referendums, albeit within the bounds of representative democracy. They tend to be initiated by political representatives, and they are most commonly merely advisory, with the representatives having the final say. Their decision whether or not to follow the referendum results should be informed not simply by observing which side won the majority, but also by taking turnout and the size of the majority into account.

Supplemental Material

sj-docx-1-psx-10.1177_00323217241266025 – Supplemental material for Beyond the Ballot: The Impact of Voting Margin and Turnout on the Legitimacy of Referendum Outcomes in Europe

Supplemental material, sj-docx-1-psx-10.1177_00323217241266025 for Beyond the Ballot: The Impact of Voting Margin and Turnout on the Legitimacy of Referendum Outcomes in Europe by Sveinung Arnesen, Troy Saghaug Broderstad, Mikael P Johannesson and Jonas Linde in Political Studies

Footnotes

Acknowledgements

We would like to thank colleagues involved in the European Internet Panel Study, participants at the 2018 ZEW/SFB Conference ‘Understanding Policy Preferences using Experimental Methods’, the ECPR General Conference 2018, the 2019 Norwegian Political Science Conference, and the 2020 Scancor seminar, Stanford University. We gratefully acknowledge support from the DIGSSCORE research infrastructure at the University of Bergen.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This research was supported by the Research Council of Norway (grant #262986).

ORCID iD

Sveinung Arnesen

Supplemental Material

Additional Supplementary Information may be found in the online version of this article.

A. An Overview of Referendums on European Integration.

A.1. EU Referendums in the Sample Countries.

Table A.1: Referendums on European integration 1972-2016.

B. Data Description.

B.1. The EIPS Infrastructure.

Table B.1: Field dates and sample sizes of the 2017 EIPS survey.

B.2. Descriptive Statistics of the EIPS Sample.

Table B.2: Descriptive statistics of the 2017 European Internet Panel (EIPS) sample.

C. Additional Information on Design.

C.1. Pre-Treatment Measure: Preference for EU Membership.

Table C.1: Panel variable names and values: Preference for EU membership.

Table C.2: Distributions: Preference for EU membership.

Figure C.1: Example screenshot of German fielding of experiment, pre-treatment question on opinion.

C.2. Main Experimental design.

Figure C.2: Example screenshot of German fielding of experiment where treatments are not shown.

Table C.3: Panel variable names and values: Turnout treatment.

Table C.4: Panel variable names and values: Majority size treatment.

Table C.5: Panel variable names and values: Winner treatment.

Table C.6: Panel variable names and values: Post-treatment measure.

Table C.7: Distributions: Turnout treatment.

Table C.8: Distributions: Majority size treatment.

Table C.9: Distributions: Winner treatment.

Table C.10: Distributions: Post-treatment measure.

C.3. Questionnaire for Brexit Experiment.

D. Additional Results.

D.1. Reproduced Results with Marginal Means (MM).

Figure D.1: The Marginal Mean (MM) of thinking that the government should follow an EU referendum by country, outcome favourability, majority size, and turnout.

Figure D.2: The Marginal Mean of thinking that the government should follow the referendum by referendum outcome, country, and EU membership preference.

D.2. Interaction effects.

Figure D.3: The interaction effect of outcome favorability, majority size, turnout, and satus quo bias on the probability of thinking that the government should follow the referendum.

Figure D.4: The interaction effect of outcome favorability, satus quo bias, and majority size, on the on the probability of thinking that the government should follow the referendum.

D.3. Heterogeneous Effects by EU Membership Preference.

Figure D.5: The effect of EU membership preference and referendum outcome on the probability of thinking that the government should follow the referendum—mean estimates.

Figure D.6: The effect of EU membership preference and referendum outcome on the probability of thinking that the government should follow the referendum—loess fit.

Figure D.7: The effect of EU membership preference and referendum outcome on the probability of thinking that the government should follow the referendum—predicted probabilities

D.4. Additional Results from the Brexit Referendum Experiment.

Table D.1: Share who believed Remain would been more likely in given scenario if all eligible voters had turned out to vote (in a future [second] referendum).

E. Additional Experiments (Robustness Checks).

E.1. Experiments Using Other Policy Issues.

Figure E.1: Other referendum issues.

Figure E.2: Other referendum issues.

E.2. Experiment Using Continuous Treatment Values.

Figure E.3: Two Dimensions of Majority Rule.

F. References.

Notes

Author Biographies

Sveinung Arnesen is a Research Professor at NORCE – Norwegian Research Centre and Professor II at the Department of Government, University of Bergen. His research revolves around political behaviour and democracy.

Troy Saghaug Broderstad is an Associate Professor in political science at UiT – The Arctic University of Norway and a Researcher III at NORCE – Norwegian Research Center. His research interests involve political representation, elite behaviour and democracy.

Mikael P Johannesson is a Researcher at NORCE – Norwegian Research Centre, focusing on survey and experimental studies on political behaviour.

Jonas Linde is a Professor of Political Science at the Department of Comparative Politics, University of Bergen. His research interests involve political support, quality of government and anti-immigration attitudes.

References

Alesina

Passarelli

(2019) Loss Aversion in Politics. American Journal of Political Science 63 (4): 936–947.

Anderson

(2003) The Psychology of Doing Nothing: Forms of Decision Avoidance Result from Reason and Emotion. Psychological Bulletin 129 (1): 139–167.

Anderson

Blais

Donovan

, et al. (2005) Losers’ Consent: Elections and Democratic Legitimacy. Oxford: Oxford University Press.

Arnesen

(2017) Legitimacy from Decision-Making Influence and Outcome Favourability: Results from General Population Survey Experiments. Political Studies 65 (Suppl. 1): 146–161.

Arnesen

Broderstad

Johannesson

, et al. (2019) Conditional Legitimacy: How Turnout, Majority Size, and Outcome Affect Perceptions of Legitimacy in European Union Membership Referendums. European Union Politics 20 (2): 176–197.

Bansak

Hainmueller

Hopkins

, et al. (2017) The Number of Choice Tasks and Survey Satisficing in Conjoint Experiments. MIT Political Science Department Research Paper No 2017-6, Stanford University Graduate School of Business Research Paper no. 17-15. Stanford, CA: Stanford University.

Bechtel

Hangartner

Schmid

(2016) Does Compulsory Voting Increase Support for Leftist Policy? American Journal of Political Science 60 (3): 752–767.

Bengtsson

Mattila

(2009) Direct Democracy and Its Critics: Support for Direct Democracy and ‘Stealth’ Democracy in Finland. West European Politics 32 (5): 1031–1048.

Blais

Morin-Chasse

Singh

(2017) Election Outcomes, Legislative Representation, and Satisfaction with Democracy. Party Politics 23 (2): 85–95.

10.

Blom

Felderer

Gebhard

, et al. (2018) German Internet Panel, Wave 29 (May 2017). Mannheim: GESIS.

11.

Breiman

(2001) Random Forests. Machine Learning 45: 5–32.

12.

Brie

Dufresne

(2020) Tones from a Narrowing Race: Polling and Online Political Communication during the 2014 Scottish Referendum CamPaign. British Journal of Political Science 50 (2): 497–509.

13.

Burn-Murdoch

(2024) Brexit: Voter Turnout by Age. Available at: https://www.ft.com/content/1ce1a720-ce94-3c32-a689-8d2356388a1f (accessed July 13, 2024).

14.

Christensen

(2019) Boosting Political Trust with Direct Democracy? The Case of the Finnish Citizens’ Initiative. Politics and Governance 7 (2): 173–186.

15.

Christensen

Karjalainen

Nurminen

(2015) Does Crowdsourcing Legislation Increase Political Legitimacy? The Case of Avoin Ministeri··o in Finland. Policy & Internet 7 (1): 25–45.

16.

Cialdini

Goldstein

(2004) Social Influence: Compliance and Conformity. Annual Review of Psychology 55: 591–621.

17.

Dahlberg

Linde

(2017) The Dynamics of the Winner–Loser Gap in Satisfaction with Democracy: Evidence from a Swedish Citizen Panel. International Political Science Review 38 (5): 625–641.

18.

De Vreese

Boomgaarden

(2005) Projecting EU Referendums: Fear of Immigration and Support for European Integration. European Union Politics 6 (1): 59–82.

19.

Donovan

Karp

(2006) Popular Support for Direct Democracy. Party Politics 12 (5): 671–688.

20.

Druckman

Lupia

(2000) Preference Formation. Annual Review of Political Science 3 (1): 1–24.

21.

Dunford

Kirk

(2016) How Did Turnout Affect the EU Referendum Result? Available at: https://www.telegraph.co.uk/politics/2016/06/24/how-did-turnout-affect-the-eu-referendum-result/ (accessed July 13, 2024).

22.

Easton

(1965) A Systems Analysis of Political Life. Hoboken, NJ: John Wiley & Sons.

23.

Esaiasson

Gilljam

Persson

(2012) Which Decision-Making Arrangements Generate the Strongest Legitimacy Beliefs? Evidence from a Randomised Field Experiment. European Journal of Political Research 51 (6): 785–808.

24.

Esaiasson

Persson

Gilljam

, et al. (2019) Reconsidering the Role of Procedures for Decision Acceptance. British Journal of Political Science 49 (1): 291–314.

25.

Ferrin

Kriesi

(eds) (2016) How Europeans View and Evaluate Democracy. Oxford: Oxford University Press.

26.

Gilley

(2009) The Right to Rule: How States Win and Lose Legitimacy. New York: Columbia University Press.

27.

Hainmueller

Hopkins

Yamamoto

(2014) Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices Via Stated Preference Experiments. Political Analysis 22 (1): 1–30.

28.

Hobolt

(2006) Direct Democracy and European Integration. Journal of European Public Policy 13 (1): 153–166.

29.

Hobolt

Tilley

Leeper

(2020) Policy Preferences and Policy Legitimacy after Referendums: Evidence from the Brexit Negotiations. Political Behavior 44: 839–858.

30.

Ivarsflaten

NCP Team (2017) Norwegian Citizen Panel Wave 9 (data file). Bergen: University of Bergen.

31.

Jung

J-H

Tavits

(2021) Do Referendum Results Change Norm Perceptions and Personal Opinions? Electoral Studies 71: 102307.

32.

Kern

Marien

Muradova

(2024) The Effect of Accumulated Losses on Perceptions of Legitimacy. European Political Science Review. Epub ahead of print 25 January. DOI: 10.1017/S175577392300036X.

33.

Kriesi

(2005) Direct Democratic Choice: The Swiss Experience. Lanham, MD: Lexington Books.

34.

Kuran

(1997) Private Truths, Public Lies: The Social Consequences of Preference Falsification. Cambridge, MA: Harvard University Press.

35.

Landemore

Elster

(2012) Collective Wisdom: Principles and Mechanisms. Cambridge: Cambridge University Press.

36.

Lazarsfeld

Berelson

Gaudet

(1944) The People’s Choice. Duell, Sloan & Pearce.

37.

Leeper

Hobolt

Tilley

(2020) Measuring Subgroup Preferences in Conjoint Experiments. Political Analysis 28 (2): 207–221.

38.

Lijphart

(1997) Unequal Participation: Democracy’s Unresolved Dilemma. American Political Science Review 91: 1–14.

39.

Lind

Tyler

(1988) The Social Psychology of Procedural Justice. Boston, MA: Springer.

40.

Loveless

(2020) When You Win, Nothing Hurts: The Durability of Electoral Salience on Individuals’ Satisfaction with Democracy. Political Studies 69: 538–558.

41.

Lupia

McCubbins

(1998) The Democratic Dilemma: Can Citizens Learn What They Need to Know? Cambridge: Cambridge University Press.

42.

Marien

Kern

(2018) The Winner Takes It All: Revisiting the Effect of Direct Democracy on Citizens’ Political Support. Political Behavior 40 (4): 857–882.

43.

Martinsson

Andreasson

Markstedt

, et al. (2018) Technical Report Citizen Panel 28 – 2017 (data file): Gothenburg: University of Gothenburg, LORE.

44.

Morel

(2017) Types of Referendums, Provisions and Practice at the National Level Worldwide. In: Morel

Qvortrup

(eds) The Routledge Handbook to Referendums and Direct Democracy. London: Routledge, pp.27–59.

45.

Morel

Qvortrup

(2017) The Routledge Handbook to Referendums and Direct Democracy. New York: Routledge.

46.

Morisi

(2018) Choosing the Risky Option: Information and Risk Propensity in Referendum Campaigns. Public Opinion Quarterly 82 (3): 447–469.

47.

Mutz

(1998) Impersonal Influence: How Perceptions of Mass Collectives Affect Political Attitudes. Cambridge: Cambridge University Press.

48.

Nadeau

Martin

Blais

(1999) Attitude Towards Risk-Taking and Individual Choice in the Quebec Referendum on Sovereignty. British Journal of Political Science 29 (3): 523–539.

49.

Noelle-Neumann

(1974) The Spiral of Silence a Theory of Public Opinion. Journal of Communication 24 (2): 43–51.

50.

Persson

Esaiasson

Gilljam

(2013) The Effects of Direct Voting and Deliberation on Legitimacy Beliefs: An Experimental Study of Small Group decision-Making. European Political Science Review 5 (03): 381–399.

51.

Quattrone

Tversky

(1988) Contrasting Rational and Psychological Analyses of Political Choice. American Political Science Review 82 (3): 719–736.

52.

Qvortrup

(2002) Government by the People: A Comparative Study of Referendums. Manchester: Manchester University Press.

53.

Robinson

(2020) When Should We Cluster Experimental Standard Errors?, Preprint, 1–64, Available at: https://ts-robinson.com/static/pub_pdfs/robinson-whenshouldwe-2020.pdf (accessed July 15, 2024).

54.

Samuelson

Zeckhauser

(1988) Status Quo Bias in Decision Making. Journal of Risk and Uncertainty 1 (1): 7–59.

55.

Scarrow

(2001) Direct Democracy and Institutional Change. Comparative Political Studies 34 (6): 651–665.

56.

Scharpf

(1999) Governing in Europe: Effective and Democratic? Oxford: Oxford University Press.

57.

Schuck

ART

De Vreese

(2015) Public Support for Referendums in Europe: A Cross-National Comparison in 21 Countries. Electoral Studies 38: 149–158.

58.

Sheffer

Loewen

Soroka

, et al. (2018) Nonrepresentative Representatives: An Experimental Study of the Decision Making of Elected Politicians. American Political Science Review 112 (2): 302–321.

59.

Sherif

(1936) The Psychology of Social Norms. Oxford: Harper.

60.

Starke

Lunich

(2020) Artificial Intelligence for Political Decision-Making in the European Union: Effects on Citizens’ Perceptions of Input, Throughput, and Output Legitimacy. Data & Policy 2: e16.

61.

Strebel

Kubler

Marcinkowski

(2019) The Importance of Input and Output Legitimacy in Democratic Governance: Evidence from a population-Based Survey Experiment in Four West European Countries. European. European Journal of Political Research 58 (2): 488–513.

62.

Surowiecki

(2004) The Wisdom of Crowds. London: Abacus.

63.

Tilley

Hobolt

(2023) Brexit as an Identity: Political Identities and Policy Norms. PS: Political Science & Politics 56 (4): 546–552.

64.

Tyler

(2006) Why People Obey the Law. Princeton, NJ: Princeton University Press.

65.

Van der Eijk

Rose

(2021) Winner-Loser Effects in Contentious Constitutional Referenda-Perceptions of Procedural Fairness and the Brexit Referendum. British Journal of Politics and International Relations 23: 104–120.

66.

Verge

Guinjoan

Rodon

(2015) Risk Aversion, Gender, and Constitutional Change. Politics & Gender 11 (3): 499–521.

67.

Weber

(2009) The Theory of Social and Economic Organization. New York: Simon & Schuster.

68.

Werner

(2020) If I’ll Win It, I Want It: The Role of Instrumental Considerations in Explaining Public Support for Referendums. European Journal of Political Research 59 (2): 312–330.

69.

Wratil

Wackerle

(2023) Majority Representation and Legitimacy: Survey-Experimental Evidence from the European Union. European Journal of Political Research 62: 285–307.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.69 MB