Sage Journals: Discover world-class research

Abstract

This conjoint study investigates the type of mandate a referendum confers in the political decision-making process. While a majority of citizens in general believe that the government should follow the results of a referendum on European Union membership, its perceived legitimacy in the eyes of the public heavily depends upon the level of turnout, the size of the majority, and the outcome of the specific referendum in question. Thus, whether a referendum legitimizes a political decision in the eyes of the public is conditional upon these three dimensions.

Keywords

Conjoint survey experiment European Union legitimacy Norway referendums

Introduction

The growing popularity and use of referendums and legislative initiatives has led to an increased interest in questions related to different perspectives on democracy and democratic legitimacy (Donovan and Karp, 2006; Ferrin and Kriesi, 2016; Hobolt, 2006; Morel, 2017; Qvortrup, 2018; Scarrow, 2001). Referendums have become increasingly important in the political decision-making process in Europe, and issues such as membership, key policies, ratification of treaties, and constitutional documents are the most voted-on issues in the world (De Vreese and Boomgaarden, 2005; Hobolt, 2009).

Despite the existence of a large body of literature on referendums, questions of the legitimacy of referendums as a political decision-making procedure are still understudied. A virtue of democratic decision-making procedures is that the foundations on which a decision is made are thought to enhance the legitimacy of that decision (Dahl, 1989), and legitimate political decisions in turn facilitate implementation of the decision outcome (Tyler, 2006). One of the most pertinent research questions yet to be answered is how variations in turnout and majority size affect the implications and legitimacy of referendum outcomes (Hobolt, 2006). In addition, the question about the extent to which the outcome affects legitimacy beliefs among those participating in the referendum should be addressed. In actual referendums, these critical dimensions—the turnout, the size of the winning majority, and the outcome—are subject to variation. We therefore empirically investigate how such variation affects legitimacy beliefs. Do people think that the government should follow any result once the issue is decided on in a referendum, or do they change their opinion when learning about the attributes of the specific referendum?

Utilizing a conjoint experimental design, we propose a hypothetical scenario in which membership in the European Union (EU) is put to a referendum. The experiment is performed within the Norwegian Citizen Panel (NCP)—an online, probability-based survey panel of the Norwegian population. The question regards whether Norway should join the EU, an issue that was previously subject to referendums in 1972 and 1994.

The respondents are given different scenarios, where the level of turnout, size of majority, and outcome are varied. Some respondents are not exposed to this information before they are asked to assess whether the government should follow the outcome, thereby allowing for comparisons of ex ante and ex post assessments of the legitimacy of a referendum as a political decision-making procedure.

The results reveal an important insight into the dynamics of politics. When varying turnout, majority size, and outcome favorability, the share of citizens that think the government should follow the result spans from virtually everyone to only a quarter of the population. Hence, holding an advisory referendum on EU membership does not automatically provide stronger legitimacy to the decision, but is instead conditional upon these three properties of the referendum in question.

Democratic decision-making and legitimacy

The aggregation of individual preferences of citizens in society into the formation of a collective decision outcome constitutes a core research field in the social sciences. Fundamental questions of social choice theory concern how conflicting interests are reconciled so as to facilitate cooperation among the group members (Arrow, 2012; Sen, 2017). The central concept for the analysis of collective decisions is that of an aggregation rule, where individual inputs—such as votes—are transformed into a collective output. The classic example is majority voting between two options, where the group selects the option receiving the most votes. Majority voting is based on the simple principle of political equality, i.e. equal chances of participation and equal power. It encourages the expression of sincere personal beliefs, rather than conformity, and it is a decision rule that is easily executed. Majority rule is popular across the full spectrum of human groups, from hunter-gatherer tribal societies to modern industrial democracies, and indeed, among several nonhuman social animals as well (Conradt and List, 2009; Conradt and Roper, 2003; Couzin et al., 2005; Hastie and Kameda, 2005).

Much work has investigated whether and how decision-making rules could be arranged to aggregate individual preferences to achieve the most beneficial outcomes for the collective (Sen, 2017). Our perspective is related but different, focusing on the legitimacy of collective decisions, i.e. the degree to which citizens are willing to comply with the outcome of the majority decision. Hence, we do not focus on the quality of the outcome of the decision, but rather on the extent to which the affected individuals view the decision as a mandate for action. The crucial question is: What makes a citizen comply with the interests of the community when this interest collides with his or her personal interests.

The legitimacy of collective decisions

Max Weber (2009) established legitimacy as one of the central concepts in understanding the survival of political regimes. He defines legitimacy as a conviction on the part of persons subject to authority that it is right and proper and that they have some obligation to obey, regardless of the basis on which this belief rests. Legitimacy is regarded as a reservoir of loyalty upon which leaders can draw, giving them the discretionary authority they require to govern effectively. In a political system in which the governing group bases its activity on a principle that the members of the system consider to be adequate grounds for obeying their rulers, power is said to be legitimate, and citizens willingly comply with the authority’s decision. This focus on compliance emphasizes the voluntary aspects of power, imposing considerable influence on the effectiveness of authorities in the hands of those they lead, i.e. the citizens (Tyler, 2006).

A distinction can be made between normative and empirical perspectives on the concept of legitimacy (Habermas, 2015). Normative legitimacy refers to what third-party analysts view as legitimate at the system level, emphasizing formal system properties and policy output criteria for democratic legitimacy, including accountability, efficiency, and procedural fairness. On the system level, legitimacy refers to the acceptability of legislation according to abstract normative criteria. The empirical perspective focuses on the individual level and the evaluation by citizens. It is therefore a more subjective way of assessing legitimacy (Weatherford, 1992). Thus, in contrast to the normative criteria, this individual-level version of legitimacy, highlighting public orientations, experiences, and expectations, maps onto the general category of ‘perceived legitimacy’ or ‘empirical legitimacy’ (Mansbridge, 2015; Thompson, 2008; Tyler, 2006). Hence, the aim of this particular study is to evaluate democratic decision processes based on empirical legitimacy, from the perspective of the citizens that are affected by the outcome.

The virtue of democratic decision-making procedures is considered a legitimizing attribute of democratic regimes (Dahl, 1989; Habermas, 2015). Such procedures include the implementation of the preference aggregation rules used to determine collective outcomes and the discursive structure of opinion- and will-formation through deliberation among citizens. Decisions made of, by, and for the people are thought to bring legitimacy to the system, sometimes referred to as ‘input-oriented’ legitimacy (Scharpf, 1999). In recent years, a growing body of survey and experimental research has been devoted to disentangling the micro-level mechanisms concerning whether, how, and why such democratic procedures legitimize decision outcomes (Arnesen, 2017; Christensen et al., 2015; Esaiasson, 2011; Esaiasson et al., 2016; Marien and Kern, 2018; Persson et al., 2013). These studies provide a complementary, empirical approach in examining central questions in studies of democratic procedures that traditionally have focused on the system level. We position our study in this tradition.

The concept of referendums

We focus on referendums since this is the most representative manifestation of majority rule used in most contemporary democracies (Morel and Qvortrup, 2017; Qvortrup, 2018). Direct democratic institutions interact with the key elements of representative democracies (Leemann and Wasserfallen, 2016), and different forms of direct democratic initiatives have become increasingly common in contemporary Europe (Donovan and Karp, 2006; Scarrow, 2001). There is also evidence of a growing demand for direct democracy among the European public (Bowler et al., 2007; Donovan and Karp, 2006; Ferrin and Kriesi, 2016). Referendums are the most frequently used form of direct participation, and many European countries have used it to decide on participation in the European integration process (Hobolt, 2006; Hug and Sciarini, 2000), most recently manifested by the British ‘Brexit’ vote in 2016.

Attempts to increase citizen influence through direct democratic instruments have sometimes been viewed as a response by the elites to growing demand for alternative forms of participation. Thus, the expansion of direct democracy could be viewed as an attempt to ‘save’ representative democracy on behalf of the representatives (Donovan and Karp, 2006). Moreover, major changes in partisan dealignment and a shift toward postmaterialist voting has caused changes in the landscape of representative democracy, which have been advanced as explanations to the increasing use of referendums (Butler and Ranney, 1994; Setälä, 1999). Since the early 1970s, voters have become more volatile and more likely to shift parties between elections (Dassonneville, 2012). The growing risk of defection by disaffected voters in elections has made governments more willing to bring issues directly before the electorate. Consequently, political parties have become more willing to let citizens decide on contested issues. This logic has brought on referendums on a wide range of issues, such as nuclear power (in Austria, Bulgaria, Italy, and Sweden), gay marriage (Croatia, Ireland, and Slovenia), and different aspects of European integration, which were subject to more than 50 referendums between 1972 and 2016 (Tilindyte, 2016).

Three determinants of referendum mandate

From a procedural perspective, referendums could very well have the potential to increase fairness perceptions of political decisions compared to more indirect democratic procedures. Procedural fairness theory argues that citizens evaluate authorities with respect to their ability to make subjects realize that the decisions they make are based on fair decision-making procedures and their ability to deliver favorable outcomes for all (Lind and Tyler, 1988; Thibaut and Walker, 1975). The message from this literature to political science is that people indeed share common perceptions of what constitutes a fair decision-making procedure and that if such procedures are applied, people will be more likely to accept the decision (Daly and Tripp, 1996; Levi et al., 2009). In referendums, all voters have an equal chance to participate, voice their opinion, and have an influence on the specific decision outcome. Hence, the theoretical point of departure is the expectation that the citizens view any free and fair referendum as an equally strong mandate for implementing the outcome of the referendum.

Yet, people may also have more nuanced perceptions of the legitimacy of referendums, distinguishing between referendums based on the specific properties of each of them. We investigate whether citizens believe that an advisory referendum always provides a mandate for decision-making, or whether this mandate is conditional on properties of the referendums. We direct our attention to the size of majority, the level of turnout, and the favorability of the outcome.¹

Size of majority

Psychology scholars and political scientists have since Sherif (1936) and Lazarsfeld et al. (1948) observed that people tend to conform to the opinions of others. In a review of the literature on social influence and conformity, Cialdini and Goldstein (2004) identify the two main mechanisms for conformity to be informational and normative. Individuals tend to treat the knowledge of others’ opinions either as pure information relevant for their own opinions, or as a social pressure signaling what opinions they should hold themselves. While some people remain unaffected, others will align with the majority opinion out of a social desire to conform or a rational evaluation that the majority is more likely to be right. The theoretical underpinnings of these worries come both from rational choice theory and from established social psychological mechanisms at the individual level. First, from a rational choice perspective, citizens are in general cognitive misers, i.e. they rely on social cues from sources they perceive as knowledgeable as informational heuristics (Lupia and McCubbins, 1998). Faced with uncertainty and limited information, people use the majority opinion as a cognitive shortcut when making up their own mind. In certain circumstances, it can be rational for an individual to adhere to the majority position, acknowledging that others in general are more knowledgeable than they are themselves. Condorcet’s (1785) jury theorem implies that if members of a group are individually able to predict an uncertain outcome at a level better than chance, then a group of sincere voters, relying on a majority decision rule, will approach perfect accuracy as the number of members increases. People might thus comply with the majority if they believe in the collective’s ability to make better choices than the individual; this is also known as collective intelligence and the consensus heuristic mechanism (Landemore and Elster, 2012; Mutz, 1998; Surowiecki, 2004).

Second, the mechanism of normative social influence asserts that holding and expressing divergent attitudes is a social cost that people, all else equal, desire to avoid if they can (Kuran, 1997; Noelle-Neumann, 1974). Information about where the majority stands on an issue may thus impose social pressure on individuals to conform to the majority opinion.

The logic both of the informational and the normative social influence on conformity imply that size matters: If an individual uses the majority opinion for identifying the ‘correct’ view, this signal becomes stronger the larger the size of the majority is. Equally, the normative social pressure is stronger when the minority is vastly outnumbered than when the minority and majority are close to equal in size. However, citizens need not change their individual opinion to accept a collective decision outcome. A common-good basis of evaluation requires that citizens make a distinction between their own self-interest and the shared interests of the political community (Gilley, 2009). When outcomes offend our sensibilities or harm our interests, our response is conditioned by the concern of whether they are consistent with the shared interests of our political community. The size of majority is in this context a measure of certainty of what these shared interests are. A decision made in a referendum with low turnout and a small majority (of those participating) could arguably be seen less as a result of the general will than a referendum with high turnout and a strong majority. The larger the difference between the majority and minority, the more certain one can be that one of the outcomes is favored over the other by the group as a whole. The larger the majority, the more costly the effort will be for the minority group to challenge the decision outcome.

Acknowledging that political views vary among members of a society and that one’s own individual preferences have to yield from time to time is part of the democratic game. Coming to terms with an unfavorable outcome is—all else equal—likely to be easier the larger the size of majority.

H1: The smaller the size of the majority, the less legitimate the citizens perceive the referendum to be.

Level of turnout

A decision made in a referendum with low turnout and a small majority could arguably be seen less as a result of the general will than a referendum with high turnout and a strong majority. However, it is rare that the majority of votes in a referendum corresponds to the majority of the people. Several countries have quorum rules for turnout levels, implying a threshold of participation that must be met for the outcome of the referendum to be valid. There is a great deal of variation among countries with regard to quorum rules (for an overview, see Morel, 2017). In the European context, the Netherlands constitutes an interesting example. Since 1 July 2015, a referendum on any piece of primary legislation may be requested by the public before it enters into force. For a referendum to be held, the request needs to be signed by 300,000 citizens. To reject the law, turnout has to reach 30%. In the first vote, the Dutch voters rejected an EU–Ukraine association accord, with a turnout of 32% (and a majority of 62%). However, most countries do not apply turnout or approval quorums in referendums on EU-related issues. Indeed, in most referendums concerning EU or European Economic Community membership and various aspects of European integration, sizable majorities of the electorate have turned out to vote. Nevertheless, a substantial number of referendums have seen low levels of turnout. In 9 out of the 54 referendums in the period from 1972 to 2016, turnout fell below 50%. Three referendums recorded turnout levels of 35% or lower. Furthermore, in an additional 18 referendums, the turnout rates were between 50 and 60%, i.e. relatively low levels (Tilindyte, 2016).

We hypothesize that the legitimacy of a referendum is, to some extent, conditional on the level of turnout. The lower the turnout, the greater the deviation from the ‘will of the people’ (Qvortrup, 2002: 172). Low turnout contributes to a smaller majority and thus arguably to a lower level of perceived legitimacy:

H2: The lower the level of turnout, the less legitimate the citizens perceive the referendum to be.

Outcome favorability

Since they will have to be governed by those they disagree with, electoral losers are crucial players in the democratic game. Many studies have shown that individuals having voted for losing parties express lower levels of political support and trust than those voting for parties ending up in government (Anderson et al., 2005). The experimental evidence to date also suggests that outcome favorability overshadow the influence of procedural assessments (Arnesen, 2017; Esaiasson et al., 2016; Skitka et al., 2003). The importance of outcome favorability is illustrated with data from the 1994 Norwegian EU referendum survey. The study surveyed Norwegian citizens before, during, and after the 1994 referendum (see Moen et al., 2012; Saglie, 2000). Before the referendum, the respondents were asked whether they were for or against a Norwegian application for membership in the EU. After the referendum, they were asked about what decision-making procedure they preferred, i.e. decision by the parliament or popular vote. The data presented in Table 1 show that the ‘no’ voters—the referendum winners—overwhelmingly thought that the referendum result should be followed. Conversely, four out of 10 ‘yes’ voters preferred a decision made by the government.

Table 1.

Norway 1994 EU membership referendum.

	Referendum outcome
Decision-making preference	Unfavorable (%)	Favorable (%)
Prefer parliament	38	7
Prefer referendum	62	93
Sum	100	100

N Unfavorable = 794, N Favorable = 1322

Source: Moen et al. (2012).

Following this, we expect that the losers will be less willing than the winners to view the referendum as a mandate for action.

H3: Losers perceive the referendum as less legitimate than winners do.

Further, outcome favorability may also color citizens’ perceptions of what is a legitimate level of turnout or majority size. Individuals who receive an unfavorable outcome will assess objective procedural arrangements more negatively than those who receive a favorable outcome. Many social psychological experiments show that outcomes should matter more in the absence of fair procedures (Brockner, 2002; Brockner and Wiesenfeld, 1996). Daly and Tripp (1996) argue that in cases in which the procedural characteristics are lacking, procedural fairness is more sensitive to self-interest concerns. In politics, the true fairness or unfairness of a decision-making procedure is often a matter of debate (Doherty and Wolak, 2012), and with this wiggle room, losers have the opportunity to motivate their reasoning in a self-serving direction when assessing the decision-making process (Esaiasson et al., 2016). For instance, a recent study of a local referendum in Belgium revealed that the winners of the referendum thought the referendum was more fair than the losers thought (Marien and Kern, 2018). We therefore also explore potential interaction effects between outcome favorability and the two other dimensions (majority size and turnout) to investigate whether losing induces a perceptual bias when assessing the size of the majority and level of turnout as legitimizing factors of a referendum.

Research design

We explore how people evaluate referendums using a conjoint experiment in a probability-based online survey. The experiment portrays a scenario in which Norwegian EU membership is again subject to a referendum.

The case of Norway

Norway has held two referendums on European integration. In 1972, Norwegians voted on whether Norway should apply for membership in the European Community. A majority of 53.5% decided against applying for membership, and 79.2% of the electorate voted. In 1994, another referendum was held concerning membership in the EU. A total of 52.2% of the voters once again said ‘no,’ with a turnout of 88.6%.

In Norway, referendums are only advisory, but it is customary to follow the result. Thus, our hypothetical referendum should be regarded as a case of ad hoc referendum (Qvortrup, 2018). Hence, the case selection is based on the familiarity of the scenario for the Norwegian population and the possible generalizability to cases in other European countries.

Survey embedded conjoint experiment

The observational data from the 1994 Norwegian EU survey illuminate an important part of our research question, but it does not tell the full story. We only know how the pro-EU voters reacted to an unfavorable outcome, since both referendums ended with a rejection of applying for EU membership. Would the anti-EU voters have reacted differently if they had ended up on the losing side? Moreover, both referendums had a very high turnout level; how would the citizens have reacted if the turnout had been significantly lower? And how would the citizens have reacted to a large majority? To address these questions, we conduct an experiment that varies these conditions in imagined scenarios.

As our aim is to present the respondents with several different ex post attributes of the referendum and let them evaluate its legitimacy, the appropriate design to apply is a conjoint experiment. While regular experiments typically expose subjects to one or two treatments, conjoint experiments can handle complex choice situations with multiple treatments given simultaneously to subjects (Hainmueller et al., 2014). According to one of the pioneers of survey experiments in political science, Paul Sniderman (2018), conjoint experiments are perhaps the most promising design innovation in survey experiments. Contrary to traditional full factorial experiments, conjoint experiments are fractional factorial, which means that not all possible vignette variations are necessarily assigned to the respondents. The treatment effects are estimated based on the vignette sample that is randomly drawn from the vignette population. It can therefore handle a wider range of treatments within the same set-up than full factorial experiments are able to.

Conjoint experiments are also known as factorial survey experiments and vignette analysis (Rossi et al., 1974) and have been used in social science research for decades, although sparingly. Their applications include among others the evaluation of fairness of income (Alves and Rossi, 1978), explaining attitudes toward gender pay gaps (Auspurg et al., 2017), and in the study vote choice (e.g. Carnes and Lupu, 2015; Kirkland and Coppock, 2018). We should note that we apply a ratings-based conjoint, where respondents are presented with only one profile at a time and asked to assess whether they consider that referendum should be followed or not. This protocol has been applied among other to measure how the public defines terrorism (Huff and Kertzer, 2018), but differs from choice-based conjoint experiments typically employed in political science, where respondents are presented pairs of profiles and asked to choose between them. We consider the ratings-based conjoint to more closely approximate the real-world situation we attempt to study, where citizens assess the legitimacy of a referendum one by one, and not in direct comparison to another referendum. This experimental design thus allows us to vary the favorability of the outcome as well as turnout and majority size. With a conjoint experimental design, we complement the limited observations of the real world with hypothetical scenarios.

The NCP

The experiment was implemented in the NCP during the fall of 2016, with a total of 1043 participating respondents (Ivarsflaten, 2016). The NCP is a probability-based general population online survey panel administered by the Digital Social Science Core Facility (DIGSSCORE) at the University of Bergen.² The panel recruits subjects through random sampling from the official national population registry. The appeal of survey experiments in probability samples comes from the possibility of making causal inferences from a representative sample of the population (Mutz, 2011). For further details on response rates or other methodological questions about the data, we refer to the NCP methodology reports (Skjervheim and Høgestøl, 2017).

Experimental design

Prior to the experiment, respondents are asked whether they support or oppose Norwegian membership in the EU (yes/no). Each respondent is then presented with a hypothetical referendum on EU membership. We present them with the following (translated) vignette:

We are interested in examining what mandate the government needs to make important decisions on behalf of the people. Imagine that there would be a new debate regarding EU membership for Norway and that an advisory referendum was held regarding the issue.

$|$ referendum_description

In such a case, should the government act according to the result of the referendum, even if the majority in Parliament disagrees?

Yes

where

|

referendum_description is a sentence describing the ex post properties of the referendum. The hypothetical referendum varies in the size of turnout, the size of the majority, and which side won (outcome), with each of these pieces of information having a .5 probability of being shown. The possible treatment values are shown in Table 2. The referendum had

5 \times 4 \times 3 = 60

possible descriptions. All descriptions are shown in the Online appendix. Thanks to the large number of respondents in the survey panel, each respondent only had to evaluate one task, ensuring that the units of observation are independent from each other at the respondent level.

Table 2.

The different treatments and their possible values.

Treatment	Value
Outcome	Not shown outcome
	Against EU membership won
	For EU membership won
Size of majority	Not shown majority
	51%
	55%
	70%
Level of turnout	Not shown turnout
	35%
	47%
	53%
	85%

The treatment levels are chosen to cover a wide range of possible referendums. The specific turnout levels are chosen to represent a spectrum ranging from a referendum with high turnout (85%) to one with very low turnout (35%). The high-level category is close to the actual turnout in the last Norwegian advisory referendum, which was 88.6%. The lowest level corresponds to the lowest levels of turnout in referendums on European integration. The mid-levels are intended to capture any threshold effect where slightly above or below half of the electorate turn out to vote, which has been a fairly frequent turnout rate in referendums on European integration.

The size of the majority ranges from a narrow majority (51%), a clear majority below typical supermajoritarian thresholds (55%), to a clear super-majority (70%). A majority of 70% is in-between typical supermajoritarian thresholds, such as two-thirds (i.e. 67%) or three-quarters (i.e. 75%). For size of majority and level of turnout, the respondents were randomly assigned into groups that were either shown or not shown any values on these dimensions. Those that were shown a value were randomly assigned to one of the values listed in Table 2. For the outcome dimension, the respondents were randomly assigned to either not being shown the outcome, to an outcome where the anti-EU side won, or to an outcome where the pro-EU side won.

Legitimacy is often measured as a property of an action or a decision ex post facto. It may be a value-based measure of legitimacy about how willing the respondents are to comply with the outcome (Arnesen and Peters, 2018) or it may be a behavioral measure where subjects help or hinder the implementation of a decision (Dickson et al., 2015).³ To be able to identify the effect of the referendum’s outcome, the control groups will not be shown the outcome. It makes little sense to evaluate an action or decision without knowing what that action or decision is. Instead, the respondents are asked to decide if the government should act according to the result of the referendum. As such, we apply a prospective measure of legitimacy. In addition, we add the qualifier that they should do so regardless of what the majority in Parliament wants. Hence, for our purpose, we operationalize the legitimacy of the referendum as whether it is evaluated as a necessary and sufficient condition for acting on behalf of the people.

Identification

As discussed above, the null hypothesis stipulates that support for implementing the result is independent of the type of contextual variations described above. We hypothesize that the size of turnout, the size of the majority, and the favorability of the outcome will affect the evaluation of legitimacy. Following Hainmueller et al. (2014), we test each of these by estimating the average marginal component effect (AMCE) for each treatment value, with the no-information-showed condition as control. To correctly identify the AMCEs, we make certain assumptions (see Bansak et al., 2017; Hainmueller et al., 2014), including the orthogonality of the different treatments. For example, we assume that the order in which we present the treatments (which we do not randomize since they are presented in a sentence format) does not affect the estimates.

For both the size of turnout and the size of the majority, we expect that a lower value has a lower AMCE. For example, formally, we expect that $β_{at 35 %}^{turnout} < β_{at 85 %}^{turnout}$ and $β_{of 51 %}^{majority} < β_{of 70 %}^{majority}$ . We also hypothesize that a favorable outcome will create a more positive evaluation of legitimacy than an unfavorable outcome. Formally, we expect that $β_{unfavorable}^{outcome} < β_{favorable}^{outcome}$ . To measure ‘outcome favorability,’ we match the respondents’ (pretreatment) stated preference with the outcome of the referendum, such that a favorable outcome means they are the same and an unfavorable outcome means they are not.

Results

The expected probability of thinking that the government should follow the referendum when we do not show any information about how it fared is 83%, with a 95% confidence interval of 77.5–88.7. Thus, without being given explicit information about the turnout, the size of the majority, or the winning side, a large majority would consider the referendum legitimate.

Figure 1 shows that all treatments affect people’s propensity to evaluate the referendum as legitimate. The figure shows the difference in the probability of thinking that the government should follow the referendum for each treatment value with no-information-shown as control condition (AMCEs). The results clearly show that knowing the level of turnout affects the legitimacy of the referendum, providing strong support for our first hypothesis. When the turnout is 85%, the probability is not significantly different from when we show no information about turnout. When the turnout is 35%, however, the probability drops by 33.5 percentage points [−41.7, −25.4].

Figure 1.

The effect of different referendum properties on the probability of thinking that the government should follow the referendum (AMCEs). Note: The dots are the point estimates of the AMCEs, and the bars show their 95% confidence intervals. The reference category is shown by a point on the dotted line.

The data also provide support for the second hypothesis of a positive relationship between majority size and perceived legitimacy. Going from the control condition to a majority of 51% decreases the probability by 12.4 percentage points [−19.7, −5].

The third hypothesis also receives strong support. Those on the losing side are significantly less likely to accept the decision compared to those on the winning side. Compared to the control condition, an unfavorable outcome lowers the probability by 8.2 percentage points [−14.5, −1.9] while a favorable outcome increases the probability by 10.1 percentage points [3.8, 16.5].

The bounds of referendum legitimacy

Figure 2 shows the expected probability of thinking that the government should follow the referendum for different combinations of referendum properties. The probabilities are estimated from a logistic regression model.

Figure 2.

The expected probability of thinking that the government should follow the referendum for different combinations of turnout, majority size, and outcome favorability. Note: The dots are point estimates, and the bars show their 95% confidence intervals. These estimates are from a logistic regression model. The plot is split by outcome favorability (columns) and majority size (rows). Within each cell, i.e. for referendums with a particular outcome favorability and majority size, the x-axis shows estimates at different levels of turnout.

This provides some sense of the bounds of legitimacy in our referendum case. With a high turnout and a favorable outcome, the proportion evaluating the referendum as legitimate approaches everyone. However, with low turnout and a small majority, approximately half of those who received a favorable outcome, and about one-third of those who received an unfavorable outcome, would evaluate it as legitimate. The results show that, even in a case such as Norway, where there would be no expectation of electoral misconduct, a referendum can go from being considered a clear decision mandate to clearly not so due to variation in the level of turnout, majority size, and outcome favorability.

The role of outcome favorability

Figure 3 shows the interaction effect between outcome favorability and the effect of majority and turnout. These are the conditional AMCEs, similar to Figure 1, but estimated separately for when the randomized outcome was favorable compared to unfavorable. It shows that the size of the majority affects legitimacy when the outcome is perceived as unfavorable rather than favorable, while the effect of turnout remains similar. When either perceived as favorable or not shown, the size of the majority has little or no effect on the legitimacy of the referendum. When perceived as unfavorable, however, going from the control condition to a majority of 51% decreases the probability of thinking that the government should follow the referendum by 24 percentage points [−36, −9]. These results support the notion of a perceptual bias in assessing the legitimacy of the referendum. Similar tendencies are observed in the interaction between turnout and outcome favorability, however not as pronounced.

Figure 3.

The three columns show the conditional AMCEs when the outcome is either favorable, not shown, or unfavorable. Note: The dots are the point estimates of the conditional AMCEs, i.e. the conditional effect of that particular referendum attribute on the probability of thinking that the government should follow the referendum. The bars represent 95% confidence intervals of the point estimates.

Discussion and conclusion

When defending the narrow victory in the 2017 Turkish referendum on a constitutional reform package, President Erdogan invoked a football analogy, saying ‘It doesn’t matter if you win 1–0 or 5–0. The ultimate goal is to win the game’ (Anderson and Masters, 2017). Yet, we have unveiled that the margin by which you win in a referendum indeed matters. When the turnout is low, and the size of the majority is small, the power of the parliament to carry out a decision is weaker.

Democratic political decision-making will necessarily produce winners and losers, and the crucial issue is to establish democratic procedures that the losers perceive as fair. In politics, the outcome also influences the perceptions of the decision-making procedure itself. When the referendum outcome is favorable, the referendum is viewed as a mandate for action. Yet, an identical referendum with an unfavorable outcome is perceived to be less so. Thus, the outcome itself constitutes an important part of the assessment of referendums as a legitimate political decision-making procedure. Winners and losers have different perceptions of the legitimacy of a referendum.

The outcome favorability bias works in two ways. First, it has a main effect on the perceived legitimacy of a referendum. That is, citizens are substantially less willing to accept a referendum that goes against their preferences than one that accords with their preferences. This is in itself unsurprising and in line with previous research. It might be more surprising that so many are willing to accept an unfavorable outcome on such an important issue as EU membership. Clearly, in an established democracy such as Norway, the majority of citizens are willing to adhere to democratic norms and accept that in a democratic society, unfavorable outcomes are a part of the game.⁴

Second, outcome favorability interacts with the perceptions of what is a legitimate majority size, whereby losers are more reluctant to accept a small majority. This supports the notion that unfavorable outcomes induce a perceptual bias about the fairness of the decision procedure. While the winners do not differentiate between the majority sizes, the losers have significantly more difficulties in coming to terms with a narrow loss than a clear loss.

The large discrepancy between winners and losers in how they evaluate the result of a referendum shows that getting people to agree on what is a fair democratic decision-making procedure will be difficult, if not impossible. Thus, in politics, there may be no such thing as an acceptable decision to all. Not all citizens necessarily view a victory as a victory, and in certain cases, members of parliament on the losing side may have significant public support when refusing to follow the outcome of the referendum.

All the scenarios in the experiment are realistic situations that frequently occur in referendums, and none of them question the legality of the referendum as such. Yet, the sizes of the treatment effects may be particular to the issue and the context; referendums about less polarized issues and in other contexts may be regarded differently by the citizens. Citizens’ experiences with referendums and their perceptions of public opinion distributions are likely to create anticipations about the level of turnout and the likely outcome. Citizens of, say, Switzerland may accept lower turnout levels given that they issue several referendums every three months and that their reference point of a normal turnout may be lower than for an EU referendum in Norway. A referendum with less political implications may be more acceptable because the outcome matters less to the citizens.

Nevertheless, even though we view this experiment to sufficiently answer our research question, there are possible limitations of this research design. The external validity of survey experiments is always subject to criticism. The proposed scenarios are hypothetical, and the behavior of the respondents may be different should such a scenario really happen to occur in the future. However, Hainmueller et al. (2015) successfully validated conjoint experiments regarding preferences for immigrants with a natural experiment of real-world referendums as a behavioral benchmark. Comparing the results from conjoint and vignette experiments on which attributes of hypothetical immigrants generate support for naturalization with the outcomes of closely corresponding referendums in Switzerland, they find that the effects estimated from the surveys match the effects of the same attributes in the behavioral benchmark remarkably well. Their findings suggest that conjoint experiments are likely to represent well how the subjects would have reacted in real-world referendums. Yet, we do not possess observational data that serve as a benchmark for our referendum study and must remain wary of this fact.

Further research on the legitimacy of political decision-making procedures should investigate how, if at all, citizens can agree upon how to make acceptable collective decisions. With regard to referendums, research is warranted on (a) the mechanisms that may account for the positive relationship between turnout and majority size and the legitimacy of a referendum; (b) to what extent the referendum issue and context moderate the influence of the three attributes of turnout, majority size, and outcome favorability; (c) how ex ante agreements about turnout and majority quorum rules in referendums influence acceptance of the result; and (d) what other attributes can influence the legitimacy of a referendum.

This study sheds new light on what mandate advisory referendums on EU membership confer on a government. In light of the recent Brexit vote, our study thus reveals intriguing insights that are relevant for current political events: In general, people think a referendum gives a mandate to a political decision, but people’s judgments are qualified by the level of turnout and the size of the majority.

Supplemental Material

Supplemental material for Conditional legitimacy: How turnout, majority size, and outcome affect perceptions of legitimacy in European Union membership referendums

Supplemental Material for Conditional legitimacy: How turnout, majority size, and outcome affect perceptions of legitimacy in European Union membership referendums by Sveinung Arnesen, Troy S Broderstad, Mikael P Johannesson and Jonas Linde in European Union Politics

Footnotes

Acknowledgements

We are grateful to Mike Tomz and Paul Sniderman for highly valuable comments to previous versions of the article. We also would like to thank the participants at the 2017 Barcelona-Gothenburg-Bergen Workshop on Experiments in Political Science and at the 2017 Annual Conference of the European Political Science Association for their input.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was partially funded by the Research Council of Norway (project number 262986).

Supplemental material

Supplemental material for this article is available online.

Notes

References

Alves

Rossi

(1978) Who should get what? Fairness judgments of the distribution of earnings. American Journal of Sociology 84(3): 541–564.

Anderson

Masters

(2017) Exclusive: Erdogan denies dictator charges. CNN. Available at: https://www.google.no/amp/s/amp.cnn.com/cnn/2017/04/18/europe/erdogan-turkey-interview/index.html (accessed 18 April 2017).

Anderson

Blais

Shaun Bowler

et al. (2005) Losers’ Consent: Elections and Democratic Legitimacy. Oxford: Oxford University Press.

Arnesen

(2017) Legitimacy from decision-making influence and outcome favourability: Results from general population survey experiments. Political Studies 65(1): 146–161.

Arnesen

Peters

(2018) The legitimacy of representation. How descriptive, formal, and responsiveness representation affect the acceptability of political decisions. Comparative Political Studies 51(7): 868–899.

Arrow

(2012) Social Choice and Individual Values. New Haven, CT: Yale University Press.

Auspurg

Hinz

Sauer

(2017) Why should women get less? Evidence on the gender pay gap from multifactorial survey experiments. American Sociological Review 82(1): 179–210.

Bansak K, Hainmueller J, Hopkins DH, et al. (2018) The Number of Choice Tasks and Survey Satisficing in Conjoint Experiments. Political Analysis 26(1): 112–119.

Bowler

Donovan

Karp

(2007) Enraged or engaged? Preferences for direct citizen participation in affluent democracies. Political Research Quarterly 60(3): 351–362.

10.

Brockner

(2002) Making sense of procedural fairness: How high procedural fairness can reduce or heighten the influence of outcome favorability. Academy of Management Review 27(1): 58–76.

11.

Brockner

Wiesenfeld

(1996) An integrative framework for explaining reactions to decisions: Interactive effects of outcomes and procedures. Psychological Bulletin 120(2): 189.

12.

Butler

Ranney

(1994) Referendums Around the World: The Growing Use of Direct Democracy. Washington DC: The AEI Press.

13.

Carnes

Lupu

(2015) Rethinking the comparative perspective on class and representation: Evidence from Latin America. American Journal of Political Science 59(1): 1–18.

14.

Christensen

Karjalainen

Nurminen

(2015) Does crowdsourcing legislation increase political legitimacy? The case of Avoin Ministeriö in Finland. Policy & Internet 7(1): 25–45.

15.

Cialdini

Goldstein

(2004) Social influence: Compliance and conformity. Annual Review of Psychology 55: 591–621.

16.

Condorcet

(1785) Essay on the Application of Analysis to the Probability of Majority Decisions. Paris: Imprimerie Royale.

17.

Conradt

List

(2009) Group decisions in humans and animals: A survey. Philosophical Transactions of the Royal Society of London B: Biological Sciences 364(1518): 719–742.

18.

Conradt

Roper

(2003) Group decision-making in animals. Nature 421(6919): 155.

19.

Couzin

Krause

Franks

et al . (2005) Effective leadership and decision-making in animal groups on the move. Nature 433(7025): 513.

20.

Dahl

(1989) Democracy and Its Critics. New Haven, CT: Yale University Press.

21.

Daly

Tripp

(1996) Is outcome fairness used to make procedural fairness judgments when procedural information is inaccessible? Social Justice Research 9(4): 327–349.

22.

Dassonneville

(2012) Electoral volatility, political sophistication, trust and efficacy: A study on changes in voter preferences during the Belgian regional elections of 2009. Acta Politica 47(1): 18–41.

23.

De Vreese

Boomgaarden

(2005) Projecting EU referendums: Fear of immigration and support for European integration. European Union Politics 6(1): 59–82.

24.

Dickson

Gordon

Huber

(2015) Institutional sources of legitimate authority: An experimental investigation. American Journal of Political Science 59(1): 109–127.

25.

Doherty

Wolak

(2012) When do the ends justify the means? Evaluating procedural fairness. Political Behavior 34(1): 301–323.

26.

Donovan

Karp

(2006) Popular support for direct democracy. Party Politics 12(5): 671–688.

27.

Esaiasson

(2011) Electoral losers revisited – How citizens react to defeat at the ballot box. Electoral Studies 30(1): 102–113.

28.

Esaiasson

Persson

Gilljam

et al . (2016) Reconsidering the role of procedures for decision acceptance. British Journal of Political Science. DOI: 10.1017/S0007123416000508.

29.

Ferrin

Kriesi

(eds) (2016) How Europeans View and Evaluate Democracy. Oxford: Oxford University Press.

30.

Gilley

(2009) The Right to Rule: How States Win and Lose Legitimacy. New York: Columbia University Press.

31.

Habermas

(2015) Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. Hoboken, NJ: John Wiley & Sons.

32.

Hainmueller

Hangartner

Yamamoto

(2015) Validating vignette and conjoint survey experiments against real-world behavior. Proceedings of the National Academy of Sciences 112(8): 2395–2400.

33.

Hainmueller

Hopkins

Yamamoto

(2014) Causal inference in conjoint analysis: Understanding multidimensional choices via stated preference experiments. Political Analysis 22(1): 1–30.

34.

Hastie

Kameda

(2005) The robust beauty of majority rules in group decisions. Psychological Review 112(2): 494.

35.

Hobolt

(2006) Direct democracy and European integration. Journal of European Public Policy 13(1): 153–166.

36.

Hobolt

(2009) Europe in Question: Referendums on European Integration. Oxford: Oxford University Press.

37.

Huff

Kertzer

(2018) How the public defines terrorism. American Journal of Political Science 62(1): 55–71.

38.

Hug

Sciarini

(2000) Referendums on European integration: Do institutions matter in the voter’s decision? Comparative Political Studies 33(1): 3–36.

39.

Ivarsflaten, EDT (2016) Norwegian citizen panel wave 7. Technical report. Data available at The Norwegian Center for Research Data, first NSD edition. Bergen: University of Bergen and UNI Research Rokkan Centre.

40.

Kirkland

Coppock

(2018) Candidate choice without party labels. Political Behavior 40(3): 571–591.

41.

Kuran

(1997) Private Truths, Public Lies: The Social Consequences of Preference Falsification. Cambridge, MA: Harvard University Press.

42.

Landemore

Elster

(2012) Collective Wisdom: Principles and Mechanisms. Cambridge: Cambridge University Press.

43.

Lazarsfeld PF, Berelson B and Gaudet H (1948) The people's choice: How the voter makes up his mind in a presidential campaign. New York, NY: Columbia University Press.

44.

Leemann

Wasserfallen

(2016) The democratic effect of direct democracy. American Political Science Review 110(4): 750–762.

45.

Levi

Sacks

Tyler

(2009) Conceptualizing legitimacy, measuring legitimating beliefs. American Behavioral Scientist 53(3): 354–375.

46.

Lind

Tyler

(1988) The Social Psychology of Procedural Justice. New York and London: Plenum Press.

47.

Lupia

McCubbins

(1998) The Democratic Dilemma: Can Citizens Learn What They Need To Know? Cambridge: Cambridge University Press.

48.

Mansbridge J (2015) A minimalist definition of deliberation. In: Heller P and Deliberation and Rao V (eds) Development: Rethinking the Role of Voice and Collective Action in Unequal Societies. Washington DC: World Bank Group, pp.27–50.

49.

Marien

Kern

(2018) The winner takes it all: Revisiting the effect of direct democracy on citizens’ political support. Political Behavior 40(4): 857–882.

50.

Moen

Øyangen

Jenssen

(2012) The Norwegian 1994 EU Referendum Study. 2nd ed. Bergen: NSD.

51.

Morel

(2017) Types of referendums, provisions and practice at the national level worldwide. In: Morel

Qvortrup

(eds) The Routledge Handbook to Referendums and Direct Democracy. London: Routledge, pp.27–59.

52.

Morel

Qvortrup

(2017) The Routledge Handbook to Referendums and Direct Democracy. London and New York: Routledge.

53.

Mutz

(1998) Impersonal Influence: How Perceptions of Mass Collectives Affect Political Attitudes. Cambridge: Cambridge University Press.

54.

Mutz

(2011) Population-Based Survey Experiments. Princeton, NJ: Princeton University Press.

55.

Noelle-Neumann

(1974) The spiral of silence a theory of public opinion. Journal of Communication 24(2): 43–51.

56.

Persson

Esaiasson

Gilljam

(2013) The effects of direct voting and deliberation on legitimacy beliefs: An experimental study of small group decision-making. European Political Science Review 5(3): 381–399.

57.

Qvortrup

(2002) Government by the people: A comparative study of referendums. Manchester: Manchester University Press.

58.

Qvortrup M (2018) Introduction: Theory, practice and history. In: Qvortrup M (ed) Referendums Around the World. Basingstoke: Palgrave Macmillan, pp.1–18.

59.

Rossi

Sampson

Bose

, et al. (1974) Measuring household social standing. Social Science Research 3(3): 169–190.

60.

Saglie

(2000) Values, perceptions and European integration: The case of the Norwegian 1994 referendum. European Union Politics 1(2): 227–249.

61.

Scarrow

(2001) Direct democracy and institutional change. Comparative Political Studies 34(6): 651–665.

62.

Scharpf

(1999) Governing in Europe: Effective and Democratic? Oxford: Oxford University Press.

63.

Sen

(2017) Collective Choice and Social Welfare. Expanded edition. London: Penguin UK.

64.

Setälä

(1999) Referendums in Western Europe – A wave of direct democracy? Scandinavian Political Studies 22(4): 327–340.

65.

Sherif

(1936) The Psychology of Social Norms. Oxford: Harper.

66.

Skitka

Winquist

Hutchinson

(2003) Are outcome fairness and outcome favorability distinguishable psychological constructs? A meta-analytic review. Social Justice Research 16(4): 309–341.

67.

Skjervheim Ø and Høgestøl A (2017) Norwegian citizen panel, seventh wave methodology report. Ideas2Evidence.

68.

Sniderman

(2018) Some advances in the design of survey experiments. Annual Review of Political Science 21: 259–275.

69.

Surowiecki

(2004) The Wisdom of Crowds. London: Little, Brown Books.

70.

Thibaut

Walker

(1975) Procedural Justice: A Psychological Perspective. Hillsdale, NJ: Erlbaum.

71.

Thompson

(2008) Deliberative democratic theory and empirical political science. Annual Review of Political Science 11: 497–520.

72.

Tilindyte

(2016) Referendums on EU issues. 582.041. Brussels: European Parliamentary Research Service (EPRS).

73.

Tyler

(2006) Why People Obey the Law. Princeton, MA: Princeton University Press.

74.

Weatherford

(1992) Measuring political legitimacy. American Political Science Review 86(1): 149–166.

75.

Weber

(2009) The Theory of Social and Economic Organization. New York: Simon and Schuster.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

6.54 MB

0.00 MB

Conditional legitimacy: How turnout,majority size,and outcome affect perceptions of legitimacy in European Union membership referendums

Abstract

Keywords

Introduction

Democratic decision-making and legitimacy

The legitimacy of collective decisions

The concept of referendums

Three determinants of referendum mandate

Size of majority

Level of turnout

Outcome favorability

Research design

The case of Norway

Survey embedded conjoint experiment

The NCP

Experimental design

Identification

Results

The bounds of referendum legitimacy

The role of outcome favorability

Discussion and conclusion

Supplemental Material

Supplemental material for Conditional legitimacy: How turnout, majority size, and outcome affect perceptions of legitimacy in European Union membership referendums

Footnotes

Acknowledgements

Funding

Supplemental material

Notes

References

Supplementary Material