Referenda as commitment devices – an experimental approach

Abstract

This research note evaluates the claim that referenda can serve as useful commitment devices in international negotiations. More specifically, we relied on individual-level survey data to test the claim that governments can successfully “tie their hands” to policy choices by calling referenda on political issues. Our empirical analysis relied on original survey data collected in April 2019 in Belize. In so doing, we took advantage of an unusual political event. On 8 May (shortly after our survey), Belizean citizens participated in a countrywide plebiscite. During this vote, they decided to send their country’s territorial dispute with Guatemala for adjudication to the International Court of Justice. From a research perspective, this event allowed us to assess the effect of disregarded referendum results in a highly salient political environment. Our experimental analysis suggested that individuals do reprimand their governments for failing to implement a majority vote (a) even if this choice precipitates a person’s favored substantive outcome, and (b) irrespective of an individual’s preferred party.

Introduction

This research note evaluates the claim that referenda can serve as useful commitment devices in international negotiations. Increasingly, states have held referenda on critical foreign policy questions ranging from international treaties (Ireland 2012), to ongoing border disputes (Slovenia 2010), and EU membership (UK 2016). While there are many reasons for governments to hold these plebiscites, a frequently hypothesized key motivation is to improve a state’s bargaining leverage. According to this logic, the prospect of a national referendum enhances the credibility of a government’s commitment, elevates its clout at the negotiation table, and thereby increases its capacity to extract concessions (Cheneval and Ferrín, 2018). While the argument is intuitive, the empirical evidence for it is mixed. Some studies suggest that the announcement of a referendum can indeed help a state to achieve better negotiation outcomes (Hug and Schulz, 2007). Other analyses cast doubt about the effectiveness of this strategy (Oppermann, 2013). Thus, the utility of referenda as commitment devices is empirically disputed.

A key assumption underpinning the “referenda as commitment device” claim is the assertion that governments incur a political cost from disregarding the outcome of a popular vote. If leaders do not face a public backlash for refusing to implement a plebiscite, then holding a referendum on a given issue would not increase a state’s bargaining power. In this study, we relied on survey data to test the assumption that governments can successfully “tie their hands” to policy choices by calling referenda on political issues. To the best of our knowledge, this paper constitutes the first investigation into this question.

Referenda as commitment devices

While previous scholarship has not systematically examined the attitudinal consequences of disregarded referendum results, two existing bodies of research implicitly offer competing predictions about people’s reactions in these scenarios.

To begin, experimental scholarship on “audience costs” investigates voter reactions when political leaders back out of meaningful foreign policy commitments (Kertzer and Brutger, 2016). These studies deploy variations of a hypothetical foreign policy scenario in which respondents are asked to convey their level of support for leaders who either follow through or renege on their foreign policy commitments. The research design in this literature deliberately deploys neutral language and avoids mention of actual political leaders or partisan affiliations (e.g., Kertzer and Brutger, 2016). The underlying motivation for this methodological choice is to minimize the degree to which responses reflect a subject’s partisan evaluation of the leader, as this could contaminate the experiment by masking the degree to which they care about shifts in policy. Relying on this approach, studies on audience costs suggest that voters punish governments for backing out of prior policy pronouncements. This implies that citizens should also respond negatively to disregarded referendum outcomes, given that ignoring such a vote is a quite extreme form of policy reversal (one that may even be argued to violate democratic norms).

However, when applied to the literature on foreign policy referenda, these studies are not without problems. In national referenda, partisan dispositions are potentially important lenses through which people interpret political decisions, given that the issues under consideration tend to be highly salient. This is significant because it constitutes a potential external validity concern in existing research on audience costs. Recent public opinion research on so-called flip-flopping strongly suggests that the presumed mechanism underpinning audience costs – that voters punish executives who break with previous foreign policy commitments – washes out during the tug-of-war of real-world political campaigns. Put differently, once the psychology of party identification enters the picture, flip-flops seem to have little net effect on public opinion; voters often forgive or rationalize policy changes by an executive that they currently support, and opponents have already formed a robust negative evaluation (McDonald et al., 2019). Indeed, there is evidence that flip-flopping may even elicit increased support from both supporters and detractors if the shift is in the direction of the individual voter’s prior preference (Croco, 2016). This is true even when people have reason to believe that the switch in policy is politically motivated (McGraw et al., 2002). Taken as a whole, this body of research suggests that substantial sectors of the electorate may not punish a government’s decision to reject the outcome of a referendum. If true, then there are credible reasons to doubt the utility of referenda as commitment devices.

Data and methods

To evaluate these competing claims, we relied on original survey data collected April 2019 in Belize. ¹ In so doing, we took advantage of an unusual political event. Shortly after our survey (8 May), Belizean citizens participated in a countrywide referendum in which the majority of voters (55.4%) decided to send their state’s territorial dispute with Guatemala for adjudication to the International Court of Justice (ICJ). This means that the World Court will now rule on the future border between both states. This case allowed us to explore the effect of referenda in a highly salient political environment. Belize’s national government had actively campaigned in favor of ICJ adjudication. The main opposition party had advocated against sending the case to the court.

Prior to the plebiscite, we conducted a survey in Belize (3–6 April 2019). In order to collect our sample, we partnered with PollBelize, a local survey firm. Relying on random-digit-dialing of cellphone numbers, PollBelize collected a representative sample of the country’s adult population (n = 787).

From a respondent’s perspective, government rejection of a referendum can present itself in two ways: (a) the government refuses to implement a referendum result when the subject votes with the minority; (b) the government refuses to implement a referendum result when the subject votes with the majority. In this paper, we only focus on the former scenario. We did this for two reasons. First, this choice allowed us to maximize statistical power. Second, this scenario was more interesting conceptually. If citizens punish a government for disregarding a plebiscite, even if this choice is consistent with their own substantive policy preferences, this would provide strong evidence for the referenda as commitment devices argument. In other words, if people punish the government for reneging on an undesirable commitment, they should be just as likely to punish their leadership for walking back on a desirable promise.

Our statistical analysis was based on an experiment. Empirically, we needed to isolate the effect of the government’s rejection of a referendum vote from the effect of a person obtaining her desired substantive policy (ICJ adjudication yes/no). We achieved this in the following way. First, respondents were asked how they planned to vote in the plebiscite. Answer options were “Yes” (in favor of ICJ), “No” (opposed to ICJ), “I don’t know,” and “I will not vote.” Second, we presented “yes-” and “no” voters with a potential post-referendum scenario that resulted in their preferred substantive outcome. However, for some respondents the final policy choice was portrayed as the government’s decision to implement a referendum result (the control group) while for others, the substantive outcome was depicted as the government rejecting a referendum vote (the treatment). The two (randomly assigned) vignettes for ICJ supporters were:

Group 1 (government implements majority vote): Suppose that the majority of Belizeans vote in favor of going to the ICJ during the referendum, and the government decides to accept the outcome of the referendum and take the case to the ICJ.

Group 2 (government rejects majority vote): Suppose that the majority of Belizeans vote against going to the ICJ during the referendum, but the government decides to reject the outcome of the referendum and take the case to the ICJ.

ICJ opponents were randomly assigned to Groups 3 or 4.

Group 3 (government implements majority vote): Suppose that the majority of Belizeans vote against going to the ICJ during the referendum, and the government decides to accept the outcome of the referendum and not take the case to the ICJ.

Group 4 (government rejects majority vote): Suppose that the majority of Belizeans vote in favor of going to the ICJ during the referendum, but the government decides to reject the outcome of the referendum and not take the case to the ICJ.

After hearing this prompt, we asked individuals whether they would approve (coded as “1”), disapprove (coded as “0”), or neither approve nor disapprove (also coded as “0”) of the government’s decision.

Results

To begin, we conducted difference-of-means tests. For both ICJ supporters and ICJ opponents, we compared response patterns across the “implemented majority vote” and “rejected majority vote” conditions. Findings are presented in Figure 1.

Figure 1.

Treatment effects for referendum rejection: (a) ICJ opponents and (b) ICJ supporters.

We see that individuals punish the government for failing to implement a referendum result. For ICJ opponents (Figure 1(a)), the estimated approval-rate-difference was 18.9% (t = 2.78; p < 0.01), for ICJ supporters (Figure 1(b)) it was 38.6% (t = 7.60; p < 0.01). ² These results suggest that – on aggregate – citizens penalize leaders for disregarding a plebiscite even if this step brings about a person’s preferred substantive outcome. This provides initial evidence for the claim that referenda allow governments to “tie their hands” to policy choices. In theory however, it is possible that all of the observed backlash occurs exclusively among opponents of the sitting government. If true, this would suggest that the substantive political cost in such a scenario remains quite limited. If only opposition supporters condemn the failure to implement a majority vote, the “real world” policy implications of such a move would be modest. In order to explore these dynamics, we interacted our binary treatment-indicator with a variable that measures people’s partisan identities. The latter was coded as “1” for subjects who identify with Belize’s majority party in government, “2” for followers of an opposition party, and “3” for political independents. Details are provided in the Online Appendix. Taken as a whole, our statistical models did not find any evidence for an interactive relationship. In other words, people punished their leaders for rejecting an undesirable referendum result, regardless of their own political affiliations.

Conclusion

In this research note, we have contributed to the research agenda on referenda as commitment devices. In particular, we tested the claim that rulers incur costs from disregarding the results of a given plebiscite. Relying on original survey data from Belize, we found that individuals did reprimand their governments for failing to implement a majority vote (a) even if this choice precipitated a person’s favored substantive outcome, and (b) irrespective of an individual’s preferred party. Our survey design did not allow us to investigate the underlying causal mechanisms: do people punish governments because they dislike policy flip-flops or because disregarding a referendum is a violation of democratic norms? Nevertheless, our findings are important, and have implications for both domestic and international politics. Domestically, our results suggest that referenda can enable governments to tie their hands to substantive policy choices and to “lock in” decisions that are politically controversial. Globally, our findings imply that calling a plebiscite could potentially increase a country’s bargaining power.

Supplemental Material

Online_Appendix – Supplemental material for Referenda as commitment devices – an experimental approach

Supplemental material, Online_Appendix for Referenda as commitment devices – an experimental approach by Florian Justwan, Sarah K. Fisher, Ashley Kerr and Jeffrey D. Berejikian in Research & Politics