The electoral consequences of policy-making in coalition governments

Abstract

Does policy-making in coalition governments affect cabinet parties’ electoral performance? Previous research shows that about two thirds of all propositions promised in coalition agreements are actually enacted. But does non-compliance with pledges made in the agreement affect subsequent electoral support? Drawing on the literature on performance voting, we expect that non-compliance is punished by voters, but the effect is conditioned by the importance of the issue and by a parties’ seniority status within the coalition. To test our argument, we have compiled the new comparative COALITIONPOLICY dataset on the enactment of more than 7000 policy pledges by 217 cabinet parties in 19 Western and Eastern European countries from 2000 to 2015. Across a broad range of models, we find no effect of pledge fulfillment on subsequent electoral performance. If anything, adherence to policy-promises only increases support for the prime minister party. These results have important implications for our understanding of political representation, coalition governance, and electoral competition in multiparty democracies.

Introduction

Does policy-making in coalition governments affect cabinet parties’ subsequent electoral performance? Before coalition governments take over executive offices, they typically engage in intensive coalition negotiations and publish a comprehensive coalition agreement in which they provide a detailed account of the policy reforms they plan to enact in government. The adoption of coalition agreements is typically a highly visible event that is accompanied by extensive media coverage so that voters receive a lot of information about the major promises that multiparty governments make in their coalition agreements. However, previous research shows that only about two thirds of all the policy reforms promised in coalition agreements were actually enacted (Moury, 2011, 2013). How does non-compliance with the promises made in coalition agreements affect subsequent electoral support? In this study, we address this question by leveraging a novel comparative dataset that maps the fulfillment of policy pledges made in coalition agreements across 19 European democracies.

Previous research devoted most attention to election pledges made in pre-election party programs. Researchers have examined when and why political parties do (not) fulfill the promises they have made in their manifestos (Artés, 2013; Costello and Thomson, 2008; Naurin, 2011, 2014; Praprotnik, 2016; Royed, 1996; Schermann and Ennser-Jedenastik, 2014; Thomson, 2001; Thomson et al., 2017). Our work also builds on the vast literature on retrospective economic and performance voting (Ferejohn, 1986; Fiorina, 1981; Key and Cummings, 1966). Importantly, Matthieß (2020) posits voters rewarding of party accomplishments and their attribution of competence based on perceptions of pledge performance as two mechanisms linking pledge fulfillment to voting behavior. These mechanisms should be at play when evaluating coalition parties based on their agreement commitments. However, both mechanisms rely on the fact that voters behave rationally, policy-seeking (Downs, 1957), and can clearly allocate pledges to governing parties (Powell and Whitten, 1993). If these assumptions are violated, rational retrospective evaluations by constituents can be impeded. Matthieß (2020) shows that the fulfillment of election pledges has important electoral implications. She finds that higher fulfillment leads to better performance at elections. Thus, voters reward parties for complying with their promises while sanctioning parties that do not fulfill their election pledges. Prior coalition research has by contrast studied coalition agreements in terms of bargaining success (Däubler and Debus, 2009; Quinn et al., 2011), as a control mechanism (e.g., Bowler et al., 2016; Eichorst, 2014; Indridason and Kristinsson, 2013; Klüver et al., 2023; Klüver and Bäck, 2019; Krauss, 2018; Müller and Kaare, 2000; Müller and Strøm, 2008; Timmermans, 2006), and with regard to their impact on policy-making during the legislative term (e.g., Bäck et al., 2017; Moury, 2011). However, the electoral impact of coalition agreement enactment remains unclear.

Even though the negotiation of a coalition contract is one of the most important political events during government formation, we know surprisingly little about the electoral consequences of not adhering to its policy propositions. While coalition negotiations are confidential, the presentation of the coalition agreement is highly visible. Coalition parties typically organize big press conferences to announce what has been negotiated in the past weeks. Media outlets report about the bargain struck between coalition partners and monitor the fulfillment of policy pledges throughout the legislative term.

In this article, we address this important question and shed new light on the electoral consequences of non-compliance with policy pledges for individual cabinet parties. We do so based on the new COALITIONPOLICY dataset which maps policy pledges made by 217 parties in 86 coalition agreements across 19 Western and Eastern European countries. Their performance is evaluated based on the fulfillment of more than 7000 individual pledges made between 2000 and 2015. We match this rich dataset with election results to investigate the consequences of (not) enacting the coalition agreement for electoral support. Counter-intuitively, we find that the enactment of coalition agreements does not increase subsequent vote share across a large number of different model specifications. Voters do not seem to reward or punish coalition parties for (non-)compliance with the coalition contract. These results are relevant for our understanding of political representation and coalition governance in multiparty democracies.

Research design

Dataset

We test our argument based on the newly compiled COALITIONPOLICY Dataset which maps policy pledges made in coalition agreements based on a hand-coded content analysis by country experts. The dataset covers 86 coalition agreements negotiated in 19 European countries between 2000 and 2015. We elaborate on the document selection and country sample in Appendix Section A. Our selection of countries follows established standards in coalition research and only includes European democracies that were governed by coalition cabinets which have publicly released a coalition agreement at least once in the time period under investigation (see Andersson et al., 2014). As discussed in Appendix Section A, we draw on official government programs that fulfill certain criteria whenever agreements only cover technical content (see Klüver and Bäck, 2019). We also provide more graphical evidence on the distribution of pledge performance across our sample in Appendix Section C.

Capturing electoral performance using lagged dependent variables

In the main manuscript, we assess the electoral performance of cabinet parties using a lagged dependent variable (LDV) approach. For each governing party, the vote share (scaled 0–100) at the election that follows the cabinet formation constitutes our outcome of interest. Crucially, we control for the vote share of each party during the election that preceded coalition formation. Intuitively, we therefore capture the effect of policy performance during the governing term on the vote share at the following election while controlling for the vote share at the previous election.

Election results are obtained from the ParlGov database (Döring and Manow, 2018). To test the robustness of our findings, we rerun the main analysis using (a) the vote ratio, and (b) the absolute difference between two subsequent election results as dependent variables (see Appendix Sections G and F). The results do not change substantially across these outcomes.

Explanatory variables: Policy performance

We measure the policy performance of individual cabinet parties by tracking how many of the policy pledges made in the coalition agreement are actually fulfilled during their time in office. To do so, we rely on a novel, extensive content analysis of coalition agreements. For each cabinet party, we identify specific policy pledges to which it has committed itself during the coalition negotiations. A policy pledge is defined as a promise to carry out an action or to produce a policy outcome whose fulfillment can be objectively verified (see Royed, 1996). To ensure reliability, we follow previous research and distinguish between objectively verifiable hard pledges and more vague statements that contain soft or no pledges (Matthieß, 2020; Thomson et al., 2017). We discuss our coding process and reliability checks in more detail in Appendix Section D.

In order to measure each parties’ share of fulfilled hard pledges, we rely on two additional steps conducted by the country experts. First, they coded the enactment of the policy pledges as either fully fulfilled or not fully fulfilled. The evaluation is based on national legislative databases, official statistics (for instance, from national statistical offices, Eurostat, or government reports and websites), and articles of trustworthy newspapers.

Second, the experts coded if a pledge found in the coalition agreement can be attributed to a specific cabinet party. They checked if a hard pledge was already mentioned in their election manifestos (obtained from the Manifesto Project (Lehmann et al., 2021)). The coders examined the campaign manifestos for mentions of every hard pledge identified in the agreement. This allows us to dissect which party actually pushed for a pledge to be part of the mutual agreement. According to a responsible party model, we expect that a cabinet party which is more successful in codifying their propositions in the coalition agreement and then translating them into policy should be rewarded by its supporters. Out of 13,756 unique hard pledges identified in the agreements, our coders were able to attribute 7’299 pledges (or 55%) to at least one cabinet party. These 7’299 agreement pledges constitute the basis of all subsequent analyses.

We then calculate two party-specific independent variables: First, party pledge count is a simple count of the number of policy pledges in an agreement that have been mentioned by a respective party and fulfilled throughout the legislative term of the coalition government. Second, we take the count and divide it by the total number of hard pledges in the coalition agreement to calculate the party pledge share. This variable takes the overall number of pledges made in the agreement into account.

Measuring issue salience for parties

In addition, we also pay attention to the divergent salience of different policy areas across parties. It is likely that voters particularly reward or punish coalition parties for (not) fulfilling pledges made in policy areas relevant for the party (see e.g., Plescia et al., 2021). To do so, we are breaking our data down to the party-issue level for some analyses. We do so based on the assignment of a policy field for every hard pledge in our sample. Our coders assigned a policy code based on the Manifesto Project (Lehmann et al., 2021) to all policy pledges which we identified in the coalition agreements. Following Klüver and Spoon (2016), we identified 11 pivotal issue areas for which the Project’s codebook includes positional categories. Appendix Section H provides an overview of the issue areas and associated categories.

Based on matching the coalition agreement data with the manifesto data, we generate three different salience variables: Manifesto salience specifies how salient the general issue was to the specific cabinet party prior to the election. We do so by calculating the share of text a party dedicated to an issue area in its entire party manifesto. In essence, this measure assesses how relevant the general issue has been to a specific cabinet party before the election. Pledge salience specifies how salient a specific pledge was to the cabinet party prior to the election. We do so by calculating the average amount of words that each cabinet party has devoted to pledges of a specific issue area. For instance, a party manifesto could refer over pages about economic policy, but only briefly mention specific economic hard pledges codified in the coalition agreement. This measure assesses how relevant the actual pledges made in an issue area have been to a cabinet party prior to the election. Agreement salience specifies how salient an issue area was in the mutual coalition agreement drafted after the election. It is based on the share of text in the agreement that was dedicated to an issue area relative to its overall length. Note that this measure also considers parts of the agreement that deal with a policy field even when it does not contain hard pledges. Thus, this measure assesses how relevant an issue area has been for the entire cabinet after the election.

By generating measures of salience and pledge fulfillment across the specific policy fields, our data is able to capture multidimensional complexity that goes along with coalition bargaining and promise keeping. Quite often, parties can hold very similar positions in one policy field—that is, European Integration—while being far apart on other issues such as economic redistribution. More detailed descriptive statistics on our dependent and explanatory variables can be found in Section A of the Appendix.

Additional control variables

Beyond having the prior vote share as a crucial control in the LDV-models, we also account for a number of potential confounders. First, we control for party status in the coalition as previous research has shown that junior parties generally lose votes at the next election (Klüver and Spoon, 2020). We thus incorporate the simple dichotomous PM party variable that receives the value of “1” whenever the prime minister was appointed by a party and “0” if a coalition party is a junior partner in the government. We also interact this measure with our salience variables. Second, since previous research on economic voting has shown that macroeconomic performance is an important predictor of incumbents’ electoral performance (Lewis-Beck et al., 2000), we also control for economic growth. Economic Growth is based on IMF data on yearly real GDP growth, operationalized in GDP percentage change between the beginning and at the end of a government’s term in office.

In addition, we control for cabinet-level attributes that can impact the clarity of responsibility. We account for the ideological distance of the cabinet, measured as the log value of the difference between the left- and rightmost party. The value is based on the distance between the two most extreme election manifestos on the rile-measure (Lehmann et al., 2021). Next, we also include the absolute number of cabinet parties and a minority cabinet dummy. Finally, models without fixed effects also include a dummy variable that denotes if a country is located in Central and Eastern Europe (CEE) to account for differences across established Western and younger East European democracies.

Data analysis

In what follows, we test the effect agreement pledge fulfillment across a large range of model specifications. First, we estimate party-level models without taking different issue areas into account. Since pledges in some policy areas might be more relevant for the electoral performance of parties than pledges made in other areas, we then estimate issue-level models. All standard errors are clustered at the country level.

Party-level models

Figure 1 presents results from 16 regression models using the LDV approach. Full tables can be found in Appendix Section E, where models 1–4 in Table 7 contain country fixed effects and models 5–8 in Table 7 omit the fixed effects. For each variable, we plot the estimate with country fixed effects (upper point) and without country fixed effects (lower point). The bars represent the 95% confidence intervals. The upper part of the figure shows null effects for our two independent variables derived from regular regression models. The lower half of Figure 1 shows the interaction between our independent variables and the prime minister party dummy. Figure 1 illustrates that neither party pledge share, party pledge count nor the interaction effects lead us to believe that compliance with policy pledges in coalition agreements has a substantive effect on subsequent electoral performance. The effects we find are substantially small and mostly do not reach conventional levels of statistical significance. If at all, we find a tendency suggesting that prime minister parties can profit from pledge enactment.

Figure 1.

Coefficient plot for the party-level models using the LDV approach (95% CI).

Issue-level models

For the issue-level models, we again estimate linear regression models with and without country fixed effects. The corresponding tables can be found in Appendix Section E. The upper plot suggests that pledge performance within specific issue categories does not influence the vote ratio.

More interesting than the overall null effects (which mimic the results from the party-level analysis) are the interactions with regard to salience. We interact the independent variables on pledge fulfillment with our three measures for policy salience to test whether the fulfillment of pledges in specific policy areas has an effect on the electoral performance. Just as for the party-level models, we estimated every model with and without country fixed effects. The lower part of Figure 2 shows that we do not find any meaningful interaction effects. None of the coefficients show confidently that pledge performance matters for electoral performance, irrespective of how salient a pledge may have been to a party or coalition. Only the interaction between the absolute pledge count and pledge salience results in a marginal, yet significant positive effect.

Figure 2.

Coefficient plot for the issue-level models using the LDV approach (95% CI).

Appendix Sections G and F show similar null-result patterns for models with vote differential or vote ratio as alternative outcomes. In nearly all models, pledge fulfillment does not have a systematic effect on coalition parties’ electoral performance. We also assess if our null-findings are driven by the diverse country sample that also covers younger democracies in Central and Eastern Europe (CEE). In Appendix Section I, we show that the results of our main analysis are robust even when we exclude cases from CEE.

Conclusion

In this article, we have addressed the question whether the (non-)fulfillment of policy pledges made in coalition agreements has electoral implications for coalition parties. Based on the newly compiled COALITIONPOLICY Dataset mapping the fulfillment of policy pledges by more than 200 cabinet parties across 19 West and East European democracies, we conclude that the enactment of coalition agreements does not garner electoral payoffs. Citizens appear to cast their vote irrespective of a cabinet parties’ adherence to its policy propositions.

These findings have important implications for our understanding of coalition governance, electoral accountability, and political representation. First, with regard to coalition governance, our results suggest that coalition parties can make policies without having to fear electoral repercussions if they do not stick to the agreement. In terms of vote-maximization, coalition agreements do not seem to dictate the policy agenda throughout the legislative term. Voters do not punish coalition parties for non-compliance. The parties have some flexibility to deviate from the negotiated agreement without jeopardizing their reelection. Hence, governments can respond to current developments such as economic downturns, international crises, or natural disasters without having to stick to the pledges they have made before taking office.

Second, while we find that the enactment of coalition agreements cannot explain the electoral performance of coalition parties, Matthieß (2020) showed that the fulfillment of campaign pledges made in election manifestos does positively effect on electoral performance. Even when we restrict our country sample to Western democracies, mimicking the sample of Matthieß (2020), we do not find a similar effect with regard to coalition agreements (see Appendix Section I). Based on the implications by Matthieß (2020) with regard to the significant effect of party manifestos and the null-findings brought forth in our analysis, we conclude that voters do hold government parties accountable for their performance in office, but that voters use the election manifesto rather than coalition agreements as a yardstick. Thus, voters reward political parties for implementing their election pledges, but they do not seem to evaluate the enactment of the coalition agreement. Future research could build on this finding and the work by Matthieß (2020) to trace the entire chain of accountability from election manifestos to coalition agreements to actual policy-making.

Overall, coalition agreements are important documents, but appear to matter more for conflict management within the cabinet than for electoral competition. Voters do not reward or sanction compliance with the agreement, so coalition parties are insulated from electoral scrutiny when implementing policy. Prior research shows that coalition agreements are effective control instruments for managing preference divergence across parties. While they give coalition parties an instrument to constrain their partners, coalition contracts play little role in electoral competition.