The relative effectiveness of overlapping international institutions: European Union versus United Nations regulations of air pollution

Abstract

Which types of international institutions display higher ability to change states’ behaviour? This article assesses the relative environmental effectiveness of a management-based (‘soft’) and an enforcement-based (‘hard’) international agreement: the United Nations Sofia Protocol and a European Union directive. Using difference-in-differences analysis, we find that the European Union directive is more effective in inducing emissions reductions than the United Nation’s Sofia Protocol. We propose that the European Union’s enforcement capacity is a likely driver of the directive’s effectiveness. The article makes two contributions to existing literature. First, we provide causal evidence on the relative importance of overlapping international institutions in regulating environmental policy outcomes, elucidating how an apparent emissions-reducing effect of a ‘soft’ United Nations Protocol is in fact driven by the existence of overlapping ‘hard’ European Union regulation. Second, we demonstrate how states’ enthusiasm for emissions regulations can explain the relative effectiveness of soft and hard law institutions.

Keywords

Compliance environmental politics European Union hard law international organizations regime effectiveness Sofia Protocol soft law United Nations

Introduction

International collective action problems are often targeted by multiple international institutions, and regime complexity increasingly permeates the global politics of issue areas such as trade, energy, defence and climate cooperation. Yet the relative effectiveness of parallel international efforts in which two or more regimes target the same problem has hitherto not been examined in existing regime effectiveness literature. Under overlapping regimes, which types of international institutions display higher ability to change states’ behaviour?

A key distinguishing feature of international institutions is whether they are based on ‘soft’ or ‘hard’ law (Abbott and Snidal, 2000). Although both soft- and hard-law institutions can incur legally binding obligations and highly precise provisions, only hard-law institutions delegate enforcement powers for interpretation and implementation of relevant rules to a supranational body.¹ While managerialists and enforcement theorists have extensively debated the effectiveness of soft- versus hard-law institutions (Chayes and Chayes, 1993; Downs et al., 1996; Tallberg, 2002), the relative effectiveness of these two types of institutions in regulating a specific collective action problem has not been systematically tested. Given the widespread adoption of hard law by contemporary international institutions (Koremenos, 2016: 237), a central question is whether these institutions are more effective in altering states’ behaviour than soft-law institutions.

Leveraging an institutional overlap between United Nations (UN) and European Union (EU) efforts to regulate nitrogen oxide (NO_x) emissions, this article develops a novel approach for assessing the relative effectiveness of international institutions. By using the difference-in-differences (DID) estimator on data concerning the 1988 Sofia Protocol under the UN Convention on Long-Range Transboundary Air Pollution (CLRTAP) and the EU’s Large Combustion Plant (LCP) Directive, we test whether the soft-law Sofia Protocol or the hard-law EU directive is most effective in inducing NO_x emissions reductions.

Theoretically, we argue that states’ enthusiasm for cooperation is a scope condition for soft-law agreements to induce behavioural change (Victor, 2011). Since soft-law agreements do not alter states’ material incentives for compliance, we propose that agreements such as the Sofia Protocol are unlikely to induce behavioural change among reluctant states. In contrast, hard-law agreements such as the EU LCP Directive can elicit compliance among both enthusiastic and reluctant states by threatening the latter with sanctions. Hence, we hypothesize that the EU LCP Directive is more effective than the Sofia Protocol in inducing NO_x emissions reductions.

Our DID analyses show that the Sofia Protocol did not significantly affect the emissions of non-EU members. EU members, in contrast, deepened their emissions reductions after 1988 compared to the counterfactual scenario. We attribute this differentiated effect to the EU’s LCP Directive targeting NO_x emissions. Moreover, our analyses suggest that the EU directive reduced emissions even in reluctant countries. Such an effect cannot be found for Sofia. In sum, our findings indicate that the EU directive has been more effective than the Sofia Protocol in inducing emissions reductions; and, given the two institutions’ differing approaches to enforcement, we argue that the EU’s sanctioning capacity is a likely driver of this result.

This article makes two contributions to existing literature on the effectiveness of international institutions. First, we provide causal evidence that a hard-law institution induces higher environmental effectiveness in direct comparison with a soft-law regime governing the same problem. Although the effectiveness of both soft-law and hard-law institutions has previously been empirically assessed (e.g. Bratberg et al., 2005; Zangl, 2008), this article is the first to leverage institutional overlap to construct a direct comparison that arrives at the relative effectiveness of hard versus soft international law. Notably, the resulting findings indicate that previous studies attributing emissions reductions to the Sofia Protocol overestimate the true effectiveness of soft law on environmental policy outcomes. Second, we introduce an original measure of states’ enthusiasm for emissions regulations, and empirically demonstrate its usefulness in accounting for the relative effectiveness of soft- and hard-law institutions. The differentiated effect of enthusiasm in soft- and hard-law institutions confirms a central theoretical conjecture in the enforcement literature (Victor, 2011).

Acid rain, the CLRTAP and the EU

NO_x are potent air pollutants, causing acidification and other environmental problems far from their sources, which are industrial and power plants as well as transport vehicles. Because air pollution is a regional problem (as opposed to climate change), a group of mainly European states adopted the UN CLRTAP in 1979. Since then, nine protocols to the CLRTAP have been negotiated and adopted by a range of states. The 1988 Sofia Protocol was the first CLRTAP protocol to target NO_x, obliging parties to stabilize emissions at 1987 levels by 1994.

Notwithstanding the CLRTAP, the EU deemed it necessary to have its own agreement covering NO_x emissions during the 1980s. Hence, in parallel with negotiations on the Sofia Protocol, EU member states prepared directives targeting the two major NO_x emissions sources: large combustion plants and transport. Having been adopted in November 1988, the LCP Directive (88/609) required new large combustion plants to comply with the emissions limit values for NO_x and other pollutants (see EUR-Lex, 1988). Furthermore, the LCP Directive included national emissions reduction plans, implying that emissions of existing plants were also targeted.

With partly overlapping memberships, both the Sofia and EU negotiations faced difficulties due to diverging preferences between enthusiastic and reluctant states (see Levy, 1993). Under the Sofia negotiations, enthusiastic states such as Sweden, Austria, Switzerland and West Germany favoured a uniform 30% reduction in the NO_x emission levels of 1985. The Soviet Union and Eastern European countries were opposed to any reductions. The UK and France called for emissions freezes; and the US sought credit for previous reductions. The Sofia compromise was to freeze emissions at 1987 levels. However, a group of 12 enthusiastic states² wanted to go beyond this target; hence, in addition to signing the Protocol, they also signed the so-called NO_x Declaration, in which they pledged to reduce emissions by 30% before 1998.

Management and enforcement: how international institutions induce behavioural change

Effective environmental agreements are characterized by three features: broad participation (among relevant actors), deep commitments and high compliance (Barrett, 2008; Tørstad, 2020). Since shallow commitments are easier to comply with, one key debate in the regime effectiveness literature concerns whether enforcement mechanisms are necessary to sustain deep commitments and high compliance simultaneously (Chayes and Chayes, 1993; Downs et al., 1996; Tallberg, 2002; Tørstad, 2020). Proponents of enforcement hold that states cooperate only as long as the (expected) marginal private benefit of taking an action (e.g. mitigation) equals or exceeds the (expected) marginal private costs of taking that action. Consequently, states will not deviate from business as usual (BAU) trajectories absent enforcement mechanisms that restructure compliance incentives (Aakre et al., 2016; Barrett, 2003; Downs et al., 1996). That is not to say that enforcement scholars are surprised that compliance with international agreements is generally high. High compliance rates are not, however, taken as evidence that cooperation is effective (Young and Levy, 1999). In contrast, the enforcement school argues that high compliance is often caused by shallow commitments that would have been complied with even absent an agreement. Deep commitments, requiring higher deviations from BAU, enhance incentives to shirk: hence, deeper agreements require more substantial enforcement mechanisms to elicit compliance (Downs et al., 1996).

Managerialists, on the other hand, argue that states often deviate from scenarios dictated by net private benefits and costs to comply (Young and Levy, 1999). Barriers to non-compliance instead include treaty ambiguity, capacity shortages and unforeseen circumstances (Chayes and Chayes, 1993). Hence, institutions can be managed in certain ways to increase the likelihood of compliance and/or effectiveness, through measures such as transparency, capacity-building and clear expectations. The most important reason states have a ‘general propensity to comply’ is that norms, not sanctions, drive state behaviour (Chayes and Chayes, 1993; Tveit, 2018). Since the pacta sunt servanda norm is strong in the international community, states usually do what they can to honour agreements: enforcement mechanisms are therefore largely seen as unnecessary, or even contra-productive, for institutional effectiveness (Chayes and Chayes, 1993).

The Sofia Protocol represents a managerial agreement in that it relies solely on ‘soft’ mechanisms such as normative pressure, monitoring and information-sharing to induce compliance with its provisions. Although econometric studies of environmental treaty effectiveness have largely found that similar managerial treaties have little or no effect on state behaviour (Ringquist and Kostadinova, 2005; Vollenweider, 2013), two studies of the Sofia Protocol argue that the Protocol caused substantial emissions reductions among its signatories (Bratberg et al., 2005; Isaksen, 2020).

In 1988, when the Sofia Protocol was adopted, the EU’s LCP Directive also entered into force. However, given that the LCP Directive entered into force simultaneously with the Sofia Protocol, we argue that the potential effect of this directive should be considered when evaluating the effectiveness of the Sofia Protocol. Because many (yet not all) parties to the Sofia Protocol are also EU members, the NO_x emissions reductions that previous research attributes to Sofia may in fact have been caused by the above-mentioned EU directive.

The preambles of the LCP Directive and Sofia Protocol express the same intention: to reduce NO_x emissions among their signatories and further the overarching CLRTAP goals. We therefore propose that the LCP Directive and Sofia Protocol can be viewed as comparable emissions reductions instruments based on their overlapping objectives, and that NO_x emissions reductions can serve as a useful measure of their relative effectiveness. Yet, despite their common objectives, the two agreements took widely differing approaches to regulate NO_x emissions that place them on different ends on the soft-/hard-law spectrum in international politics.

First, the LCP Directive specified differentiated individual emissions reduction targets for states and technical requirements for how much emissions combustion plants could emit. The directive hence contained more precise obligations than the Sofia Protocol, which only called for a general freezing of 1987 levels of emissions and did not specifically regulate the source of emissions.

Second, the advanced EU compliance system behind the LCP Directive ensured that non-compliers would be threatened with sanctions. In contrast to the ‘soft’ Sofia Protocol, EU directives are monitored and sanctioned by the European Commission and ultimately the European Court of Justice (ECJ). The Commission enforces non-compliance through infringement procedures. Over the period 1988–1996, the EU opened a total of 481 infringement cases in the environmental policy sector (Börzel, 2021). If the Commission deems that an infringement persists despite more lenient measures (informal discussions, a letter of concern, a ‘reasoned opinion’), it may appeal to the ECJ. If the ECJ agrees with the Commission, the member state is obliged to carry out whatever measures the ECJ decides to ensure compliance. If the ECJ deems it necessary, it may ultimately impose a financial penalty on the non-compliant state. Hence, Tallberg (2002) argues that the infringement procedure and the potential ensuing sanctions constitute ‘a highly deterrent mechanism’, and Alter (2000) describes the EU as ‘perhaps the most “legalized” international institution in existence’.

In sum, the LCP Directive’s precise obligations, coupled with the EU compliance system that is based on a substantial degree of sanctions (or threats thereof), render it an example of a hard-law agreement backed by strong enforcement powers. Since many Sofia Protocol signatories are also EU members, these countries’ emissions are regulated by both Sofia and EU law. In our empirical analysis, we aim to establish whether EU countries’ emissions decreased at a different rate from the emissions of other Sofia signatories, and hence evaluate whether NO_x emissions reductions are attributable to the managerial-based Sofia Protocol or enforcement-based EU law. By doing so, we contribute an assessment of the relative environmental effectiveness of management-based and enforcement-based international institutions.

Although our theoretical discussion evolves around the notion of compliance, our dependent variable in the empirical analysis (NO_x emissions) strictly captures environmental effectiveness. There is often substantial overlap between these concepts, but compliance is neither a necessary nor a sufficient condition for policy effectiveness as unambitious goals may require little policy change (Avrami and Sprinz, 2019). The management versus enforcement debate is nonetheless key to understanding variation in institutional effectiveness, as measures that induce compliance contribute (by definition) towards the effectiveness of the institutional arrangement under which it is established (Tørstad, 2020). It is important to note, however, that although the Sofia Protocol and the EU LCP Directive had the same environmental effectiveness goal (to reduce NO_x emissions), their legal requirements for compliance differed. In the following, we therefore mainly aim to compare the institutions’ environmental effectiveness.

Our analysis goes beyond a simple comparison of management versus enforcement by testing whether effects are differentiated for ‘enthusiastic’ states that have strong intention to comply and ‘reluctant’ states with little or no such intention (Victor, 2011). In the empirical analysis, we leverage the existence of the NO_x Declaration to test whether the hard-law EU LCP Directive is more effective than the soft-law Sofia Protocol to induce compliance among both enthusiastic and reluctant states. Since signatories of the NO_x Declaration pledged higher emissions cuts than strictly necessitated by either the EU or Sofia, we test whether the ‘enthusiastic’ signatories of this declaration are affected differently from (‘reluctant’) non-signatories by hard and soft law. As discussed above, a potential benefit of hard law is that it restructures material incentives to render it more attractive for reluctant states to comply. In the absence of sanctions, states’ enthusiasm towards (deep) cooperation is hence likely to be a scope condition for compliance. Since soft-law agreements do not alter states’ material incentives for compliance, reluctant states are unlikely to alter cost-benefit ratios based on soft-law agreements such as the Sofia Protocol. In contrast, hard-law agreements such as the EU LCP Directive can elicit compliance among both enthusiastic and reluctant states, by threatening the latter with sanctions in case of non-compliance. We therefore expect that the EU LCP Directive is effective in inducing NO_x emissions reductions among both enthusiastic and non-enthusiastic states, whereas we only expect the Sofia Protocol to be effective for enthusiastic states.

Data and operationalization

Our observations cover 23 countries between 1980 and 1996 (country-years). As Table 1 shows, 4 countries in our sample are members of only the CLRTAP, whereas 10 are both CLRTAP members and Sofia participants.³ Of those 10 countries, three became EU members on 1 January 1995. Consequently, they are included in the EU member group in 2 of 17 years of observation. Eight countries are EU members throughout the period.

Table 1.

Participation in Convention on Long-Range Transboundary Air Pollution (CLRTAP), Sofia Protocol and the European Union (EU) 1980–1996.

Only CLRTAP	CLRTAP and Sofia	CLRTAP and EU	CLRTAP, EU, and Sofia
Iceland	Austria*	Belgium	Denmark
Turkey	Bulgaria		France
Poland	Czech Republic		Germany
Romania	Slovakia		Ireland
	Finland*		Italy
	Norway		Netherlands
	Sweden*		Spain
	Switzerland		United Kingdom
	Russia
	Hungary

Entered the EU in January 1995.

We follow the data operationalization of Bratberg et al. (2005) and use changes in NO_x emissions from the previous year as the dependent variable. The main explanatory variable of interest is participation, a dummy separating Sofia participants from non-participants. Country-years of participants score 1 on participation from 1989 onwards. Countries are classified as participants if they signed and ratified Sofia no later than 31 December 1994. The EU LCP Directive came into force in November 1988. Because 1989 is the first complete year in which states’ emissions may have been affected by those directives, EU members would score 1 on a binary EU law treatment variable from 1989 onwards, and 0 prior to 1989. Hence, there would be perfect correlation between such an EU law treatment variable and the participation variable. Consequently, we do not include any additional EU law variables in our data set; instead, we run our models with and without EU countries in the sample to capture any potential effects of EU law on emissions.

To test whether effects are differentiated based on enthusiasm, we include a dummy that separates post-1988 units of signatories of the NO_x declaration from other country-years. This declaration of 12 countries not only to ratify Sofia, but to reduce emissions by 30% by 1998, could indicate a willingness to reduce emissions (i.e. enthusiasm) that eventually influences policies. All models include several controls, listed in Supplemental Material I. Supplemental Material I also provides information about the data sources for all variables. Except for dummies and the state capacity measure, all variables are in logs and first differences. Because our model is in first differences, any time-invariant heterogeneity is removed.

Estimation and empirical strategy

To evaluate the relative effectiveness of the Sofia Protocol and the LCP Directive, we fit several DID regression models on the data described above. Seeking to mitigate selection effects and unobserved confounders, quantitative studies of the effectiveness of international environmental agreements have converged on the use of panel data and quasi-experimental designs such as the DID (e.g. Bratberg et al., 2005; Vollenweider, 2013) or synthetic control estimators (e.g. Almer and Winkler, 2017; Isaksen, 2020) to produce credible counterfactuals (Angrist and Pischke, 2009). Since evaluating differences in national emissions trends is an analytical problem that is particularly prone to both selection effects and unobserved confounders, quasi-experimental approaches such as the DID estimator should provide more credible causal estimates than standard multiple regression methods given that its identification assumptions are met.

The main idea behind DID is that an untreated group, for example, states choosing not to participate in an environmental protocol, can be used to control for effects of factors that affect all units (both treated and untreated) over time. The key assumption for identifying a causal treatment effect in DID analysis is that there are no omitted time-varying confounders: that is, absent treatment, treated units would have experienced the same change in the outcome variable that we observe for non-treated units (Angrist and Pischke, 2009). One way to test whether this assumption holds in practice is to examine whether pre-treatment trends in the variable of interest (here: NO_x emissions) substantively differ between treated and untreated units. Parallel trends prior to treatment indicate that the DID estimator can credibly elicit the causal effect of treatment status.

In its general form, the DID estimator is

Δ = E (Y \begin{matrix} 1 \\ i t \end{matrix} - Y \begin{matrix} 0 \\ i, t - 1 \end{matrix} | D i = 1) - E (Y \begin{matrix} 0 \\ i t \end{matrix} - Y \begin{matrix} 0 \\ i, t - 1 \end{matrix} | D i = 0)

(1)

where $Y \begin{matrix} 1 \\ i t \end{matrix}$ is the score on the outcome variable of unit i at time t if the unit is treated, whereas $Y \begin{matrix} 0 \\ i t \end{matrix}$ is the unit’s score if it has not been treated D_i is the treatment variable (treated and non-treated units score 1 and 0, respectively). Hence, in our DID application, Y_it is the emissions of a country at time t, whereas states that became parties to Sofia no later than 31 December 1994 score 1 on D_i. In Equation (1), the first term equals the difference between emissions before and after treatment among signatories. The second term is the corresponding difference in emissions among non-treated units. The treatment effect equals the difference between those two differences.

When applied to a dataset having more than two observations of each unit, the DID becomes

Δ = E ((Y \begin{matrix} 1 \\ i t \end{matrix} - Y \begin{matrix} 0 \\ i, t - 1 \end{matrix}) - (Y \begin{matrix} 0 \\ i, t - 1 \end{matrix} - Y \begin{matrix} 0 \\ i, t - 2 \end{matrix}) | D i = 1) - E ((Y \begin{matrix} 0 \\ i t \end{matrix} - Y \begin{matrix} 0 \\ i, t - 1 \end{matrix}) - (Y \begin{matrix} 0 \\ i, t - 1 \end{matrix} - Y \begin{matrix} 0 \\ i, t - 2 \end{matrix}) | D i = 0)

(2)

In our application, treatment takes place in period s, and we assume that treatment may have an effect in all observed periods thereafter. In addition to Y_it, our model includes three dummy variables, D_i, D_it and S_it. Units in the treatment group score 1 on D_i throughout the time-series. All units score 0 on D_it until the treaty is signed, which is assumed to take place in 1988.⁴ In year s and thereafter, units in the treatment group score 1 on D_it, whereas non-treated units still score 0.⁵ All units score 0 on S_it until the protocol is signed.⁶ Thereafter, non-signatories score 1, whereas signatories score 0.⁷

As Table 2 indicates, D_i distinguishes country-years of states that at some point become signatories from country years of non-signatory country-years (regardless of time), whereas S_it distinguishes post-agreement country-years of non-signatories from all other units and D_it distinguishes post-agreement from pre-agreement country-years within the treatment group. The present application of the DID estimator is illustrated by Figure 1.

Table 2.

The scores of signatories and non-signatories on core dummy variables.

	s ⩾ t (pre-agreement)		s < t (post-agreement)
	Signatories	Non-signatories	Signatories	Non-signatories
S_it	0	0	0	1
D_it	0	0	1	0
D_i	1	0	1	0

Figure 1.

Difference-in-differences (DID) applied to emissions data before and after agreement (for α₀ = 0).

The period between t = s ‒ 1 and t = s is the (last) pre-agreement period, whereas t = s to t = s + 1 is the (first) post-agreement period. Note that Figure 1 includes only two periods: one pre-agreement and one post-agreement period. In contrast, the countries we study are observed multiple times both before and after signing. Here β₁ is the difference between pre- and post-agreement slopes for non-signatories. In Figure 1, emissions go down both before and after t = s, but emissions reductions per time unit are larger in the post-agreement period. Hence, in Figure 1, β₁ is negative. Likewise, γ is the difference between the pre- and post-agreement slopes for signatories. Hence, the treatment effect equals γ – β₁. The common trends assumption holds as long as there are no pre-agreement differences between the treated and non-treated groups; that is, if α₀ equals 0.

Thus, we estimate the model:

Y_{i t} - Y_{i, t - 1} = β_{0} + β_{1} S_{i, t - 1} + α_{0} D_{i} + γ D_{i, t - 1} + V_{i t,}

(3)

where V_it is the error term (note that V_it = U_it – U_i,t_-1).

The model also includes a vector of control variables, X_it – X_i,_t-1. Because all variables except dummies are in first differences, all models include country fixed effects (FE). Hence, no country-specific constants θ_i are included in Equation (3). The model’s core coefficients and their interpretations are presented in Table 3.

Table 3.

Core coefficients of the estimated model.

Coefficient	Interpretation
α₀	Estimates if the pre-agreement emissions trajectories of signatories and non-signatories differ. If no such differences exist (i.e. if α₀ = 0), the trends are parallel until treatment, thereby suggesting that any treatment effect can be interpreted causally.
β₁	Estimates post-agreement slope change among non-signatories.
γ	Estimates post-agreement slope change among signatories. The treatment effect is γ – β₁.

Results

Table 4 lists the results of nine regression models.⁸ Model 1 includes all units, and therefore does not distinguish EU members from non-members. Because γ is estimated at −0.025, and β₁ at −0.018, the treatment effect equals −0.007. Substantively, this estimate implies that Sofia participants reduced their NO_x emissions 0.7% more per year than non-participants did. However, the treatment effect is not statistically significant (t-values are not reported in Table 4 but equals −0.29 for the treatment effect). Hence, although the post-agreement slope change for all Sofia participants (γ) is statistically significant, Model 1 does not reveal a clear treatment effect.

Table 4.

The effects of the Sofia Protocol and European Union (EU) law on nitrogen oxide (NO_x) emissions.

	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6	Model 7	Model 8	Model 9
γ	−0.025** (0.009)	−0.016 (0.015)	−0.022 (0.014)	−0.014 (0.015)	−0.019 (0.011)	−0.018 (0.012)	0.013 (0.018)	−0.039** (0.012)	−0.038* (0.016)
α₀	−0.010 (0.018)	−0.020 (0.022)		−0.008 (0.016)
β₁	−0.018 (0.021)	−0.020 (0.022)	−0.006 (0.017)
Controls	V	V	V	V	V	V	V	V	V
EU applicant						−0.007 (0.021)	0.009 (0.022)
NO_x declaration							−0.053* (0.023)		−0.002 (0.014)
Treatment effect	−0.007 (0.022)	0.004 (0.026)	−0.016 (0.014)	−0.014 (0.015)	−0.019 (0.011)	−0.018 (0.012)	0.013 (0.012)	−0.039** (0.012)	−0.038* (0.016)
Constant	0.014 (0.017)	0.014 (0.018)	0.000 (0.011)	(0.015) (0.011)	−0.002 (0.009)	−0.002 (0.009)	0.003 (0.009)	0.021 (0.014)	0.021 (0.014)
Country fixed effects	V	V	V	V	V	V	V	V	V
N	230	129	129	129	129	129	129	101	101
R ²	0.330	0.412	0.407	0.408	0.407	0.407	0.433	0.172	0.172
adj. R²	0.300	0.362	0.362	0.363	0.367	0.362	0.385	0.110	0.100

Standard errors in parentheses.

*p < 0.05, **p < 0.01.

Model 2 is identical to Model 1, except that N differs. To assess if γ in Model 1 is driven by EU law, Model 2 excludes all EU members from the sample. The difference between the results for γ between Models 1 and 2 reveals that excluding all EU members from the sample diminishes the estimate for γ. Although γ is estimated at −0.025 in Model 1, it is only −0.016 (and statistically insignificant) in Model 2. This substantial weakening of γ suggests that estimating the effect of Sofia participation without taking EU law into account⁹ entails a risk of biased results. Surprisingly, when EU members are not in the sample, our estimates indicate that emissions reductions were larger in states that did not sign and ratify Sofia (β₁ = −0.020) than in those who did (γ = −0.016). Overall, Model 2 indicates that the treatment effect of Sofia is null when EU members are excluded.

Models 3 and 4 test if α₀ and β₁ differ significantly from zero when one of them is estimated and the other is left out. That they do not, has two implications. First, the consistently insignificant estimates for α₀ suggests that the main DID identifying assumption holds: the pre-agreement slopes of Sofia participants are no different from those of non-participants. In other words, the two groups move in parallel until treatment. Second, as β₁ does not differ from zero, the treatment effect reduces to γ.

Model 5 shows that the treatment effect of Sofia does not reach statistical significance even when β₁ and α₀ are excluded. Moreover, the effect of Sofia participation is slightly weakened when the EU applicant dummy is included (Model 6).

The inclusion of the NO_x declaration dummy, our proxy for enthusiasm to undertake emissions cuts, is more consequential. Model 7 shows that when that voluntary declaration is controlled for, the estimate for Sofia participation flips from negative to positive (yet statistically insignificant). Hence, any potential effect of Sofia vanishes when we separate post-1988 country-years of NO_x declaration signatories from the other country-years. In contrast, the NO_x declaration estimate is negative and statistically significant. In other words, the Sofia Protocol has no effect when we control for enthusiasm; but enthusiasm has a significant effect on NO_x emissions reductions.

Models 8 and 9 include only country-years of EU members. They reveal two insights. First, in the EU-only sample, γ is substantially higher than in previous models, and statistically significant. Model 8 suggests that EU countries reduced their post-1988 NO_x emissions by 3.9% annually, compared to the counterfactual scenario.¹⁰ Second, in contrast to that for Sofia, the estimate for EU law is not sensitive to the inclusion of the NO_x declaration dummy (enthusiasm). Importantly, the finding that the effectiveness of the EU directive is invariant to states’ level of enthusiasm to undertake emissions reductions suggests that the EU’s enforcement capacity is a driver of the emissions reductions among EU members. Since enthusiastic states can be expected to comply with targets, the key rationale for having enforcement mechanisms is that they induce behavioural change among reluctant states (Victor, 2011). Hence, our results showing that enthusiasm dampens the effect of the unsanctioned Sofia Protocol but has no effect on the enforcement-based EU directive are in line with the theoretical conjecture proposed by enforcement theorists.

Supplemental Material II reports several sensitivity checks, which are all consistent with our main results. Supplemental Material III also shows which countries legally complied with the Sofia Protocol and those which did not. Non-compliers with Sofia include EU members such as Ireland, Italy and Spain, which all complied with the EU Directive. Although our DID analysis does not by itself establish that sanctions (or threats thereof) caused emissions to decline more rapidly in the EU, the case of Ireland’s attitudes to Sofia and the EU LCP Directive illustrates the theoretical enforcement conjecture described above. First, parliamentary records indicate that Irish politicians at the time viewed complying with the EU Directive as more acute than complying with the Sofia Protocol (Dáil Éireann, 1989). In a 1989 parliamentary debate, Ireland’s environmental minister stated that the Directive ‘must be implemented’, but only ambiguously referred to implementing Ireland’s Sofia commitments. Irish elected officials also asked several questions about the government’s compliance with the directive, but none about whether the country was complying with the Sofia Protocol. Second, Ireland only complied with the EU LCP Directive after an infringement case was opened. After the LCP Directive entered into force in 1989, an infringement case against Ireland was opened in 1991 because Ireland had failed to notify implementing measures. The year after, in 1992, the EU terminated the case and reported that Ireland was now in compliance with the LCP Directive (EUR-Lex, 1992). In sum, Ireland appears to have been reluctant towards both the Sofia Protocol and the EU Directive, and only complied with the latter after being threatened with sanctions by the EU. Yet its non-compliance with the Sofia Protocol over the commitment period 1989–1994 went unmentioned and unsanctioned by the CLRTAP.

Conclusion

This article assesses the relative environmental effectiveness of a management-based (‘soft’) and an enforcement-based (‘hard’) international institution: the CLRTAP’s Sofia Protocol and the EU. Our DID analysis shows that that the post-1988 NO_x emissions of EU members deviated substantively from the counterfactual scenario, whereas non-EU CLTRAP members’ emissions did not. Although our analysis cannot categorically establish the causal mechanism driving the emissions reductions in the EU, we propose that the positive treatment effect in the EU sample is attributable to a major EU directive targeting NO_x emissions and that the EU’s enforcement system is a likely driver of the directive’s effect. In contrast, we find that the Sofia Protocol, which relies solely on a non-enforcement managerial compliance mechanism, had no effect on emissions in participating countries.

Our analysis contributes to the literature on effectiveness and compliance in international institutions by providing evidence that compliance enforcement can be an effective factor in inducing behavioural change on states (Chayes and Chayes, 1993; Downs et al., 1996; Ringquist and Kostadinova, 2005; Tallberg, 2002). Theoretically, we demonstrate the importance of which types of states, reluctant or enthusiastic, are induced to cooperate by different cooperative arrangements (Victor, 2011). In support of this argument, our findings show that the hard-law agreement led both reluctant and enthusiastic states to undertake emissions reductions; but that the soft-law agreement examined here only had a positive effect among enthusiastic states. If generalizable to other settings, these results suggest that states’ enthusiasm towards emissions reductions may be a scope condition for soft-law agreements to induce real emissions reductions. Given that the enthusiasm of states is a more consequential effectiveness factor in soft-law regimes than hard-law regimes, the former are likely to have more variable effectiveness prospects than the latter; and would also be more exposed if major countries are unenthusiastic towards cooperation.

Three caveats are worth highlighting. First, we stress that our analysis is unable to categorically establish that the threat of sanctions drove compliance with the LCP Directive. One plausible alternative explanation is that EU countries wanted to maintain good cooperative relations within the union and were concerned about the long-term reputational effects of not complying with an EU directive. We have argued that EU’s sanctioning capacity likely was an important factor since the EU opened infringement cases¹¹ that appear to have induced compliance with the LCP Directive, and because our regression models show that the EU induced unenthusiastic states to undertake emissions reductions. Nonetheless, since our research design is inapt for inferring states’ compliance motives at large, we propose that future research on the effectiveness of international institutions should examine this question more systematically across institutions.

The second caveat is that the Sofia Protocol did not contain capacity-building measures, which managerial theorists have argued could help non-enforced agreements elicit compliance (Chayes and Chayes, 1993). Hence, it may not have been an ‘ideal’ managerial treaty. Although a recent study fails to confirm positive effects of capacity on compliance across a range of CLRTAP protocols (Tveit, 2021), we do not preclude the possibility that capacity-building measures under the Sofia Protocol could have reduced emissions.

Third, it is important to acknowledge that our analysis does not preclude that the CLRTAP, Sofia Protocol and LCP Directive were in some ways mutually reinforcing agreements.¹² The creation of the Sofia Protocol and of the EU LCP Directive were interconnected, and since both the Directive and Sofia refer to the CLRTAP in their preambles it appears reasonable to assume that both agreements were influenced by the larger UN convention. We nonetheless propose that it is analytically possible to extricate the effects of Sofia and the EU directive based on the near-perfect overlap in timing of the agreements, as well as their comparable but different memberships. Crucially, despite synergistic effects in the establishment phases of the Sofia Protocol and LCP Directive, we do not have any reason to believe that there were compliance spillover effects that confound our analysis of their relative environmental effectiveness. We discuss this point further in Supplemental Material IV, where we also provide a sectoral breakdown of EU countries’ emissions which further indicates that the LCP Directive is likely a primary driver of the EU emissions reductions. In sum, the environmental effectiveness of the LCP Directive appears robust even when accounting for potential spillover effects from Sofia. However, given that both the EU Directive and Sofia were established in the shadow of the CLRTAP, this convention may have paved the way for the effectiveness of EU law in reducing NO_x emissions.

Supplemental Material

sj-docx-1-ips-10.1177_01925121221145496 – Supplemental material for The relative effectiveness of overlapping international institutions: European Union versus United Nations regulations of air pollution

Supplemental material, sj-docx-1-ips-10.1177_01925121221145496 for The relative effectiveness of overlapping international institutions: European Union versus United Nations regulations of air pollution by Andreas Kokkvoll Tveit and Vegard Tørstad in International Political Science Review

Supplemental Material

sj-pdf-2-ips-10.1177_01925121221145496 – Supplemental material for The relative effectiveness of overlapping international institutions: European Union versus United Nations regulations of air pollution

Supplemental material, sj-pdf-2-ips-10.1177_01925121221145496 for The relative effectiveness of overlapping international institutions: European Union versus United Nations regulations of air pollution by Andreas Kokkvoll Tveit and Vegard Tørstad in International Political Science Review

Footnotes

Acknowledgements

We are most grateful to Jon Hovi and three anonymous reviewers, as well as participants at a panel at the annual Midwest Political Science Association (MPSA) Conference, Chicago, April 2018, for constructive comments and suggestions. Any remaining errors are the authors’ responsibility.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

ORCID iD

Vegard Tørstad

Supplemental material

Supplemental material for this article is available online.

Notes

Author biographies

Andreas Kokkvoll Tveit is a senior researcher at the Institute of Transport Economics, Oslo, Norway. His research interests include international cooperation, transport and environmental politics, and the development, implementation and acceptance of public policy. He holds a PhD in political science from the University of Oslo (2019).

Vegard Tørstad is an assistant professor at the Department of Political Science, University of Oslo. His main interests include climate and environmental politics, global governance and international institutions. He holds a PhD in political science from the European University Institute (2021).

References

Aakre

Stine

Helland

Leif

Hovi

Jon

(2016) When Does Informal Enforcement Work? Journal of Conflict Resolution 60(7): 1312–1340.

Almer

Christian

Winkler

Ralph

(2017) Analyzing the Effectiveness of International Environmental Policies: The case of the Kyoto Protocol. Journal of Environmental Economics and Management 82: 125–151.

Alter

Karen J

(2000) The European Union’s Legal System and Domestic Policy: Spillover or Backlash? International Organization 54(3): 489–451.

Angrist

Joshua D

Pischke

Jörn-Steffen

(2009) Mostly Harmless Econometrics: An Empiricist’s Companion. Oxford: Princeton University Press.

Avrami

Lydia

Sprinz

Detlef F

(2019) Measuring and Explaining the EU’s Effect on National Climate Performance. Environmental Politics 28(5): 822–846.

Barrett

Scott

(2003) Environment and Statecraft. The Strategy of Environmental Treaty-Making. Oxford: Oxford University Press.

Barrett

Scott

(2008) Climate Treaties and the Imperative of Enforcement. Oxford Review of Economic Policy 24(2): 239–258.

Börzel

Tanja A

(2021) Berlin Infringement Database. Version 2021.1. Berlin: Freie Universität Berlin. Available at: https://www.polsoz.fu-berlin.de/en/polwiss/forschung/international/europa/bid/index.html.

Bratberg

Espen

Tjøtta

Sigve

Øines

Torgeir

(2005) Do Voluntary International Environmental Agreements Work? Journal of Environmental Economics and Management 50(3): 583–597.

10.

Chayes

Abram

Chayes

Antonia Handler

(1993) On Compliance. International Organization 47(2): 175–205.

11.

Éireann

Dáil

(1989) Dáil Éireann Debate – Tuesday, 14 November 1989. Available at: https://www.oireachtas.ie/en/debates/debate/dail/1989-11-14/45/?highlight%5B0%5D=plant&highlight%5B1%5D=pollutant&highlight%5B2%5D=emissions.

12.

Declaration on the 30 Per Cent Reduction of Nitrogen Oxide Emissions (1988) Environmental Policy and Law 18(6): 234.

13.

Downs

George W

M Rocke

David

N Barsoom

Peter

(1996) Is the Good News about Compliance Good News about Cooperation? International Organization 50(3): 379–406.

14.

EUR-Lex (1988) Council Directive 88/609/EEC of 24 November 1988 on the Limitation of Emissions of Certain Pollutants into the Air from Large Combustion Plants. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:31988L0609&qid=1539599691494&from=EN .

15.

EUR-Lex (1992) Tenth Annual Report on the Monitoring of the Application of Community Law. Available at: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A51993DC0320.

16.

Isaksen

Elisabeth Thuestad

(2020) Have International Pollution Protocols Made a Difference? Journal of Environmental Economics and Management 103: 102358.

17.

Koremenos

Barbara

(2016) The Continent of International Law. Cambridge: Cambridge University Press.

18.

Levy

Marc A

(1993) European Acid Rain: The power of tote-board diplomacy. In Haas

Peter M

Keohane

Robert Owen

Levy

Marc A

Gasser

Les

(eds) Institutions for Earth: Sources of Effective International Environmental Protection. Cambridge, MA: MIT Press, 75–132.

19.

Ringquist

Evan J

Kostadinova

Tatiana

(2005) Assessing the Effectiveness of International Environmental Agreements: The case of the 1985 Helsinki Protocol. American Journal of Political Science 49(1): 86–102.

20.

Tallberg

Jonas

(2002) Paths to Compliance: Enforcement, management, and the European Union. International Organization 56(3): 609–643.

21.

Tørstad

Vegard

(2020) Participation, Ambition and Compliance: Can the Paris agreement solve the effectiveness trilemma? Environmental Politics 29(5): 761–780.

22.

Tveit

Andreas Kokkvoll

(2018) Norms, Incentives, or Deadlines? Explaining Norway’s noncompliance with the Gothenburg Protocol. Global Environmental Politics 18(1):76–98.

23.

Tveit

Andreas Kokkvoll

(2021) Does Capacity Increase Compliance? Examining evidence from European cooperation against air pollution. International Environmental Agreements 21(2): 323–345.

24.

Victor

David G

(2011) Global Warming Gridlock: Creating More Effective Strategies for Protecting the Planet. Cambridge: Cambridge University Press.

25.

Vollenweider

Jürg

(2013) The Effectiveness of International Environmental Agreements. International Environmental Agreements: Politics, Law, and Economics 13(3): 343–367.

26.

Young

Oran

A Levy

Marc

(1999) The Effectiveness of International Environmental Regimes. In Young

Oran R

(ed.) The Effectiveness of International Environmental Regimes. Cambridge, MA: MIT Press, 1–32.

27.

Zangl

Bernhard

(2008) Judicialization Matters! A comparison of dispute settlement under GATT and the WTO. International Studies Quarterly 52(4): 825–854.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB

0.39 MB