Sage Journals: Discover world-class research

Abstract

Efforts to evaluate third-party peacebuilding interventions are welcome but many studies rely on experimental approaches that might be at odds with the theories that underpin the discipline. Rigorously evaluating interventions ill-suited to experimental analyses is just as important, however, especially when programmes adopt novel approaches. In this article, we employ an instrumental variables approach to evaluate one such intervention – the EU Programme for Peace and Reconciliation (PEACE II). Following contemporary peacebuilding theories, PEACE II disseminated funds to grassroots organizations via unique intermediate funding bodies and an innovative open competition. Splitting Northern Ireland into 582 wards, we merge panel data on individuals’ perceptions of neighbourhood quality with PEACE II’s accounts. One-stage analyses show that individuals in treatment regions report significantly elevated perceptions. Two-stage approaches, accounting for biases arising from the rollout method, show no significant relationship. Post-estimation analyses imply that funding did not reach areas with the poorest observable indicators. We thus remain agnostic on the effectiveness of the funded projects but conclude that, despite solid theoretical foundations, weaknesses in the application of these theories hampered potential positive impacts. Future interventions can learn from this and should ensure stronger ties between the theoretical base and how these theories are applied to funding disbursement.

JEL Codes: O18; F35

Keywords

Civil society grassroots impact evaluation instrumental variables Northern Ireland peacebuilding political violence randomized control trial

Introduction

In recent years, the number of third-party peacebuilding interventions has grown markedly (Scherrer, 2012) as has the breadth of programming that comes under the peacebuilding umbrella (UN, 2009). The uptick in attempts to rigorously analyse the impact of these interventions is also welcome, yet the literature most commonly defaults to one of two typologies: those that focus on ‘aid’ as a whole (Azam and Thelen, 2008; Böhnke and Zürcher, 2013; Gutting and Steinwand, 2017; Nielsen et al., 2011; Savun and Tirone, 2011; Young and Findlay, 2011); or those using experimental approaches that require, amongst other things, randomized programme rollout¹ (Ackett et al., 2011; Blattman and Annan, 2011; Blattman et al., 2014, 2017; Fearon et al., 2008; Gaarder and Annan, 2013; Gilligan et al., 2013; Malhotra and Liyanage, 2005; Puri et al., 2017). Although these studies are incredibly useful, the former group lacks nuanced information on what works or why it works at the programme level. The latter group relies on approaches that might be at odds with the programmatic application of contemporary peacebuilding theories that espouse the idea that peacebuilding should come from the bottom up (Belloni, 2001; Byrne, 2001; Lederach, 1997; Saunders, 2000).

Although bottom-up programmes do not inherently contradict the requirements of experimental impact evaluations, practical implementation features of these programmes often do.² Intermediate bodies, for example, that play an important role linking beneficiary communities with donors are likely to interact with the communities they serve. Therefore, a trade-off may exist between the quality of the intervention and the quality of the learning that can be developed from it. On the one hand, there remains a need for robust impact evaluations (Blum, 2011; Nelson, 2008; Wittkowski, 2014). On the other, optimal programme design complicates, or even renders impossible, the analytical approaches that have most commonly facilitated such evaluations.

In this article, we tackle this trade-off by conducting an ex post impact evaluation of a large-scale peacebuilding programme in Northern Ireland (the EU Programme for Peace and Reconciliation, or PEACE II), which focussed on integrating Track III grassroots and civil society actors into the country’s peace process (Byrne et al., 2009; Byrne, 2001; Racioppi and O’Sullivan See, 2007). This aim to involve civil society matches peacebuilding theories but implies three considerations at the operational level: that individuals and/or organizations must self-select their participation in the intervention; that they must be involved in designing and implementing the projects that receive funding; and that intermediate bodies linking donors and beneficiaries may be required to bridge cultural, social and trust gaps. Although the first two requirements are achievable within a randomized framework, it is less obvious how practical or desirable randomizing such intermediate bodies would be in. As per Deaton (2009, 2010), this should not deter the development of impact evaluations, however, especially when the programme has unique features and may act as a model for future interventions. It is therefore important to understand whether or not PEACE II achieved it aims, not just in terms of the programme itself or its scale³ but also in terms of how these novel design features performed and of how they could be expected to perform in other scenarios.

Specifically, PEACE II used a bespoke mechanism that identified and funded projects through the adoption of decentralized and multi-tiered decision making and the creation of intermediate funding organizations (Potter and Egerton, 2011). Intermediate funding organizations solicited applications for support from a wide range of civil society actors before evaluating them against objective criteria. Not only does this ensure the involvement of non-elite actors in the peace process but it should mean that funding is granted to the strongest proposals. This approach has frequently been mooted as a model for the implementation of peacebuilding programmes elsewhere (Racioppi and O’Sullivan See, 2007).⁴ Against this backdrop, PEACE II has largely been ignored or mentioned only in passing in the academic literature (Arthur, 2000; Byrne, 2001; Hayward, 2007a, b; O’Dowd and McCall, 2007; Tannam, 2006, 2007) and empirical analyses of its impact are entirely absent.⁵ To close these knowledge gaps, we explore the impact of PEACE II on individuals living in treated communities. We first ask whether or not PEACE II resulted in a measureable positive impact on the communities that received funding. In doing so, we are the first to rigorously analyse the impact of PEACE II. Subsequently, we contextualize how – if at all – the bottom-up funding mechanisms and intermediate funding bodies influenced outcomes.

We use a unique database built from panel data on individuals’ perceptions of neighbourhood quality from the British Household Panel Survey (BHPS) matched to a record of PEACE II spending by year in each of Northern Ireland’s 582 electoral wards. We conduct one-stage analyses using fixed effects (FE) and random effects (RE) estimators and show a positive and significant relationship between receipt of treatment and perceptions of neighbourhood quality. Noting the potential biases that arise from selection into the funding competition, the evaluation of funding applications and potential quality differences across intermediate funding bodies, we employ a novel instrumental variables approach using the spatial distribution of historical violence in Northern Ireland (Ferguson, 2017; Ferguson and Michaelsen, 2015). Disaggregating violence to electoral ward level and using RE2SLS models, we show no significant relationship between PEACE II programmes and perceptions of neighbourhood quality once biases arising from the rollout method are accounted for. In combination with post-estimation analyses focussing on observable regional ‘hardships’ such as deprivation, this implies that the elevated perceptions of neighbourhood quality were pre-existing. Not only does this call into question the success of PEACE II but it also highlights weaknesses in the bottom-up rollout methodology used. Funds often did not reach the neediest neighbourhoods and, even when they did, structural weaknesses in the capacities of organizations in those neighbourhoods may have limited the performance of the treatment.

Our results are of obvious importance to the peacebuilding literature as we present one of the first quantitative impact evaluations of a theoretically developed intervention. Further, we do so in the context of a unique programme with features that are likely to be used in future, both in continuing programming in Northern Ireland and in new programmes elsewhere. More generally the results are also of interest to other fields where this bottom-up programming and intermediate funding organizations are desirable. Specifically, the concerns raised in this article with regard to targeting and funding mechanisms are analogous to those in a wide range of other environments and to programmes with a wide range of other motivations, for example, community-driven development programmes (CDD) programmes (see King et al., 2010; Wong, 2012), which aim to build community cohesion from grassroots and often involve the use of intermediate organizations that encourage individual participation in the process.⁶

In the case of PEACE II, despite a strong theoretical motivation, the application of bottom-up programming was not a guarantor of success. In peacebuilding settings, this gap between theory and implementation should be of particular concern. Given the difficulty of measuring peace as an outcome (Fearon et al., 2008), it is likely that it is also difficult to measure its absence as an input in funding decisions. Similarly, given the damage that conflict can do to social cohesion (Colletta and Cullen, 2000), concerns arise with the assumptions that underpinned PEACE II’s approach. Those neighbourhoods worst affected by violence and most in need of funding are also likely to be those with the weakest social cohesion and are thus also those least capable of identifying their problems or providing solutions to them. Differences may have been present in management and implementation capacity between the recipients of funds in different conflict exposure strata. In turn, structural biases may have arisen in the process that reduced the capacity of the most damaged neighbourhoods to participate in the competition at all, prevented funds reaching the most damaged neighbourhoods and limited the effectiveness of the programmes in the neighbourhoods that did receive funding. Future interventions using this kind of funding mechanism, should therefore carefully consider how they can more strongly tie together academic theories with real-life funding decisions. Furthermore, checks should be built into these processes to ensure that funding gets to the areas where it is needed most and that recipients receive the required guidance to successfully use the funds and to implement their plans.

The rest of this article is set out as follows. In the next section we discuss the background to PEACE II and the features of the intervention that make it worthy of evaluation as well as making a short review of the wider findings of third-party peacebuilding interventions. In the third section we present our data and methodological approach. In the fourth section we present our results and in the final section we provide our conclusions.

Background

Following multiple paramilitary ceasefires in 1994 and the signing of the Belfast (Good Friday) Agreement in 1998, the state of the Northern Irish conflict was probably ‘ripe’ (Zartman, 1985, 2000) for pro-peace interventions. At the same time, significant distrust remained, both between the nationalist and unionist communities and between grassroots and elites, implying a need for social, as well as political, renewal. In part to benefit from the ripeness of the moment and in part to deal with the social and psychological legacy of the conflict on Northern Ireland’s citizens, the EU developed the Special Support Programme for Peace and Reconciliation in Northern Ireland. In combination, these programmes, which split into PEACE I and PEACE II,⁷ contributed over €1 billion towards building peace through a unique decentralized funding mechanism (Racioppi and O’Sullivan See, 2007). This decentralization of funding was designed to ensure that civil society was involved in efforts to address the legacy of conflict by ensuring that non-governmental organizations (NGOs), voluntary groups, community groups and individuals were included in the peace process. Such approaches build on major peacebuilding theories (Lederach, 1997; Saunders, 2000), which note that in the long term, programmes are only likely to succeed when they involve stakeholders in all strata and at all levels of society. PEACE I and then PEACE II were specifically implemented with these goals in mind and their funding mechanisms designed to ensure, not only that funding reached non-elite actors, but also that these actors were involved at multiple stages of the funding and programming process.

In many ways, these concerns share at least some overlaps with the motivation behind so-called community-driven development interventions (King et al., 2010; Wong, 2012). Like PEACE II’s programming, CDD programmes aim to put their beneficiaries at the centre of the decision-making process. As King et al. (2010) note, this is established through three specific design components. The first is that beneficiary communities are involved in the ‘selection, design and implementation’ of the intervention. The second is that programmes include funding for the implementation of the intervention. The third is that they work through new or pre-existing community organizations in order to establish community participation.

At the same time, the PEACE programmes were designed to be innovative in responding to three particular peacebuilding challenges (Racioppi and O’Sullivan See, 2007) that go beyond typical CDD programming: the timing of the intervention when the conflict was ripe for third party engagement; encouragement of multilevel engagement across all social strata; and addressing of the social and psychological aspects of the conflict, as well as its structural impacts. The second and third of these aims required harnessing voluntary actors to engage in the peace process and to support the efforts of political elites. In turn, these requirements necessitated the development of a unique funding mechanism. These funding schemes involved not only the EU but also the national British and Irish governments, local governments in Northern Ireland and a range of non-governmental bodies. A number of pioneering intermediary sub-national funding bodies were subsequently developed in order to bridge the gaps between elite funders and grassroots beneficiaries, as well as to overcome contestations about funding decisions made at the state level.⁸

This began with the development of District Partnerships (DPs) during PEACE I. These DPs were composed of trade unionists, business people, community and voluntary leaders and politicians from Northern Ireland’s local councils. Within the general focuses of the programme, DPs could set their own strategies and make their own funding decisions, which often involved aiding communities in setting their own priorities and developing their own solutions. Within PEACE II, these DPs formed into more permanent Local Strategy Partnerships (LSPs), which served a similar purpose, although were compositionally slightly different. LSPs maintained a role in distributing funds but had significantly more autonomy and were able to set their own goals and objectives. Finally, throughout the funding period, Intermediate Funding Bodies (IFBs) were developed to focus specifically on the wider aims of sustained involvement of grassroots actors. IFBs were set up to work directly with a range of NGOs on specific projects and were charged with ensuring the inclusion of marginalized populations. In combination, DPs, LSPs and IFBs formed a highly decentralized and unique funding delivery system. Thus, whilst similar to CDD programmes in that these local organizations aimed to harness community participation, the roles, powers and responsibilities of these organizations are significantly broader. The performance of these bodies, therefore, merits deeper study.

Particularly given the strong links of these mechanisms to key peacebuilding theories and their specific aim to engage a broad spectrum of actors, this multi-tiered and decentralized approach is likely to act as a model for future third-party interventions. As such, it is important to understand both the impact of the funded projects themselves and the role of these unique funding mechanisms and intermediate bodies in those outcomes. Given the assumptions implicit in the design of these bodies, such evaluations are doubly important. First, decentralization implies that there are no structural differences in the capacities of local individuals and groups to engage with the funding bodies, to identify their needs or to provide solutions to them. This contrasts with the impacts of conflict literature, which implies that the communities most affected by violence are those that suffer the greatest psychological, economic and social damage (Blattman and Miguel, 2010). Similarly, assumptions are made that the worst affected communities can be a priori identified by these intermediate bodies, which may not be the case given the notorious difficulty in objectively observing abstract concepts like peace. Finally, it also implies that there are no structural differences across conflict-exposure strata in the capacity of recipient groups to organize and manage their grants and to ensure funds are optimally spent to deliver their aims. In this context, the general lack of attention paid to these programmes, is of significant concern.

More generally, however, the idea that peace can be built through third-party interventions is supported by a growing body of literature (see Gaarder and Annan (2013) for a review of sorts). Blattman and Annan (2011), for example, show significantly improved outcomes in a programme focussing on the reintegration of ex-militants in Liberia. Gilligan et al. (2013) show similar effects in Burundi, with participants showing significantly lower incidences of poverty and, accordingly, probably having fewer incentives to engage in antisocial behaviour. Fearon et al. (2008) show the positive impact of a CDD programme in DRC, whilst King et al. (2010) suggest that there are weakly positive impacts of CDD on social cohesion in their study of eight programmes. Malhotra and Liyanage (2005) show positive effects on empathy towards outgroups owing to attendance at peace workshops in Sri Lanka. See, also, Avdeenko and Gilligan (2015) and Fearon et al. (2008) for more evidence on CDD programmes.

Blattman et al. (2017) show positive and significant effects from a joint cognitive behaviour therapy and cash transfer programme in Liberia, whilst Lyall et al. (2017) show potential positive outcomes in an analysis of an employment for peacebuilding programme in Afghanistan.⁹ Okunogbe (2016) shows positive effects on attitudes towards outgroups from a programme that randomly assigns university students to places in Nigeria where their own ethnicity is not a majority. Cilliers et al. (2016) show strengthened social relationships from a reconciliation programme in Sierra Leone, yet note that this came at the expense of participants’ psychological wellbeing. Ahmed (2017) shows positive economic and peace outcomes from the programming of a number of international NGOs in the Blue Nile area of Sudan.

In this context, this evaluation adds to the growing body of evidence on the effectiveness of peacebuilding programmes, whilst shedding light on the performance of a programme with a grouping of unique and highly integrated intermediate bodies that linked communities and their needs to donors.

Data and methods

In the absence of inbuilt quantitative data collection within PEACE II, we create a large database, collected from multiple sources, in order to evaluate the programme. This database is the result of three large-scale data collection/collation efforts. First, we collect survey data at the individual level from the British Household Panel Survey (BHPS) from its inception in Northern Ireland in 2001 until the end of PEACE II programmes in 2007. We match this to a unique and detailed record of all applications for PEACE II funding, which includes the location of the programme, the range of its expected impact, its budget and the success or failure of the application. This data was provided to us upon special request by the PEACE II programme monitoring committee and, to our knowledge, has never before been used in quantitative research. Finally, we develop highly disaggregated data of historical violence in Northern Ireland based on the work of Sutton (1994) and codified to electoral ward level in Ferguson (2017) and Ferguson and Michaelsen (2015). In combination, this database constitutes an unmatched record of data pertinent to post-conflict Northern Ireland. To this we add local economic and deprivation data available from the Northern Ireland Statistics and Research Agency (NISRA).

As per Fearon et al. (2008), the use of secondary data poses certain problems in a peacebuilding context. Chief amongst these is the determination of a suitable outcome variable for empirical analysis. This variable must satisfy two constraints: that it is reasonable to believe that it (latently) captures peace-related outcomes; and that it can reasonably be expected to move in response to the programme’s targets. In the case of PEACE II, this is complicated by the fact that the intervention aimed to build peace via two distinct impact channels: improved economic performance and increased social cohesion. Any outcome variable, therefore, must be expected to move equally with improvements in either of these channels, or in both, as very few funded projects can be linked to a single impact channel. Furthermore, however, we argue that establishing economic or social cohesion effects, alone, is insufficient to measure ‘peace’. Rather, a successful impact on peace should improve this outcome variable over and above the direct but narrower programme effects.

In this way, we consider ‘peace’ first in terms of positive peace, which implies effective social systems and norms that allow constructive management of disagreements (Galtung, 1969), rather than merely the absence of violence. Therefore, peace is considered as the maintenance of the social contract and the commitment to it by the state, sub-national groups and individuals. We view this definition as encompassing, therefore, not just Track I political talks but also the Track III processes that underpin the approaches taken to the PEACE interventions.

We propose that latent perceptions of neighbourhood quality is a suitable proxy¹⁰ as both economic opportunity and social cohesion are robust correlates (Clark and Hunter, 1992; Ellaway et al., 2001). At the same time, however, it is easy to believe that more (positively) peaceful neighbourhoods are more desirable places to live. As such, we have prior expectations that perceptions of neighbourhood quality should improve were the PEACE II interventions successful but that changes in this indicator should not be fully explained by only economic and social outcomes. More generally, it is plausible that people prefer living in peaceful neighbourhoods, satisfying the first constraint.

We source information on these latent perceptions from the first seven waves of the BHPS survey in Northern Ireland. The BHPS is an annual survey of individuals living in the UK, with Northern Ireland oversampled to ensure representativeness in the country. The survey is conducted with approximately 3500 individuals in 2000 households in Northern Ireland. Using data from 2001, the first year for which BHPS data is available, to 2007, the first year after PEACE II ended, we generate an unbalanced panel of n×T = 11,664 individual-years in 523 of Northern Ireland’s 582 electoral wards. As PEACE II aimed to improve the entire community in which a project was funded, rather than just impacting on the project’s participants, the use of secondary survey data is, more appropriate than data from participants.

We focus on two questions of interest in the survey. The first asks individuals whether or not they like the neighbourhood in which they live; the second asks about whether or not they would like to move house.¹¹ Owing to incredibly low variation in the first of these questions, we use the latter for our main analyses.¹² In doing so, we follow the residential mobility literature, stretching back to Speare (1974). Speare (1974) shows that residential satisfaction – including perceptions of latent neighbourhood quality – act as intervening variables in individuals’ desire to move house. Furthermore, he suggests that, once all other intervening variables are accounted for, the residual will measure individuals’ perceptions about the quality of their neighbourhood. Throughout this literature, a number of other intervening variables are mooted, which we include as control variables. These include employment status (Böheim and Taylor, 2003), housing quality (Clark and Huang, 2004) and individual characteristics, such as age, gender and education (Zaiceva and Zimmermann, 2009). By controlling for these individual and locational heterogeneities, individuals’ desire to move home becomes a solid proxy for latent perceptions of neighbourhood quality.¹³ The BHPS includes data on these other variables of interest. We thus garner information on: age; gender; employment status; education; housing tenure type; housing quality; previous moves; and variables of observable neighbourhood quality from the survey. To this data, we add: regional deprivation data sourced from the NISRA multiple deprivation index; regional employment rates; local average incomes; population density; and the proportion of young males (Urdal, 2006) in each ward. We match this information to the ward in which an individual lived in each wave of the survey.

We link these perceptions and control variables to the spatial variation in PEACE II spending (Figure 1). Our PEACE II database includes information on all 12,000+ applications for PEACE II funding, including: the success of the application; the amount of funding delivered; the start and end dates of the funding period; and the location(s) where the project expected to have impacts. We restrict our interest to successful applications from Northern Ireland and to those whose impact was expected to have an impact within a single electoral ward.¹⁴ This constitutes some 85% of the projects that were funded in Northern Ireland.¹⁵ Given the nature of funded projects, there are few good reasons to believe that impacts should spill over in space. Most projects focus on developing very local infrastructure within the ward, or even within sub-ward areas.¹⁶ Our main analyses, therefore, do not consider the potential for spatial effects.¹⁷

Figure 1.

PEACE II spending by electoral ward. Source: authors’ construction of PEACE II monitoring committee data.

We derive our main treatment variable of interest from this data: the total project expenditure, by project start year, in each ward. For robustness of results, we generate three further definitions of treatment: the number of funded projects by ward-start year; total expenditure by ward-end year; and the number of projects of ward-end year. As shown in Table 1, total spending by ward-start year varies between €0 and €4 million, whilst the number of projects varies between 0 and 9. Of Northern Ireland’s 582 wards, approximately 80% received some form of funding. In line with the Special EU Programmes Body’s suggestion that impacts may only become obvious in the longer term, we generate further treatment variables that look at spending lagged over the previous three years.

Table 1.

Descriptive statistics of PEACE II and violence data

	Mean	Standard deviation	Minimum	Maximum
Spending (€000s)	154.85	283.63	0	3346
Treatments	1.54	1.18	0	9
Deaths	3.67	7.88	0	81
Deaths/1000 inhabitants	1.21	2.40	0	25
Deaths ≥ 1 (median)	0.61	0.49	0	1
Deaths ≥ 4 (mean)	0.27	0.44	0	1
Deaths ≥ 9 (top decile)	0.11	0.31	0	1
N	11,664
I	1704

Note: Data based on authors’ construction of Sutton (1994).

We write the basic relationship between PEACE II and individual-level perceptions of neighbourhood quality as a hierarchical model:

S T A Y_{i j k t} = β_{1} T R E A T_{k t} + β_{2} X_{i t} + β_{3} Y_{j t} + β_{4} Z_{k t} + u_{i t} + v_{j t} + w_{k t} + ϵ_{i j k t}

(1)

where i is the subscript for individual-level variables, j for house-level variables and k for ward-level neighbourhood variables. STAY is a binary variable that determines whether or not an individual wishes to move house at time t, whilst TREAT indicates the degree of treatment a neighbourhood as received. X, Y and Z are vectors of exogenous control regressors at individual, household and ward level, respectively, whilst u, v and w are unobserved heterogeneities at each level.

Owing to the bottom-up nature of PEACE II’s funding decisions, it is difficult to believe that expenditure was randomly distributed throughout Northern Ireland. Significant biases are therefore likely to arise should we employ only linear models. For example, should spending have successfully targeted the neighbourhoods most in need, ordinary least squares models may report a false negative impact of the programmes as, although perceptions may have improved, they are likely to remain lower than in unaffected neighbourhoods. Alternatively, by asking people to come together, some threshold of neighbourhood cohesiveness may have been required, leading to upwardly skewed ordinary least squares estimates as funding only reached neighbourhoods with already elevated quality perceptions. At individual and neighbourhood levels, other concerns arise. These include unobserved preferences and unobserved indicators of neighbourhood quality. To overcome these biases, we follow an instrumental variables approach (see Angrist and Pishke, 2009). This approach requires at least one ‘instrument’ that is correlated with the endogenous treatment variable but not with the regression error term. Such approaches are common in impact evaluation (see Abadie et al., 2002; Atoyan and Conway, 2006; Deaton, 2009; Chen and Ravallion, 2003; Duryea and Morrison, 2004; Khandker et al., 2010; Maredia et al., 2000).

We construct two instruments based on the spatial variation of historical violence in Northern Ireland. This is a geo-coded event count database of all fatal events directly related to The Troubles, based on Sutton (1994). This database links each event to an electoral ward in Northern Ireland and then sums deaths in that ward over the entire period of the conflict and includes approximately 3000 fatalities. As shown in Table 1 the mean number of fatalities in an electoral ward is just under four but with significant variation: almost 200 wards experienced no fatal violence at all, whilst some experienced in excess of 75 fatalities. From this data we generate the deaths instrument, which is total fatalities in a given electoral ward. We also generate sldeaths, which is a spatially weighted lag of violence that occurred elsewhere in Northern Ireland¹⁸ (see Figure 2).

Figure 2.

Spatial distribution of historical violence in Northern Ireland. Source: authors’ construction of Sutton (1994).

We implement the IV approach by regressing the endogenous variable on all exogenous regressors and on the instruments, denoted IV:

T R E A T_{k t} = α_{0} + α_{1} X_{i t} + α_{2} Y_{j t} + α_{3} Z_{k t} + α_{4} I V_{k} + η_{k}

(2)

where η is the regression error term and α_i the regression coefficients.

In order for the instruments to be valid, α₄ must be correlated with TREAT but not with the regression error term. Put differently, $α_{4} \neq 0$ and $C O V (I V, ϵ) = 0$ . As we have two instruments and only one endogenous regressor, our analysis is overidentified and the validity of the instruments can be shown statistically. At the same time, given the use of two instruments derived from the spatial distribution of historical violence, it is prudent to discuss this qualitatively. First, as violence ended a decade before the beginning of our analysis, we see no reason to believe that it should be a direct determinant of individuals’ perceptions about the current quality of their neighbourhood.¹⁹ As such, any residual effects are indirect. Violence could cause persistent regional deprivation, low-quality services, higher crime, poor social relations and so on. We control for these effects using data from the Northern Ireland Multiple Deprivation Measure (NIMDM), which includes ward-level information on seven domains of deprivation: income deprivation; employment deprivation; health deprivation and disability; education, skills and training deprivation; proximity to services; living environment;²⁰ and crime and disorder. As such, the inclusion of these additional controls and the use of violence that ended long before our sample period offer evidence that refutes the conceivable sources of bias from our deaths instrument. As sldeaths relates to violence that takes place elsewhere in the country, it is more obviously exogenous. As shown in Table A2, our instruments pass the Sargan–Hansen J-test of instrument validity, further reinforcing the appropriateness of our empirical strategy.²¹

We estimate our one-stage analyses using typical RE and FE approaches and our two-stage models using simple two-stage least squares (2SLS) and random effects two-stage least squares (RE2SLS) owing to Balestra and Varaharajan-Krishnakumar (1987).

Results

Our first results link the spatial variation of PEACE II funding to historical regional violence. The results of these artificial first stages are presented in row 1 of Tables A1 and A2 for the deaths instrument and row 2 for the sldeaths instrument. Table A1 shows the first stage of our 2SLS specification and Table A2 for the RE2SLS models. These results show that both instruments are strong predictors of all formations of our treatment variable. The Cragg–Donald F-statistics for all models are higher than the 1% Stock–Yogo thresholds, showing the strength of the estimates and suggesting that minimal bias will arise from their use. We are unable to reject the null hypothesis of valid over-identifying restrictions in the Sargan–Hansen J-test, supporting the exogeneity of our instruments. Unsurprisingly, historical intensity of violence and its spatial lag are positive and accurate predictors of the locations that received the largest proportions of PEACE II expenditure.

Truncated results from the main analyses are presented in Table 1 for spending by ward-start year and in Table 2 for the number of funded projects. Columns 1 and 2 show the results from the one-stage FE analysis, column 2 for the one-stage RE and columns 3 and 4 from the 2SLS and RE2SLS analyses, respectively. The one-stage analyses suggest a positively and strongly significant impact of PEACE II expenditure on perceptions of neighbourhood quality. Taken in isolation, this would imply that the programmes were successful in delivering their goals. In the two-stage analyses, however, the sign of the coefficient changes and when individual-, house- and ward-level unobservables are accounted for in the RE2SLS approach, the coefficients become insignificant. This implies that PEACE II had no discernible causal impact on individuals’ perceptions of neighbourhood quality. Given that economic and social cohesion, as well as peace factors, are expected determinants of this variable, this implies that PEACE II did not achieve its broad or narrow aims.

Table 2.

Baseline analysis – effect of total expenditure by start year

	(1)	(2)	(3)	(4)
	FE	RE	2SLS	RE2SLS
Spending	1.50 × 10⁸***	1.2 × 10⁸***	−4.15 × 10⁸*	−3.13 × 10⁹
	(4.55 × 10⁹)	(4.40 × 10⁹)	(2.23 × 10⁸)	(2.57 × 10⁸)
Individual	Yes	Yes	Yes	Yes
House	Yes	Yes	Yes	Yes
Ward	Yes	Yes	Yes	Yes
N	11,664
I	1704

Note: Standard errors in parentheses. ***p < 0.01; **p < 0.05; *p < 0.1. N = Observations; I = individuals.

Table 3.

Effect of number of projects by start year

	(1)	(2)	(3)	(4)
	FE	RE	2SLS	RE2SLS
Treatments	0.004**	0.003*	−0.013*	−0.0001
	(0.002)	(0.002)	(0.008)	(0.010)
Individual	Yes	Yes	Yes	Yes
House	Yes	Yes	Yes	Yes
Ward	Yes	Yes	Yes	Yes
N	11,664
I	1704

Note: Standard errors in parentheses. ***p < 0.01; **p < 0.05; *p < 0.1. N = observations; I = individuals.

The full results from the analysis by spending in start year can be found in Table A6. By and large, the controls display anticipated signs and levels of significance anticipated in the literature to date. Only marital status and employment status buck this trend. Individuals prefer to stay in higher-quality houses and in higher-quality locales. Individuals with higher education are more likely to desire mobility than less-educated counterparts. Those who own their homes and those in larger residences are more likely to wish to stay than those who feel that their homes lack space. Local factors such as high deprivation and higher population density are also correlated with preferences to move. Given the work stemming from Speare (1974), these results embed our findings within this wider literature, whilst also supporting the validity of our approach.

Results are robust to alternative specifications of the treatment variable, showing no deviations from the headline results in our preferred RE2SLS analyses (see Tables A3 –A5). These results also show that lagged spending is insignificant in all specifications. These findings stand in contrast to theoretical priors, which would anticipate stronger peacebuilding results from bottom-up grassroots-focussed interventions. Given the upward skew on the coefficients stemming from rollout-related biases, however, the most likely driver of these results is that funds did not make it to the neighbourhoods with the lowest levels of latent quality. Simple post-estimation analyses confirm this. The unconditional correlation between PEACE II spending and the NIMDM is only 16%, despite the most deprived wards in Northern Ireland experiencing the worst of the violence (Ferguson and Michaelsen, 2015).

In this context, we remain agnostic on whether or not the funded projects were inherently flawed. Rather, we note that our results are most likely a direct result of failures in the rollout methodology employed during PEACE II. At best, this implies difficulties in identifying the areas most in need but, at worst, it implies a structural issue that systematically excluded applications from the communities most in need.

Conclusions

PEACE II was a six-year multi-million Euro programme designed to build on the Track I political process to bring a long-term and sustainable peace to Northern Ireland. During the programme, almost €1 billion was spent on community-instigated projects with the aim of reconciling past differences and developing Northern Ireland economically and socially. Despite the huge outlay, the programme’s unique features and the idea that it should act as a model for future peacebuilding programming, however, impact evaluations and specific focus on it in any academic literature have been largely absent. In this article, we close this gap but show no significant impact of the intervention once we control for potential sources of bias and other intervening variables. Our post-estimation analyses suggest that this occurred, at least in part, because spending did not reach the communities most in need. This relates to three critical assumptions built into PEACE II’s rollout method: that the communities most in need were equally capable of identifying their needs and the solutions to those needs; that they were equally capable of putting together successful applications for funding; and that they are equally capable of managing and using funds received.²² Given the damage conflict is known to cause in exposed communities, any one of these assumptions could seem strong. Thus, despite being built on solid peacebuilding theories, the application of these theories in real-life scenarios limited potential impacts.

This finding provides important information for future programmes, both in Northern Ireland and elsewhere, especially if PEACE II’s approach is to be used as a model for other interventions. Caution must be urged in the application of bottom-up programming, particularly when there is a competitive aspect to how funds are disseminated. Despite the strong theoretical grounds, targeted spending can only maximize benefits if it reaches those it intends to target. This is uniquely difficult in peacebuilding settings, not least because peace itself is so difficult to observe or measure. In turn, future programmes not only need to ex ante target the regions or individuals most in need but must also have appropriate inbuilt checks and balances. These checks and balances should ensure at least three things: first, that spending is reaching the communities most at need; second, that those communities are not excluded by the rollout competitions employed; and third, that those communities have access to appropriate forms of help in identifying their problems, in designing projects to overcome them, and in suitably managing funds that are successfully received.

More generally these results also show that more evidence is also required on the links between peacebuilding theories and their application in the field, particularly when that application applies to large-scale third-party peacebuilding interventions. Specifically, this work should focus on what these theories imply for programme design and, in turn, on how programme design can incorporate these theories without jeopardizing performance. Development of practical advice that bridges academic research and programme implementation in the field, therefore, is required. Such work will further strengthen the development of bottom-up peacebuilding programmes and ensure that the next generations of interventions are capable of delivering stronger and more robust outcomes.

Supplementary Material

Supplementary Material, CMP766865_online_appendix – Money can’t buy love but can it buy peace? Evidence from the EU Programme for Peace and Reconciliation (PEACE II)

Online Appendix tables A8 and A9

Footnotes

Appendix

Table A7.

Inclusion of spatial lag of expenditure

	(1)	(2)	(3)
	FE	RE	RE2SLS
spending	1.43 × 10⁻⁸***	1.2 × 10⁻⁸***	9.10 × 10⁻⁹
	(4.69 × 10⁻⁹)	(4.54 × 10⁻⁹)	(4.27 × 10⁻⁸)
slspending	5.47 × 10⁻⁷	−4.75 × 10⁻⁷	−3.15 × 10⁻⁷
	(9.51 × 10⁻⁷)	8.94 × 10⁻⁷	(2.26 × 10⁻⁶)
Constant	0.151	−5.356**	−5.358*
	(3.900)	(2.800)	(2.815)
N	11,664
I	1704

Note: Standard errors in parentheses. ***p < 0.01; **p < 0.05; *p < 0.1.

Acknowledgements

The authors are grateful to three anonymous reviewers, the participants of the 8th Annual HiCN Workshop in Aix-en-Provence and the 2013 Joint PSS-ISA conference in Budapest and to Topher L. McDougal and Maren M. Michaelsen for valuable comments. We especially thank the PEACE II Programme Monitoring Committee for providing the detailed application and funding information that made this research possible.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Supplemental material

All data, replication materials, and instructions regarding analytical materials upon which published claims rely, along with the Online Appendix, are available online through the SAGE CMPS website:

Notes

References

Abadie

Angrist

Imbens

(2002) Instrumental variables estimates of the effect of subsidized training on the quantiles of trainee earnings. Econometrica 70(1): 91–117.

Ackett

Borsenberger

Dickes

Sarrancino

(2011) Measuring and validating social cohesion: A bottom up approach. CEPS Working Paper No. 2011-08. Center for European Policy Studies, Brussels.

Ahmed

(2017) Impact of development interventions on peace building in Blue Nile State – Sudan. Available from: http://repository.sustech.edu/bitstream/handle/123456789/18421/Impact%20of%20Development%20..........pdf?sequence=1&isAllowed=y

Alderman

(2002) Do local officials know something we don’t? Decentralisation of targeted transfers in Albania. Journal of Public Economics 83(3): 375–404.

Anderson

(2000) Economic voting and political context: A comparative perspective. Electoral Studies 19(2): 151–170.

Angrist

Pischke

(2008). Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton, NJ: Princeton University Press.

Arthur

(2000) Special Relationships: Britain, Ireland and the Northern Ireland Problem. Belfast: Blackstaff Press.

Atoyan

Conway

(2006). Evaluating the impact of IMF programs: A comparison of matching and instrumental-variable estimators. The Review of International Organizations 1(2): 99–124.

Avdeenko

Gilligan

(2015) International interventions to build social capital: Evidence from a field experiment in Sudan. American Journal of Political Science 109(3): 427–449.

10.

Azam

Thelen

(2008) The roles of foreign aid and education in the war on terror. Public Choice 135(3): 375–397.

11.

Balestra

Varadharajan-Krishnkumar

(1987) Full information estimations of a system of simultaneous equatiuons with error component. Econometric Theory 3(2): 223–246.

12.

Barrett

Carter

(2010) The power and pitfalls of experiments in development economics: Some non-random reflections. Applied Economic Perspectives and Policy 32(4): 515–548.

13.

Baston

Monnat

(2015) Distress in the desert: Neighbourhood disorder, resident satisfaction and the quality of life during the Las Vegas Foreclosure Crisis. Urban Affairs Review 51(2): 205–238.

14.

Basu

(2002) The new empirical development economics: Remarks on its philosophical foundations. Economic and Political Weekly XL 40: 4336–4339.

15.

Baum

(2012) sspecialreg: Stata module to estimate binary choice model with discrete endogenous regressor via special regressor method. Available from: http://ideas.repec.org/c/boc/bocode/s457546.html

16.

Becker

(1968) Crime and punishment: An economic perspective. In: The Economic Dimensions of Crime. Basingstoke: Palgrave Macmillan, pp. 169–217.

17.

Belloni

(2001) Civil society and peacebuilding in Bosnia and Herzegovina. Journal of Peace Research 38(2): 163–180.

18.

Blattman

Annan

(2011) Reintegrating and employing high-risk youth in Liberia: Lessons from a randomised evaluation of a landmine action agricultural training programme. Poverty Action Document. Available from: http://www.poverty-action.org/sites/default/files/blattman_annan_ex-com_reintegration_ipa_liberia_1.pdf

19.

Blattman

Miguel

(2010) Civil war. Journal of Economic Literature 48(1): 3–57.

20.

Blattman

Fiala

Martinez

(2014) Generating skilled self-employment in developing countries: Experimental evidence from Uganda. The Quarterly Journal of Economics 129(2): 697–752.

21.

Blattman

Jamison

Sheridan

(2017) Reducing crime and violence: Experimental evidence from cognitive behavioural therapy in Liberia. American Economic Review 107(4): 1165–1206.

22.

Blum

(2011) Improving peacebuilding evaluation: A whole field approach. United States Institute of Peace Special Report. Available from: https://www.usip.org/sites/default/files/resources/Improving_Peacebuilding_Evaluation.pdf

23.

Böheim

Taylor

(2003) Tied down or room to move? Investigating the relationships between housing tenure, employment status and residential mobility in Britain. Scottish Journal of Political Economy 49(4): 369–392.

24.

Böhnke

Zürcher

(2013) Aid, minds and hearts: The impact of aid in conflict zones. Conflict Management and Peace Science 30(5): 411–432.

25.

Byrne

(2001) Consociational and civic society approaches to peacebuilding in Northern Ireland. Journal of Peace Research 38(3): 327–352.

26.

Byrne

Arnold

Fissuh

Standish

Irvine

Tennent

(2009) The EU PEACE II Fund and the International Fund for Ireland: Nurturing cross-community contact and reconciliation in Northern Ireland. Geopolitics 14(4): 630–652.

27.

Chen

Ravallion

(2003) Hidden impact? Ex-post evaluation of an anti-poverty program. World Bank Policy Research Working Paper, 3049.

28.

Cilliers

Dube

Siddiqi

(2016) Reconciling after civil conflict increases social capital but decreases individual well-being. Science 352: 787.

29.

Clark

Huang

(2004) Linking migration and mobility: Individual and contextual effects in housing markets in the UK. Regional Studies 38(6): 617–628.

30.

Clark

Hunter

(1992) The impact of economic opportunity, amenities and fiscal factors on age-specific migration rates. Journal of Regional Science 32(3): 349–365.

31.

Colletta

Cullen

(2000) Violent Conflict and the Transformation of Social Capital: Lessons from Cambodia, Rwanda, Guatemala, and Somalia. Washington, DC: World Bank.

32.

Conning

Kevane

(2002) Community-based targeting mechanisms for social safety nets: A critical review. World Development 30(3): 375–394.

33.

Deaton

(2009) Instruments of development: Randomization in the tropics, and the search for the elusive keys to economic development. National Bureau of Economic Research Working Paper no. w14690.

34.

Deaton

(2010) Instruments, randomisation and learning about development. Journal of Economic Literature 48: 424–455.

35.

Deglow

(2016) Localised legacies of civil war: Postwar violent crime in Northern Ireland. Journal of Peace Research 53(6): 789–799.

36.

Duryea

Morrison

(2004). The effect of conditional transfers on school performance and child labor: Evidence from an ex-post impact evaluation in Costa Rica. Inter-American Development Bank Working Paper no. 505.

37.

Ellaway

Macintyre

Kearns

(2001) Perceptions of place and health in socially contrasting neighbourhoods. Urban Studies 38(12): 2299–2316.

38.

Fearon

Humphreys

Weinstein

(2008) can development aid contribute to social cohesion after civil war? Evidence from a field experiment in post-conflict Liberia. American Economic Review: Papers and Proceedings 99(2): 287–291.

39.

Ferguson

(2017) Just the two of us? Civil conflicts, pro-state militants and the violence premium. Terrorism and Political Violence 29(2): 296–322.

40.

Ferguson

Michaelsen

(2015) Money changes everything? Education and regional deprivation revisited. Economics of Education Review 48(C): 129–147.

41.

Gaarder

Annan

(2013) Impact evaluation of conflict prevention and peacebuilding interventions. Policy Research Working Paper no. 6496. Washington, DC: World Bank.

42.

Galtung

(1969) Violence, peace and peace research. Journal of Peace Research 6(3): 167–191.

43.

Gilligan

Mvukiyehe

Samii

(2013) Reintegrating ex-rebels into civilian life: Quasi-experimental evidence from Burundi. Journal of Conflict Resolution 57(4): 598–626.

44.

Gutting

Steinwand

(2017) Donor fragmentation, aid shocks and violent political conflict. Journal of Conflict Resolution 61(3).

45.

Hayward

(2007a) Building on the EU’s Legacy: Cross-border cooperation in Ireland. Administration 55(3): 51–74.

46.

Hayward

(2007b) Mediating the European ideal: Cross-border programmes and conflict resolution on the island of Ireland. Journal of Common Market Studies 45(3): 675–693.

47.

Khandker

Koolwal

Samad

(2010) Handbook on Impact Evaluation: Quantitative Methods and Practices. Washington, DC: World Bank.

48.

King

Samii

Snilstveit

(2010) Inventions to promote social cohesion in Sub-Saharan Africa. Journal of Development Effectiveness 2(3): 336–370.

49.

Lederach

(1997) Building Peace: Sustainable Reconciliation in Divided Societies. Washington, DC: USIP Press, p. 4.

50.

Lewbel

(2012) An overview of the special regressors method. In: Racine

Ullah

(eds), Handbook of Applied Nonparametric and Semi-Parametric Econometrics and Statistics. Oxford: Oxford University Press.

51.

Ludwig

Duncan

Gennetial

Katz

Kessler

Kling

Sanbonmatsu

(2012) Neighbourhood effects on the long-term well-bring of low-income adults. Science 337(6101): 1505–1510.

52.

Lyall

Zhou

Imai

(2017) Reducing insurgent support among at-risk populations: Experimental evidence from cash transfers and livelihood training in Afghanistan. Available from: https://imai.princeton.edu/research/files/invest.pdf

53.

Malhotra

Liyanage

(2005) Long-term effects of peace workshops in protracted conflicts. Journal of Conflict Resolution 49(6): 908–924.

54.

Maredia

Byerlee

Anderson

(2000) Ex post evaluation of economic impacts of agricultural research programs: a tour of good practice. Available from: http://www.researchgate.net/profile/Jock_Anderson/publication/252875679_Ex_Post_Evaluation_of_Economic_Impacts_of_Agricultural_Research_Programs_A_Tour_of_Good_Practice/links/00b4953bbd5774dab2000000.pdf

55.

Mercy Corps (2015) Does youth employment build stability? Mercy Corps Evidence Paper. Available from: https://d2zyf8ayvg1369.cloudfront.net/sites/default/files/MercyCorps_AfghanistanINVEST_ImpactEvaluation_2015.pdf

56.

Nelson

(2008) Are we ready for RCTs? Monday Developments. Available from: http://www.3ieimpact.org/media/filer_public/2012/05/07/AreWeReadyforRCTs.pdf

57.

Nielsen

Findley

Davis

Candland

Nielson

(2011) Foreign aid shocks as a cause of violent armed conflict. American Journal of Political Science 55(2): 219–232.

58.

NISRA (2005) Attitudinal survey: A NISRA report for the distinctiveness working group, PEACE II Monitoring Committee. Available from: http://www.seupb.eu/Libraries/PEACE_Programme_Evaluations/Attitudinal.sflb.ashx

59.

O’Dowd

McCall

(2007) The voluntary sector: Promoting peace and cooperation. In: Coakley

O’Dowd

(eds), Crossing the Border: New Relationships Between Northern Ireland and the Republic of Ireland. Dublin: Irish Academic Press.

60.

Okunobe

(2016) Does exposure to other ethnic groups promote national integration? Evidence from Nigeria. Harvard University Working Paper. Available from: http://scholar.harvard.edu/oye/publications/impact-interethnic-migration-okunogbe

61.

Potter

(2013) Evaluation of the PEACE III Programme. Northern Ireland Assembly Research and Information Service. Briefing Paper 71/13 NIAR 681-11.

62.

Potter

Egerton

(2011) The EU PEACE and INTERREG Programmes in Northern Ireland. Northern Ireland Assembly Research and Information Service. Briefing Paper 126/13 NIAR 189-13.

63.

Powell

Whitten

(1993) A cross-national analysis of economic voting: Taking account of the political context. American Journal of Political Science 37(2): 391–414.

64.

Puri

Aladysheva

Iversen

Ghorpade

Brück

(2017) Can rigorous impact evaluations improve humanitarian assistance? Journal of Development Effectiveness 9(4): 519–542.

65.

Racioppi

O’Sullivan See

(2007) Grassroots peacebuilding and third-party intervention: The European Union’s Special Support Programme for Peace and Reconciliation in Northern Ireland. Peace and Change 32(3): 361–390.

66.

Saunders

(2000) Interactive conflict resolution: A view for policy makers on making and building peace. In: Stern

Druckman

(eds), International Conflict Resolution After the Cold War. Washington, DC: The National Academies Press, .

67.

Savun

Tirone

(2011) Foreign aid, democratization and civil conflict: How does democracy aid affect civil conflict? American Journal of Political Science 55(2): 233–246.

68.

Scherrer

(2012) Measuring the impact of peacebuilding interventions on rule of law and security institutions. SSR Papers no. 6.

69.

Sirgy

Cornwell

(2002) How neighbourhood features affect quality of life. Social Indicators Research 59(1): 79–114.

70.

Speare

(1974) Residential satisfaction as an intervening variable in residential mobility. Demography 11(2): 173–188.

71.

Sutton

(1994) Bear in Mind these Dead: An Index of Deaths from the Conflict in Ireland. Belfast: Beyond the Pale.

72.

Tannam

(2006) Cross border cooperation between Northern Ireland and the Republic of Ireland: Neo-functionalism revisited. The British Journal of Politics and International Relations 8(2): 256–276.

73.

Tannam

(2007) Public policy: The EU and the Good Friday Agreement. In: Coakley

O’Dowd

(eds), Crossing the Border: New Relationships Between Northern Ireland and the Republic of Ireland. Dublin: Irish Academic Press.

74.

UN (2009) The challenge of sustaining peace. United Nations. Available from: http://www.un.org/pga/wp-content/uploads/sites/3/2015/07/300615_The-Challenge-of-Sustaining-Peace.pdf

75.

Urdal

(2006) A clash of generations? Youth bulges and political violence. International Studies Quarterly 50(3): 607–629.

76.

Wittkowski

(2014) A question of plausibility. Or: The art of evaluating peacebuilding interventions. Center for International Peace Operations Policy Brief, March.

77.

Wong

(2012) What have been the impacts of world bank community-driven development programs? CDD impact evaluation review and research implications. Available from: http://documents.worldbank.org/curated/en/967431468161087566/pdf/695410WP0SW0CD00Box370017B00PUBLIC0.pdf

78.

Young

Findlay

(2011) Can peace be purchased? A sectoral-level analysis of aid’s influence on transnational terror. Public Choice 149(3–4): 365–381.

79.

Zaiceva

Zimmermann

(2009) Scale, diversity and determinants of labour migration in Europe. Oxford Review of Economic Policy 24(3): 428–452.

80.

Zartman

(1985) Ripe for Resolution. Oxford: Oxford University Press.

81.

Zartman

(2000) Ripeness: The hurting stalemate and beyond. In: Stern

Druckman

(eds), International Conflict Resolution After the Cold War. Washington, DC: The National Academies Press.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB