Sampling bias in environmental peacebuilding research

Abstract

Sampling bias poses enormous challenges to research as it can undermine the validity of results, cause significant knowledge gaps, result in research that is driven by convenience rather than need and limit the ability to provide policy advice. In this study, I assess potential sampling biases in the rapidly emerging research field of environmental peacebuilding based on a systematic review of the peer-reviewed literature. The field frequently engages with difficult cases which are hard to access and/or unlikely to confirm theories, hence reducing concerns about sampling by convenience and result validity. By contrast, there are clear knowledge gaps regarding countries highly vulnerable to environmental change, regarding the Pacific region and Latin America, regarding intra-state peacebuilding dynamics in Asia, and regarding countries with weak or isolated civil societies. Voices from the Global South and female scholars are relatively well represented, but further improvements are desirable. Addressing these issues would support environmental peacebuilding in challenging one-sided environment-conflict narratives, adding to the wider peacebuilding literature, and providing policy advice on sustainable peace.

1. Introduction

Recently, the environmental security literature has seen various controversies about sampling biases, for instance in climate change research (Hendrix, 2017; Singh, 2022) and climate-conflict research (Adams et al., 2018; Ide et al., 2018; Levy, 2018). In this context, sampling refers less to the selection of cases by individual studies, which was a key issue in earlier debates around environmental conflicts (Gleditsch, 1998). Rather, selection bias, as discussed by the above-mentioned studies and understood in this article, is concerned with the selection of cases studied by a research field as a whole, and the implications this selection has for knowledge production and representation across the field. While several prior studies have touched upon sampling issues in environmental peacebuilding work (e.g. Johnson et al., 2021; Krampe, 2017), this article provides the first systematic assessment of sampling bias in this research field.

An unbalanced or patchy selection of the sample of cases to be analysed by a research field can be problematic in at least three respects:

• (1) It can affect the validity of the results if cases where a certain phenomenon or causal link does (not) exist are strongly over- or under-represented in the sample, or if overwhelmingly ‘easy cases’ (which are likely to provide support for the theoretical assumptions) are studied. Individual studies might focus on easy cases as a plausibility probe and to establish possible causal mechanisms, but if such cases are overrepresented across the research field as a whole, the resulting knowledge becomes less reliable (Flyvbjerg, 2006). Likewise, best practices identified based on a limited number of cases might work less well (or not at all) in other contexts.

• (2) Biased case selection can lead to important knowledge gaps if theoretically or politically important regions are rarely studied. For instance, a region’s accessibility (through data availability, English being widely spoken and safe field research options) rather than theoretical considerations or ‘objective’ needs can shape case selection. In the worst case, this can result in countries with the greatest need for evidence-based policymaking being studied least (Hendrix, 2017).

• (3) Strongly focusing on certain regions, countries or projects can lead to stigmatisation or depoliticisation. Critics have argued, for example, that environmental security research’s strong focus on Africa risks reproducing colonial narratives of the continent as incapable to manage its resources and as naturally violent (Verhoeven, 2014). Likewise, a one-sided focus on environmental cooperation in a certain region or project can obscure underlying injustices and inequalities. Reynolds (2017), for instance, argues that a focus on the positive aspects of cross-boundary cooperation between Israelis and Palestinians can cloud the view for how such cooperation perpetuates the marginalisation of Palestine.

Before proceeding, a brief introduction to the research field under study here is due. Environmental peacebuilding can be defined as ‘the multiple approaches and pathways by which the management of environmental issues is integrated in and can support conflict prevention, mitigation, resolution and recovery’ (Ide, Bruch, et al., 2021, pp. 2–3) or ‘the sustainable management of natural resources before, during or after conflict, emphasizing the potential for environmental governance – especially cooperative governance between conflict actors – to support peace and stability’ (Krampe et al., 2021, p. 2). According to proponents of the approach, cooperative environmental governance can contribute to peace by promoting economic development, by establishing institutions that facilitate collective action, by building trust and understanding between (potential) conflict parties, and by managing environment-related tensions (Ide, 2019; Johnson et al., 2021).

Environmental peacebuilding is part of the wider research on environmental security. It complements existing work on resource conflicts and climate conflicts by focusing on cooperative environmental governance and peaceful adaptation to climate change, hence challenging one-sided narratives about environment-conflict linkages. At the same time, environmental peacebuilding adds a focus on important topics such as climate change, water security and high-value natural resources to the wider literature on peacebuilding (Barma, 2016; Krampe, 2017). The respective research has considerable potential to inspire policy debates about environmental conflict prevention, conflict-sensitive environmental governance, adaptation to climate change and sustainable peacebuilding.

Research on environmental peacebuilding has developed rapidly in recent years (see Figure 1), including work on gender and environmental peacebuilding (Yoshida & Céspedes-Báez, 2021), more fine-grained data sources (Grech-Madin et al., 2018), the environmental impacts of peacebuilding (Bakaki & Böhmelt, 2021; Murillo-Sandoval et al., 2021) and grassroots environmental peacebuilding (Huda, 2021; Ide, Palmer, et al., 2021). Scholars studying the topic are overwhelmingly using qualitative, small-N research methods, usually based on field research. This makes the selection of cases – and the associated issues or sampling, research focus and representation – particularly acute.

Figure 1.

Cumulative number of empirical environmental peacebuilding studies in peer-reviewed journals.

This is the first study to systematically survey sampling issues in the research field of environmental peacebuilding. In particular, I focus on three core questions: (1) Do needs and access issues impact environmental peacebuilding research? (2) Do sampling patterns result in knowledge gaps on certain regions? (3) Does sampling bias affect the validity of insights from environmental peacebuilding research? Given its relevance when considering issues of bias and representation in research, this study will also briefly discuss (4) the extent to which female scholars and scholars from the Global South have a voice in academic debates on environmental peacebuilding. To address these four issues, I describe my methods in the next section before presenting and discussing key results and eventually drawing a conclusion.

2. Methods

To analyse potential sampling biases and representation issues, this study focuses on peer-reviewed journal articles with a clear empirical and regional focus. Peer-reviewed articles in reputable journals are the ‘gold standard’ outcome of academic research and often form the core of evidence-based knowledge in a research field. ¹ Empirical studies actively select and produce evidence of certain countries or regions. ² Finally, I include only studies with a clear regional focus because large-N, cross-case studies on environmental peacebuilding are rare and focus on global datasets (Barquet et al., 2014; Ide, 2018).

I used a two-staged procedure to identify the relevant studies (see Figure 2 for a visual summary). In the first stage, I drew on the Scopus database to scan for journal articles published before 1 January 2022. Specifically, I run a ‘title + abstract + keywords’ search with the following term:

‘environmental peacemaking’ OR ‘environmental peacebuilding’ OR ‘environmental peace’

Figure 2.

Flow diagram summarising the identification process of relevant studies.

This search term captures all articles framing themselves as contributions to debates about environmental peacebuilding while keeping the sample manageable. The inclusion of search terms such as ‘environment AND diplomacy’ or ‘environment AND cooperation’ would have yielded a large number of studies unrelated to environmental peacebuilding (while many relevant studies using these terms were found during the second stage of the identification process; see below). After removing 24 duplicates, false positives (articles identified by Scopus that were not dealing with environmental peacebuilding at all) and articles with no clear empirical and regional focus, 32 articles remained.

In the second stage, I then scanned four recent review or conceptual articles on environmental peacebuilding (Dresse et al., 2019; Ide, 2019; Johnson et al., 2021; Krampe et al., 2021) and discussed the first-stage list with several researchers active in the field to add missing empirical publications with a clear regional focus. The inclusion criteria were (1) the peer-reviewed journal article framed itself actively as a contribution to the environmental peacebuilding literature and/or (2) the peer-reviewed journal article engaged actively with the environmental peacebuilding literature beyond the citation of a few publications. During the second stage, I added 14 studies, resulting in an overall sample of 46 articles. ³

Subsequently, I read through all 46 articles and noted down the countries and world regions they are dealing with. I only considered the countries/regions on which the articles presented novel empirical research. In addition, I recorded for each article whether it studies environmental peacebuilding within states, between states or a mixture of both (e.g. in the Israeli–Palestinian context).

3. Results and discussion

Overall, the peer-reviewed empirical literature on environmental peacebuilding focused on 47 countries until the end of 2021 and produced 104 country case investigations. The latter refer to a study providing empirical evidence for a certain country. ⁴ Figure 3 visualises the countries studied, while Tables 1 and 2 summarise the most frequently researched countries and regions.

Figure 3.

Map of countries studied by the environmental peacebuilding literature.

Table 1.

Countries covered by three or more journal articles.

Country	Studies
Israel	12
Palestine	10
Colombia	5
Jordan	5
India	4
China	3
Cyprus	3
Democratic Republic of Congo (DRC)	3
North Korea	3
South Korea	3
Lebanon	3
Pakistan	3
Sierra Leone	3

Table 2.

Regions studied by the environmental peacebuilding literature.

Region	Studies
Middle East	37
Asia	29
Africa	20
Europe	9
Latin America	8
North America	1
Pacific	0

The most prominent cluster of cases in the environmental peacebuilding literature is transboundary water cooperation between Israel (12 studies), Jordan (5) and Palestine (10), followed by research on Colombia after the 2016 peace agreement (5), and water- or energy-related cooperation between India (4) and its neighbours. Together, these five countries account for more than one-third of the 104 investigations. Not surprisingly, then, the Middle East (37) and Asia (29) are the most frequently studied regions, followed by Africa (20). Colombia is an outlier as apart from the country’s post-agreement period (five studies), there is very limited attention to Latin America. It is noteworthy that peer-reviewed studies on environmental peacebuilding, despite being such a young field, already cover 47 different countries. When it comes to the geographical scope of the empirical studies, 21 focus on intra-state environmental peacebuilding, 13 on international peacebuilding and 12 on both.

It is interesting to note that there are clear differences between the countries studied by environmental peacebuilding research and those analysed by climate-conflict scholars according to Adams et al. (2018). When comparing the top 15 entries of both lists, ⁵ only three countries appear on both (Israel, India and Palestine), while many countries frequently studied in environmental peacebuilding research are not covered by climate-conflict scholars at all (Colombia, Cyprus, Korea North and South, Lebanon, Sierra Leone) and vice versa (Kenya, Egypt, Nigeria). Consequentially, when only considering the countries covered by at least one body of literature, the correlation between both lists is weak (0.197). While no specific research field can be blamed for it, this discrepancy still inhibits the possibilities for dialogue and for comparing results between environmental peacebuilding and climate-conflict research.

The remainder of this section discusses the results through the lens of three questions before briefly considering concerns over the representation of female and non-Western scholars in the research field.

3.1. Do needs and access issues impact environmental peacebuilding research?

Previous studies have found that scholars working on climate change and conflict tend to focus on accessible countries for which high-quality data and scholarly networks are available, where English is widely used, and where foreign scholars face relatively limited risks to their personal safety (Adams et al., 2018). Furthermore, Hendrix (2017) shows that those countries which are most vulnerable to climate change and which consequentially have the most urgent needs for scientific knowledge (to inspire policy action) are often understudied. Can we find similar patterns for environmental peacebuilding research?

There is mixed evidence that access issues shape case selection in the research field. Israel, Jordan and Palestine – which form the most intensively studied cluster – are easy to access for many international researchers and relatively safe, with low levels of armed conflict intensity and crime. Crucial environmental peacebuilding actors in the region such as EcoPeace Middle East or the Arava Institute publish extensive information on their work and projects, including in English. ⁶ This facilitates the establishment of networks and the gathering of data. Strong local research partners and the widespread use of English also facilitate access (yet, Israel also receives a disproportional amount of attention in the general international relations literature, see Hendrix & Vreede, 2019). Likewise, Colombia saw a considerable inflow of international aid and personnel after the peace agreement, while India is relatively safe, and English is widely used there.

However, looking beyond the very top of the list reveals a more nuanced picture. Several of the countries covered by two or three studies pose enormous language (e.g. Pakistan, South Korea), entry (e.g. North Korea, Syria) and security (e.g. Afghanistan, DRC, Sudan) challenges for many researchers. The respective studies are often the result of impressive networking activities by scholars in the Global North or outstanding research efforts by scholars from the Global South (e.g. Castro, 2018; Kibaroglu & Sayan, 2021; Song & Hastings, 2020). This provides evidence that several environmental peacebuilding researchers go the long way to gather insights from difficult-to-access countries. ⁷

An ambivalent picture emerges when it comes to whether environmental peacebuilding research samples according to need. Of the 17 countries with an active armed conflict in 2020, 59% are covered by the literature, in addition to several fragile and/or post-conflict countries such as Iraq, Pakistan, Sierra Leone or Timor-Leste. Environmental peacebuilding research hence focuses on countries where expertise on transforming conflicts and establishing peace is clearly needed.

The picture changes when considering environmental stress. In theory, if countries face high levels of environmental stress or of vulnerability to environmental change, ⁸ there should be considerable demand for (scientific) knowledge on cooperative environmental governance, conflict-sensitive natural resource management and environmental conflict prevention (Singh, 2022). This is particularly the case as environmental peacebuilding, at least ideally, addresses not only conflict-related but also environmental problems.

However, the correlation of the number of studies per country with ND-GAIN’s climate change vulnerability index (0.037) and the amount of renewable freshwater resources per capita (−0.078) is almost zero and not statistically significant. Likewise, on 12 of the 20 countries most vulnerable to climate change and on 14 of the 20 countries most water scarce, not a single study exists (see Supplemental Material). This indicates that environmental peacebuilding research is often not focusing on those countries where environmental stress is greatest.

3.2. Do sampling patterns result in knowledge gaps?

In the best case, a research field is capable of providing knowledge on all relevant countries and regions. Such relevance can be derived from scientific concerns. For instance, if a phenomenon (e.g. civil war) occurs in several geographical areas which are quite distinct from each other (e.g. Central African Republic, Philippines, Ukraine), we would require evidence from most or all of these regions for valid global inferences. Alternatively, relevance can be a function of interest from decision-makers. Politicians in the European Union, for instance, are more interested in research on environmental migration in North Africa than in similar research for Southeast Asia (Methmann & Rothe, 2014).

Regarding environmental peacebuilding, it is always possible to argue for additional studies on a particular country due to its dire environmental problems, history of organised violence, or geopolitical relevance. The purpose of this sub-section, however, is to point out systematic blind spots on a supra-national level, which result in knowledge gaps for the respective world regions.

The most obvious knowledge gap in the environmental peacebuilding literature concerns the Pacific region. With the partial exception of Timor-Leste (two studies), which is commonly considered as part of Southeast Asia, there is not a single peer-reviewed article on environmental peacebuilding in the Pacific Island countries. This is despite their high vulnerability to environmental change (such as sea-level rise) and a history of political instability in various countries (e.g. Fiji, Nauru, Solomon Islands). Potential explanations for this include the rather small population and the geopolitical irrelevance of the region. ⁹

Likewise, apart from post-agreement Colombia, there is only one study on environmental peacebuilding in Latin America. ¹⁰ This is surprising given the history of civil wars and political unrest in countries such as Guatemala, Haiti and Peru (each not covered by a single study), as well as the entanglement of these conflicts with issues such as land ownership or disaster responses. A lack of Spanish-language skills among Western researchers and a tendency of local scholars to rather use political ecology and environmental justice frameworks (meaning their work is not captured by the search procedure described in Section 2) might explain this knowledge gap. The low availability of the Spanish-language literature via Scopus most likely also contributed to this finding. One should note that limited attention to the Pacific region and Latin America is also a characteristic of the climate-conflict literature (Adams et al., 2018).

While Asia is generally rather well covered, there is a clear focus on international relations between India and its neighbours. Despite past or ongoing insurgencies, studies on intra-state environmental peacebuilding in countries such as India, Myanmar, Nepal, Pakistan or Sri Lanka are lacking. There is also no peer-reviewed study on Central Asia even though concerns about the Aral Sea Basin were key in early debates about environmental peacebuilding (Weinthal, 2002). A considerable knowledge base on Africa exists. Yet, surprisingly, no environmental peacebuilding study covers conservation cooperation in southern Africa and intra-state dynamics around Lake Chad, even though these regions are intensely studied by related research on transboundary conservation (Marijnen et al., 2020) and climate security (Daoust & Selby, 2022).

3.3. Does sampling bias affect the validity of insights?

Based on existing evidence, a number of review articles have recently concluded that environmental cooperation can catalyse peacebuilding processes between and within states, even if only as a minor factor, only through certain mechanisms, and only in specific contexts (for details, see Ide, 2019; Johnson et al., 2021; Krampe et al., 2021). Particularly as these insights are derived from a relatively small set of cases, the question remains whether sampling biases in the research field cast doubt on their validity and generalisability.

The most important concern regarding validity is a confirmation bias: Environmental peacebuilding research might focus on ‘easy cases’ that are relatively safe, free from acute violence, well accessible, and hence most likely to see the occurrence of both environmental cooperation and peacebuilding. This concern is not justified. Researchers focus on numerous difficult cases of intractable conflicts (e.g. India–Pakistan, Israel–Palestine, North Korea–South Korea) or acute violence (e.g. Afghanistan, DRC, Sudan), and often conclude that environmental management has at best a very minor impact on high-level peace processes. This indicates that there is no confirmation bias in the environmental peacebuilding literature.

That said, many studies find a positive impact of environmental cooperation on local, every day or low-level peace processes (e.g. Castro, 2018; Huda, 2021). This might well be caused by an overrepresentation of cases with strong civil society ties and well-established community cooperation, such as Cyprus, Israel–Jordan–Palestine or Lebanon. To corroborate the interlinkages between cooperative environmental governance and local peace, more studies need to focus on cases where civil societies are weak (intra-state level) or not well connected (international level). This would also help to make the voices of those people heard who conduct environmental peacebuilding practices in very difficult contexts, such as Iran (e.g. Samiee, 2022). Furthermore, broadening the set of cases under study would also increase the generalisability of results from environmental peacebuilding research, which are mostly derived from a small number of cases and countries (see Table 1 and Section 3.2).

3.4. How are voices from the Global South and female scholars represented in environmental peacebuilding research?

Gender and postcolonialism are rising and important topics in the environmental peacebuilding literature (e.g. Vélez-Torres & Lugo-Vivas, 2021; Yoshida & Céspedes-Báez, 2021). In the light of this, it is crucial to assess the representation of female and ‘Southern’ scholars in the research field. International relations scholars have long shown that an under-representation of certain groups is not only problematic in terms of fairness but might also result in an ignorance of valuable epistemological, ontological and empirical perspectives that can advance knowledge (Acharya, 2014; Maliniak et al., 2013).

To identify the number of female authors, I manually investigated the first names of all researchers in my sample and then used Sumner’s (2018) automatised Gender Balance Assessment Tool (GBAT). ¹¹ The resulting GBAT score of 41.08 indicates that 41.08% of all authors of peer-reviewed environmental peacebuilding studies are women (or, to be more precise, have a female name). This is broadly in line with Johnson et al.’s (2021) sample of intra-state environmental peacebuilding research, where 48% of all studies have at least one female author.

While a clear gender gap in the research area exists, it is less pronounced than in related fields. Recent reviews of the climate-conflict and climate-migration literatures, for instance, yield GBAT scores of 18.41 (Scartozzi, 2021) and 31.03 (Šedová et al., 2021). ¹² When related to the overall group of researchers and practitioners in the field, the representation gap is even smaller. As of January 2022, 40.24% of the professional members of the Environmental Peacebuilding Association ¹³ have female-associated names (which could very well be a sign of significant entry barriers for women to the research field). In other words, women account for approximately 40.24% of the self-identified environmental peacebuilding scholars and produce 41.08% of the peer-reviewed articles.

When checking the publicly available personal profiles, I found that 32.6% of the first authors in my sample have significant academic roots in the Global South (i.e. they received a BA, MA or PhD degree from, or worked for at least 2 years at universities in Africa, Asia, Latin America, the Middle East or a Pacific Island country). On the one hand, this shows that voices from the Global South are active and established in environmental peacebuilding research. This is good news not only from an equality and fairness point of view, but also from the perspective of knowledge generation: Scholars from the Global South often have networks, academic socialisations, language skills and cultural awareness that allow them to contribute new theories and insights on places hard to study for Northern researchers (see Section 3.1). On the other hand, voices from the Global South still account for less than one-third of all first authors of peer-reviewed articles, indicating a clear need for further improvement, particularly given the far larger population of the Global South (even though the number of active researchers per capita is arguably rather small in many low- and middle-income countries).

4. Conclusion

This article conducted a systematic review of sampling biases in the peer-reviewed, empirical literature on environmental peacebuilding based on 46 journal articles published until December 2021. Environmental peacebuilding research does well by focusing on hard-to-access and ‘difficult’ cases. ¹⁴ Consequentially, there is limited evidence that sampling patterns affect the validity of results or that sampling is strongly based on convenience. The field does also relatively well when it comes to the inclusion of voices from the Global South and ensuring the representation of female scholars, even though further improvements on these fronts are desirable.

Resonating similar calls in climate change research (Hendrix, 2017), environmental peacebuilding should pay more attention to countries highly vulnerable to environmental change. Big knowledge gaps also exist regarding an environment-peace nexus in the Pacific region (covered by no study at all) and Latin America (apart from post-peace agreement Colombia), mirroring similar concerns in climate-conflict research (Adams et al., 2018). Intra-state environmental peacebuilding dynamics in Asia also deserve to be studied in greater details, particularly given the recent history of civil violence in the region. Finally, cases with strong civil society ties and well-established community cooperation are currently overrepresented in environmental peacebuilding research.

These knowledge gaps and sampling biases should be addressed by future research for a number of reasons. First, there is evidence that conflict dynamics, climate-conflict linkages and peacebuilding contexts strongly vary between different world regions and even between countries in the same region (Hao et al., 2022; Mross et al., 2021). We can therefore not assume that insights from a small number of cases (e.g. Israel–Jordan–Palestine, post-2016 Colombia, international cooperation between India and its neighbours) are necessarily applicable on a wider scale. Likewise, good practices that have been identified as working well in such places could work less well elsewhere. Second, and relatedly, large knowledge gaps on particular regions constrain the ability of environmental peacebuilding research to provide tailored policy advice to decision-makers and practitioners in these regions. Third, such advice on sustainable and conflict-sensitive environmental governance could be in particular high demand among countries facing severe environmental problems, which currently do not receive particular attention. Fourth, over-sampling cases with well-established civil societies could tempt scholars to overestimate the impact of environmental cooperation on local or everyday peace processes.

An important concern highlighted by Hendrix (2017) and Singh (2022) is that sampling biases can be self-reinforcing. Once a sufficient amount of (good) publications on a country or region are available, various incentives exist for scholars to focus on these very countries or regions: (1) existing initiatives and actors in the country/region can be traced more easily, (2) more relevant data on the case(s) exist, (3) the case(s) can be more easily portrayed as important or ‘paradigmatic’ cases (Flyvbjerg, 2006, p. 232) and (4) the authors of previous studies can provide scholars moving into the field with contacts, access to networks, advice and even mentoring opportunities. Consequentially, funders, reviewers, editors and supervisors should take into account that analyses of previously understudied cases, even if their execution is harder and patchier, can provide more novel insights on environmental peacebuilding than well-conducted work on established cases (which benefited from the four advantages outlined). ¹⁵

An additional way to deal with the knowledge gaps resulting from sampling biases is to draw on research providing insights on environmental peacebuilding, yet not using the term (or engaging with other work on the issue). Examples include studies on climate security (Petrova, 2022), disaster diplomacy (Kelman et al., 2018) and natural resource management (Ojha et al., 2019). Such publications were not covered by the analysis presented here. However, these literatures can suffer from similar sampling biases (Adams et al., 2018). Furthermore, it remains uncertain whether insights from studies not explicitly engaging with the topic can be easily transferred to debates about environmental peacebuilding.

By addressing the knowledge gaps and sampling biases outlined in this article, environmental peacebuilding research can realise its considerable potential to challenge one-sided environment-conflict narratives, to bring knowledge on environmental stress and natural resources to the peacebuilding literature, and to provide advice on peace-enhancing responses to environmental challenges.

Supplemental Material

sj-xlsx-1-eas-10.1177_27538796221143850 – Supplemental material for Sampling bias in environmental peacebuilding research

Supplemental material, sj-xlsx-1-eas-10.1177_27538796221143850 for Sampling bias in environmental peacebuilding research by Tobias Ide in Environment and Security