Sage Journals: Discover world-class research

Abstract

While previous research has focused on terrorist attacks and natives’ attitudes towards immigration, we examine the effect of anti-refugee attacks on refugees’ attitude towards the host country. We use survey data from the 33rd wave of the German Socio-Economic Panel as the fieldwork period overlapped with the infamous anti-refugee riots in Bautzen and as the survey includes a refugee sample. Making use of this natural experiment, we find significant and negative short-term effects of the riots on respondents’ perception of Germany, as well as low geographic variation. Such natural experiments in the form of unexpected events during survey design offer social scientists the possibilities to identify causal effects from observational survey data as they split respondents into a control and treatment group. Given the vast amount of (cross-)national survey data, often including specific subsamples, our study demonstrates the great potential of natural experiments for sociological research on minority groups in society.

Keywords

ethnic violence natural experiment refugees SOEP survey data unexpected events during survey design

Introduction

In the last decade, many European (and non-European) countries experienced a large influx of refugees and large parts of the host societies were welcoming and supported their integration into society (Akrap, 2016). However, there was also a dark side in the form of increasing numbers of hate crimes towards refugees and ethnic minorities (Pai, 2020). Little is known about the causal effects of such hate crimes on refugees: While both Ben Farhat et al. (2018) and Bouhenia et al. (2017) document and describe experiences with violence and psychological distress of refugees using interview data collected in Greece and in France, respectively, and Schwitter and Liebe (2022) use population survey data to provide correlational evidence that refugees feel less positive about Germany in regions with a high accumulated share of arson attacks on refugee homes, the identification of causal effects remains challenging with such data and analytical approaches. A better understanding on how refugees perceive their host country and how this perception can change over time provides important insights into the factors that influence refugees’ social integration and their overall well-being.

While randomised experiments are considered the gold standard in causal analysis (Rubin, 2008), they are often extremely difficult or even unethical to implement, especially when researching sensitive topics. Particularly in such instances, naturally occurring experiments can offer a valuable and useful study design. Unlike controlled experiments, natural experiments occur when circumstances or events create a situation where individuals are exposed to experimental and to control conditions depending on factors outside of the researchers’ control. Thus, in natural experiments, the assignment of individuals to different conditions is determined by external factors. They often arise from naturally occurring events, policy changes, geographical variation or historical incidents (see Dunning, 2012). Unforeseen and salient events during the fieldwork of public opinion surveys can also work as such external factors and split the sample of respondents into a treatment – those surveyed after the event – and a control group – those surveyed before the event (Muñoz et al., 2020); they are also called unexpected events during survey design (UESD). When specific assumptions are met, causal effects can be assessed with this setup.¹ The advantage of natural experiments lies in their ability to approximate the conditions of a controlled experiment while taking advantage of existing real-world situations. By leveraging these situations, researchers can observe the effects of an independent variable (the external factor) on variables of interest by comparing the control and the treatment group, providing insights into causal relationships.

In this research note, we consider the riots in the East-German town of Bautzen to be an unexpected event during the survey fieldwork of the 33rd wave of the German Socio-Economic Panel (SOEP). Employing a quasi-experimental approach through this setup will allow us to assess potential causal effects of the incident on refugees’ perception on welcomeness. This extends previous work on the attitudes of refugees towards their host country as it is the first study that shows effects of violent attacks on refugees’ attitudes in an experimental framework. In the next section, we will introduce our study context before we then present the effects found. We will conclude with a discussion of UESD and point to further example studies to highlight the method’s applicability.

Study Context, Natural Experiment and Statistical Approach

A Riot on Refugees as an Unexpected Event During Survey Design

The Eastern German town of Bautzen became infamous for an extreme event of ethnic violence in 2016: after a series of events, including arson attacks on a hotel building initially designated for use as refugee accomodation in February, as well as several national demonstrations throughout the summer, tensions reached a peak on the night of September 14/15 2016. This culminated in a ‘hunt for refugees’ when about 80 violent local right-wing extremists clashed with 15–20 young unaccompanied refugees (Deutsche Welle (DW), 2016). This incident, during which several people were verbally harassed and physically wounded, has received notable media attention. Ethnic violence has not come to a halt in Bautzen after this incident; in December of the same year, another refugee accommodation was attacked. In the following, we will assess the effect of the Bautzen riots of 15 September 2016 on the refugee population in Germany.

Ethnic violence can be seen as a perceived ‘realistic threat’ (Stephan and Stephan, 2000: 25; also Stephan et al., 2002) to the well-being and physical safety of refugees, and we expect that extreme incidents of violence towards refugees such as riots, which were reported in local and national media, will lead refugees to feel more negatively towards their host country. Past research has made use of group-threat theory to explain levels of prejudice against ethnic minorities, arguing that they have been perceived as threats (see, for example, Legewie, 2013). In a meta-analytical review, Riek et al. (2006) discussed how different threat types, including realistic threats, lead to more negative outgroup attitudes. Against this background, we argue that refugees consider the population of the host country as a threat after experiencing or hearing about an incident and that they thus develop more negative attitudes about the host country. However, the temporal duration of such an attitudinal change might vary with the study context and needs to be explored (e.g. Frey, 2022; Legewie, 2013).

Our analysis is based on the German SOEP (Goebel et al., 2019; Socio-Economic Panel (SOEP), 2019), a longitudinal household survey of the German population. Since 2016, the SOEP also surveys refugees, sampled from the Central Register of Foreign Nationals (Brücker et al., 2016, 2019). The sample includes individuals who have entered Germany since January 2013 with an asylum application. The fieldwork period of the 2016 SOEP survey overlapped with the riots in Bautzen, which took place on the night of 14/15 September 2016 (see Figure 1). This circumstance allows us to assess whether the riots had an impact on the attitudes of refugees by comparing respondents interviewed prior to the attacks (control group) to those after (treatment group). For the 2016 survey, the sample includes 4465 individuals nested in 3289 households. After removing observations with missing answers in relevant variables (see below), the sample for the full fieldwork period consists of 4078 respondents in 3022 households (1742 respondents in the control group, 2336 in the treatment group). For the fieldwork period ±80 days around the riots (the longest bandwidth we consider in this study), the sample consists 1314 respondents nested in 1022 households (699 respondents in the control group, 615 in the treatment group).

Figure 1.

Fieldwork of SOEP 2016 relative to Bautzen riots on 15 September 2016.

Dependent Variable: Perceived Welcomeness

We use one attitudinal variable on perceived welcomeness to assess the impact of the Bautzen riots on refugees and examine whether the attack impacted their attitudes towards Germany. This variable provides a holistic measure, particularly given the background of the German ‘welcome culture’ and has been used in previous research (see Frey, 2022; Schwitter and Liebe, 2022). In the survey, refugees are asked how welcome they currently feel in Germany, and they can answer on a scale from 0 (not at all) to 4 (completely). The overwhelming majority of respondents feels welcome in Germany; in the full sample, 61.6% of respondents in the control group and 63.5% of respondents in the treatment group feel completely welcome in Germany; 23.8% and 22.3% for the most part, 11.7% and 10.5% in some respects, 2.1% and 2.6% barely, and 0.8% and 1.1% not at all. Given this skewed distribution and the small sample size for some response categories, we simplify the analysis by collapsing categories and dichotomising the variable (completely/for the most part vs. the rest).

Experimental Design

We expect the Bautzen riots to work as an exogenous stimulus; respondents are randomly assigned to the treatment and the control group, depending only on the timing of their interview. The main independent variable in our analysis thus forms the treatment indicator, which takes a value of 0 if a refugee was interviewed before the Bautzen riot (i.e. before 15 September) and 1 if the interview took place after. Given that the attacks took place during the night, respondents interviewed on 15 September are in the treatment group.

To assess a causal effect, two assumptions must hold in the UESD: excludability and temporal ignorability (Muñoz et al., 2020). The excludability criterion states that the differences in attitudes are solely due to the treatment effect and that there is no other channel than the riot. This cannot be guaranteed as time-varying variables systematically related to the outcome may bias the estimate. To limit such biases and capture temporal fading of effects, we consider different bandwidths around the event (between ±80 days and ±4 days around the event). We conduct imbalance analyses for different time frames around the event date to check whether the UESD fully satisfies the temporal ignorability assumption, that is, whether control and treatment group differ significantly in terms of specific variables other than the outcome (see Supplemental material). The present UESD does not fully satisfy the ignorability assumption. To relax the assumption, we include information on the number of prior interview attempts (see a similar approach in, for example, Frey, 2022). To further control for differences in reachability and survey design, we control whether respondents live in East or West Germany, and for geographic proximity to Bautzen (measured as the logarithm of the Euclidean distance to respondents’ districts centre), age, gender, marital status, citizenship, German language skills, type of accommodation, asylum application status and years already living in Germany. In addition to conditioning on these variables in the model estimation, we further match the sample using entropy balancing (Hainmueller, 2012; Hainmueller and Xu, 2013). We also check whether there is a pre-existing negative time trend in our dependent variable before the Bautzen riot; this is not the case (see Online Supplemental Material).

Under these assumptions, the causal effect of the Bautzen riot on attitudinal change can be estimated as follows:

Y_{i} = α + β_{1} T_{i} + β_{2} X_{2 i} + \dots + ε_{i}

with $T_{i}$ being a binary variable differentiating the control and the treatment group, that is, being an indicator of time, so that $β_{1}$ identifies the estimated effect of the riot, conditional on covariates $X_{i}$ . We employ linear probability models with cluster-robust standard errors to account for heteroskedasticity and clustering at the household level.²

Effects of Riots on Refugees’ Attitudes towards the Host Country

How did the Bautzen riots affect the perceived welcomeness of refugees in Germany? In short bandwidths around the event – comparing respondents interviewed, for example, in the eight days before the attack with those interviewed in the eight days after the attack – we find a significant and negative treatment effect of 7.5%. This means, the refugees surveyed feel significantly less welcome in Germany after the violent Bautzen riots. In Figure 2, we plot the treatment effect of Bautzen (model including controls) when increasing the bandwidth from four days around the Bautzen riot to 80 days around the riots in steps of four-day bandwidths (see, for a similar plotting approach, Frey, 2022).

Figure 2.

Treatment effect across different bandwidths, showing 5% significance level (thick black line) and 10% significance level (thin grey line). Plotted are coefficient estimates of a linear probability model. Plotting approach follows Frey (2022).

Figure 2 captures the fading effect over time: while the treatment effect of Bautzen is strong and significant in the recent aftermath (with the exception of the 12-day bandwidth where the effect is not as pronounced), it weakens in strength and significance over time (see Table A.4 in the Supplemental material for the numerical results). Please note that the effect does not reach conventional significance thresholds when using a bandwidth of four days, due to the small sample size.

For better interpretation, we calculate average predictive margins based on these models. We find that the average probability to feel welcome in Germany lies at 90.7% four days before the Bautzen attack and drops to 83.3% in the four days after the attack. This difference decreases the larger the bandwidth: the average probability to feel welcome in the 16 days before the riots lies at 88.6% and drops to 82.9% after the attack; in the 28 days before the riots, it lies at 86.6% and drops to 82.7%; and when comparing those interviewed 80 days before and after the Bautzen attack, there is a less than 2% difference with an average predicted probability of 85.8% to feel welcome in Germany if surveyed before and 84.5% if surveyed after. We also checked whether the negative treatment effect is significantly weaker the further away from Bautzen respondents live, however, we only find a significant interaction effect between the treatment and the distance from Bautzen in the very recent aftermath (up to 12 days after the event; see Online Supplemental Material).³

Discussion and Conclusion: Making Use of Natural Experiments

Making use of an unexpected event and extending previous work, this study provides a more direct test of the causal effect of an incident of ethnic violence on refugees’ attitudes towards Germany. Our negative treatment effect shows that violence against refugees influences their attitudes in the aftermath, highlighting the immediate and profound impact of negative events on refugees’ perception of their host country. While we do not know yet what the individual consequences of such negative changes in attitude might be, it has shown to been accompanied with clinically relevant declines in mental health (Frey, 2022). The negative effect on the perceived welcomeness in Germany fades in strength and significance across time as the different bandwidths analysed suggest. While some other UESD studies have found long-term effects of extreme events (e.g. lasting ‘rally effects’ on support for George W Bush after 11 September 2001, see Hetherington and Nelson, 2003), it is not unusual that changes in attitudes are only temporary (see, for example, Frey, 2022). These findings suggest that negative events do not necessarily have lasting negative impacts on those affected; resilience and adaptation mechanisms might enable people to restore their opinions. Future research on this topic should investigate in more detail how refugees navigate the challenges of integrating into a host country when being met with hostility instead of hospitality.

Natural experiments offer a valuable tool to explore causal effects, and UESD studies have emerged as a promising area of research in the field of discrimination more generally (see, for example, Boomgaarden and De Vreese, 2007; Czymara and Schmidt-Catran, 2017; Echebarria-Echabe and Fernández-Guede, 2006; Frey, 2022; Jakobsson and Blom, 2014; Larsen et al., 2020; Legewie, 2013; Muñoz et al., 2020; Nägel and Lutter, 2020; Nussio et al., 2019; Schmidt-Catran and Czymara, 2020). In these experiments, distinct, unexpected events (most often terrorist attacks) occurring within the fieldwork of a survey programme were used as a quasi-experimental stimulus. Examples include:

Frey (2022) who, based on data from the SOEP and the ALLBUS (German General Social Survey; allgemeine Bevölkerungsumfrage der Sozialwissenschaften), found German respondents to feel increasingly hostile against refugees, as well as refugees to experience more discrimination, feel less welcome and suffer declines in mental health in the aftermath of a series of terrorist attacks (instead of attacks on refugees) in 2016.

Larsen et al. (2020), Nägel and Lutter (2020) and Schmidt-Catran and Czymara (2020) employ the terrorist attack on the Berlin Christmas market in 2016 as a salient and unexpected event during the fieldwork of the European Social Survey (ESS). While Larsen et al. (2020) do not find an effect of the Berlin attack on attitudes towards refugees (they do not state their bandwidth), Schmidt-Catran and Czymara (2020) report that the change in attitudes was lagged. Nägel and Lutter (2020) find that immediately after the event, there is a decline in the acceptance of refugees by respondents with a right-wing political attitude only; as time progresses, this worsening of attitudes towards refugees can also be observed in the general population.

Legewie (2013) finds considerable cross-national and regional variation in the effect and its temporal duration of the terrorist attack in Bali in October 2002 on attitudes towards immigration, based on ESS data from 65 regions in nine European countries. The effect of the Bali attack is found to be larger in regions with an increasing unemployment rate, and the immigrant share and potentials for contact with immigrants further shape the response to the event.

National and cross-national population surveys such as the British Social Attitudes survey, the United Kingdom Household Longitudinal Study (Understanding Society), the ESS and the SOEP used in this research note form essential sources of observational data for social scientists to better understand (changing) attitudes and behaviours of society. When the fieldwork of these studies overlaps with unforeseen and salient events, they can be used to estimate causal effects under certain assumptions. Many national (panel) surveys also include youth samples and systematic oversamples of subgroups; for example, the SOEP includes not only a specific sample covering refugees, but has also oversampled high-income households, households with different migration backgrounds and households with a household head who identifies as lesbian, gay, bisexual, transgender or non-binary. This allows researchers to analyse the effect of events on specific subgroups with high statistical power. While this UESD study design is not new, our research note demonstrates its potential in the context of research on ethnic violence and refugee integration. Given the vast amount of (cross-)national survey data available to researchers, the UESD type of natural experiments is a promising research strategy in several still unexplored research fields.

Supplemental Material

sj-docx-1-soc-10.1177_00380385231206072 – Supplemental material for Using Natural Experiments to Uncover Effects of Anti-Refugee Riots on Attitudes of Refugees

Supplemental material, sj-docx-1-soc-10.1177_00380385231206072 for Using Natural Experiments to Uncover Effects of Anti-Refugee Riots on Attitudes of Refugees by Nicole Schwitter and Ulf Liebe in Sociology

Footnotes

Data availability statement

The data that support the findings of this study are available from the German Socio-Economic Panel. Restrictions apply to the availability of these data, which were used under licence for this study. Data can be accessed via the SOEPremote servers with the permission of German Socio-Economic Panel.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

ORCID iDs

Nicole Schwitter

Ulf Liebe

Supplemental material

Supplemental material for this article is available online.

Notes

Nicole Schwitter is an honorary research fellow at the University of Warwick. Her main research interests include integration and discrimination, social networks and computational social science.

Ulf Liebe is Professor at the Department of Sociology at the University of Warwick. His research interests comprise experimental methods, theory comparison, discrimination and prosociality.

References

Akrap

(2016) Germany’s response to the refugee crisis is admirable. But I fear it cannot last. The Guardian. London, 6 September. Available at: https://www.theguardian.com/commentisfree/2015/sep/06/germany-refugee-crisis-syrian (accessed 27 September 2023).

Ben Farhat

Blanchet

Bjertrup

, et al. (2018) Syrian refugees in Greece: Experience with violence, mental health status, and access to information during the journey and while in Greece. BMC Medicine 16(40): 3–12.

Boomgaarden

De Vreese

(2007) Dramatic real-world events and public opinion dynamics: Media coverage and its impact on public reactions to an assassination. International Journal of Public Opinion Research 19(3): 354–366.

Bouhenia

Ben Farhat

Coldiron

, et al. (2017) Quantitative survey on health and violence endured by refugees during their journey and in Calais, France. International Health 9(6): 335–342.

Brücker

Croisier

Kosyakova

, et al. (2019) Second wave of the IAB-BAMF-SOEP Survey: Language skills and employment rate of refugees improving with time. IAB Brief Report 3/2019, Nürnberg.

Brücker

Rother

Schupp

, et al. (2016)IAB-BAMF-SOEP-Befragung von Geflüchteten: Flucht, Ankunft in Deutschland und erste Schritte der Integration. IAB-Kurzbericht 24/2016, Nürnberg.

Czymara

Schmidt-Catran

(2017) Refugees unwelcome? Changes in the public acceptance of immigrants and refugees in Germany in the course of Europe’s ‘immigration crisis’. European Sociological Review 33(6): 735–751.

Deutsche

Welle (DW)

(2016) Eastern German town of Bautzen witnesses clashes between locals and refugees. Available at: https://www.dw.com/en/eastern-german-town-of-bautzen-witnesses-clashes-between-locals-and-refugees/a-19553867 (accessed 27 September 2023).

Dunning

(2012) Natural Experiments in the Social Sciences: A Design-Based Approach. Cambridge, MA: Cambridge University Press.

10.

Echebarria-Echabe

Fernández-Guede

(2006) Effects of terrorism on attitudes and ideological orientation. European Journal of Social Psychology 36(2): 259–265.

11.

Frey

(2022) Getting under the skin: The impact of terrorist attacks on native and immigrant sentiment. Social Forces 101(2): 943–973.

12.

Goebel

Grabka

Liebig

, et al. (2019) The German Socio-Economic Panel study (SOEP). Jahrbücher für Nationalökonomie und Statistik/Journal of Economics and Statistics 239(2): 345–360.

13.

Hainmueller

(2012) Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political Analysis 20(1): 25–46.

14.

Hainmueller

(2013) Ebalance: A Stata package for entropy balancing. Journal of Statistical Software 54(7): 1–18.

15.

Hetherington

Nelson

(2003) Anatomy of a rally effect: George W. Bush and the war on terrorism. PS: Political Science & Politics 36(1): 37–42.

16.

Hofstetter

(1969) Political disengagement and the death of Martin Luther King. The Public Opinion Quarterly 33(2): 174–179.

17.

Jakobsson

Blom

(2014) Did the 2011 terror attacks in Norway change citizens’ attitudes toward immigrants? International Journal of Public Opinion Research 26(4): 475–486.

18.

Larsen

Cutts

Goodwin

(2020) Do terrorist attacks feed populist Eurosceptics? Evidence from two comparative quasi-experiments. European Journal of Political Research 59(1): 182–205.

19.

Legewie

(2013) Terrorist events and attitudes toward immigrants: A natural experiment. American Journal of Sociology 118(5): 1199–1245.

20.

Lever

(1969) The Johannesburg station explosion and ethnic attitudes. The Public Opinion Quarterly 33(2): 180–189.

21.

Muñoz

Falcó-Gimeno

Hernández

(2020) Unexpected event during survey design: Promise and pitfalls for causal inference. Political Analysis 28(2): 186–206.

22.

Nägel

Lutter

(2020) The Christmas market attack in Berlin and attitudes toward refugees: A natural experiment with data from the European Social Survey. European Journal for Security Research 5: 199–221.

23.

Nussio

Bove

Steele

(2019) The consequences of terrorism on migration attitudes across Europe. Political Geography 75: 102047.

24.

Pai

(2020) The refugee ‘crisis’ showed Europe’s worst side to the world. The Guardian. London, 1 January. Available at: https://www.theguardian.com/commentisfree/2020/jan/01/refugee-crisis-europe-mediterranean-racism-incarceration (accessed 27 September 2023).

25.

Riek

Mania

Gaertner

(2006) Intergroup threat and outgroup attitudes: A meta-analytic review. Personality and Social Psychology Review 10(4): 336–353.

26.

Rubin

(2008) For objective causal inference, design trumps analysis. The Annals of Applied Statistics 2(3): 808–840.

27.

Schmidt-Catran

Czymara

(2020) ‘Did you read about Berlin?’: Terrorist attacks, online media reporting and support for refugees in Germany. Soziale Welt: Zeitschrift für Sozialwissenschaftliche Forschung und Praxis 71(2–3): 305–337.

28.

Schwitter

Liebe

(2022) Does living in districts with higher levels of ethnic violence affect refugees’ attitudes towards the host country? Empirical evidence from Germany. Ethnic and Racial Studies 45(15): 2793–2821.

29.

Sicinski

(1969) Dallas and Warsaw: The impact of a major national political event on public opinion abroad. The Public Opinion Quarterly 33(2): 190–196.

30.

Socio-Economic Panel (SOEP) (2019) Data for years 1984–2017, version 34. DOI: 10.5684/soep.v3y.

31.

Stephan

Boniecki

Ybarra

, et al. (2002) The role of threats in the racial attitudes of Blacks and Whites. Personality and Social Psychology Bulletin 28(9): 1242–1254.

32.

Stephan

(2000) An integrated threat theory of prejudice. In: Oskamp

(ed.) Reducing Prejudice and Discrimination. Mahwah, NJ: Lawrence Erlbaum Associates Publishers, 23–45.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.45 MB