Abstract
The question of how social media usage impacts societal polarization continues to generate great interest among both the research community and broader public. Nevertheless, there are still very few rigorous empirical studies of the causal impact of social media usage on polarization. To explore this question, we replicate the only published study to date that tests the effects of social media cessation on interethnic attitudes (Asimovic et al., 2021). In a study situated in Bosnia and Herzegovina, the authors found that deactivating from Facebook for a week around genocide commemoration in Bosnia and Herzegovina had a negative effect on users’ attitudes toward ethnic outgroups, with the negative effect driven by users with more ethnically homogenous offline networks. Does this finding extend to other settings? In a pre-registered replication study, we implement the same research design in a different ethnically polarized setting: Cyprus. We are not able to replicate the main effect found in Asimovic et al. (2021): in Cyprus, we cannot reject the null hypothesis of no effect. We do, however, find a significant interaction between the heterogeneity of users’ offline networks and the deactivation treatment within our 2021 subsample, consistent with the pattern from Bosnia and Herzegovina. We also find support for recent findings (Allcott et al., 2020; Asimovic et al., 2021) that Facebook deactivation leads to a reduction in anxiety levels and suggestive evidence of a reduction in knowledge of current news, though the latter is again limited to our 2021 subsample.
Introduction
Social media is altering the social, political, and economic landscapes of countries around the world in profound ways. With the number of users of social media platforms continuing to rise globally, the impact of social media on a range of political outcomes has become a source of strident debates both inside and outside of academia (Lorenz-Spreen et al., 2023; Mosleh et al., 2022). Among the questions receiving perhaps the most attention are those that focus on how social media is affecting social and political polarization (Van Bavel et al., 2021; González-Bailón et al., 2023; Simchon et al., 2022). Understanding social media’s impact on polarization is perhaps nowhere more important than within ethnically segregated post-conflict areas, where social media content and online interactions may have an outsized impact on fragile group relations.
Yet, most research on the effects of social media has been conducted on users in advanced democracies, with a primary focus on political polarization, that is the relationship between supporters of opposing political parties, as opposed to polarization across ethnic groups (Tucker et al., 2018; Persily et al., 2020). Moreover, most of the existing evidence about the effects of social media is correlational (Lorenz-Spreen et al., 2023), and points to the positive association between social media usage and increased political polarization within established democracies. Obtaining causal estimates is challenging because social media activity is endogenous to a variety of factors, and because—given the extent to which social media usage has already penetrated most societies—the population of interest are users who are already using social media. To overcome this challenge, researchers have begun to utilize what are known as “deactivation studies,” which are based on assigning users in the treatment group to deactivate (i.e., not use) social media accounts for a particular period of time, while the control group remains active as usual.
The largest Facebook deactivation study published to date, conducted on a sample of U.S. users, finds that deactivation led to a significant reduction in the levels of political polarization, together with a smaller decrease in partisan affective polarization, which was measured by calculating the difference between a respondent’s thermometer warmth toward her own party and her warmth toward the other party (Allcott et al., 2020). Following the same deactivation design, but with a focus on interethnic attitudes as the main outcome, Asimovic et al. (2021) find that deactivation during the 7 days around the commemoration of the Srebrenica genocide in Bosnia and Herzegovina (hereafter BiH) led to more negative attitudes toward ethnic outgroups, a surprising finding (and contra the pre-registered expected outcome) given the received wisdom that social media usage exacerbates political polarization (although see Barberá, 2020). Further analysis showed that the negative effects of deactivation were driven by users in ethnically homogeneous areas, for whom deactivating from social media may have meant moving away from any exposure to individuals from other ethnic groups. The study in BiH (Asimovic et al., 2021) found—as did the earlier deactivation study conducted on U.S. users (Allcott et al., 2020)—that deactivation led to a significant improvements in users’ subjective well-being and a decrease in their levels of news knowledge, that is, the ability to answer factual questions about recent news correctly.
To assess the scope conditions of these findings, we implement the same deactivation design within a different ethnically polarized setting. We selected Cyprus, an island country in the Eastern Mediterranean that continues to grapple with legacies of conflict, power politics in the region, and high levels of ethnic segregation. As is the case in BiH, Cyprus is characterized by the absence of armed conflict but an abundance of interethnic tensions (Adamides, 2020). Unlike in BiH, however, there is a higher level of geographical separation and a language barrier between the two main ethnic groups (Greek Cypriots and Turkish Cypriots) that contributes to even lower levels of intergroup contact (Sahin et al., 2020; Sahin, 2022). Deactivation in the original study in BiH (Asimovic et al., 2021) was conducted during a week around the anniversary of the 1995 Srebrenica genocide of Bosnian Muslims. Each year on July 11th, victims identified over the previous year are buried in a public funeral ceremony and commemorated. This, however, is accompanied by heated discussions across the region about the past conflict, as genocide denial and interethnic tensions remain omnipresent. Although no two events are the same, we aimed to mimic the design from BiH, and thus situated our replication study during a week around July 20th, which is the commemoration period of the highly contested conflict in 1974 that led to the current partition of Cyprus between predominantly Greek Cypriot community (southern part of the island) and Turkish Cypriot community (northern part of the island) (Psaltis et al., 2017b; Smeekes et al., 2017; Psaltis et al., 2017a). We ran the study in both 2020 and 2021 to minimize the risk of pandemic-related events of 2020 shaping our findings in unique ways.
Our main finding is that, counter what was found in BiH, we cannot reject the hypothesis of no effect of deactivation on group attitudes in Cyprus. While some outcomes are estimated to be in a positive direction (e.g., deactivation improving outgroup attitudes), none survive pre-registered multiple comparison corrections. We do find some support—within our 2021 subsample—for a pre-registered conditional hypothesis, which was designed to see if we could replicate arguably the most important insight from the study in BiH: the heterogeneity of users’ offline networks does indeed shape the effect of deactivation across both BiH and Cyprus. We are also able to replicate some of the secondary outcomes from the original study: deactivation of Facebook in Cyprus did lead to a significant decrease in anxiety. We also observed suggestive evidence of a reduction in levels of news knowledge, but this was limited to our 2021 subsample.
Replicating the effects of Facebook deactivation
While the relationship between social media usage and intergroup relations has been discussed widely, conclusive judgments about the direction and strength of these relationships are difficult to make due to a paucity of causal empirical evidence. Arguments highlighting the positive potential of social media for group relations largely draw on the argument that social media may lower barriers to contact with members of other groups, more easily create perceptions of equal status, and reduce individuals’ anxiety around interethnic interactions (Amichai-Hamburger et al., 2017; Bagci et al., 2021). These effects are hypothesized to be particularly meaningful within segregated or homogeneous areas, where users have limited opportunities to connect with others outside of the group with which they identify. Facebook, in particular, has affordances that make it conducive to extended and vicarious contact (i.e., observing intergroup interactions or learning that an ingroup member befriended an outgroup member) which has been found to positively affect intergroup relations (Mazziotta et al., 2011; White et al., 2021).
On the other hand, research has also suggested that social media platforms may be exacerbating polarization through algorithms that limit exposure to counter-attitudinal news (Levy, 2021), while amplifying hate speech and content expressing outgroup animosity (Rathje et al., 2021; Harel et al., 2020; Tellidis et al., 2016). More recently, Lorenz-Spreen et al. (2023) analyzed 496 published articles and found a strong association between digital media use and lack of trust in institutions, as well as greater exposure to misinformation. Moreover, unstructured and short interethnic interactions may easily turn negative, thus exacerbating rather than ameliorating group conflict (Paolini et al., 2010).
Considering these competing theoretical predictions, we pre-registered our main hypotheses in line with the findings from BiH: 1. H1: Social media deactivation decreases outgroup regard, that leads to more negative attitudes toward ethnic outgroups. 2. H1a: The negative effect of social media deactivation on outgroup regard will be stronger for users within more ethnically homogeneous networks.
In addition to our primary outcome of interest—outgroup attitudes—we also pre-registered predictions that we would replicate two additional findings from prior research that Facebook deactivation would lead to improvements in subjective well-being (H2) and a decrease in knowledge of the news (H3).
Research context
This replication took place in Cyprus, a European country with a complex history of intergroup conflict and cooperation, dealing with tenuous relations between its two largest ethnic groups (Greek Cypriots and Turkish Cypriots), and high level of geographical segregation. In 1974, following a coup d’etat instigated by the Greek military junta, Turkey intervened militarily and captured the northern portion of the island. While there is variation in how similarly citizens perceive these events, the official narrative is clearly divided: Turkish Cypriot officials refer to the intervention as a peacekeeping operation that liberated their community from Greek oppression and impeded the enosis (union) of Cyprus with Greece, while Greek Cypriot officials consider it to be an illegal invasion (Smeekes et al., 2017). Following large population displacement within the island, the northern part proclaimed independence and established the Turkish Republic of Northern Cyprus (TRNC) in 1983. However, the United Nations rejected the declaration of independence, and currently only Turkey recognizes TRNC as a separate country, while the Republic of Cyprus (the southern part of the island) is a full member of the European Union (Psaltis et al., 2016).
The events of 1974 are commemorated annually on July 20th with a celebration known as “Peace and Freedom Day” in the localities in the North, while the South observes it as a day of mourning (Eyerman et al., 2015). Following the design from BiH, we situated our replication during this commemorative period in Cyprus, asking users to deactivate from Facebook during a week around July 20th. 1 Importantly, for this study, Cyprus has one of the highest social media usage rates in Europe, with only 4% of internet users without a social media account (Hootsuite, 2021). Facebook is the most widely used platform, with 86% of those with a social media account being active Facebook users (Hootsuite, 2021). We ran this study in two waves, in 2020 and 2021, to account for any potential differences caused by the pandemic (SI, Section A.1).
Treatment assignment and recruitment
We recruited participants through Facebook advertisements that were circulated across the island in both Greek and Turkish (SI, Section A). Participants were informed that the study was about the effects of social media—but without specifying our outcomes of interest—and were offered the same amount of money to participate regardless of treatment assignment. All participants had to be 18 years or older, active Facebook users, and be born or living in Cyprus. Unlike Asimovic et al. (2021), we did not restrict our sample to individuals who reported following news or political pages on Facebook and reported minimal or no usage of Twitter or Reddit. Instead, we examine the heterogeneity in effect size across this dimension (see SI, Section G.1).
After participants completed a baseline survey, we randomized those with complete answers into treatment and control groups and e-mailed participants with their treatment assignment. The treatment group was asked to deactivate from Facebook (and received instructions on how to do so), while the control group was asked to remain active as usual. Deactivation was monitored via a Python script that automatically checked Facebook URLs several times a day, in addition to manual checks by our team. After a week, we e-mailed participants with the Qualtrics link to the post-treatment survey. Our main outcome of interest is attitudes toward ethnic outgroups (i.e., outgroup regard), captured as a composite index from a set of pre-registered indicators (listed in SI, Table A6): a feeling thermometer, social distance, meta-stereotypes, outgroup trait ratings, and attitudes toward cooperation. In our main results (Figure 1), the composite index of outgroup regard was calculated as a sum of z-scores of the associated indicators. However, we also report the results with a composite index calculated as a principal component score (SI, A6), with the results not substantially different. As pre-registered, we present results at both the indicator and composite index levels. ITT effect of deactivation on outgroup regard, with a full set of controls: gender, age, employment status, weekly Facebook usage, and the perceived importance of country and ethnic membership to one’s identity. For operationalization of these indicators, see SI, Table A6. Each coefficient is reported with its corresponding 95% confidence intervals based on robust standard errors and is standardized relative to the standard deviation of the control group (N=543).
Estimation strategy
With the following regression, we estimate the intent-to-treat effect of Facebook deactivation on our outcomes of interest:
Results: Effects of deactivation on outgroup regard, news knowledge, and subjective well-being
As a result of an intent-to-treat analysis where we pool the data and control for the same covariates as in the original BiH study, we are not able to reject the null hypothesis of no effect on outgroup regard following Facebook deactivation in Cyprus (ITT: β = .095; SE = .087). Pooling the data across the 2 years, our sample size in Cyprus (N = 543) is larger than the sample size in the original study in BiH (N = 354), and we are powered to detect an effect size larger than 0.24 SD (SI, Section B). As such, we do not attribute our inability to replicate the effect from BiH in Cyprus to issues of statistical power. We recognize that we do not have adequate power to detect the statistical significance of smaller effects, but such effects would also be of a more limited substantive significance. Interestingly, though, we observe heterogeneity in the effects of deactivation across years. Deactivation had a more positive, albeit imprecisely estimated, effect on outgroup regard in 2020, a particularly complex period for group relations in Cyprus: our research coincided with the pandemic, the closing of checkpoints between the two sides of the island, and other developments that drew further attention to the divisions between the Greek-speaking and Turkish-speaking communities (see SI, Section A.1). In 2021, comparatively a less tumultuous period, there was no detectable effect of deactivation on outgroup attitudes. 2
Similarly to previous deactivation studies (Asimovic et al., 2021; Allcott et al., 2020), deactivation in Cyprus significantly increased the amount of time spent with friends and family, as well as watching TV and engaging in non-technological activities (Figure 2). (Left) Effect of deactivation on the activities users’ engaged in during the research. (Right) Histogram of treated group’s responses in the final survey (multiple responses permitted): “In the last week, relative to what is typical for you, have you spent more or less of your free time…”.
Measures of user network composition from the baseline survey.
Testing for an Interaction effect between outgroup regard and an indicator of offline network homogeneity (included in the model as a sum score of survey items capturing users’ offline networks in the first column; and as a principal component score in the second column). Higher values indicate a more homogeneous offline network. Full results are in SI, Table A13.
∗p < .1; ∗∗p < .05; ∗∗∗p < .01.
We conducted an additional test of the moderation hypothesis (H2) employing a different proxy for the heterogeneity of offline networks—living in the capital, based on the assumption that those in the capital would have more diverse offline networks than those living outside of Nicosia 3 —that allowed us to use the entire sample in the analysis. Reassuringly, within the 2021 subsample, we observed the same pattern as with the first indicator: a significant interaction between the treatment and the proxy for offline heterogeneity, revealing a more negative effect of deactivation for users who do not reside in the capital. Interestingly, this heterogeneity was not observed in 2020, as deactivation had a more positive effect on users’ attitudes toward the outgroup. We suspect, however, that the difference in results between 2020 and 2021 could be due to the particular dynamics of the 2020 summer due to the start of the COVID-19 pandemic (SI, Section A.1).
The peculiarities of the summer of 2020 may also explain why the knowledge of the news (our secondary outcome) was less impacted by the treatment of deactivation for our participants in 2020, compared to 2021. On a pooled sample, deactivation did not affect the level of news knowledge (ITT: β = −.087; SE = .081). However, the deactivation did lead to a more substantial reduction in the level of news knowledge (ITT: β = −.211; SE = .134) in 2021, unlike in 2020 (ITT: β = 0.098, SE = 0.117). One possible explanation for this over-time discrepancy is that the need to be informed about the ever-changing developments around the pandemic may have made users particularly interested in following the news, regardless of whether or not they could access Facebook. Finally, we find that deactivation led to a significant decrease (ITT: β = −.264; SE = .086) in anxiety (H3); this finding holds regardless of whether we pool data from both years or limit the sample to 2020 (ITT: β = −.249; SE = .114) or 2021 (ITT: β = −.271; SE = .136). The effect of deactivation on reducing anxiety is the one with the largest effect size among the findings from BiH, and this effect was also found in the Allcott et al. (2020) deactivation study on U.S. users.
Finally, since our measurement of offline network heterogeneity differs from the approach used in Asimovic et al. (2021), we conducted an additional exploratory analysis (not pre-registered) that enables us to characterize the offline environment using the same measures in both contexts. We rely on nationally representative samples collected in Bosnia and Herzegovina (2020, N = 3637) and Cyprus (2017, N = 1602) by the Centre for Sustainable Peace and Democratic Development. 4 Within each country, we divided the sample to capture responses from individuals in regions with below-median scores on the “outgroup feeling thermometer,” as well as regions with median and above-median scores (SI, Section F). We observed a similar pattern in both Cyprus, and Bosnia and Herzegovina, where deactivation led to more negative outgroup regard among individuals in regions with higher polarization (i.e., lower “outgroup feeling thermometer” scores), and either positive or neutral outgroup regard in regions with lower polarization (i.e., higher “outgroup feeling thermometer” scores). Building on previous evidence from BiH, this analysis further highlights the significance of considering users’ offline networks when assessing the impacts of their social media usage.
Discussion and Conclusion
We cannot reject the null hypothesis of no effect of deactivation in Cyprus, but we find a similar pattern as in Bosnia and Herzegovina (Asimovic et al., 2021) regarding the interaction between the heterogeneity of users’ offline networks and the deactivation treatment. The original study conducted in BiH (Asimovic et al., 2021) presents a compelling argument: social media deactivation may reduce ethnic polarization in contexts where offline intergroup contact is limited or nonexistent, and, therefore, where users’ offline echo chambers may be stronger than their online ones. For the online environment to provide more opportunities for intergroup contact, however, one of the basic prerequisites is that users are able to access outgroup content and understand it. Incidental exposure to the outgroup on social media, which is a common explanation for how social media may break down online echo chambers (Bakshy et al., 2015; Nanz et al., 2020; Barnidge, 2020), is likely to be much less prevalent in contexts with larger language barrier between polarized groups. For example, in BiH, where there is basically no language comprehension barrier to speak of between the dominant ethnic groups, online content from outgroups is likely to be relatively accessible online. Within countries like Cyprus, however, where the two communities speak different languages and are in highly segregated informational environments, the online space (for the most part) provides few opportunities for engagement with individual members of the outgroup.
Having observed this dynamic online, we pre-registered (in our 2021 update) our expectation that users who were able to communicate in the language of the outgroup would respond to the deactivation treatment more negatively (i.e., report larger decrease in outgroup regard) compared to users who do not have any knowledge of the outgroup’s language (SI, Section G). We thus gathered data on users’ fluency in Turkish, Greek, and English as a proxy of their ability to communicate with the outgroup. Within our (2021) sample, only 2% of participants reported being fluent in the language of the outgroup, while 10% of participants reported being able to understand but not speak nor write in the language of the outgroup. Thus there is unfortunately not enough variation in language comprehension in our sample to test how language ability moderates the effect of deactivation. However, we observe that those participants who have at least some knowledge of the outgroup language are significantly more likely to be interacting with the outgroup, as well as significantly more likely to be exposed to outgroup content on Facebook. This aligns with our intuition that higher language barriers between polarized groups in Cyprus may be dampening the potential of social media to expose users to one another, at least inadvertently. Future research should explore the role of language barriers in shaping the effects of social media usage, and particularly in post-conflict societies with ethnic divides that are at least somewhat reflected in language proficiency. 5
As with any replication study, the question of the quality of research implementation is central to the reliability of the findings. In addition to following the same process as in the original study (Asimovic et al., 2021), we are encouraged by the fact that we were able to replicate secondary findings from the previous study in terms of deactivation reducing users’ anxiety. Although we do not observe a reduction in the knowledge of the news on the pooled sample through deactivation, we do detect a reduction within the 2021 subsample, albeit with less precision compared to the results from BiH. From the research conducted so far, though, it is clear that the effects of social media deactivation in different countries may vary depending on contextual circumstances that we are only starting to identify. One that clearly emerges from the empirical findings in Asimovic et al. (2021) and our replication in Cyprus is the heterogeneity of offline networks. Other characteristics that we posit are important include the level of language barriers between polarized groups, or more broadly, the ease of accessing the outgroup online. Exploring the role of these and other contextual conditions in shaping the effects of social media usage is a promising avenue for future research. We hope these efforts will result in more precise expectations around the effects of social media’s usage and inform future interventions that re-envision social media environments in a way that contributes to the goal of fostering, rather than diminishing, peacebuilding, and stability (Vollhardt, 2012).
Supplemental Material
Supplemental Material - Replicating the effects of Facebook deactivation in an ethnically polarized setting
Supplemental Material for Replicating the effects of Facebook deactivation in an ethnically polarized setting by Nejla Asimovic, Jonathan Nagler, and Joshua A Tucker in Research & Politics
Footnotes
Acknowledgments
This publication was made possible in part by a grant from the American Political Science Association (Doctoral Dissertation Improvement Research Grant, awarded to Asimovic). The statements made and views expressed are solely the responsibility of the authors. The Center for Social Media and Politics at New York University is generously supported by funding from the National Science Foundation, the John S. and James L. Knight Foundation, the Charles Koch Foundation, the Bill and Melinda Gates Foundation, the Hewlett Foundation, Craig Newmark Philanthropies, the Siegel Family Endowment, and NYU’s Office of the Provost. The team at the Centre for Sustainable Peace and Democratic Development (SeeD) shared with us their survey data from BiH and Cyprus, for which we are deeply grateful. We also thank the members of NYU Center for Social Media and Politics, and participants of NYU Comparative Politics Workshop, the 2021 Annual Meeting of the American Political Science Assocation; and the 2021 Annual Meeting of the Mid-West Political Science Association for their excellent comments and suggestions. We thank Daniela Donno, Osman Kisaoglu, and members of Home for Cooperation for their contribution to this project. For excellent technical support, we thank Megan Brown; and for their on-the-ground research assistance, we are indebted to Suzan Kisaoglu, Marita Anastasi, and Katerina Kousoulidou.
Author contributions
N.A. conceived of the idea for the project. N.A. designed the study and wrote its pre-analysis plan with feedback from J.A.T. N.A. collected the data, analyzed the data, and wrote the draft of the paper. N.A., J.A.T. and J.N. oversaw the infrastructure to support the study, and all of the authors contributed to revisions of the manuscript.
Declaration of conflicting interests
N.A. and J.N. declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. J.A.T. received a small fee from Facebook to compensate him for administrative time spent in organizing a 1-day conference for approximately 30 academic researchers and a dozen Facebook product managers and data scientists that was held at NYU in the summer of 2017 to discuss research related to civic engagement; the fee was paid before any data collection for the current project began. He did not provide any consulting services nor any advice to Facebook as part of this arrangement. N.A. was the recipient of a Meta PhD Research Fellowship that partially covered living and conference travel expenses from mid-2022 to mid-2023, which includes the period spent revising this paper. The fellowship was not used for any part of the data collection.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the American Political Science Association, National Science Foundation, and the NYU Center for Social Media and Politics, which is generously supported by the John S. and James L. Knight Foundation, the Charles Koch Foundation, the Bill and Melinda Gates Foundation, the Hewlett Foundation, Craig Newmark Philanthropies, the Siegel Family Endowment, and NYU’s Office of the Provost.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
