Abstract
How can we measure the resource mobilization (RM) efforts of social movements on Twitter? In this article, we create the first ever measure of social movements’ RM efforts on a social media platform. To this aim, we create a four-conditional lexicon that can parse through tweets and identify those concerned with RM. We also create a simple RM score that can be plotted in a time series format to track the RM efforts of social movements in real time. We use our tools with millions of tweets from the United States streamed between November 28, 2018, and February 11, 2019, to demonstrate how our measure can help us estimate the saliency and persistency of social movements’ RM efforts. We find that our measure captures RM by successfully cross checking the variation of this score against protest events in the United States during the same time frame. Finally, we illustrate the descriptive and qualitative utility of our tools for understanding social movements by running conventional topic modeling algorithms on the tweets that were used to compute the RM score and point at specific avenues for theory building and testing.
Keywords
On Saturday, August 12, 2017, a “Unite the Right” rally is organized to protest the removal of confederation era icon, General Robert E. Lee (AlJazeera 2017). Considered one of the “largest white supremacist events in recent U.S. history,” it sought to literally unite ultranationalist groups under one banner. What made it even more memorable for U.S. politics is the fact that counter-protesters from the political Left made sure to demonstrate against the event at the same location as Far- Righters. The clashes that ensued between the two groups led to one dead, 19 injured, and the declaration of a local state of emergency by the City of Charlottesville and the County of Albemarle (AlJazeera 2017; Yan, Sayers, and Almasy 2017).
Both proponents and opponents of Unite the Right organized vast and intricate campaigns of resource mobilization (RM) on social media platforms (SMPs) like Twitter before, during, and after the protests of Charlottesville. Twitter accounts and hashtags used by Far-Right demonstrators have been targeted and denounced by “Antifa” (short for anti-fascist) activists before the Charlottesville mobilization and even more so afterward in an effort to identify culprits of violence (Parham 2017; Shieber 2017). At the same time, Twitter conducted purges of Neo-Nazi and Alt-Right accounts on its site, in an effort to curb mobilizations of such ideological tendencies on its platform (Linton 2018; Mathias 2017). Starting in late 2018, Antifa protesters were mobilizing support and resources again in an effort to stop the next Unite the Right mobilization, using such hashtags as #stopthenextUTR or #NoNewKKK.
The Charlottesville upheavals are by no means a uniquely Western phenomenon, or an isolated occurrence around the world, even if they are symptomatic of the Far-Right’s resurgence in Western politics. In the United States, the mass public often gathers to protest old or new issues, led by old or new social leaders, but increasingly takes on SMPs to mobilize resources and support for their cause. Black Lives Matter, March for Our Lives, and Occupy Wall Street are notable examples of social movements that made extensive use of SMPs to organize their discontent and, in turn, deeply affected the politics and economics of the United States in the past few years (Salamon 2018; Vega 2016). But social movements also affect developing economies ruled by democratic or authoritarian systems. The revolts of the Arab Spring, the “Fees Must Fall” movement of South Africa, and many others are modern testaments that social movements affect developed and developing societies alike. In the Middle East alone, the revolts of the Arab Spring were massively organized through SMPs and deeply shook the foundations of politics in the region (Eltantawy and Wiest 2011; Wolfsfeld, Segev, and Sheafer 2013).
The scientific community still lacks a quantitative measure of the RM activities of social movements on SMPs despite the latter’s rising centrality in the mobilization of resources and support for contentious action. Resources on SMPs may range from access to funding and physical capital to human capital, social-organizational infrastructure, and cultural references (Edwards, McCarthy, and Mataic 2018:80). The literature on social movements and SMPs is very recent and still struggles to yield substantial results for theory building. It emerges largely in the 2010s from the first heavy uses of SMPs by social movements involved in the Global Occupy protests and the Arab Spring (Eltantawy and Wiest 2011; Kidd and McIntosh 2016; Laer and Aelst 2010; Tufekci and Wilson 2012). Some social scientists and computer scientists have studied whether and how SMPs contribute to protest recruitment (Barberá and Steinert-Threlkeld 2019; González-Bailón et al. 2011; Jost et al. 2018; Metzger and Tucker 2017) and a few more have studied protest organization and recruitment (Barberá et al. 2015; Cha et al. 2010; Sun et al. 2009). Few managed to generate some theoretical insights, but even these remained limited (Langer et al. 2019; MacDuffee Metzger et al. 2016; Munger et al. 2019). At the same time, many computer scientists created intricate models designed to predict various social movement activities like protests (Bahrami et al. 2018; Compton et al. 2013; Korolov et al. 2016). While these models may be of use for computer science–driven questions, we find, like others before us, that the research design practices on which they are based can be problematic for answering theory-driven questions dominant in social sciences (Steinert-Threlkeld et al. 2015).
We find that open-source scientific data on social movements’ activities in the Age of Information are lacking, and this impairs the advancement of theoretical scientific debates on the matter. Yet, measuring and potentially forecasting the RM efforts of social movements on SMPs would have several benefits for scholars, policymakers, and financiers alike. For scholars, such measures and forecasts would help in better understanding and explaining why and when social movements arise. They would also help in explaining what contributes to the success or failure of such movements when taken in conjunction with other sociopolitical and economic variables. After all, RM does matter for the very success of contentious actions according to the vast literature on social movements. Scholars have discussed this matter through one of the three interconnected theoretical frameworks used to explain the success or failure of contentious action—resource mobilization theory (RMT; Abdul Reda 2016; Wiktorowicz 2004).
How can we measure the RM efforts of social movements on SMPs? In this piece, we focus on measuring the RM efforts of social movements on one SMP—Twitter. To do so, we aim to create the first ever measure of social movements’ RM efforts on an SMP. We start by developing two new tools—one for identifying tweets that seek to gather resources for social movements on Twitter and one for creating a quantitative measure of social movements’ RM activities at time
Both of the tools that we develop in this article are important empirical contributions to the scholarly literature because they open many new opportunities in the scientific study of social movements. For instance, they can be used as dependent variables when fitting models with explanatory variables openly accessible elsewhere. In fact, our RM score is significant for science because it can help evaluate new theories about the determinants of social movements’ RM efforts in a modern age where such endeavors are largely conducted online. It is also in and of itself an outcome variable that would serve to be analyzed against socioeconomic fluctuations and political announcements to advance theoretical understanding of the ways in which financial and political events affect mobilization. Likewise, the lexicon that we use to gather the tweets used to compute the RM score is also significant for science because it can help understand variations in RMs toward specific issues. When used, for instance, in complement with regular topic modeling algorithms, students of social movements could stand to gain a better understanding of the dynamics that affect RM toward key and current issues. Whether on issues of race, the presidency, immigration, or many others, fluctuations in Twitter posts on a variety of topics can be used in complement with different explanatory variables for a better theoretical understanding of RM on a variety of current issues. With large time components in the RM score, the scientific community could stand to even gain a better understanding of the effect of slow onset dynamics on social movements, in the like of climate change—like droughts—or the economy—such as delocalization of manufacture. In sum, the methods we develop in this article can be used with other, openly accessible data to better understand a plethora of dynamics related to social movements and protests, in countries where Twitter usage is largely in English and/or in the United States.
In this article, we focus on demonstrating the validity of our instruments. As a result, we mainly develop and cross check both our lexicon and our RM score here, and we do so by using millions of tweets from the United States streamed between November 28, 2018, and February 11, 2019. We present these tests, cross checks, and their results—among which that our RM score helps convincingly measure RM of social movements on Twitter because it strongly correlates with the occurrence of protests in the United States in this time frame. We also illustrate how our lexicon can be used with regular topic modeling algorithms to expand descriptive and qualitative understanding of social movement RMs on Twitter in a given time frame. We fit a topic modeling algorithm with 11 such issues for our time frame in the United States, the most salient of which were Ted Cruz’s campaign for Congressional Amendment, the Women’s March, the United Teachers Los Angeles (UTLA) and education-related strikes, and the campaigns in opposition and support of the Covington High School students. As a final step, we point at specific avenues through which the two tools that we develop in this article can be used for theory building and theory testing in the scientific study of social movements.
Theory
There is a vast and vibrant literature on social movements (Abdul Reda 2016; Kidd and McIntosh 2016; Paige 1975; Tarrow 1994; Tilly and Tarrow 2015). Yet we find no systematic attempt at formalizing the field around quantitative theoretical models to this day, despite the existence of formal theoretical approaches to the study of social movements. Social movement theories are made up of three general areas of theoretical inquiry—political opportunity structure (POS), RMT, and cultural framing (CF; McAdam 1999; Wiktorowicz 2004). According to McAdam (1999), these three theories represent the crystallization of a “consensus” on how social movements emerge around three dominant factors, namely, the existence of political opportunity, the availability of mobilizing structures, and finally an ideational framing process (McAdam 1999).
POS is interested with the structural overtures and limitations faced by social movements—degree of openness in the political system and its institutional structure (Kitschelt 1986; Meyer 2003), economic or political crises, coups d’état, wars, and so on. In fact, whatever structural conditions affect the playing field of social movements could be considered parts and parcels of the structure of political opportunities they face. For instance, McAdam’s seminal discussion of the rise of black insurgency movements in the United States outlined how an exogenous factor—the Cold War—was the key opening factor in the POS (McAdam 1999). Without that important condition to the opportunity structure, black insurgency movements may not have emerged or may not have been successful. The same can be said of industrialization, which very much leads to the intractable tensions between proletariat and bourgeoisie and expose the contradictions within capitalism (Tarrow 1994). In fact, POS is very much an operationalization of those key events in history which sometimes lead to a system breakdown or threats to ruling elites.
CF targets social movements’ attraction of popular support through their ideology. Cultural frames are tools used by movements to disseminate their ideas into society and address the key issues faced by the people in ways that seek to garner membership in the movement. One may consider that “they represent interpretive schemata that offer a language and cognitive tools for making sense of experiences and events in the ‘world out there’” (Wiktorowicz 2004:15). Therefore, CF rests on the premise that the framing of movements’ stances and goals matter for garnering popular support in the long run. After all, a movement could have ample political opportunities at hand and large resources mobilized, but an ideological framing very poorly adapted to the issues faced by the people. One case in point is the Syrian Muslim Brotherhood—it stands at the onset of the Syrian civil war of 2011 as “the best organized and best funded political group of the Syrian opposition by far” (Abdul Reda 2016:3; Khatib, Lefèvre, and Qureshi 2012:22-25). Yet despite considerable changes to its CF, it still failed to address key popular issues of the Syrian civil war of 2011 (Abdul Reda 2016). For some, this “cruelly” undermined the group’s efforts at leading the struggle against Assad’s regime despite unmatched RM capacities and an open playing field (Abdul Reda 2016).
Finally, RMT examines the impact of three sets of factors on contentious mobilization: the structure of social movement organizations (SMOs), their organizational basis, and resources. The extant scholarship has, inter alia, shed light on the organizational infrastructure of social movements and their degree of institutionalization (Kriesi and Wisler 1996), social networks and recruitment strategies (Fernandez and McAdam 1988; Snow, Zurcher, and Ekland-Olson 1980), tactical repertoires (Smithey 2009), and different types of resources available at their disposal such as material, human, social–organizational, cultural, and moral resources (Snow, Soule, and Kriesi 2007). Social networks are key here—for instance, black churches were these structures of social networking that helped mobilize black insurgents in the United States (McAdam 1999). In Saskatchewan, it was the agricultural cooperatives who helped farmers mobilize into a unique socialist movement for North America (Lipset 1971).
In more recent times, it is digital communication and social media that become the defining RM structures of social movements around the world (Diani 2000; Laer and Aelst 2010). Their effect on the organization of discontent is widely seen with the revolts of the Arab Spring, starting in 2011. A plethora of publications have focused on how social media were used to organize contentious action in Arab countries affected by the revolutions (Abdul Reda 2014; Eltantawy and Wiest 2011; Gerbaudo 2012; Howard et al. 2011; Rane and Salem 2012; Steinert-Threlkeld 2017b; Tufekci and Wilson 2012; Wolfsfeld et al. 2013). Ever since, it has been a key feature of every single social movement–related upheavals worthy of the name around the world. Twitter, Facebook, Instagram, and Google have, among others, been the key components of RMs in Iceland, Spain, and the Global Occupy movement ever since the Arab revolts (Kidd and McIntosh 2016) and political campaigns from around the world (Jungherr 2015). Such conclusions should not come as a surprise: After all, social media connect billions of people from almost every walks of life around the world in 2018 (Rahman 2018). Some estimates suggest that 3.3 billion people, or more than half the world population, used social media in January 2018 (Rahman 2018). Yet, to this day, no consistent measure of social movements’ RM capacities on social media exist, to the best of our knowledge.
Some students of digital communication, political communication, and social media have taken advantage of text-as-data methods to generate insights about the use of SMPs for organizing discontent. The thrust of this scholarship is to detect to what extent online contentious activity can generate off-line protest movements. For instance, Tufekci and Wilson (2012) use a survey of participants in the Tahrir Square protests of Egypt in 2011 to show how the use of social media increased the likelihood of joining protests. Others reveal how an individual’s decision to join a protest depends on the decisions of others in their social network (Larson et al. 2019). Overall, most studies are concerned with whether and how social media contribute to protest recruitment (González-Bailón et al. 2011; Jost et al. 2018; Metzger and Tucker 2017) or the mechanisms of user influence in protest organization (Barberá et al. 2015; Cha et al. 2010; Steinert-Threlkeld 2017b; Sun et al. 2009). Be that as it may, the field is very recent (largely post-2010) and still struggles to generate substantial theoretical insights (Langer et al. 2019; MacDuffee Metzger et al. 2016; Munger et al. 2019).
To the best of our knowledge, there are still no consistent measures of the capacities for social movements to emerge in given societies around the world. Many quantitative studies coming from the social sciences have discussed determinants of discontent and contentious action. But most focused on explaining the causes of social movements’ emergence (or lack thereof; Tertytchnaya and De Vries 2018; Tertytchnaya et al. 2018). Conversely, we would like to pique scientific interest in crafting a new measure of contentious action—one that can help quantify the chances of social movement–related activities in a specific society and potentially forecast them in the long run.
A very recent yet scarce literature has emerged in the past few years with the aim to predict various aspects of protests by using data from SMPs. Unfortunately, we join others before us in highlighting some important limitations in this literature, which remains to this day largely computer science driven. For instance, the overall trend in this literature is to select on the dependent variable by filtering tweets by keywords, hashtags, or selecting trending tweets. Such practices may be less concerning when the contentious action of interest is described only by the keyword in question (like “protest”) or the given hashtag (such as #BLM). But they are more problematic when scholars are interested in assessing the relative saliency of the issue in question and its socioeconomic determinants and/or when many different keywords and/or hashtags are used by the members of the social movement. In these situations, such practices can amount to selecting on the dependent variable—a practice in research design which has been shown to create unreliable results in the past (Geddes 1990, 1999). In such conditions, scholars need a tool that can generate representative samples of tweets about social movements’ RM activities online, without selecting on the dependent variable.
We are not alone in emphasizing the predominance of some important limitations in the literature aiming to predict various aspects of social movements using data from SMPs. Others before us have flagged the need for some new data-gathering tools that yield representative samples of online conversations around social movement mobilizations (Barberá and Steinert-Threlkeld 2019; Jungherr 2015; Salganik 2017; Steinert-Threlkeld 2018). As a result, some have previously mentioned that selecting tweets “based on protest-related topic words” may fail to give a “representative sample of online conversations” because it amounts to “selecting on the dependent variable” (Steinert-Threlkeld et al. 2015). Yet, some go as far as starting their data processing by reducing their data set to specific keywords—like “protest” and “rally” (Compton et al. 2013; Korolov et al. 2016). Not only is this a problematic case of selection bias depending on the research question being asked, but as we show later in this article, this approach fails to capture other very important keywords—like strike, march, and so on.
To avoid selecting on the dependent variable, scholars need a tool that can stream the whole Twitter application programming interface (API) for a geographical location and that is optimized as a natural language processing algorithm for the triage of tweets. Unfortunately, others have based their whole models on trending tweets only, something akin yet again to selecting on the dependent variable (Bahrami et al. 2018). In the case that trending tweets are the issue of interest, scholars still need a tool that yields a representative sample of tweets as a first step, which is then complemented by triaging for most popular tweets—by accounting for likes and retweets.
In this article, we design a data gathering algorithm specifically crafted with representative samples in mind. We do so to address a gap in the literature, as we find no previous attempt at using advanced data-driven models to measure or predict contentious action in a manner that is based on the social science theories that were developed to explain contentious action. The publications that we looked at are generally the works of computer scientists attempting to leverage SMPs’ data to predict protests. But we find no attempt at connecting social scientific theories—like POS, RMTs, and CF—to the construction of these models. We also find no consistent, open-source measures of the risks that social movements emerge in a society being registered in a data set for a range of societies. In fact, we notice that the literature builds a few different models that approach aspects of SMPs’ usefulness to protests, user centrality and influence in protest recruitment, protest prediction, but little to no theoretical integration and generalization of the models. Yet leveraging the power of big data in relation to social scientific theories of protest could potentially yield much stronger predictions that would be of worth to the academic, policy, and financial sectors.
Method and Measurement
In this article, we focus on measuring the RM efforts of social movements on Twitter. We approach social movements here in a broader sense that refers to a political claim-making bearing on other actors’ interest through coordinated actions (Tilly and Tarrow 2015). In other words, social movements mobilize resources to target specific institutions/individuals/entities, and secondly, they employ standardized methods to make their claims, such as demonstrations and petitions. As a result, our definition of social movements does not differentiate them from related concepts such as political campaigns or protest events. 1 One of the reasons why we adopt a large definition of social movements is because we notice that related concepts like political campaigns and protest events behave in a similar way online on questions of RM, thus warranting a similar conceptual approach. Moreover, our tools will remain of use to scholars interested in more focused definitions of social movements or its related concepts because the data they gather can be, in a second step, fine-tuned on either of them.
Our definition of RM encompasses the growingly different types of resources that can be marshaled by social movements in their engagement with the public sphere in the age of information. Namely, it encompasses online mobilization of material resources such as funding and physical capital but also of human resources such as voluntary participation, individual commitment, and skilled labor power; social-organizational resources such as networks and infrastructure; and cultural and moral material resources (Edwards et al. 2018:80; Staggenborg and Ramos 2016:20). What we aim to capture here is not a mere available pool of resources online but the outcome of actions by which these resources are strategically aggregated and mobilized in the aim to convert them to enable collective contentious action (Martin 2008:503). In this vein, social media provides an especially fruitful avenue for amassing financial contributions, mobilizing a large audience, and recruiting constituents for specific causes and events.
We move in four steps in the rest of this section. First, we present a computer algorithm, written in Python, that picks up Twitter posts about social movements’ RM activities based on the conceptual framework for social movements and RM that we just described. We design this algorithm based on a lexicon that we develop in the English language and which we test using Twitter posts from the United States. Second, we create a measure of social movements’ RM activities on Twitter. To assess the validity of this RM score, we conduct a range of tests which we present in this article and in its associated Online Appendix (which can be found at http://smr.sagepub.com/supplemental/). Third, we conduct some computer-assisted topic modeling of the tweets that seek to mobilize resources for social movements. Our goal here is to illustrate the qualitative utility of our data gathering lexicon for descriptive purposes. We expand on this goal in a fourth step by comparing the evolution of our RM score with the saliency of specific issues for which Americans mobilize resources on Twitter through time. Because of practical limitations, we limit our illustration of the methodological contribution of this article to the descriptive exercise illustrated in steps 3 and 4. In the Discussion section of this article, we expand on the ways in which our data gathering tool—the lexicon—and our RM score can be used by students of social movements to address the theoretical aims that drive social sciences.
It is worth mentioning that we look at Twitter posts—instead of posts on other SMPs—for a range of practical reasons. First, social movements use Twitter very profusely to organize discontent into contentious action. The protests of the past decade have proven the centrality of this SMP for organizing discontent around the world, from North America to South East Asia and passing by the Arab world (Bahrami et al. 2018; Barberá and Steinert-Threlkeld 2019; Compton et al. 2013; Steinert-Threlkeld 2018). Second, Twitter posts are easily accessible and have been used extensively in the literature on predicting protests (Bahrami et al. 2018; Korolov et al. 2016).
The Lexicon
We develop a lexicon that can identify Twitter posts that mobilize resources for social movements. The lexicon is made of four lists of keywords that each represents a unique aspect of tweets about protests; three of these lists are associated with positive conditions and the last one is associated with a negative condition. We implement the lexicon through a four-conditional algorithm—if there is at least one keyword from each of the positive lists and no keyword from the negative list in the text of a tweet, then it is categorized as a tweet that seeks to mobilize resources for social movements. Our algorithm considers regular tweets, extended tweets, retweets, and quotes when classifying tweets based on our lexicon.
To be clear, we are interested in Twitter posts that seek to organize discontent into contentious action—whether strikes, protests—but also posts that seek to gather a very wide variety of support for the movement—through fundraising, donations, and political support. In other words, the algorithm does not capture all social movement tweets but only the ones that aim at gathering resources—resources that they allocate for their claim-making. The general form of tweets that we are particularly interested in gathering is similar to these following tweets picked up by our algorithm: Why do you boycott? Members of Congress are pushing legislation against your ability to boycott and support BDS (boycott, divestment and sanctions of Israel) until Palestinians have freedom, justice and equality. Join this social media campaign: #whyiboycott https://t.co/TTyntwptfh. Come join me and Patrick Kennedy @PJK4brainhealth at our picket line in SF tomorrow 12/10.
We decide to build a lexicon for a range of reasons, chief among which is the absence of any preexisting and publicly available data set of tweets that seeks to mobilize publics in support of social movements. In fact, it is notable that there is no publicly available data set of Twitter posts about social movements to the best of our knowledge, such that researchers cannot train a machine learning model based on any preexisting data. At the same time, there is a noticeable ease associated with designing, implementing, and sharing a lexicon compared to other methods like training a machine learning model. A machine learning model would not necessarily yield more precision and would ultimately start with a preselection of tweets that would most likely be selected based on a lexicon. Ultimately, we also notice that the final version of our lexicon which we present here deals very satisfyingly with false positives, which we minimize by designing a category of negative keywords as the fourth condition of the lexicon.
We design our lexicon by iteratively reviewing the data for keywords. Our team of researchers conducted qualitative studies into the minutiae of tweets that seek to mobilize resources for social movements in the United States and identified four important components that became the four lists of keywords presented below. This initial step allowed us to identify a few important aspects that typified tweets seeking to mobilize resources for social movements. For instance, this was the particular importance of verbs and prepositions, over for instance, the adjectives that early sentiment lexicons focused on (Taboada et al. 2011:270). Subsequent iterations of this initial step also identified the often-recurring similarity between RM tweets and those that attempted to recruit support and attendance for sporting or cultural events—something which mandated the creation of a negative list of keywords. We also noted that it was sometimes better to be case specific about some keywords in our lexicon, while others could be left to vary in their inflections. 2
As a result, we noted that the current combination of keywords in the lexicon was optimal by examining thousands of tweets and cross checking our results with different configurations of the lexicon over a period of multiple months. We present results from quantitative probes into false positives and false negatives in the Online Appendix (which can be found at http://smr.sagepub.com/supplemental/) of this article for the final version of this lexicon. To be sure, we follow our previous recommendations in the process of collecting Twitter posts and in order to test the efficiency of our lexicon against false positives and false negatives. We livestream tweets from the United States using both a bounding box and a filter by geolocation, and only then do we test whether or not our lexicon does a good job of collecting tweets that seek to mobilize resources for social movements. 3 Moreover, we use a random time frame for our endeavor—we started the algorithm when practically possible for us and stopped it when it was impossible to collect more tweets because of space limitations, as the file had already reached a few terabytes in size.
The first component of the lexicon represents what can be considered as [“sit-in,” “protest,” “rally,” “picket,” “strike,” “boycott,” “march,” “fight,” “mobilization,” “petition,” “to demand,” “we demand,” “walkout,” “walk out”]
The second component of our lexicon focuses on keywords that seek to Join the Maryland HBCU students, this upcoming Saturday, December 8th as we rally in Annapolis, Maryland to DEMAND an appeal of the HBCU Inequality Court. If you can’t make it, show your support and sign the petition!?? https://t.co/Xv99bvvZFN. I’m attending a @theactionnet event: 2019 Houston MARCH. RSVP here: https://t.co/gvaEMfeyrI. Come join the Hate Busters and march to support women! [“join,” “joined,” “stand with,” “come out,” “come down,” “please come,” “support,” “help the,” “help us,” “i am going to,” “i’m going,” “im going,” “on my way,” “be there”] [“ in,” “at,” “to,” “for,” “against,” “outside,” “with,” “PM,” “AM,” “here,” “tonight,” “today,” “tomorrow”] [“box,” “game,” “play,” “ufc,” “wwe,” “twitch,” “mma,” “wrestl,” “fan,” “fans,” “marchmadness,” “march madness,” “christmas parade,” “viceroy,” “marchandiseur,” “fitness”]
RM Score
We design a simple ratio score that identifies the saliency of RM activities for social movements on Twitter based on the results yielded by parsing through the data using the lexicon. The score is computed based on the lexicon: If the tweet meets the lexicons’ conditions and originates from the United States, then it is stored in a data set of tweets predicted to be about protest in the United States and +1 is added to the number of tweets about protest overall in the United States. We then divide the total number of RM tweets by the total number of tweets at a single point in time. The RM score that we get generates an approximation of how much Twitter is being used for RM at a specific point in time. The equation that follows formalizes the computation of the score in question:
We also plot this score in a time series format to assess any peaks or declines in such activities. The equation that follows formalizes the model that we use for time series estimation of our score:
Topic Modeling
We conduct some computer-assisted topic modeling of the Twitter posts picked up by our lexicon for the geographical location of the United States in order to illustrate the qualitative and descriptive utility of the methodological tools we develop in this article. We do so to show how the tweets collected through our lexicon and used to compute the RM score can be used to better understand social movements in the United States in a given time frame. Because of space limitations, we cannot show here how the content of these tweets and the RM score we compute out of them can be used for theory building and/or testing. But we do flag some key avenues for theory-driven research on social movements based on the lexicon and the RM score that we introduce here.
We start preprocessing the tweets by removing three lists of stopwords in a manner that remains consistent with the rest of the literature. The first is a list of common stopwords included in the scikit-learn package, the second is a similar list that includes common typos and abbreviations that are common on SMPs, yet omitted by the scikit-learn package, and third, the corpus-specific stopwords that are identified by the keywords of our lexicons. We do this because this is a conventional practice in the literature working with text-as-data and tweets that has been demonstrated to allow for topic modeling algorithms to focus on more meaningful, contentful words rather than stopwords (Haselmayer and Jenny 2017; Lucas et al. 2015; Yang, Torget, and Mihalcea 2011). The reason is quite simple: Topic modeling algorithms generate lists of topics based on the repetition of words—as a result, each topic is usually represented by its top 10 words. Stopwords are not meaningful, as they largely represent types of words that we all use in sentences and which do not actually bring meaning to a text. As a result, “words such as ‘the’ are so frequent they are still likely to be prominent in many topics” and such “extremely frequent terms may overwhelm the model and reduce how well the model fits contentful terms” (Schofield, Magnusson, and Mimno 2017:432, 433). For these reasons, the literature is in agreement that “there are benefits in model quality when stopwords are removed” (Schofield et al. 2017:432). If most common stopwords are captured by the scikit-learn package, not all of them are, in particular those that can be prevalent in the type of colloquial English that often dominates SMPs like Twitter. As a second step, we therefore remove a list of stopwords that are similar to the one contained in the scikit-learn package but without apostrophes—like “dont” instead of “don’t,” or “im” instead of “I’m.”
Finally, in our situation, we find it important to also remove corpus-specific keywords considering the objective of this model—they are the ones contained in the [“sit-in,” “protest,” “rally,” “picket,” “strike,” “boycott,” “march,” “fight,” “mobilization,” “walkout,” “walk out,” “petition,” “demand,” “we demand,” “join,” “joined,” “stand,” “come,” “come,” “please,” “support,” “help,” “going,” “be there,” “ll,” “don’t,” “dont,” “im,” “did,” “don,” “amp”]
The TC-W2V method that we use for topic number optimization is developed by O’Callaghan et al. (2015) as a measure similar to the distributional similarity (DS) measures and using a word2vec model that creates a term vector
We use the NMF because it is simply more appropriate for so many short texts. We confirm that assessment by way of background analysis including modeling topics using latent Dirichlet analysis. We also optimize the number of topics using the TC-W2V method because it yielded more logically coherent topic numbers compared to other methods we tested for—such as the Mimno et al. (2011) coherence score. A manual, human analysis of the resulting categorization of tweets by our team of researchers confirmed that the TC-W2V method yielded more coherent topics and categories.
Finally, we plot the saliency of each topic through time to help us understand what issues are mobilizing people on Twitter the most and at what point in time. We do this also in a time series format formalized by the following equation:
Data and Application
We use data from the United States to test the efficiency of our methodological tools. We stream millions of tweets between November 28, 2018, and February 11, 2019, we store them in a JSON file, and we parse through it using our lexicon. We compute the RM score for the United States for the time period of November 28, 2018, to February 11, 2019. This time frame is based on practical limitations—live streaming tweets requires a substantial amount of computer storage space and tweets quickly generate very large JSON files. We therefore started the algorithm on a random date on which we had the capacity to start collecting tweets and ended it two and a half months later because of storage space limitations. Ultimately, we estimated the file would be large enough for scientific use as it already contained millions of tweets.
Figure 1 illustrates the evolution of that score between these two dates. The

Evolution of the resource mobilization Score in the United States (November 28–February 11, 2019).
To assess whether our RM score really measures RM—and not just online discussions—we cross check the variation of our RM score in relation with social movement activities happening on the ground in the same time frame and geographical location. As such, we assessed the validity of our approach to capture real-world RM by examining the correlation between the RM score and the number of protests happening in the United States during the same time period of November 28 to February 11, 2019. The number of protests that occurred between those dates is obtained from the Global Data on Events, Location, and Tone (GDELT) data set. GDELT actively monitors “the world’s broadcast, print, and web news from nearly every corner of every country in over 100 languages” and automatically collates said reports into events of various predetermined categories (Leetaru and Schrodt 2013). The major advantage of using GDELT for measuring protests is its wide geographical coverage and the extensive temporal availability of protest-related events in the data set (for similar applications, see Galla and Burke 2018; Qiao et al. 2017; Qiao and Wang 2015; Wu and Gerber 2018). GDELT has its limits though, as many critique the data set for often having too many duplicate returns (Halkia et al. 2020:43; Ward et al. 2012). To strengthen our argument in this article, we conduct an additional robustness check by rerunning the analysis with the Crowd Counting Consortium data set—a data set optimized for estimating the size of crowds rather than the number of protests and demonstrations, but which doesn’t suffer from deduplication of events like GDELT. The results of this robustness check are in the Online Appendix (which can be found at http://smr.sagepub.com/supplemental/) and remain consistent with our findings presented here.
Figure 2 presents both the total number of protests (from GDELT) in blue and the RM score in red. We can clearly see that the rolling seven-day average of RM score tracks the rolling seven-day rolling average number of protests quite well (Spearman correlation = .91, Pearson correlation = .95). Just like the RM score, the number of protests is lowest during the winter break (last week of December and the first week of January) and clearly increases following the first week of January. Thus, it is clear that social movements on the ground can largely be captured using our RM score. In the Online Appendix (which can be found at http://smr.sagepub.com/supplemental/), we present further analyses of our technique (e.g., analysis of precision) in addition to the results without smoothing.

Evolution of the Resource Mobilization Score in Comparison with the Number of Protest Events in the United States (November 28–February 11, 2019).
We showcase the usefulness of our lexicon to furthering a better descriptive understanding of social movements by analyzing the tweets used to compute the RM score with topic model algorithms. This analysis does not yield theoretical insights but showcases an added value of the tools we introduce in this article for theory building and testing—not only can we use the tweets collected by our lexicon for quantitative measures, but we can also use them for qualitative, descriptive insights. Moreover, we suggest that future research can use topic modeling of tweets collected by our lexicon in different time frames in complement with other multilevel variables to better understand determinants of RM on Twitter over different issues.
We analyze the RM tweets using non-NMF and compute optimized topics using topic coherence word2vec. The TC-W2V measure (described previously) suggests that our corpora of tweets can most optimally be categorized into 11 topics that describe the issues around which American social movements mobilize resources on Twitter during the time frame for which tweets are available. We follow the scientific convention by presenting these 11 topics along with the top 10 words that best define them in a table format—they are presented in Table 1. Moreover, we manually inspect the tweets that are categorized within these 11 topics to best understand what each topic refers to in relation to the top 10 recurrent words presented below.
Top Words per Topic
Most of the salient issues that drive American social movement mobilizations on Twitter are topically very clear and do not seem to necessarily identify a singular movement for each topic. Overall, they appear useful in understanding the most salient issues around which one (or more) movements mobilize resources during a given time frame for which tweets are available. Moreover, some of them appear linked to a clear social movement—because they largely repeat the same rhetoric and often even simply retweet the same message passed down to them—while others seem constituted of many disparate movements—because messages are more diverse, and/or the object of contention less clear. Yet, it remains interesting to note that the entirety of these tweets is used to compute the RM score presented earlier and which correlates strongly with protest events taking place on the ground. As such, even though disparate sometimes, these tweets do properly measure RM when taken as a whole because they clearly connect with social movement actions on the ground.
For instance, Topic #0 shows that Americans mobilize in support of Ted Cruz’s campaign for congressional amendment. On January 4, 2019, Cruz introduced a bill that would put limits to Senators’ appointments in the United States to two 6 years terms only and limit members of the House of Representatives to three 2-year terms. As of 2019, the appointment terms for both houses of Congress remain unlimited, and Topic #0 shows that Americans are quite active in support of putting an end to the status quo (Shamsian 2019). This is an important finding because it demonstrates that Americans are still very concerned about the “brokenness we see in Washington,” roughly two years after it deeply affected the 2016 Presidential elections (Robinson 2019). As a result, people throughout the United States mobilized support for the movement seeking a congressional amendment, mainly by retweeting the following message and calling on others to do so too: “I agree with @tedcruz…It is LONG past time for congressional term limits. Join the fight and support Ted’s constitutional amendment.” Because the campaign was organized through retweets mostly, this topic also shows an interesting pattern of rhetoric repetition, potentially coming down from movement leaders.
Topic #4 represents the many campaigns of RMs that take place to fight against some of the biggest health issues of our time such as cancer. To do so, activists mostly organize fundraising for research aimed at curing or preventing the disease altogether. They do so with tweets like the following: “We won’t ever give up fighting cancer. As Notre Dame and Oklahoma tip off at the Jimmy V Classic, join the fight!” Sometimes, they even post tweets calling on to well-known fundraising movements like “Movember”: “Now this guy knows how to participate in #NoShaveNovember! Found #Sasquatch at the Bonneville Fish Hatchery. Proud to support family, friends & everyone who’s fighting #cancer. #Bigfoot #CancerSucks #NewsBeard #LetItGrow #ChinStrapBeard #Movember.” That being said, Topic #4 is not secluded to cancer only as it also contains tweets fundraising for other major health issues like AIDS: “On #WorldAIDSDay we remember the more than 35 million who we’ve lost to this devastating disease, support the 37 million living with HIV, and recommit ourselves to fighting the stigma & finding a cure to end this destructive pandemic. #RockTheRibbon.”
Topic #2 pertains to Republicans mobilizing support against Democrats and denouncing what they view as Democrat facilitation of illegal immigration on which the government spends billions every year. It echoes the public debate that takes place in late 2018 and early 2019 regarding the financing and building of President Trump’s promised wall with Mexico to supposedly stop illegal immigration (Gramlich 2019; Reason 2017). The campaign picked up by our lexicon during the time frame for which we have tweets organized particularly against Senator Kamala Harris and heavily based on a series of retweet of the following message: “@DonaldTrumpsPe1 @KamalaHarris America spends 135 Billion dollars a year on illegal immigration. 5 Billion is 1/1000th of Americans’ yearly budget. The President is fighting for America and showing the world who Democrats care about. Democrats don’t care about the American people. Why do you support Democrats?” Just like the movement for congressional amendments, this movement is also based largely on a series of retweets and therefore shows yet another interesting pattern of rhetoric repetition that may be based on movement leaders.
Topic #8 represents the many mobilization of resources both for and against the Women’s March—some helping its organization and some denouncing it as having anti-Semitic elements. It goes to show that the public debate over the anti-Semitic controversy of the Women’s March leaders was reverberated in the public sphere as people mobilized on both sides of the debate (Glusker 2019; North 2019). As a result, some called on support for the movement by joining local Women’s March already taking place in January 2019 with tweets like “@womensmarch’s 6th California sister march is in Inland Empire, CA! Join @WomensMarchIE there on 01.19.19!” or “I just RSVP’d for the 3rd annual Women’s March. The #WomensWave is coming. Join me in DC on 1/19/19. @womensmarch.” Others flat out called on others to oppose the movement with tweets accusing its leaders of being racists in some form: “Dear @MoveOn, Y’all need to seriously rethink your decision to stand with the hypocritical racist anti-Semitic bigoted hate mongering supporters of Louis Farrakhan from the Women’s March because there’s only one side. The one that doesn’t promote hate” or “If i tell you why you should not support the (white) women’s march and you can still hit me with that the leader is different from the organization’s bullshit then I guess you’re antisemitic too.”
Topic #3 illustrates the efforts of pro-environment nongovernmental organizations and activists seeking membership and support for their cause in light of global warming. Some clearly called for people to show support for the Paris climate agreement: “Proud to see the #ClimateHeritage Mobilization @ #GCAS2018 highlighted as an action of the #EYCH2018! All working together, we’re mobilizing the heritage sector for #climateaction & support of the #ParisAgreement!! #EuropeForCulture #ClimateHeritage.” Others took this cause to elected representatives themselves by demonstrating in Washington, DC. “Wow. Tomorrow, @sunrisemvmt is descending on DC to demand US Congressional leaders support a #GreenNewDeal—because there are #NoExcuses not to take bold climate action.” Finally, others even took on this issue in relation with Indigenous rights to the lands being exploited by big multinationals: “This is so NOT cool. Please join the #PoorPeoplesCampaign to demand Congress stop the immoral and unconstitutional seizure and sale of Apache sacred grounds to two of the world’s biggest mining companies.”
Topic #9 demonstrates that President Donald Trump remained very much still one of the most divisive issues in the United States at the end of 2018 and beginning of 2019. This topic represents the many mobilizations in support and against Trump himself, which were called upon in relation to the wall, the border, Democrats, the country, and God. For instance, some sought to mobilize support for the fight against Trump’s wall and against right-wing policies toward children of illegal migrants born in the United States: “Miriam a TPS holder and Shaniqua getting ready to lobby their MD MOC against Trumps wall and to demand that the new Congress come up with a legislative solution for #Dreamers and #TPS holders. Join them by calling ur members of Congress: 1-888-204-8353 #DefundHate”
Topic #10 represents another key issue in American politics at the end of 2018 and beginning of 2019—funding of the public education sector. This topic illustrates the many efforts that went into mobilizing resources for the strikes of the education sector in the United States, and in particular those of the UTLA. As a result, tweets categorized in this topic number call for support and participation in the education strikes: “@UTLAnow San Pablo Teacher Roxana Due poses with Los Angeles artist @AltEEOB Ernesto Yerena Montejano’s poster in her likeness. Join next weekend’s #March4Ed! #strikeready #UTLAstrong.”
Topic #7 is also very evident from the top 10 words. It represents the wave of support and opposition for students of the Covington Catholic High School who got caught in an altercation at Washington, DC, with native activists. In fact, it is determined by petitions and other types of collective actions organized largely in opposition to the students before the full video of the altercation was released and, subsequently, in support of the students being reinstated in their high school. The wave of support for students emerged after the full video was released and showed the students to be innocently accused of racism in a video taken out of context, which put them at threat of being kicked out of their high school (Whitcomb 2019). As a result, many of the early tweets categorized in this topic number sought to gather opposition to the students by retweeting the following message: “@WrathOfKhan2016 Please everyone, take a moment to call this number, I’m going to: Phone Covington Catholic High School in Kentucky to complain at (859) 491-2247—‘This MAGA loser gleefully bothering a Native American protestor at the Indigenous Peoples March.’” Then, later tweets were much more supportive of the students, as a large number of them retweet the following message: “Covington Catholic High School: Stand With The #CovingtonBoys—Sign the Petition!” This topic is also based on series of retweets that show a prevalent repetition of the same rhetoric which may be tied to either movement leaders or a core of activists.
A few topics are less evident and made up of smaller issues that would not value a category of their own. Instead, they are lumped together into topics that share similar semantic characteristics. As such, Topic #1 goes to demonstrate that Americans are very concerned about their rights and freedom at home. It is made up of many tweets that seek to mobilize popular support by outlining the people’s demands through an emphasis on rights and freedoms. Some, for instance, sought to gather support for movements fighting for Trans people’s rights: “On 3/31/19 Trans/GNC people and our allies march, united, on DC for our 1st ever National Trans Visibility March on DC…hope you will all join us @transmarchondc @Caitlyn_Jenner @PointFoundation @TransDivaChandi @candiscayne @BuzzFeedLGBT.” Others took inspiration from the Yellow Vest protests taking place in France to call for a similar anti-corruption and pro-freedom movement taking place in the United States: “@kdkelly8_16 @NoLongerIgnored @PatCarr12856 @KenSahib @MagniFieri #yellowvest movement coming soon to USA and the world. WE HAVE HAD ENOUGH. The elites have robbed the people for too long now. WE WILL FIGHT FOR OUR FREEDOMS AND TO END GOVERNMENT CORRUPTION! Vive la France! We stand with you!” Others yet sought support in their fight for animals’ rights: “How can you support this horrible cruelty.@Forever21 please take a look at what is happening to these innocent lambs & decide to be #CrueltyFree or people need to #BoycottForever21 PETITION attached.”
Topic #5 largely represents one of the movements for racial justice in the United States. It is based on the “color of change” platform, which supports activists in their fight against racial inequalities in America following the events of the Hurricane Katrina in 2005 by offering them a platform to centralize petitions against such issues (“Color of Change|We Help You Do Something Real about Injustice” n.d.). As a result, many of the tweets in this topic category are posted as retweets of the following message: “I just signed this petition on OrganizeFor with @ColorOfChange. Will you join me? #O4.” As a result, this is yet another topic based on the clear repetition of the same message, thus potentially showing a clear tie between movement members and movement leaders. Finally, Topic #6 also represents a variety of smaller issues aggregated into one topic of their own because they make use of the same type of semantics. For instance, many of these tweets simply go as follows: “@JoeyRockstone Where can I join would love to rally with you.”
We plot the saliency of our topics through time to better understand the saliency of each topic through time and in relation to the RM score. The result of this computation for the top six topics is plotted in Figure 3. The

Evolution of topics in the United States (nonnegative matrix factorization topic coherence word to vector) between November 28, 2018, and February 11, 2019.
Figure 3 shows that some issues are very constrained at a specific point in time and then relatively take a back seat in the mobilization of resources that take place on Twitter. It is notable that the issues with the highest peaks are generally the ones that are more limited through time. For instance, the most salient issue is Ted Cruz’s campaign against congressional amendment (Topic #0), which peaks in mid-January 2019, but relatively disappears thereafter. The mobilization of resources by the Women’s March and by movements opposing it (Topic #9) constitute the second topic that peaks the most. This topic becomes very salient around January 20, 2019, but also relatively disappears from Twitter posts thereafter. A third topic in that category is the defense campaign for the Covington High School students—represented by Topic #7. It peaks around January 20 and also takes a backstage thereafter. On the other hand, mobilizations for education-related strikes—represented by Topic #10—are the most consistent through time while being relatively above average in saliency. They peak mid-January and rise again in saliency in early February.
When taken in comparison with the RM score plotted in Figure 2, the topic saliency plot of Figure 3 showcases the issues that are most likely to drive the RM score back into a peak during the month of January 2019. It seems that the four topics that are the most responsible for the increase in the RM score at the turn of the new year are Ted Cruz’s campaign for congressional amendments (Topic #0), the education strikes of California centered on UTLA (Topic #10), mobilizations for and against the Women’s March planned for March 2019 (Topic #8), and mobilizations in favor of the Covington High School boys (Topic #7). RMs regarding these four issues all pick up steam right after January 1, 2019, with all of them save the education strikes peaking around January 15. It is impossible to theoretically explain fluctuations in these topic saliencies in the context of this article, but it seems plausible that resource mobilizers “take a break” from social issues during the holiday season to attend more personal matters. There is indeed a noticeable flattening of the saliency of all topics a little after December 15, 2018, and right until January 1, 2019. In the Online Appendix (which can be found at http://smr.sagepub.com/supplemental/) to this article, we include a few more plots that serve to illustrate in more detail the connection between topic saliency and RM score.
Discussion and Conclusion
How can we measure the RM efforts of social movements on Twitter? In this article, we developed a lexicon that can filter tweets that mobilize resources for social movements. We also created a score that measures the saliency of RM by social movements on Twitter. After presenting these two tools, we illustrated their effectiveness by using Twitter data from the United States, which we streamed between November 28, 2018, and February 11, 2019. More specifically, we demonstrated how our RM score strongly correlates with the number of protests in the United States, indicating both acceptable sensitivity and precision. This test lent additional credence to the arguments developed since the Arab Spring of 2011 that SMPs do seem to reflect if not contribute to protest events by social movements (Coppock, Guess, and Ternovski 2016; Enikolopov, Makarin, and Petrova 2019; Larson et al. 2019; Steinert-Threlkeld 2017b; Tufekci and Wilson 2012). We also used our method in conjunction with well-acknowledged approaches to computer-assisted topic modeling to illustrate how the tools we introduce in this article can help shed descriptive and theoretical lights on social movement mobilizations on SMPs. We have focused on testing the validity of these new tools in this article and therefore haven’t used them for theory building and testing, but we still have pointed at concrete ways in which they can be used to this aim in order to advance the scientific literature on social movements. To be sure, more remains to be done to construct a holistic measure of RM altogether. But our findings matter as they show that our lexicon and RM score can be used to measure the RM activities of social movements on Twitter in the United States.
We cross validated that the lexicon and RM score that we introduced in this article actually serve to properly measure the RM efforts of social movements through a range of tests, some included in the Online Appendixes (which can be found at http://smr.sagepub.com/supplemental/). For instance, we computed an analysis of user IDs to verify whether the tweets collected by our lexicon and used to compute the RM score were only the work of a handful of users, thus potentially negating the idea that they measure actual RM.
4
In so doing, we find that the thousands of tweets in question are largely posted by individual users. These results serve to lend additional credence to the idea that our RM score actually measures RM based on the evidence presented in the body of this article that it also strongly correlates with protests. We also computed an analysis of keyword usage to better inform how our lexicon picks up tweets about social movements’ RM, and we computed additional precision and sensitivity checks. Finally, we created an alternative lexicon with the word “sign” added to the
Our tools have a number of important implications for the scientific study of social movements using big data. We previously described several issues with the scientific literature that approaches social movements’ activities on SMPs, chief among which is selection on the dependent variable for some research questions. In relation to this literature, one of the important contributions of our article is that our Twitter-based model is not built by selecting on the dependent variable. We streamed all tweets using the public Twitter API by only filtering by geolocation and only then did we optimize our lexicon through several back and forth with the data. This makes our model much more generalizable. We also find that the only other work that considers a representative sample uses hashtags to filter tweets about protests (Steinert-Threlkeld et al. 2015). This is doubtless a very good method when one is interested in these specific hashtags, but as our findings show here, such practices are not optimized to account for the diversity of tweets
The methodological tools we presented in this article open the door to a range of possibilities that have key contributions to the scholarly, policy, and financial minded inquirers. For scholarship, our RM score may be used to better understand what correlates with social movement mobilizations around the world, in democratic and authoritarian regimes, and in different time periods. For the policy world, our lexicon teamed with topic modeling enables policymakers and politicians alike to better understand the issues that mobilize the citizens who elect them. It also helps them better position themselves on such key issues. Finally, the finance-minded observer may also gain from the fruits of the method we introduce here—after all, societal upheavals stemmed by social movements are a key component of the political risk involved in investing in different societies.
To be sure, future steps need to be taken in ameliorating the current model. There are many opportunities in combining the results of our RM index with potential measures related to POSs, CF, and other RM efforts of social movements. The next step in developing our lexicon would be to translate it to a range of different languages—French, Arabic, and Spanish. Finally, the same efforts at measuring social movement mobilization should be extended to other key social media for contentious politics, like Facebook.
Supplemental Material
Supplemental Material, sj-pdf-1-smr-10.1177_0049124120986197 - Mobilizing the Masses: Measuring Resource Mobilization on Twitter
Supplemental Material, sj-pdf-1-smr-10.1177_0049124120986197 for Mobilizing the Masses: Measuring Resource Mobilization on Twitter by Amir Abdul Reda, Semuhi Sinanoglu and Mohamed Abdalla in Sociological Methods & Research
Footnotes
Acknowledgment
The authors would like to thank Ludovic Rheault and Nora Webb Williams for providing comments on preliminary versions of this paper.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to acknowledge the support of the Social Science and Humanities Research Council and the National Sciences and Engineering Research Council of Canada, and Université Mohamed VI Polytechnique for the financial support they provided for the research and publication of this article.
Supplemental Material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
