Abstract
Many scientists share preprints on social media platforms to gain attention from academic peers, policy-makers, and journalists. In this study we shed light on an unintended but highly consequential effect of sharing preprints: Their contribution to conspiracy theories. Although the scientific community might quickly dismiss a preprint as insubstantial and ‘clickbaity’, its uncertain epistemic status nevertheless allows conspiracy theorists to mobilize the text as scientific support for their own narratives. To better understand the epistemic politics of preprints on social media platforms, we studied the case of a biomedical preprint, which was shared widely and discussed controversially on Twitter in the wake of the coronavirus disease 2019 pandemic. Using a combination of social network analysis and qualitative content analysis, we compared the structures of engagement with the preprint and the discursive practices of scientists and conspiracy theorists. We found that despite substantial engagement, scientists were unable to dampen the conspiracy theorists’ enthusiasm for the preprint. We further found that members from both groups not only tried to reduce the preprint's epistemic uncertainty but sometimes deliberately maintained it. The maintenance of epistemic uncertainty helped conspiracy theorists to reinforce their group's identity as skeptics and allowed scientists to express concerns with the state of their profession. Our study contributes to research on the intricate relations between scientific knowledge and conspiracy theories online, as well as the role of social media platforms for new genres of scholarly communication.
Keywords
Introduction
With the digital transformation of science, several new genres of scientific work have emerged. One genre that has become particularly important in recent years is preprints. A preprint is an academic manuscript that is intended for publication in a peer-reviewed outlet but that has not yet been peer-reviewed. The first preprint server–a platform allowing scientists to make their preprints openly accessible online–was established in the early 1990s. Over the last decade, the number of preprint servers has increased substantially and the practice of sharing preprints has become widely accepted in most academic fields. Preprint servers create technical accessibility to preprints but unlike academic publishers lack the capabilities to create attention for newly uploaded manuscripts. As a consequence, scientists who want to generate attention for their preprints need to take care of it themselves. Many of them use social media platforms like Twitter to promote their manuscripts. To date, most research on the role of social media for preprints has focused on the intended effects of this form of information sharing. For example, sharing preprints on social media has been found to increase their likelihood of being downloaded and cited in academic publications (Fraser et al., 2020; Shuai et al., 2012) or picked up in news media (Fleerackers et al., 2022). However, surprisingly little is known about the unintended effects of sharing preprints on social media, most notably their relation to conspiracy theories, which has come to the fore during the coronavirus disease 2019 (COVID-19) pandemic (Heimstädt, 2020). One strategy to better understand the unintended effects of sharing preprints on social media is to pay attention to the key characteristic of this genre of academic output, that is, their uncertain epistemic status: In the absence of peer review, it is rather unclear to readers–compared to peer-reviewed articles–whether or not a preprint is making a contribution to scientific knowledge.
One group of social media users that might be intrigued by the uncertain epistemic status of preprints are conspiracy theorists. Conspiracy theorists use social media platforms to consume, discuss and diffuse normative narratives that explain phenomena or events by pointing to the (alleged) plots of (supposedly) powerful persons or groups (Sunstein and Vermeule, 2009; Byford, 2011). When promoting such narratives, conspiracy theorists exhibit an ambiguous relationship to science. On the one hand, they often portray scientists and scientific institutions as powerful and malevolent elite actors in order to undermine the epistemic authority of science (Jolley et al., 2020; Harambam and Aupers, 2015). On the other hand, they mimic scientific modes of reasoning and selectively enrol scientific voices that support their worldviews (Byford, 2011; Haupt et al., 2021). The uncertain epistemic status of preprints allows conspiracy theorists to mobilize these texts as scientific evidence for their normative narratives, even if scientists widely agree that a preprint is flawed. While conspiracy theorists also selectively enrol peer-reviewed research to further their goals, 1 preprints appear to be an even more susceptible target due to their uncertain epistemic status. They are susceptible because they allow researchers to present their arguments in an exaggerated form. This exaggeration creates attention for the preprint, yet would most likely be mitigated during the peer review process. However, if the review process is not yet complete, preprints can be both exaggerated clickbait and potential contributions to scientific knowledge. To better understand the unintended effects of sharing preprints on social media platforms, we set out to explore how two groups of users with presumably different attitudes towards scientific knowledge – scientists and conspiracy theorists–engage with this new genre of scientific output and its uncertain epistemic status.
We pursued this research interest through a case study of a biomedical preprint related to COVID-19, which was published on a preprint server in January 2020. After a brief discussion in the comment section of the preprint server, the community of scientists quickly agreed that the preprint was scientifically flawed. However, when the preprint spilled over to Twitter, a heated controversy over its scientific value evolved that included, among others, many scientists and conspiracy theorists (Heimstädt, 2020). We analyzed this case using a mixed-methods research design. First, we conducted a social network analysis of the debate to trace the structure of the two major groups involved (defined as follower relationships between and among scientists and conspiracy theorists) as well as their forms of engagement with the preprint (defined as tweets, retweets, replies, and quotes). Second, we conducted a qualitative content analysis of selected discussions among users to study how they discursively addressed the uncertain epistemic status of the preprint. We found that both groups are similar in size and volume of engagement but differ in their forms of engagement and the timing of their engagement. For both groups, we identified not only different discursive practices aimed at reducing the epistemic uncertainty of the preprint, but also practices aimed at maintaining epistemic uncertainty.
Our study contributes to the debate on the role of social media platforms for new genres of scientific output that have emerged from the digital transformation of science. While previous work has looked at the way in which social media platforms shape the reach of preprints, we unpack how social media platforms can turn into discursive arenas in which the epistemic status of a manuscript is claimed and challenged by scientists and non-scientists alike. How preprints become consequential beyond social media platforms depends not only on the attention they attract, but also on the struggles over their epistemic status on those platforms. We also contribute to research on the activities of conspiracy theorists online. Previous research has argued that the dissemination of conspiracy theories online is enabled by a lack of scientific expertise, particularly when conspiracy theorists imitate scientific reasoning and evidence. Our study paints a more complicated picture of the relationship between scientists and conspiracy theorists on social media platforms, suggesting that both groups engage in attempts to reduce as well as to maintain the uncertain epistemic status of preprints.
Preprints, social media, and epistemic uncertainty
One important dimension of the broader digital transformation of scientific work is the emergence of new genres of scientific output and ways to organize access to these outputs (Eve and Gray, 2020; Plantin et al., 2018). In this study, we focus on preprints as a new genre of scientific output that has become particularly important in recent years (Chiarelli et al., 2019b). Definitions of what constitutes a preprint differ slightly across academic communities. In this study, we define preprints as scientific manuscripts that are intended for publication in an academic outlet but have not yet been submitted for peer review or accepted for publication (Chiarelli et al., 2019b). The key characteristic of preprints compared to most other genres of scientific output (journal articles, books) is their uncertain epistemic status, which results from a lack of certification through formal peer review. Readers who encounter a preprint have no certainty whether this preprint makes a contribution to scientific knowledge or not. The text might or might not develop and substantiate its argument in a way that corresponds to the convention of its specific scientific community. Epistemic uncertainty hence results from a lack of information in the present about the future evaluation of an individual preprint in the peer review process (see also Pontille and Torny, 2015). Traditionally, the only academic texts with an uncertain epistemic status were those currently undergoing peer review. Exposure to these texts was restricted to the small group of academics involved in the peer review process. At the end of the process, the epistemic uncertainty was reduced by the editorial decision to either accept or reject the text. The epistemic uncertainty during the peer review process had no consequences beyond the small group of authors, reviewers and editors involved. Preprints, however, circulate much more widely and beyond the academic community, eventually giving rise to broader contestations of their epistemic status and new practices of dealing with their epistemic uncertainty.
The starting point for this wide circulation of preprints is preprint servers. In the early 1990s, arXiv was established as the first preprint server, and allowed scientists from engineering and the natural sciences to share their manuscripts online (Ginsparg, 2021; Reyes-Galindo, 2016). Throughout the 1990s and 2000s only a few academic communities used preprint servers (Acord and Harley, 2012; Chiarelli et al., 2019a). However, over the last decade the practice of sharing preprints has become more widely accepted across academic fields and the number of preprint servers has increased drastically to more than 60 in 2019 (Chiarelli et al., 2019a). When a preprint is submitted to a preprint server, voluntary administrators screen the new submission and reject the manuscript if it violates formatting requirements or clearly falls outside of the server's disciplinary scope (Ginsparg, 2021). Preprints that pass this screening receive a Digital Object Identifier (DOI) and become openly accessible on the platform. Preprint servers have been developed to allow scientists to make their manuscripts openly accessible (and discoverable via search engines) on the Internet at no cost. In doing so, preprint servers challenge and partially de-integrate existing norms and infrastructural arrangements of scholarly communication (Plantin et al., 2018). A consequence of this position in the broader ecology of scholarly publishing is that most preprint servers lack the capabilities to promote new manuscripts to academic audiences. Most of them are not linked to traditional academic databases and do not maintain comprehensive mailing lists comparable to those of academic journals. Consequently, many scientists make use of social media platforms like Twitter to promote their new preprints themselves.
Research on the role of social media platforms for the dissemination of scientific outputs has focused predominantly on peer-reviewed publications (Taylor, 2020). Only recently, researchers started to empirically examine preprints and their circulation on social media platforms. For example, it has been found that promoting preprints on Twitter strongly increases their download numbers on preprint servers and gives their peer-reviewed versions a citation advantage over peer-reviewed publications that have not previously been shared as preprints (Fraser et al., 2020; Shuai et al., 2012). It has also been found that sharing preprints on social media positively affects their coverage in news media in comparison to preprints that have been uploaded but not promoted (Fleerackers et al., 2022). Research in this line of work hence focuses on the intended and potentially positive effects of sharing preprints through social media platforms. However, they say relatively little about the relationship between these platforms and the key characteristic of preprints, that is, the uncertainty of their epistemic status.
Conspiracy theorists on social media: The role of scientific knowledge
The lack of peer review that is characteristic for preprints can have societal benefits, for example the quick dissemination of research results in times of crisis (Fraser et al., 2021), or the opportunity to publish heterodox research in fields where orthodox groups of scholars dominate editorial boards and reviewer pools (for the field of economics, see Dobusch and Kapeller, 2012). However, when publishing and promoting preprints, their uncertain epistemic status can induce unintended and potentially negative effects. One of these unintended effects is the advancement of conspiracy theories on social media platforms (Papakyriakopoulos et al., 2020; Caballero, 2020).
What exactly constitutes a conspiracy theory is subject to an ongoing academic and public debate. For the purpose of this article, we define conspiracy theories as normative narratives that explain phenomena or events by pointing to the (alleged) plots of (supposedly) powerful persons or groups (Sunstein and Vermeule, 2009; Imhoff and Lamberty, 2017). Conspiracy theorists–individuals who consume, discuss, and diffuse such narratives–identify as people who seek and promote ‘true’ knowledge (Byford, 2011). Based on this identity, they maintain a complex and ambiguous relationship to science as the societal institution that has traditionally been most strongly associated with the production of universal and trustworthy knowledge. On the one hand, conspiracy theorists often portray scientists as key actors in their narratives of secretive plots. For example, a widespread conspiracy theory claims ‘that climate change scientists fake their data in order to receive research funding’ (Jolley et al., 2020). In line with this, conspiracy theorists are ‘inherently skeptical towards conventional mechanisms of warranted belief production’ such as ‘peer reviewed journals, judicial investigations, university departments or scientific institutions’ (Byford, 2011: 89). Conspiracy theorists criticize these mechanisms of knowledge production and therefore challenge the epistemic authority of scientific institutions (Harambam and Aupers, 2015). On the other hand, conspiracy theorists ‘routinely seek to emulate mainstream scholarship and inquiry’ (Byford, 2011: 89) in order to substantiate their narratives, for example by referring to studies or experts who have allegedly been excluded from the scientific or political debate by powerful actors. These activities of conspiracy theorists are in line with research that shows how the spread of misinformation is particularly rapid when such false claims evoke scientific authority or evidence (Haupt et al., 2021). Furthermore, conspiracy theorists imitate scholarship and scientific research by adopting a scientific style of writing and by establishing organizations and journals whose façade mimics that of traditional scientific institutions (Byford, 2011).
A growing body of literature addresses the activities of conspiracy theorists on social media (Bangerter et al., 2020; Caballero, 2020; Uscinski et al., 2018). Some of this work examines the forms of engagement with scientific knowledge that social media platforms afford to conspiracy theories and theorists. For example, Papakyriakopoulos et al. (2020) show that conspiracy theories reproduced by reputable media outlets, which base their reporting on scientific facts and expertise, have a broader reach on social media platforms than conspiracy theories reproduced by alternative media outlets. Other work tries to assess if and how scientific knowledge can be used to fight the spread of conspiracy theories online. Several studies in this line of work conclude that conspiracy theories spread on social media due to insufficient interventions by scientific authorities. For example, Ahmed et al. (2020) find that in the context of conspiracy theories that link 5G technology and COVID-19, there has been a lack of scientific authority confronting these narratives and suggest more generally that ‘the assistance of […] relevant scientific experts’ might be an efficient method of counteracting misinformation (2020: 6). Other research points to the important role of fact-checking and credible sources in efforts to stop the spread of misinformation on social media platforms (Gruzd and Mai, 2020; Jung et al., 2020).
Our study departs from the following two interrelated observations on preprints and social media: First, scientists increasingly use social media platforms like Twitter to promote their preprints (Kupferschmidt, 2020). Second, conspiracy theorists use social media platforms to promote their narratives. While they claim to reject scientific institutions, they also selectively associate with scientific knowledge that allegedly aligns with their worldview (Byford, 2011). It, therefore, seems likely that conspiracy theorists try to enrol preprints for their narratives, even if the scientific community views them as failing to meet academic quality criteria. To learn more about the way in which scientists and conspiracy theorists on social media platforms controversially discuss and come to terms with the uncertain epistemic status of such preprints, we ask the following research questions:
How much do scientists and conspiracy theorists engage with controversial preprints, and how do these groups compare regarding their structure and forms of engagement? How do these scientists and conspiracy theorists discursively engage with the uncertain epistemic status of preprints?
Methods
The case of Uncanny Similarity
We address these questions through a case study of a controversial preprint related to COVID-19. In the wake of the pandemic, biomedical research on the new virus expanded rapidly. To accelerate research on this pressing issue, many researchers shared their latest results on the preprint server bioRxiv. On 31 January 2020 a team of researchers from two Indian universities published the preprint ‘Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1 gp120 and Gag’ (Pradhan et al., 2020, from here: Uncanny Similarity) on bioRxiv. In their preprint, the authors claimed to have identified similarities in the molecular structure of the new coronavirus and HIV. At the time of publication, the origin of the virus had already turned into a matter of public speculation. While most scientific experts presented zoonosis (the transmission of the virus from a non-human animal to humans) as the most likely origin of the pandemic, some also advanced the alternative hypothesis of an accidental release of the virus from a lab. Conspiracy theorists rejected these two hypotheses and instead promoted the narrative that the virus was deliberately designed and strategically released as a means of biological warfare (Heimstädt, 2020). In their preprint, Pradhan and colleagues did not explicitly refer to this conspiracy narrative, but used words and expressions that were sufficiently ambiguous to allow for such an interpretation. They not only qualified the alleged similarity between the viruses as ‘uncanny’, but also assessed such similarity as ‘unlikely to be fortuitous in nature’ (Pradhan et al., 2020: 1).
Shortly after its upload, scientists began to discuss the quality of Uncanny Similarity in the comment section that bioRxiv provides for every preprint. Dozens of comments were made within a few hours and the community quickly converged on the assessment that the study was fundamentally flawed in several ways. In response to this overwhelming criticism, the authors withdrew the preprint less than 48 h after the initial upload. In the discussion section of bioRxiv, one of the authors explained this decision and distanced himself from links between their work and the emerging conspiracy theories around the origin of the virus (@disqus_9vTYsrZnzD, 1 February 2020, 6:25 pm). Subsequently, a red banner was added to the preprint's webpage, which informed visitors of bioRxiv that ‘This article has been withdrawn’ (on 2 February 2020, 6:54 am). The formally withdrawn document as well as the discussion remained available on the server. With the withdrawal of the preprint and the authors’ statement, the discussion around Uncanny Similarity on the preprint server came to a halt. However, by that time the debate had already spilled over to Twitter, where the preprint was shared and discussed widely. The case of Uncanny Similarity thus allows us to examine a very particular kind of controversy surrounding scientific knowledge. Unlike many other controversies about scientific knowledge (Callon et al., 2011), there is little disagreement between scientists in this case. Rather, we are dealing with a controversy in which scientists and conspiracy theorists bring forward diverging evaluations of a text that claims to contribute to scientific knowledge.
Data collection
Research on controversies on Twitter oftentimes focuses on specific hashtags (Yang et al., 2017) or groups of users (Crosset et al., 2019). In line with our research questions, we moved the controversial preprint centre stage in our research design. To understand how scientists and conspiracy theorists engaged with Uncanny Similarity on Twitter, we retrieved all the tweets that referred to the preprint within one month after its publication (31 January to 29 February 2020). We retrieved the tweets via the full-archive endpoint of the Twitter API. 2 We included all tweets that either contained the preprint's title, URL or DOI. Only tweets that were written either in German or in English were further considered. We justify these sampling decisions more deeply in the next section. For each tweet we obtained the text, referenced tweets, username, timestamp (all timestamps in this paper are in UTC), as well as the number of likes, retweets, and quotes that a tweet received. In total, we retrieved 2629 tweets. From these, 557 are original tweets, 1406 are retweets, 505 are replies and 161 are quotes. 1499 of the tweets in our dataset were published between the tweet reporting the initial upload of the preprint (31 January 2020, 2:04 pm, tweet by @biorxivpreprint) and the announcement of its withdrawal on Twitter (1 February 2020, 7:27 pm, tweet by the founder of bioRxiv).
In addition to the individual tweets, we retrieved the user information (date of creation, bio text, number of followers and number of users followed) of 2234 users whose tweets were included in our dataset. A caveat to this step in our data collection is that the Twitter API only allowed us to retrieve user information that was valid at the time of data collection (February 2021), but not at the time when Uncanny Similarity circulated on Twitter (January to February 2020).
Data preparation
In order to analyze to what extent scientists and conspiracy theorists engaged with the controversial preprint on Twitter and how they compare regarding their structure and forms of engagement, we drew on a combination of social network analysis and the manual coding and classification of Twitter accounts (Ahmed et al., 2020; Gruzd and Mai, 2020).
As we are interested in the engagement of scientists and conspiracy theorists, we classified Twitter users into these two groups. We opted for manual instead of automated classification, because it allowed us to consider context and nuance when making the ethically challenging decision to classify an account as a conspiracy theorist. To allow for manual classification within our resource constraints, we reduced the number of accounts through several sampling decisions. As described in the previous section, we already limited the number of tweets by considering only tweets that directly referenced the preprint. We focused on these tweets, because directly referencing the preprint indicated a rather intensive engagement with it. In this step, we further decided to narrow our analysis to users (and their tweets) with at least 1000 followers. This reduced our sample from 2234 to 811 users that directly referenced the preprint at least once. Finally, we excluded accounts whose bios were written in languages other than English or German. Due to the language-specific filtering of tweets in the previous step, this only slightly reduced our sample from 811 to 789 accounts. Another 187 accounts were already deleted or blocked by the time of the analysis, which further reduced the sample to 602 accounts. This filtering of accounts reduced our sample of tweets from 2629 to 778 tweets, consisting of 210 original tweets, 395 retweets, 59 quotes and 114 replies.
Classification of accounts was primarily performed by the first author. The emerging coding scheme and particularly ambiguous cases were discussed with the other members of the research team. When an account could not be classified as either scientist or conspiracy theorist, we assigned it to the residual category of ‘other’. The category ‘scientist’ was applied to accounts that included links to a university or research centre, scientific titles or job positions (for example ‘Postdoc @Cambridge_Uni’) in the Twitter bio. In less clear-cut cases we engaged with the tweeting activity of an account and checked whether recent activities included references to markers such as peer-reviewed studies or scientific institutions. Classifying accounts to the category ‘conspiracy theorists’ was more challenging. Previous work on conspiracy theorists on Twitter has identified words such as ‘truth’ or ‘uncover’ as markers for this group (Ahmed et al., 2020). With regards to our dataset, we found that words and hashtags like ‘#CCPVirus’, ‘Deep State’ or ‘#TheNewGlobalOrder’ appeared frequently and served as markers that are specific to the COVID-19 context. If these markers were part of the Twitter bio, we classified the account into the category ‘conspiracy theorist’. In less clear-cut cases we again drew on the account's tweeting activities to substantiate our assessment.
Social network analysis
To develop an understanding of the engagements with preprints on Twitter, we started with a social network analysis, examining the dataset of 602 accounts and 778 tweets. To better understand the social network, we generated two different visualizations, zooming in on the follower network and the interaction network (Cha et al., 2010; Ke et al., 2017; Ribeiro et al., 2018). The follower network (Figure 1) gives us an idea of how different users are connected to each other. We identified different clusters within the follower network, using the Leiden algorithm with resolution parameter 0.0001. The Leiden algorithm (Traag et al., 2019) is used to detect communities in a network. It improves the Louvain algorithm, one of the most popular community detection algorithms, especially in terms of well-connected communities and speed. We then examined the clusters’ composition in terms of scientists and conspiracy theorists (Javed et al., 2018; Traag et al., 2019). We further computed the average cluster size, to give us some idea of their connectedness. We also examined the node degrees, their average and distribution, to get some further insights into their connectedness. The interaction network (Figure 2) gives us some insights into how different users interacted with each other. We identified different clusters within the interaction network. The clusters are formed by retweets, replies and quotes themselves; no community detection algorithm is required. We further examined the clusters’ composition in terms of tweets from scientists and conspiracy theorists. We further examined interaction activities within the groups and between different groups, and computed the average node degree. To better understand the temporal dynamics, we created two more visualizations (Figures 3 and 4). The visualizations give us an idea of how the engagement of the two groups compares over time. All software code used for retrieving and analyzing the Twitter data is available on Open Science Framework, along with lists of all retrieved tweets and accounts. 3
Follower network of large accounts with at least 1000 followers.
Interaction network of large accounts with at least 1000 followers. Arrows point from the users that posted the original tweets to the users that interacted with these tweets. An interaction is either a retweet, a reply or a quote. Arrows without shafts represent self-interactions.
Temporal structure of tweets between 31 January and 29 February 2020.
Temporal structure of tweets between publication (31 January 2020) and withdrawal of preprint (2 February 2020).
Qualitative content analysis
To develop a comprehensive understanding of the engagements with Uncanny Similarity on Twitter, we complemented the social network analysis with a qualitative content analysis of particularly salient discussions within our dataset (Caballero, 2020: 135–137). This methodological decision was inspired by calls to experiment with new combinations of quantitative and qualitative methods when studying digitally networked phenomena and information diffusion in digital publics in particular (Moats & Borra, 2018). While the social network analysis informed us about the extent to which, and patterns in which scientists and conspiracy theorists engaged with preprints, the goal of the qualitative content analysis was to understand how actors within and across these groups discursively addressed the uncertain epistemic status of the preprint. From the social network analysis we learned that a relatively small number of tweets attracted relatively lively debates (Figure 2). We, therefore, decided to follow up on these debates and focused on tweets from scientists and conspiracy theorists that were retweeted ten times or more during our observation period. In total, there are seven such tweets: five of these tweets come from scientists and two from conspiracy theorists. We included three more tweets (two from conspiracy theorists and one from a scientist) that were retweeted less frequently but were among the tweets with the largest number of replies. This resulted in a list of ten tweets that were starting points for salient discussions within our dataset.
We collected all the corresponding replies and quotes for each of these tweets, as well as discussions that evolved from these initial replies and quotes. We only included replies and quotes that were sent within our observation period (31 January to 29 February 2020). This produced a collection of 487 replies and quotes in total. This collection can be accessed through the aforementioned repository on Open Science Framework as well. We used the qualitative data analysis software MAXQDA to organize these individual tweets into separate discussions, each evolving from one of the ten original tweets. Subsequently, we analyzed these discussions as a group and in an iterative way (Flick, 2014). In the first round, we identified sets of tweets (e.g., one user replying to another) in which the epistemic uncertainty of the preprint was addressed. Noticing differences between these sets, we used the next round of analysis to find out whether a discursive practice aims at reducing or maintaining epistemic uncertainty. At the end of this round, we used the manual classification of accounts from the previous phase to link discursive practices to groups of scientists or conspiracy theorists. For each of the two groups, we identified one distinct discursive practice aimed at reducing the epistemic uncertainty of the preprint. For each group, we also identified one discursive practice aimed at maintaining the epistemic uncertainty.
Findings
Structure and forms of engagement with Uncanny Similarity
Within our sample of 602 large accounts with at least 1000 followers, 184 belong to the group of scientists and 169 to the group of conspiracy theorists. Hence, the groups of scientists and conspiracy theorists who shared the preprint during the first month were similar in size. Together, these two groups represent almost 60% of all the accounts that shared the controversial preprint. Figure 1 depicts the follower network of accounts that engaged with the preprint. The nodes in the network are accounts in our sample, the edges indicate that at least one of the accounts follows the other. Thus, the network shows the scientists’ and conspiracy theorists’ structures of engagement with the preprint.
As Figure 1 reveals, the follower network consists of two large clusters. We have analyzed these clusters by applying the Leiden algorithm. We have empirically tuned the resolution parameter to maximize the difference between scientists and conspiracy theorists within each cluster while maximizing the cluster size. For this purpose, we have designed a scoring function that we aim to maximize. The score of a cluster equals the absolute difference between scientists and conspiracy theorists that belong to that cluster, multiplied by the size of the cluster. The total score of the graph results from adding the scores of all the clusters it comprises, including those that consist of only one node. We have run the Leiden algorithm and computed this score for the values 0.1, 0.01, 0.001, 0.0001 and 0.00001. The last two values yielded the same score, and higher than the rest. We therefore set the resolution parameter to 0.0001.
The largest cluster that results from running the Leiden algorithm with the aforementioned resolution parameter comprises 260 nodes, 144 of which are conspiracy theorists (orange), 11 are scientists (blue) and 105 accounts belonging to the broad category of ‘other’ (grey). The second-largest cluster, which comprises 201 nodes, includes 157 scientists, 5 conspiracy theorists and 39 accounts from the category of ‘other’. The remaining clusters are considerably smaller. These numbers indicate that both scientists and conspiracy theorists tend to follow other members of their own community.
In Figure 1, it is clear to see that scientists are closer connected to each other than conspiracy theorists. We analyzed the average cluster size a scientist or a conspiracy theorist is in, to get a first impression of how well-connected the members of each group are. The average cluster size of a group results from adding the sizes of the clusters of each member of that group, divided by the number of members in the group. Scientists have an average cluster size of 188 and conspiracy theorists have an average cluster size of 228. Members of both these communities belong on average to larger clusters than the rest of the users, which have an average cluster size of 146. This result is to be expected, as the residual group ‘other’ is more heterogeneous.
Connectedness can be further evaluated by looking at node degrees, which show how many users each user follows (out-degree), and by how many users each user is followed (in-degree). When looking at the average node degree, which is computed by adding the incoming (outgoing) connections of each node, divided by the number of nodes, one can see that scientists are in general more connected than conspiracy theorists: Scientists have an average in-degree of 14.78 and an average out-degree of 14.10, whereas conspiracy theorists have an average in-degree of 5.99 and an average out-degree of 5.77. When looking at the distribution of how many users each user is followed (in-degree), one can see that some accounts are more visible than others. In total, 7 accounts have more than 100,000 followers, comprising six conspiracy theorists and one scientist. When leaving those accounts out, the difference between the average node degree of scientists and one of conspiracy theorists is even higher.
Within our sample of 778 tweets, we have 395 retweets, 59 quotes and 114 replies. 216 tweets belong to scientists, 278 to conspiracy theorists and 284 are classified as ‘other’. Hence, the number of tweets from scientists and conspiracy theorists were similar in size. Figure 2 displays the interaction network of our sample of 602 large accounts with at least 1000 followers, based on their form of engagement with the preprint, i.e., their tweets, retweets, replies and quotes. A node represents an account, while an edge between two nodes means that one account has interacted with a preprint-related tweet by another account. Nodes that are unconnected represent accounts that have not been interacted with from any other account in our dataset.
As Figure 2 reveals, there are two main clusters. The largest one comprises 164 users. In total, 128 of them are scientists (blue) and five conspiracy theorists (orange). The remaining 31 tweets, depicted in grey, are other users. The second-largest cluster, which comprises 85 tweets, includes 28 tweets from conspiracy theorists, four from scientists and 53 from other users. The remaining clusters are considerably smaller. These numbers indicate that both scientists and conspiracy theorists tend to retweet tweets from other members of their community. It underscores our finding that both groups keep to themselves.
Figure 2 indicates that there is very little interaction across the groups. Tweets from scientists are mainly interacted with by other scientists. Original tweets from scientists received 146 interactions from other scientists but only 15 interactions from conspiracy theorists. Original tweets from conspiracy theorists received 61 interactions from other conspiracy theorists and only 9 from scientists. The majority of tweets induces no or only very few interactions. However, a small number of tweets induces a lot of reaction. These tweets will be picked up in the qualitative analysis.
Figure 2 also indicates that scientists retweet each other more often than conspiracy theorists do. While the total number of tweets (including retweets, quotes and replies) is similar in both groups, we find that the number of original tweets from conspiracy theorists is much higher than that of scientists: 42 original tweets from scientists and 103 from conspiracy theorists. This structure suggests that although conspiracy theorists see the tweets of other conspiracy theorists, they decide to send a new tweet rather than interacting with others. Thus, the form of engagement with the preprint differs between conspiracy theorists and scientists.
This finding is also underpinned by the number of times a tweet has been interacted with. We see that tweets from scientists, with an average node out-degree of 0.91, are more frequently interacted with than the ones from conspiracy theorists, with an average node out-degree of 0.46.
To better understand the temporal structure of the preprint's circulation, we also looked at the temporal distribution of tweets sent by scientists and conspiracy theorists over time. During our entire observation period, 778 tweets were sent, 216 by scientists and 278 by conspiracy theorists. Figure 3 shows that the majority of these tweets were sent within the first three days after the publication of the preprint.
Figure 4, therefore, zooms into this initial period and shows the distribution and timing of tweets by scientists and conspiracy theorists inside this temporal bracket. The authors of Uncanny Similarity uploaded their preprint to bioRxiv at 5:54 am on 31 January 2020. The preprint first reached Twitter when it was shared by the account of bioRxiv at 2:04 pm on the same day. Shortly after the tweet, scientists joined the discussion on Twitter. This peak then decreased before scientists picked up the preprint again around 7 pm. At the same time, conspiracy theorists began to engage with the preprint. On the evening of the preprint release, both groups engaged heavily with the preprint, although the conspiracy theorists’ participation was slightly higher. The discussion on the preprint continued in both groups on 1 February 2020. In the evening, the withdrawal of the preprint was announced by Richard Server (founder of bioRxiv) on Twitter at 7:27 pm. This announcement seems to have resulted in a renewed engagement with the preprint among scientists, as seen by the increased number of tweets that evening. For the conspiracy theorists, however, the withdrawal of the preprint on 2 February 2020 at 6:54 am did not seem to play an important role, as it did not result in substantially increased tweeting activity within this group. On 2 February 2020, the day of the withdrawal, both conspiracy theorists and scientists tweeted about the preprint but to a lesser extent than on the first two days. While the tweeting activity of the scientists decreased towards the end of the day, a slight increase can be observed among the conspiracy theorists, thus indicating that with the withdrawal of the preprint the discussion among scientists came to an end, while the engagement of conspiracy theorists continued.
Discursive engagement with Uncanny Similarity
Zooming in on particularly salient discussions among scientists and conspiracy theorists, we find that the groups engage in distinctly different discursive practices but address the uncertain epistemic status of the preprint in the same two ways: reducing and maintaining.
Reducing epistemic uncertainty. Many conspiracy theorists sent original tweets that mentioned the controversial preprint and claimed its scientificity. Oftentimes, other conspiracy theorists replied to these original tweets in an affirmative tone, thereby trying to reduce the epistemic uncertainty of the preprint. For example, the user @ImperatorTruth tweeted: #CoronaVirus why is the MSM trying to downplay this information about how the Wu Flu was made in a lab for sure … https://biorxiv.org/content/10.1101/2020. 01.30.927871v1.full.pdf (@ImperatorTruth, 3 February 2020, 11:46 pm)
However, we also identified a more complex discursive practice through which conspiracy theorists tried to reduce the epistemic uncertainty of the preprint. In what we call a cascade of confirmation conspiracy theorists initially share the preprint but only hint at its potential status as scientific evidence for a conspiracy narrative. In replies and quotes, other conspiracy theorists then pick up on this hint and forge a stronger link between the preprint and the group's worldview. For example, shortly after the publication of the preprint, the account @RuthlessIndia tweeted: Breaking: Indian scientists have just found HIV (AIDS) virus-like insertions in the 2019-nCov virus that are not found in any other #coronavirus. They hint at the possibility that this #WuhanCoronavirus was designed […]. Scary if true (@RuthlessIndia, 31 January 2020, 8:40 pm) The gut feeling was that #ChinaVirus was a man made biological Warfare tool. This is now proved […]. (@amitabhafirst quoting @RuthlessIndia, 1 February 2020, 5:39 am)
Conspiracy theorists tried to reduce the uncertain epistemic status of the preprint by gradually reframing it from a ‘hint’ into a statement that is ‘proved’. The group of scientists on Twitter engaged in discursive practices geared towards a reduction of the preprint's epistemic uncertainty as well. However, instead of constructing confirmations of the scientific quality of the preprint, they sought to prove its scientific inadequacy. The way in which scientists discussed the content of the preprint with their peers can be described as an open peer review of the preprint. The openness of this form of peer review – afforded by the social media platform –, can be best illustrated in a discussion that evolved in response to the tweet in which the bioRxiv account initially shared the link to Uncanny Similarity. One of the scientists who replied to this initial tweet from bioRxiv was @stefanmgolas:
The central claims of this paper are completely false (@stefanmgolas replying to @biorxivpreprint, 31 January 2020, 9:57 pm)
Only a few minutes later, another scientist replied to this fairly general comment and provided a similarly critical but much more elaborate assessment: Yes! They used BLASTp to search databases with very short motifs, but did not report any e-values!!! I tried to replicate, I find sequences from many different viruses (including HIV-1) at very identify scores, but with e-values much higher than 10, so totally not significant!!! (@RomainStuder replying to @stefanmgolas and @biorxivpreprint, 31 January 2020, 10:20 pm)
One of the most striking aspects of this tweet is @RomainStuder's account of a failed attempt to replicate parts of the Uncanny Similarity study. In the traditional mode of academic peer review, reviewers might pose questions to authors, which they in turn are required to respond to in their revision. In this discursive pattern of open peer review, the authors are absent, but ad hoc reviewers such as @RomainStuder become the object of requests for clarification themselves–for example, when yet another scientist wants to learn more about the failed replication: ‘Which genome data are you using?’ (@PureDemocracyNZ in reply to @Romainstuder, 31 January 2020, 11:47 pm). In this discursive practice of open peer review, not only can reviewers be prompted to specify their assessment, but anyone can ‘jump in’ to respond to such open questions. In this specific situation, the scientist @ross_stalker answers this question in two consecutive tweets before @RomainStuder does: Proteome. The preprint seems very loose in its use of the terms nucleotide sequence and amino acid sequence. I find it interesting that they did a proteome alignment of 2019- nCoV and HIV1 but then didn't follow up with a genome alignment. (@ross_stalker replying to @PureDemocracyNZ, 1 February 2020, 12:20 am) If their argument is that these are inserts from HIV, and not chance alignments from point mutations, surely they should want to look at the genome, not just the amino acid sequences. (@ross_stalker replying to @PureDemocracyNZ, 1 February 2020, 12:21 am)
This discursive pattern of open peer review reveals one way in which scientists on Twitter come to terms with the uncertain epistemic status of the Uncanny Similarity preprint. In traditional academic peer review the epistemic status of a manuscript (either satisfying the scientific standards of a community or not) is established and stabilized by a clearly defined set of reviewers and editors. When controversial preprints move to Twitter, scientists perceive the need to clarify the epistemic status of these texts without an authority at hand (e.g., editors) who decide on the set of reviewers. As we have shown, scientists overcome this problem by providing individual comments on the preprint and by pushing each other to further clarify these remarks. Authority to assess the epistemic status of a preprint in open peer review does not come from being assigned the role of reviewer, but by means of self-selection and quickly making small but specific comments.
Maintaining epistemic uncertainty. A particularly surprising result of our analysis is that some members of the two groups also engaged in discursive practices that actively maintained the uncertain epistemic status of the preprint.
Conspiracy theorists maintained epistemic uncertainty through symbolic skepticism. In this discursive practice, conspiracy theorists provide a critique of the scientific accuracy of the preprint to other conspiracy theorists in order to reassure themselves and their peers of their collective identity as truth seekers. When performing symbolic skepticism, conspiracy theorists emulate procedures of scientific reasoning – not to completely disprove the epistemic status of the preprint but to keep it in suspension. An illustrative example of this dynamic unfolded around the user @arabbitorduck. We classified the user as a conspiracy theorist, as they used words like ‘Deep State’ or retweeted content from well-known conspiracy theorists like Jack Posobiec. However, shortly after the publication of Uncanny Similarity, @arabbitorduck published a number of tweets in which the user engaged critically with the study's methodological accuracy and limitations: [thread] 1/ Discussing the pre-print paper from Indian scientists regarding similarities to portions of the #WuhanCoronavirus and HIV – what they said, what it means, and what they may have wrong. (TL;DR: don't panic) (@arabbitorduck, 1 February 2020, 1:05 am) While there are claims that #coronavirus reportedly contains some elements similar to the HIV virus, there are different views on what that means. In the interest of information crowd sourcing and truth, here is an alternate assessment (@dadetrading quoting @arabbitorduck, 1 February 2020, 1:42 pm)
Through the formulation ‘in the interest of information crowdsourcing and truth’, @dadetrading expressed that the ‘alternate assessment’ might not be fully consistent with a particular conspiracy narrative, but that sharing this assessment is aligned with the identity of conspiracy theorists as a group that is inherently skeptical and dedicated to an impartial search for truth.
For scientists, maintaining the uncertain epistemic status of Uncanny Similarity became a conduit for launching into a broader process of professional reflection. For example, when the scientist @cshperspectives shared a link to the comment on bioRxiv in which one of the authors of Uncanny Similarity announced the withdrawal of the preprint (@cshperspectives, 1 February 2020, 8:27 pm), other scientists used this tweet as a launchpad to reflect on the scientific community's reaction to the controversial publication: rapid self-correction in science. without the $25 billion intermediary. (@drdevangm quoting @cshperspectives, 1 February 2020, 1:38 pm)
Very impressive- #preprints and #OpenReview work. Tweets of re-analyses by @trvrb and others really helped. Cautionary tale for researchers tempted to push out papers without thinking of consequences? #2019nCoV #MoreHasteLessSpeed (@nicolamlow replying to @cshperspectives, 2 February 2020, 9:32 am)
What cuts across these tweets is that they do not engage with the content of the preprint, but simply use the broader debate as a conduit to reflect on the norms and procedures of scientific knowledge production. For example, the user @drdevangm portrays the case of Uncanny Similarity as support for the idea that scientists can live up to their professional standards without the commercial services of large academic publishing companies (‘$25 billion intermediary’). On a different note, the user @nicolamlow interprets the situation as one that has exposed the harmful consequences of increased pressure to ‘push out papers’.
Besides the state and future of the review process, scientists also used the preprint's epistemic uncertainty to reflect professionally upon other issues. One of these issues was the question of how accountability in the field of science have changed and need to be reconfigured in response to the rise of preprint servers: This highlights a key issue with preprints. Deeply flawed & ridiculously speculative, but it's the latest fuel for #nCoV2019 conspiracy theories. Just like socmed needs to act on misinformation, servers like @biorxivpreprint have a responsibility here too. (@whereisdaz, 1 February 2020, 9:28 am)
Discussion
There is a broad interdisciplinary debate on the spread of conspiracy theories on social media platforms. With our study we contribute to one aspect of this debate, that is, the role of scientists and scientific knowledge in this process (Byford, 2011; Jolley et al., 2020). Previous work has already pointed out that the spread of misinformation is particularly pronounced when it is associated with scientific authority (Haupt et al., 2021). With our study, we show that such association is shaped by the epistemic status of a scientific output. We demonstrate that preprints are easily enrolled by conspiracy theorists due to their uncertain epistemic status. Previous research has further attributed agency to combat the spread of conspiracy theories to authority figures ‘with a sizable following’, such as government accounts, public figures, and scientists (Ahmed et al., 2020: 6). Other studies have indicated that the spread of conspiracy theories can be potentially reduced by fact-checking and directing people to credible scientific information sources (Gruzd and Mai, 2020; Jung et al., 2020). In our study we have looked closely at the way in which conspiracy theorists and scientists engage with a controversial preprint. Our research design did not allow us to assess the effect of the groups’ interactions on users not belonging to either of the groups. However, our analysis of the temporal patterns of engagement (see Figure 3) indicates that conspiracy theorists kept on engaging with the preprint after scientists had left the social media platform – an observation that casts doubt on the effectiveness of the scientists’ intervention as authoritative experts (Ahmed et al., 2020; Haupt et al., 2021). One explanation for this, which emerges from our social network analysis as well as the qualitative content analysis, is that the two groups engaged mainly with other members of their own group (see Figure 1 and 2). When users from one group engaged with an original tweet from another group, these engagements were either widely ignored (scientists reacting to tweets by conspiracy theorists) or twisted and reinterpreted in what we have called cascades of confirmation (conspiracy theorists quoting tweets from scientists). In summary, scientific engagement does not have a clear positive effect on the suppression of conspiracy narratives. However, it is also clear from our analysis that scientists’ tweets did not serve as ‘oxygen of amplification’ for conspiracy narratives either (Phillips, 2018).
Our study further contributes to research on the role of social media platforms for new genres of scientific output, especially genres whose access is not restricted by a paywall. Thus far, research on preprints has focused on the positive and intended effects that emerge when scientists actively share preprints on social media platforms like Twitter (Fleerackers et al., 2022; Fraser et al., 2020; Shuai et al., 2012). The primary contribution of our study to this debate is that we provide a rigorous empirical account of an unintended effect of sharing preprints on social media, that is, their vulnerability to be mobilized by conspiracy theorists. Our findings leave no doubt that conspiracy theorists attempt to mobilize preprints to imbue their claims with scientific authority and to reaffirm their collective identity. However, a conclusive assessment of this risk of preprints is difficult, since our analysis has also shown that when conspiracy theorists mobilize non-peer-reviewed preprints on social media, this can draw in large numbers of scientists who see it as their duty to openly peer-review the manuscript. While in the case of Uncanny Similarity, such a performance of open peer review seemed to have little effect on the conspiracy theorists, we can speculate that it could increase other Twitter users’ trust in science. Besides the mentioned unintended effects of the sharing of preprints on social media, our study also identified a new type of advantage for scientists. In many academic fields, the recent rise of preprints and preprint-servers has unsettled established modes of knowledge production (Chiarelli et al., 2019a). When preprints spill over to Twitter, scientists might feel encouraged to openly discuss with their peers how this new genre changes established modes of knowledge production and how professional norms and practices need to be adjusted.
Limitations, future work and practical implications
There are some limitations from our study's research design that must be considered when interpreting the results. First, our analysis focused on the dyadic relationship between scientists and conspiracy theorists. Future work should also look at the effect of the interaction between scientists and conspiracy theorists on other audiences on social media platforms, e.g., the ones we collectively refer to as ‘other’. Second, our findings need to be interpreted in light of our sampling procedure. It seems possible that this sampling procedure resulted in an underrepresentation of conspiracy theorists in our sample due to the suspension of accounts during the COVID-19 pandemic (Twitter Inc., 2021). Assuming that suspended conspiracy theorists return to Twitter with new accounts, they must rebuild their follower base and are therefore more likely to fall below our sampling threshold of 1000 followers. Due to our need for manual coding of accounts, we only analyzed tweets that made direct reference to the preprint. Therefore, no conclusions can be drawn from our study about how scientists and conspiracy theorists use likes and replies as a means to interact with tweets that make direct reference to the preprint. Third, our study traced the engagement with the preprint on one digital platform only. To better understand how controversies around preprints emerge and settle (Venturini, 2010), future work should trace preprints backward and forward, e.g., linking the debate on Twitter with debates in the comment sections of preprint servers or other social media platforms like Reddit.
Finally, our study allows us to discuss some practical implications for the governance of preprint servers. An important governance issue for preprint servers is content moderation. To date, most preprint servers limit moderation of preprints to the initial submission process. Our study raises the question of whether a server has the responsibility to act upon a preprint that passed the initial round of moderation. When the authors of Uncanny Similarity withdrew their preprint, the team behind bioRxiv decided to keep the preprint available on the platform and added a banner, which informed visitors that the preprint had been withdrawn. However, our findings raise the question whether preprint servers should remove preprints, which are deliberately misleading or unintentionally fuel misinformation campaigns. This more drastic approach to moderation has recently been taken by the preprint server SocArXiv. In response to a controversial preprint, the governance team decided to remove the preprint file and replaced it with a ‘tombstone’ that included the paper's metadata and a link to a statement that explains the decision (Marcus, 2022).
Footnotes
Acknowledgments
An earlier version of this paper was presented at STS Conference Graz, at Weizenbaum Conference, and at the Center for Interdisciplinary Research (ZiF) at Bielefeld University. The authors would also like to thank Katharina Berr, Sebastian Koth, and Cornelius Puschmann for their feedback and advice.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been partially funded by the Federal Ministry of Education and Research of Germany (BMBF) under grant no. 16DII138 (Weizenbaum Institut)
