How People Weave Online Information Into Pseudoknowledge

Abstract

Misinformation has found a new natural habitat in the digital age. Thousands of forums, blogs, and alternative news sources amplify fake news and inaccurate information to such a degree that it impacts our collective intelligence. Researchers and policy makers are troubled by misinformation because it is presumed to energize or even carry false narratives that can motivate poor decision-making and dangerous behaviors. Yet, while a growing body of research has focused on how viral misinformation spreads, little work has examined how false narratives are in fact constructed. In this study, we move beyond contagion inspired approaches to examine how people construct a false narrative. We apply prior work in cognitive science on narrative understanding to illustrate how the narrative changes over time and in response to social dynamics, and examine how forum participants draw upon a diverse set of online sources to substantiate the narrative. We find that the narrative is based primarily on reinterpretations of conventional and scholarly sources, and then used to provide an alternate account of unfolding events. We conclude that the link between misinformation, conventional knowledge, and false narratives is more complex than is often presumed, and advocate for a more direct study of this relationship.

Introduction

Digital misinformation has been widely recognized as a significant and serious modern problem (Howell, 2013). In a recent Pew Report (Anderson & Rainie, 2017), roughly half of the 1,116 experts surveyed voiced pessimistic responses to the following question:

In the next 10 years, will trusted methods emerge to block false narratives and allow the most accurate information to prevail in the overall information ecosystem? Or will the quality and veracity of information online deteriorate due to the spread of unreliable, sometimes even dangerous, socially destabilizing ideas?

The question suggests that the danger of misinformation is its ability to carry or inspire false narratives. Unlike a piece of fake news that cascades through social media, false narratives are not just passed from person to person. They are also constructed from disparate pieces of information that are drawn from multiple sources, building over time to become part of a plausible reality. Researchers have devoted significant attention to how misinformation spreads, how to detect it, and how to reduce its credibility. The goal of our study is to provide a richer description of how people collectively use different media to construct and adapt false narratives online.

Prior work examining the dynamics of false narratives online has often adopted methods that are widely used to study the diffusion of digital artifacts, such as hashtags and memes (Dodds & Watts, 2004). These methods treat a digital artifact as a kind of contagion which spreads via a series of social contacts on a social media platform (e.g., Bessi et al., 2015). These methods can yield great insights about the dynamics of an information contagion in social networks but do not reveal how the underlying narratives might change. Different methods are required to provide a richer account of how people engineer and adapt false narratives online.

Following previous work (Introne, Iandoli, DeCook, Yildirim, & Elzeini, 2017), we refer to false narratives that have begun to take on the heightened status of a plausible reality within a community as pseudoknowledge (PK). PK is accepted by the community as being at least as plausible as conventional knowledge but (as we show here) is based in part on unconventional epistemic strategies that include extrapolating from unconventional sources of evidence (e.g., pictures from a fictional movie or other bodies of PK), rejection of rigorous research (e.g., see the “defense of pseudoknowledge” [DOPK] code, below), and the unwarranted use of conventional knowledge (e.g., referencing theories about quantum teleportation as evidence of “stargates”).

We extend this previous work by illustrating how PK draws upon the broader media ecosystem, and orient our analysis around three research questions:

1. What is the internal structure of PK?

2. What criteria dictate the changes that are made to PK?

3. How do people use information arrayed across the media ecosystem to enrich PK?

To answer these questions, we analyze a popular online discussion about the existence of alien stargates on Earth, or stargate theory (SGT), spanning roughly 10 years and 6,800 posts, and involving 1,025 contributors. Through this analysis, we find a diverse population of users that engage in a kind of participatory storytelling, generating PK that conforms to a model of narrative (Stein & Glenn, 1979). Over time, this underlying narrative changes in response to attacks and other social dynamics within the forum. We also find that collaborators draw from a wealth of information sources to substantiate and enrich the narrative, including mainstream media, scholarly work, popular fiction, and other PK. The false narrative emerges as a compelling story, anchored in a dense thicket of information drawn from many conventional and unconventional sources.

In the following, we review recent literature that examines PK, and then discuss how a narrative framework can enrich such analyses. We then outline our analytical approach and introduce the forum. In our analysis, we first describe the construction of PK within the forum and then examine how it is enriched by sources outside of the forum itself. Finally, we conclude with a broad discussion about the implications of our work for emerging research on fake news and false narratives.

Review

Conspiracy theories, pseudoscience, and pseudohistory—each of which is a different kind of PK—are ways of explaining complex external events and circumstances in the world. Like scientific theories or media portrayals of world events, these theories can be understood as narratives that connect disparate events into a coherent pattern. Unlike scientific theories or carefully reported journalism, the coherence of PK may be judged more rigorously against internal beliefs rather than externally established facts (Evans, 2001; Wood, Douglas, & Sutton, 2012). In this way, PK is woven out of diverse pieces of information, some of which are objectively false. In this sense, PK, as we define it here, is a kind of false narrative.

Many researchers study how people construct, interpret, and use narratives (e.g., Braddock & Horgan, 2016; Bruner, 1986; Chatman, 1980; Fisher, 1985; Griffiths & Tenenbaum, 2009; Hastie & Pennington, 2000; Kuhn, 1991; Schank & Abelson, 1995; Stein & Glenn, 1979; Trabasso & van den Broek, 1985). A central question in this literature is what makes one more compelling than another? We could hope that people, like detectives, strive to uncover the true causal chain of events underlying a narrative by applying some rational, logico-argumentative process. This is unfortunately not how we typically behave (Kuhn, 1991). Although people are capable of sophisticated argumentation (Mercier & Sperber, 2011), these skills are usually only applied when we are motivated by social context and to protect existing beliefs (Kolstø & Ratcliffe, 2007; Mercier & Sperber, 2011). The lack of careful analysis might not be problematic if online narratives had clear provenance or were uniformly based in verifiable facts, but this is also clearly not the case. If people are not careful consumers, and there are no clear indicators of narrative credibility, what criteria do people use to determine which narratives to believe?

The cognitive psychologist Jerome Bruner (1986, 1991) suggested that narrative and argumentative reasoning are two fundamental modes of human thought and are subject to different criteria. Arguments are judged according to their veracity, or whether or not they are true, whereas narratives are judged according to their verisimilitude, or whether or not they seem plausible (Bruner, 1986). In light of the wealth of empirical evidence documenting people’s reluctance to evaluate information critically (Mercier & Sperber, 2011), and the challenges people face in establishing the credibility of web-based information, it is plausible that the criteria governing our judgments of narrative verisimilitude outcompete more deliberative modes of processing when evaluating online narratives. Moreover, because PK is not constrained by conventional standards of evidence, it is likely that it can be adapted more freely to enhance its verisimilitude or in response to social pressures.

Bruner did not establish a set of rules for verisimilitude, but a rich line of research in cognitive science indicates that the pattern of causal relationships among events in a narrative affects understanding, retention, recollection, and perceptions of its coherence (Bartlett, 1932; Kintsch & van Dijk, 1978; Mandler & Johnson, 1977; Rumelhart, 1975; Schank & Abelson, 1995; Stein & Glenn, 1979; Trabasso & Sperry, 1985; Trabasso & van den Broek, 1985). As we illustrate below, such work can be used to examine how people manipulate online narratives.

Research on the dynamics of online false narratives has focused on how they spread, rather than their internal structure. One series of studies has examined the dynamics of conspiracy narratives on Facebook, leading to a range of findings (Bessi, Caldarelli, Del Vicario, Scala, & Quattrociocchi, 2014; Del Vicario et al., 2016; Zollo et al., 2015). For instance, both conspiracist and scientific content tend to spread among people who are similarly polarized toward that type of content, but interest in conspiracy narratives (measured as the number of people with whom specific posts are re-shared) persists for longer periods of time (Del Vicario et al., 2016). Individuals who engage with conspiracist content are less critical of content they prefer (Bessi et al., 2014) but reject obvious attempts to debunk conspiracy theories and subsequently redouble their engagement with conspiracist content (Bessi et al., 2014; Zollo et al., 2017).

Other research illustrates how false narratives identified by digital signifiers like hashtags can change over time. Starbird et al. (2016) examined how individuals change their interpretations of rumors that are tracked via hashtags, introducing a distinction between a narrative and the hashtag that signifies its presence. They classified Twitter postings that carried rumors during a crisis, and identified distinct temporal variations in the content of people’s posts about the hashtag. Such mutability becomes more obvious when multiple media platforms are considered. For example, news coverage of the #pizzagate incident—a hashtag associated with shooting that was motivated by an online false narrative—illustrates how the underlying story was elaborated over time, picking up new pieces of misinformation as it traveled (Silverman, 2016).

How people use the broader online media ecosystem to engineer or adapt false narratives has not been widely studied. In one of the few such studies, Starbird (2017) found that a variety of alternative news outlets operate in concert to promote broader political belief systems, which are woven out of false narratives stemming from current events. Different sites play different roles in promulgating false narratives, with some being used as sources of evidence, while others combine evidence to craft richer stories (Starbird, 2017).

Our contribution extends Starbird’s (2017) work to examine how an online community uses different types of online resources to support and adapt a narrative. Ours is a rich and primarily qualitative investigation, intended to show how one community manipulates a false narrative to motivate future avenues of inquiry, rather than make general claims.

Methods

For our study, we sought to identify a long-standing forum discussion in an established community of individuals who were focused on maintaining PK, avoiding highly politicized and ideological conversations because we feared that the presence of strong external motivations might obscure other aspects of PK creation. For pragmatic and ethical reasons, we sought a forum with terms that would allow analysis, that was publicly visible without an account login, and did not by design reveal identifying information about its users.

Searching for the keywords “conspiracy forum” yielded more than 16 million resources, and after an investigation of these, we selected a site called “AboveTopSecret.” The site’s “about” page indicates that it hosts discussions about “a range of alternative topics such as conspiracies, UFOs, paranormal, secret societies, political scandals, new world order, terrorism.” ¹ From the many forums hosted on the retrieved site, we identified the discussion with the most replies in the “Aliens and UFOs” forum category. The forum discussion (Undo, 2006) began in 2006 and was last active (at the time of writing) in August 2016.

To analyze the conversation, we scraped all of the data from the forum into a database. At the time of scraping, the forum included 6,878 posts contributed by 1,025 unique users, consisting of roughly 1.3 million words (approximately 2,600 single-spaced pages). We applied three kinds of content analyses to classify and interpret content posted in the form: discourse analysis (to analyze the flow of the conversation), narrative analysis (to analyze the structure of and changes to PK), and context analysis (to capture how external resources were used). We then examined our labeled data using visualization and statistical analysis to develop further insights. Our content analytic methods and one visualization are described in the following sub-sections, while our other visualizations and statistical tests are presented along with our findings.

Discourse Analysis

Various discourse analytic coding procedures exist (e.g., Voss, 2005; Wise & Chiu, 2011), but none consider the narrative construction process we were interested in. Thus, we developed our own set of codes, drawing upon Glaser’s (1965) constant comparative technique and following procedures commonly used for developing grounded theory (Strauss & Corbin, 1998).

We began coding using narrative construction and argumentation (as described by Bruner, 1986) as sensitizing concepts. We made no effort to produce mutually exclusive codes, because individual posts might contain multiple kinds of knowledge construction activity. Initially, the fourth author worked through a large number of posts (approximately 300, sampled from across the entire conversation) to develop an initial taxonomy. Subsequently the first, second, and last authors jointly worked through the data starting from the earliest post and working forward in batches of 50 to a 100 posts at time, following Glaser’s (1965) constant comparative technique, until a point of saturation was reached (roughly 500 posts). The team revised the codebook again and then jointly recoded these posts using the final taxonomy. These posts were used in the narrative analysis task, described further below.

To establish intercoder reliability, the second and last authors coded an additional 300 posts sampled randomly from the remainder of dataset, and we used Cohen’s Kappa statistic to assess intercoder reliability. We obtained values of .9 or greater in all but two of the developed categories (see Appendix A for exact values). Kappa values greater than .6 indicate substantial agreement, and values greater than .9 are nearly perfect. The two categories exhibiting relatively low intercoder reliability were “Troll”(.67) and “Cheer” (.59). These categories were challenging to code because they depended heavily on inferences about the poster’s intent.

Our final coding scheme includes 14 codes, and definitions for each of these are provided in Appendix A. We offer additional detail and examples of four of the more specialized codes below.

Narrate

Posters developed stories through a process of iterative retelling and innovation. Some aspects of the story stabilized over time (at least temporarily), and were retold in response to questions, or in the process of considering new evidence or potential modifications. When a poster retold a stable portion of a story, we coded this as narration.

We used Stein and Glenn’s story grammar (Stein & Glenn, 1979) and Chatman’s taxonomy of story content (Chatman, 1980), to help distinguish story content from other information. For example, in the following passage, Undo relates the consequences of the Annunaki’s (the “race” of aliens in the SGT) interference on Earth, setting the stage for a divine intervention:

One of the reasons is that everything was polluted. the earth was polluted, the genome of the animals, polluted. hybrids of all kinds were rampant and apparently some of the combinations were very bad. (Undo)

Stitch

Stitching involved an attempt to attach new evidence to an established story. To qualify as stitching, a poster must establish a link between some evidence and a specific story element. Stitching might result in elaborating a story or rejecting the proposed evidence. For example, in the following, Justyc proposes a connection between Neal Stephenson’s (2000) novel Snow Crash and the Tower of Babel myth:

Interesting thread undo. Thank you i was wondering if you had any ideas or theories about what it was that happened somewhere around 3000 bc that made writing suddenly take a 90 degree turn. up until then, it would have been read from top to bottom but then it was changed from left to right. can this be somehow connected to the nam-shub of enki, who was said to have “changed the speech” (Justyc)

Mutate

A mutation is a suggested or actual change to the underlying story. We require that a mutation modifies overall episodic structure of the story, as defined by Stein and Glenn (1979). Thus, while “stitching” might support or elaborate of some aspect of the story, “mutation” only occurred when the change produced some structural modification or addition to the story. For example,

. . . the real big hint is the same reference as the epic of gilgamesh gate guards, the scorpion men (which doesn’t mean i think they are scorpions, but i do beileve they have tails on them, rather than on their flying contrivances). for anyone who has read the entire thread, remember the image of the vase from abydos? the tall beings have tails. my first thought was that these were the forerunners of the pharaohs, who were partially reptilian, such as nimrod’s entourage, but it’s difficult to say. i keep vacilliating between the idea that the hybridization/modifications made to nimrod were either machine or reptilian (perhaps a mixture of the two, explaining why he claims he’s 2/3rds “god,” et.al, 1 part human and ALSO 1 part machine/1 part reptilian). (Undo)

The implications of this proposed mutation have significant consequences for the overall storyline.

In some cases, mutation occurred offline and could only be identified by comparing a story from one point to the next.

Defense of Pseudoknowledge (DOPK)

DOPK posts defended the epistemic strategies of the poster and were offered as a foundational argument for reasoning about PK. For example,

Some myths are not myths at all. And some myths are purely myth. But when German Higher Criticism exploded onto the scene, all ancient history was labeled myth. Bar none. Every country on the planet, every people, have histories. They are considered lies, fables and myths at the university level, not because they have evidence they were lies, fables and myths, but because they don’t believe in the subject matter. You can’t dissect a text correctly if you have no respect for it. Your view and translation will always be slanted to your own paradigm. (Undo)

Narrative Analysis

To examine how the narrative changed, we applied a schema that was developed to capture cognitive representations of narrative (Stein & Glenn, 1979) to the first 500 posts that were coded during the initial discourse analysis effort. We restrict our examination to this window because conversation dynamics in this window are representative of those in other regions of the conversation, and an extended analysis of subsequent changes to the narrative is beyond the scope of this article.

Stein and Glenn’s (1979) schema represents narratives as a composition of one or more episodes that can be chained together in various ways. Each episode captures the intentional actions of the main character(s) in response to an event and the effects of these actions. Some elements of the schema may be omitted in telling a story, but an adult listener will infer them. In this article, we apply the compressed format for episodes that was introduced in Introne et al. (2017), which consists of four elements: Initiating Event, Internal Response (a character’s internal reaction to an event), Action, and Consequence. We applied these codes to the data as follows:

1. Select the subset of posts containing narration and/or mutation;

2. Extract text related to story content from these posts;

3. Divide text into propositions (following Stein & Glenn, 1979);

4. Rearrange propositions to follow the sequence of events in the underlying story;

5. Apply our story.

The first and third authors worked independently on Steps 2 through 5 and then resolved differences in consultation with the other co-authors. In general, there were few substantive differences in the coding.

Contextual Analysis

To understand how forum participants used external resources (i.e., hyperlinks), we developed and applied a classification of resource types and also coded the discourse contexts (using the discourse analysis codes described above) where each resource was used.

First, we collected all of the URLs that were included in any posts in the forum (omitting reply links). We analyzed links from the entire conversation because we sought a comprehensive summary of how participants used these resources. This produced 2,484 unique links, and from these we extracted the URL of the base domain, yielding 415 unique URLs. We used these to develop a taxonomy of resource types. We started with four initial types, based on our previous analysis: PK (seen in posts assigned the “Other PK” discourse code), conventional history (a commonly observed type of evidence), conventional science (another type of evidence), and news (often used in “Current Event” posts). The first author examined each site at the time it was posted using the Internet Archive, which maintains archived snapshots of many pages on the Internet. New resource types were added as sites were encountered that did not fit neatly into existing categories. Three sites could not be identified, and these were classified as “unknown.”

The first and second authors discussed this taxonomy and worked together to develop the final taxonomy of 15 resource types and criteria used to distinguish each category, captured in Appendix B. Finally, to ensure consistency, the first author reapplied the final taxonomy, discussing any borderline cases with the second author.

To apply the discourse codes to links, the second author coded each of the original 2,484 links in a first pass, and the first and second authors reviewed the codes and discussed any ambiguous cases. Each link was coded separately, so that different links in the same post might have different codes.

Visualizing the Use of Online Media

We developed several visualizations to understand how different resources were used in the context of PK construction. We examined the distributions of the discourse analytic codes and resource type codes individually, and then examined how well the discourse codes and resource type codes correlated. More details on these visualizations are provided in the “Findings” section.

To get a better sense of the overall ecosystem, we constructed a network to examine relationships among different resources. Following Starbird (2017), we were interested in how resources were related by the activity of contributing to the conversation, and so infer a relationship between resources that are included in the same post. We used these relationships to generate a graph from the 415 unique domains identified (see Figure 4 in the “Findings” section). Each domain is a node in graph, and each edge’s weight is scaled to the number of posts where the pair of nodes it connects appeared.

To improve readability of the visualization, only those nodes belonging to the giant component (the largest portion of the graph in which any node is reachable from any other node by traversing links) and that were used more than once are included, restricting the graph to roughly 28% of all nodes. We labeled the 20 most frequently mentioned sites in the graph (each of these appear in 10 or more posts), and Table 1 provides detail for these sites. Finally, we scaled node and font sizes according to the number of times that domain was referenced (henceforth, its weight), and colored each node according to its resource type.

Table 1.

Description of the Top 20 Most Frequently Cited Domains in Figure 4.

Site	Description	Type	Times cited
files.abovetopsecret.com	File hosting for AboveTopSecret	PK	158
thelivingmoon.com	The Pegasus Research Consortium (Zorgon is a member)	PK	126
thestargates.com	Site Undo uses to host her e-books	PK	69
abovetopsecret.com	Alternative topics message board	PK	61
landoflegends.us	Previous incarnation of the Pegasus Research Consortium	PK	49
en.wikipedia.org	An online encyclopedia	Reference	42
news.bbc.co.uk	The BBC	News	22
oi.uchicago.edu	The Oriental Institute Museum at University of Chicago	Historical	19
psd.museum.upenn.edu	The Pennsylvania Sumerian Dictionary	Historical	19
upload.wikimedia.org	Wikimedia’s platform for file storage	Reference	17
fas.org	The Federation of American Scientists	Science	15
artapprentice.net	The Secrets of Sumer	PK	13
livius.org	Articles on ancient history	Historical	11
ancientegyptonline.co.uk	Site devoted to Egyptian history	Historical	11
americanantigravity.com	Site devoted to pseudoscience	PK	11
arxiv.org	Site used to host scientific research articles	Science	11
etcsl.orinst.ox.ac.uk	Text corpus of Sumerian literature	Historical	10
globalsecurity.org	Curates media articles related to military and security matters	News	10
phouka.com	Blog and resource page focusing on a variety of topics	Other media	10
youtube.com	Video aggregator	Other media	10

PK: pseudoknowledge.

Findings

By way of introduction, the SGT is a particular version of an “Ancient Aliens” theory developed by the thread initiator, who goes by the username “Undo.” At the time the forum begins in March 2006, Undo had written several freely available e-books on the subject and was working on a third. She initiates the thread to share the SGT and proceeds to present the theory for other forum members.

Undo begins with two basic assumptions: first, that mainstream historians have mistakenly classified religious writings as mythical (in particular, the Book of Genesis), while in fact they were historical accounts of an alien visitation; and second, that the timeline of ancient history established by mainstream historians is incorrect. She proceeds to identify parallels between religious texts and other ancient myths to produce a theory about events concerning the interaction between humans and the alien visitors (the Annunaki, whom she contends were the fallen angels described in Genesis). A central component of her story is that the Annunaki first arrived on Earth via one of several “stargates,” which are interplanetary portals that still exist on Earth. Undo believes that the stargates’ locations are kept secret by various world governments, but her theory focuses on the historical evidence of the stargates’ existence. The timeline of her story incorporates and provides alternate explanations for the Genesis Flood and fall of the Tower of Babel, which are both stories in Abrahamic religions.

Undo is well informed about the SGT and source materials she draws upon, which include the Epic of Gilgamesh, the Bible, other ancient alien theories, and other historical writings. She is responsive to and patient with other forum visitors, who respond to her theory with both enthusiasm and disdain. In the following, we use our discourse analysis to illustrate the flow and tone of the conversation, and our narrative analysis illustrates how the forum participants collectively constructed PK throughout the conversation.

The Evolution of PK

We organize our analysis of the evolution of the SGT around three phases of activity that occurred in during the first 500 posts in the forum: the “birth” of the story, a response to an attack, and an innovation inspired by a forum visitor. We refer to two visualizations based on our discourse and narrative analyses: Figure 1 visualizes the discourse codes, and Figure 2 visualizes a portion of the story constructed during this time.

Figure 1.

Conversation dynamics across the first 500 posts in the forum. Each vertical column is a single post, and marks indicate the codes assigned to that post. The y-axis is split into those discourse codes that are mostly oriented toward argumentation (top) and those that focus on the narration (bottom). Note that the scale of the x-axis shifts after the hiatus (there was no forum activity in this period), because posting prior to the hiatus was much more rapid. Different phases of activity, referenced from the article text and in some cases corresponding to the story in Figure 2, are indicated across the top of the visualization.

Figure 2.

Visualization of the story constructed during the conversation; dashed boxes were not explicit in the forum and were inferred by the authors. Episodes beneath the “mutations” line are subsequent changes to the story following its initial presentations. All mutations followed the hiatus shown in Figure 1.

The Birth of the Story

Undo begins her story with a burst of posts peppered with evidence and narrative (see the “Narrate” and “Evidence” axes leading up to March 7 on in Figure 1). These posts introduce the story, covering Episodes 1, 2, and 3 in Figure 2. Other posters respond in a variety of ways; many are supportive, but others attack Undo and her story (see the “Attack” axis in Figure 1). Undo defends the story against these (see the “Support” axis in Figure 1), occasionally deploying a DOPK argument. Some posters propose mutations or attempt to stitch evidence to the story, but Undo never accepts these additions. Undo is the sole architect of the story during these first several weeks.

Response to an Attack

Following this initial presentation, Undo adds a sub-episode (Episode 2.1), explaining that the Nephilim (alien–human hybrids) maintain the Annunaki bloodline through the practice of droit du seigneur (the right of the lord to claim the first night of a newly wed woman). As evidence, Undo cites a portion of the Epic of Gilgamesh and provides pictures of elongated skulls from various cultures as evidence of alien–human hybridization. Several posters respond negatively to these assertions.

One poster, named Indellkoffer, is familiar with the ancient texts Undo references and indicates that she ² has degrees in ancient history and literature. She attacks both the “elongated head” evidence and the practice of droit du seigneur. These attacks are persistent, verbose (approximately 5,600 words over the course of 11 posts), and appeared to be grounded in conventional historical theory.

Several posters offer supportive comments to Undo but do not specifically defend the SGT from Indellkoffer’s attack. However, one poster directly addresses the argument and offers advice:

Indellkoffer has given some very good skepticism in this, and it would seem that the theory that the Annunaki (either directly, or through the greatly influenced Osiris/et al guy) impregnated the wives-to-be before the husbands could get there, is going out of our view of plausibility. It doesn’t demerit your general theory at all, it just crosses a few things out. It could come back, given some new evidence—but don’t expect it to. Clinging to it and holding to something that was only tenuously supposed in the beginning is a good mixture for making yourself look less knowledgeable than you are. You need to adapt to this situation, take it in stride, stop focusing on the children/firstborns, and spend more time with the folks in this thread and the elongated heads issue. If that begins to cave, move back to the general stuff. (Viendin, emphasis added)

Viendin’s advice for handling Indellkoffer, and especially the emphasized portion, could be interpreted as a general strategy for PK maintenance in an online forum, and we return to this point in our discussion. Following the interaction with Indellkoffer, there is a 3-month hiatus during which no posts are made. When Undo does return to the conversation, she summarizes her story in a single post and replaces Episode 2.1 (Figure 2) with a new explanation of how the Annunaki maintained their bloodline:

Yet, according to the sumerian texts, the Igigi were the Greater gods and didn’t come to the earth, but remained up in the sky, circumnavigating the globe. So how is it they “Deflowered” anyone while being uninvolved in the activities of the humans on the planet? Abduction, my friends, abduction. And they are still at it today. (Undo)

This example illustrates not only how debunking can overcome components of PK but also that a devoted maintainer can salvage a story by changing it in ways that preserve its overall structure. Below, we examine how the story changes when it is not under threat.

Innovation

As the conversation progressed, forum visitors proposed numerous modifications to the story, but Undo rarely accepted them. One exception to this occurred following the hiatus between March and July. At the beginning of July, a new poster named Zorgon began posting, responding to earlier posts and offering support for Undo’s story (see Figure 1). Zorgon is active on another site (which he links to frequently), which maintains PK about a variety of topics (e.g., cryptozoology, zero-point energy, aliens, and paranormal phenomena). One of the topics Zorgon maintains on that site is about Martian civilizations.

Undo returns to the forum to reply to Zorgon and resumes narrating the story. The forum becomes more hospitable to Undo during this period; there are fewer attacks, and other posters defend her story. Activity focuses on stitching evidence to the story and suggesting mutations. In most cases, posters frame their offerings as questions for Undo. Undo is highly responsive and engages other posters in considering these contributions but only rarely accepts them as part of the story or its accumulated body of supporting evidence.

The only mutation she allows in the first 500 posts is a suggestion by Zorgon that a stargate was used to drain water from Mars to flood the Earth. The contribution unfolds over a series of posts, culminating in the following exchange (noted in Figure 1; the episode is captured in Figure 2):

. . . Now you remember our talk about the water? Well there was TONS of water on Mars . . . NASA has proof and so do I Haven’t found the stargate up there yet but thats gonna be tough! . . . So if there was water and its now gone . . . [Nasa’s current puzzle: where did it go] and if you take that stargate and open one end on Earth . . . the other at the bottom of the Martian Sea . . . Voila instant drain . . . wipe out Mars by sucking up the water . . . flood the Earth with all that extra water . . . short circuit the gate and Kaboom! Two worlds wrecked in a short time . . . (Zorgon)

. . . It wouldn’t surprise me at all about the Mars flood idea, because, from what I can tell, Enki is accused of creating the flood in at least one instance. It fits with my theory that Ra was modelled after Enki and his E.ABZU and that the appearance of Ra in the Legend of Ra and Hathor is in fact, a memory and metaphorical reference (however obscure) to Enki’s E.ABZU . . . Also in the Legend of Ra and Hathor, Ra’s “Eye” as a weapon, creates a flood of “red blood,” which fits with the idea that Mars might be involved somehow (something I was avoiding without sufficient textual evidence, but which fits the pieces of scattered information if you connect the dots). (Undo)

Unlike others, the Mars episode is never tightly integrated into the story because its initiating event is never specified and could not be inferred, and we searched for other mentions of episode to see whether an event was ever developed. Querying our database for the words “Mars” and “flood” yielded 25 additional posts, with the most recent occurring in June 2014 (nearly 8 years after its initial presentation). We could not identify a clear initiating event but observed that Undo almost always credited Zorgon with the episode:

. . . My theory (which Zorgon, btw, originally suggested as regards Mars and a gate), was that the big gate in the persian gulf (and potentially others in big bodies of water on earth) was siphoning water from another body of water, off planet, in particular Mars. because in the legend of the destruction of mankind, the water is said to be “red,” which could’ve been a reference to water from mars . . . (Undo)

Because it is poorly integrated in the story, and because Undo is careful about crediting Zorgon for the episode, we hypothesize that Undo’s inclusion of the Mars episode is a way of thanking Zorgon for his support and providing him with a sense of ownership and a sanctioned role in the story’s maintenance. The episode also generates several discussions about alien civilizations, and new posters come to the forum and participate in these. We reflect further on these observations below.

Visualizing the Media Ecosystem

The forum conversation about the SGT draws upon a broader online context. To understand how external resources were used, we analyzed these sites as described in the “Methods” section and then developed several visualizations to explore the data. Figure 3a presents the distribution of unique sites across the different resource categories; color indicates the average number of times each site was linked to in each category. Roughly 25% of the 415 domains referenced were devoted to PK, and these were referenced far more frequently (see the legend in the figure) than other sites. About 40% of the domains referenced were focused on more conventional forms of knowledge (reference, science, theological, academic, historical, and news categories). The remaining sites were a mix of blogs, personal pages, pictures, and some types of social media.

Figure 3.

Distribution of the codes: (a) distribution of resource types over 415 unique domains, (b) distribution of discourse codes across 2,484 links, and (c) correlations between resource types and discourse codes, obtained from residuals of a chi-square analysis. Raw counts in each cell are provided in Appendix C.

Figure 3b illustrates the distribution of discourse context codes. Colors indicate how codes were distributed over sites using a metric called skew (Introne & Goggins, 2015). Skew is the normalized complement of entropy. A skew of zero indicates that all of the sites in a category were used the same number of times, whereas a skew approaching one indicates that a small number of sites were used far more than the others. Higher skews indicate the presence of a long-tail distribution. Thus, Figure 3b indicates that the majority of links (roughly 70%) were used as some form of evidence, but that a few domains were used far more frequently than the others. The fact that skew tends to increase as the number of links in a category increases suggests a “rich get richer” pattern in the use of sites by category.

Figure 3c illustrates how different resource types are used across discourse contexts. To generate the figure, we first log-scaled the data (because of the uneven distributions reported above) and then plotted the standardized residuals from a chi-square test comparing the coding schemes. The residuals reflect the correlation between individual codes (Agresti, 1996) but do not capture raw counts—for instance, tourism and PK both co-occur with narrative the same number of times, but because the relative frequency of PK is much greater across all categories, it is relatively less well correlated with narrative than is tourism. For reference, the underlying counts for all cells are provided in Appendix C.

The preceding analysis reveals several trends; in the following, we italicize specific resource types for clarity. Information from historical sites, and to a lesser degree theological and tourism web pages (which often provide images of archaeological sites), is commonly used when narrative is presented. PK is rarely used in narrative contexts or as evidence but is commonly used in conversation that is explicitly about other PK. For example, Undo often linked to other “ancient aliens” theories when explaining the differences between those and the SGT. Other posters linked to PK when using the SGT to build support other PK theories (e.g., a rash of alien abductions). Links to PK also occurred in conversations seeking to explain current events (the “Explanation” code), such as the 2007 troop surge during the war in Iraq and the 2010-2011 international response to Somali pirates Gulf of Aden. In both of these cases, posters hypothesized that stargates were the real reason for military action.

Scientific sites played an important role as evidence, but forum participants mostly used pictures when attempting to adapt the narrative to new evidence (via “stitching”). Finally, news and other media (see Appendix B for definitions of these categories) are often used when discussing current events, but other media also plays a role in discussions about other PK.

Figure 4 illustrates relationships between resources referenced from the forum, and Table 1 describes of the top 20 most heavily referenced domains. Nodes in Figure 4 are organized using a force-directed layout algorithm, whereby nodes repel one another unless they are constrained by links, which act like springs to hold nodes together. Thus, heavily interconnected nodes tend to cluster together. Figure 4 reveals the existence of a single, central component, consisting primarily of historical sites and other conventional (non-PK) sources. One PK-oriented site called The Stargates (thestargates.com), where Undo hosts her e-books, is present in this cluster, but others are peripheral to the central cluster.

Figure 4.

Network of relationships among domains that co-occur in posts. Only domains appearing more than once and are part of the giant component are shown (28% of 415 total domains). Node and font size are scaled to the number of times a domain appeared, and color indicates the resource type. The top 20 most frequently cited cites are labeled and described in Table 1.

This kind of organization is called a core–periphery structure, wherein nodes that are peripheral to a central cluster, called the core, are connected to nodes in the core but are not well connected to one another (Borgatti & Everett, 2000). This pattern appears because each PK site collects alternative (pseudohistorical or pseudoscientific) versions of more conventional knowledge across a range of topics. Posters select links from PK sources and present them along with conventional sources. For example, in one post, Undo summarizes the scientific support for stargates, posting eight links to conventional sources, including one link to peer-reviewed research on quantum teleportation (Yeo, Liu, Lu, & Yang, 2004). Precisely how such resources support the existence of stargates is not explained; for example, Undo does not explain what quantum teleportation has to do with the transfer of matter across interstellar distances. Along with these sources, but not differentiated from them, Undo provides two links to sites hosting a variety of PK: one is to an interview with a physicist who promotes fringe science and the other to a page of links purported to support the existence of stargates.

This type of juxtaposition is common in the forum, explaining the visual organization of graph in Figure 4 and illustrating how forum members typically build support for the theory. Supporters of the SGT subsequently refer to this accumulated mass of resources as supporting research, without ever explaining exactly how it supports the theory. For example, consider Undo’s reaction to one attacker:

. . . Just because it’s in print on the web doesn’t mean it’s real, but when the research is presented and the facts are presented with the results of the test, and the procedure, we call that PROOF. Where’s yours? (Shugo)

It’s so completely frustrating. You have no idea. There’s a ton of really good research and reference material in this thread, including data from government research labs, air force labs, etc, in .pdf files and news articles, for your perusal and you didn’t read a one of them didya? (Undo)

In summary, we find that support for the SGT is primarily drawn from conventional sources across the web that are reinterpreted and blended with PK. Together, this mass of resources is used to defend the theory against attacks.

Discussion

Unlike a single piece of fake news that is passed from person to person across a social network, online PK is dynamic and drawn from multiple sources. We find that participants engaged in a conversation about PK debate, enrich, and adapt an underlying story. They support this narrative using information drawn from many conventional sources across the web, occasionally enhanced by other sources of PK. In the following, we re-examine our initial research questions in light of these findings.

Like other narratives, the SGT follows an episodic organization. In the initial phases of SGT’s development, the narrative evolves at episodic boundaries—an attack results in a replaced episode, and a contributor’s interests lead to the addition of a peripheral sub-episode. This growth pattern leads us to wonder if the changes play a functional role in an evolutionary sense. For example, by replacing Episode 2.1 following Indellkoffer’s attack, Undo inoculates the evolving narrative against similar attacks in the future. The case of the Mars episode is more nuanced. By incorporating this episode, Undo may increase Zorgon’s commitment to the story. At the same time, the episode opens the door to contributions from other Mars enthusiasts, increasing opportunities for growth. Future work might consider this evolutionary perspective more rigorously.

Viendin’s advice to Undo following Indellkoffer’s attack also seems to suggest that there may be a hierarchy of importance within the episodic structure of the narrative—might some aspects of the narrative be sacrificed to preserve others that are more critical? Understanding whether (and why) some episodes are more important than others might inform new approaches for debunking more dangerous false narratives, such as those advocating jihadist extremism (Braddock & Horgan, 2016).

In analyzing contributors’ use of external resources, we found that the SGT is undergirded by many sources of conventional knowledge, although the relevance of this material is often dubious. Over time, these sources are referred to en masse as “research” that supports the theory. Skeptics who demand concrete evidence are directed to this research and scolded for not reading it before posting. In most cases, this strategy deflects all but those few energetic detractors (such as Indellkoffer) who have the time and expertise required to sift through and argue about this material’s validity or relevance.

The SGT develops on reinterpreted information from conventional sources and is henceforth used as a lens on the world. The narrative becomes the basis of PK-oriented explanations of current events and is also used to reflect on other bodies of PK. These latter activities are a widespread and important mode of participation. Forum visitors do suggest new evidence (often in the form of pictures taken from other sites) and possible mutations, but Undo is very selective about what she allows. Yet participants are free to apply the story to interpret events in the world, and this happens frequently. Extrapolating from this result, the ability of a story to explain current events may play an important role in generating engagement among visitors.

Our analysis has not focused on how or whether the SGT travels across sites, but we have not encountered any significant evidence that this takes place. Here, we find that the Internet helps to support the growth of the SGT in other ways. It provides a rich trove of online resources that can be woven together to create an imposing thicket of evidence. This evidence may be misused or reinterpreted, but the fact that it is drawn from resources that appear to be credible may be enough. Discovering an underlying misinterpretation or misuse of evidence requires careful analysis, and it is rare for forum visitors to engage in this manner. For the instance of PK we have focused on, a critical enabling property of the web is not hyperconnectedness but the existence of hyperlinks. These may be amassed, providing readers with some assurance that the theory is grounded in credible information, perhaps freeing them to evaluate the story on the basis of its verisimilitude.

Finally, by virtue of being publicly accessible, roughly 1,000 people over the course of a decade are able to participate in the creation and maintenance of the SGT. They bring diverse resources drawn from across the web to build support for the theory, either by providing direct evidence for the narrative or by demonstrating its explanatory power. In essence, the Internet makes it easy to crowdsource the creation and maintenance of PK.

Limitations

Our study is an initial case study of a single forum and may not be representative of other venues of collective PK construction. One factor that may differentiate our analysis site from others is Undo’s position as a self-published author with a vested interest expanding her readership. She brings a story that was already well developed to the forum, and her role as an author influences the social dynamics that arise. More generally, the motivations of contributors are certain to play a role in the dynamics of PK construction, and it is unclear whether the social roles adopted here are similar to the social roles that emerge in other forums. Consideration for the mix of motivations among participants engaged in PK construction is a critical factor to consider in future work.

Another factor that influences the development of the SGT is the existence of many other similar theories beyond the forum. Many of the posters are aware of these theories, and Undo is careful to differentiate the SGT from them. Future work should consider how any given PK-based narrative relates to other narratives that it may compete with or draw upon.

Finally, in choosing to explore the use of context, we have neglected a more extensive analysis of narrative dynamics beyond the first 500 posts, and it is quite likely we will encounter novel patterns in the remaining data. It is therefore important not to generalize our results prematurely. Nonetheless, this study has led a number of potentially fruitful routes for future inquiry.

Conclusion

Like fake news and misinformation, online PK thrives on the Internet. Yet, unlike such digital contagions, PK is a dynamic and evolving entity that may come to inhabit a virtual place for a time, drawing support from the rich and diverse pools of online information that surround it. PK changes in response to the dynamics of the sociotechnical system it inhabits and may be used by its supporters to make sense of the world. Over time, individuals who flow in and out of an online conversation about PK bring resources that may enrich it in a manner that makes it more credible, more resilient, and more engaging.

We believe this brings an important perspective to the emerging conversation about fake news and false narratives. Thus far, researchers have focused on how misinformation spreads, how to detect it, and how to reduce its credibility. Underlying this work is the implicit assumption, surfaced in the recent Pew Survey cited above (Anderson & Rainie, 2017), that misinformation and fake news carry or reinforce false narratives. This may be the case, but our findings demonstrate that fake news is certainly not a requirement for false narratives. Rather, the Internet allows the architects of false narratives to manufacture credibility by drawing information from many credible sources. The role misinformation and fake news might play in this process is not well understood, and is an important research area for the future.