Sage Journals: Discover world-class research

Abstract

Search engines, such as Google or Yandex, shape social reality by informing their users about current and historical phenomena. However, there is little research on how search engines deal with contested memories, which are subjected to ontological conflicts known as memory wars. In this article, we investigate how search engines circulate information about memory wars related to the Holodomor, a mass famine caused by Soviet repressive politics in Ukraine in 1932–1933. For this aim, we conduct an agent-based audit of four search engines—Bing, DuckDuckGo, Google, and Yandex—and examine how their top search results represent the Holodomor and related memory wars. Our findings demonstrate that search engines prioritize interpretations of the Holodomor aligning with specific sides in the memory wars, thus becoming memory warriors themselves.

Keywords

algorithms genocide Holodomor memory wars search engines Ukraine

Introduction

Memory wars are ontological conflicts that deal with the interpretation of the past in the public sphere and often involve institutional actors (e.g. states; Mälksoo, 2015). Such conflicts are particularly intense in the case of ‘fractured’¹ (Bernhard and Kubik, 2016) memory regimes, which rely on exclusive historical narratives to legitimize their status and mobilize supporters. To sustain these exclusive narratives, regimes engage in memory wars to subjugate alternative interpretations of the past through legal prohibition of specific narratives (Koposov, 2017) or the subversion of grassroot memory practices (Fedor, 2017). In some cases, memory wars can justify the use of extraordinary measures (e.g. lethal force; Gaufman, 2015) by presenting the alleged distortions of the past as an existential identitarian threat. One example of such a case is Russia’s war against Ukraine, where the revision of the Soviet narrative of the Great Patriotic War in Ukraine has been utilized by the Kremlin to emphasize the existential threat for Russophone Ukrainians and justify the use of violence to counter it (Gaufman, 2015; Makhortykh, 2018).

Similar to other types of ontological conflicts, memory wars are affected by the rise of digital technologies. The unprecedented connectivity between producers and consumers of memories enabled by online platforms expands the range of actors involved in these memories’ contestation. Together with anonymity and possibilities for low-cost engagement with the past (e.g. linking or sharing the piece of content), individuals have more possibilities to partake in memory wars. Besides their less top-down nature, these online memory wars—or ‘web wars’ (Rutten et al., 2013)—enable new forms of contestation varying from hashtag protest campaigns (Bosch, 2017) to cyberattacks against heritage institutions (Shanapinda, 2018) to animations challenging the hegemonic nature of memory-related festivities (Makhortykh and Sydorova, 2022).

However, these digital transformations go beyond new possibilities to contest the past offered to human actors. They also enable the rise of non-human actors who are involved in memory-making and, potentially, memory contestation (Makhortykh, 2021). These actors are algorithmic systems used by platforms (e.g. YouTube) to curate an unprecedented volume of online content. While sometimes assumed to be more objective than humans (see Pethig and Kroenung, 2022), algorithms are shaped by the values of their designers and the data they are applied to. This dependency makes algorithmic systems subject not only to occasional errors, but also to systematic bias resulting in distorted treatment of certain subjects. The potential for such malperformance is particularly high for ontologically contested subjects (e.g. race; see Noble, 2018), and it means that algorithmic systems might be particularly prone to errors and bias in the case of memory wars.

In this article, we examine how one particular class of algorithmic systems, namely search engines, deals with information about the Holodomor, a human-made famine in the 1930s that took a particularly heavy toll on the Ukrainian population of the Soviet Union. Our choice is attributed to the intense contestation of Holodomor-related memories between Ukrainian, Russian, and Western historiographies, which makes the selection of prioritized sources a challenging task for search engines. The situation is further complicated by the tendency of some search engines (e.g. Yandex, a Russian search engine) to amplify pro-Kremlin narratives (Kravets and Toepfl, 2021), which can have implications for their curation of memory-related information. To investigate the performance of search engines under these circumstances, we conducted a comparative audit of search outputs for the query ‘Holodomor’ in Latin and Cyrillic script for four major search engines: Bing, DuckDuckGo, Google, and Yandex.

The rest of the article is organized as follows: First, we introduce our case study, the Holodomor, and discuss memory wars related to it. Then, we discuss the relationship between search engines and collective memory and introduce the methodology of our agent-based audit. Then, we present our findings about the types of sources prioritized by different search engines and their treatment of ontologically contested aspects of the Holodomor. Finally, we discuss the implications of our findings for understanding the role of algorithmic systems in the context of memory wars, and the limitations of the study.

The Holodomor in the context of memory wars

The Holodomor took place in 1932–1933 and was caused by Stalin’s politics of collectivization, which resulted in a mass famine in Soviet Ukraine and the Kuban region inhabited by Ukrainians, as well as several other parts of the Soviet Union (e.g. Kazakhstan and Volga region). Among the affected regions, Ukraine suffered the heaviest toll of the famine, resulting in millions of deaths² attributed not only to brutal grain requisitions, but also to the internal blockades that intentionally prevented Ukrainians from leaving their villages to search for food (Dyck, 2021). The purpose of this forced starvation is presumed to have been to suppress resistance to collectivization in Ukraine and to prevent further nationalist uprisings, which had occurred frequently in the 1920s (Applebaum, 2017).

Until Perestroika in the late 1980s, the subject of the Holodomor remained a taboo topic in the Soviet Union. While there was a growing recognition of the artificial nature of the famine in the West from 1950s to 1960s (mainly through the efforts of Ukrainian émigré groups; Applebaum, 2017), in Soviet Ukraine, the public debate was only initiated by the local intelligentsia in the 1980s. However, the Holodomor only started to be recognized as an all-Ukrainian tragedy on an official level after the regaining of Ukraine’s independence in 1991. The 60th anniversary of the Holodomor was marked by Ukrainian officials, including then President Kravchuk, and became an important milestone in Holodomor remembrance, followed by the establishment of the Day of Remembrance of Famine Victims in 1998 (Kasianov, 2008).

The period of Yushchenko presidency (2005–2010) marked another important stage in institutionalizing Holodomor remembrance, both in Ukraine and worldwide. Not only was the law recognizing the Holodomor as a genocide adopted in 2006 (Klymenko, 2016), but new public commemoration practices were also established and popularized around this time (e.g. the minute of silence and the practice of lighting candles in memory of the victims). Altogether, it resulted in the growing recognition of the Holodomor as a genocide in Ukrainian society.

The recognition of the Holodomor as an important element of collective remembrance in Ukraine and the establishment of new memory practices on the local and international level have also made Holodomor memory the subject of ontological conflicts. Some of these memory wars involve Ukrainian scholars and developed on the local level, whereas others relate to foreign politics and affect the relationship between Ukraine and other countries (mainly Russia). While it would be hard to establish an exhaustive list of ontological conflicts related to the Holodomor, we summarize some of the ones that we find of particular relevance for the current study below.

One of the key mnemonic conflicts relates to the interpretation of the Holodomor. While the Holodomor is recognized as the genocide in Ukraine and a number of Western countries, this claim was earlier contested within Ukraine and remains contested by a number of other countries, particularly Russia (Klymenko, 2016). The main argument here relates to the claim that not only Ukraine, but also other parts of the Soviet Union, suffered from the famine. Hence, the Holodomor was not necessarily a genocide against Ukrainian people but should be treated as a crime of the Soviet leadership. Thus, arguments about the Holodomor being a genocide are treated as a ‘common thesis of the Ukrainian nationalistic propaganda’ that undermines the communal suffering of the Soviet people (Nevedov, 2018: 17).

Besides the contestation of the status of the Holodomor as a genocide, which at times can also be attributed to the intent of avoiding parallels between it and other genocides (e.g. the Holocaust; Dyck, 2021), there are also claims denying the criminal nature of the Holodomor. Such denialist claims are often made by the Russian journalists-cum-conspiracy theorists who argue that the Holodomor was due either to natural causes or the efforts of enemies of the people (and, thus, cannot be blamed on the Soviet leadership; Prudnikova and Chigirin, 2013), or that it is another lie of the corrupt West aiming to pit Ukrainians against their Slavic brothers (Rambler, 2017).

Another subject of memory wars concerns the number of Holodomor victims. Because of the fragmentary data, existing estimates are usually based on statistical techniques that model the number of fatalities and vary rather broadly. In Ukrainian scholarship, the estimates range from 3.9 to 4.5 million victims (Rudnytskyi et al., 2015) to 7 (Marochko, 2017) or even 10.5 million (Petryshyn et al., 2021). Russian historiography tends to align with more conservative estimates, such as 2.9–3.5 million (Marchukov, 2008), or even lower numbers (e.g. 1–2 million; Shubin, 2008).

The already challenging task of estimating the human losses is complicated by the politicization of the number of Holodomor victims. In the case of Ukraine, it is reflected in the tendency of some institutions to propagate claims about the higher number of victims, such as in the case of President Yushenko’s claims of 10 million victims or the recent statements of the Ukrainian Holodomor Museum about the even higher number (Gadzins’ka, 2021). The lower numbers based on demographic estimates (Rudnytskyi et al., 2015) are labeled by some Ukrainian public figures as Russian propaganda aiming to downplay the importance of the Holodomor (see Gadzins’ka, 2021).

One more subject of mnemonic contestation is related to the comparability of the Holodomor with other atrocities. While such comparisons can facilitate understanding of the Holodomor, they also raise ethical concerns (e.g. by challenging the notion of each genocide being unique; Jonassohn, 1998). The Holodomor is often compared with other genocides, in particular, the Holocaust, to emphasize the genocidal nature of the former or stress its significance (e.g. by claiming that the number of victims is higher for the Holodomor). Such ‘competitions of victims’ (Jilge, 2006) are commonly criticized by scholars for their potential to undermine the significance of the atrocities that are compared to each other (Dyck, 2021) or even defend past crimes (e.g. by claiming that the Holocaust was justified because of the Jewish involvement in the Holodomor; Makhortykh, 2019).

In the case of Russia, the main emphasis is laid on comparisons between the Holodomor and other Soviet famines that occurred in 1932–1933, for instance, in Kazakhstan and the Volga region. Unlike the comparisons between the Holodomor and the Holocaust, the main purpose of the comparison in this case is to downplay the genocidal nature of the Holodomor by arguing that similar events occurred in other parts of the Union (Gudz’, 2019).

The algorithmic turn in memory wars

Technological infrastructures are increasingly recognized as an important factor in shaping collective memories. A number of studies (e.g. Mayer-Schönberger, 2011; Van House and Churchill, 2008) discuss the unprecedented possibilities for capturing and storing information about the past granted by digital infrastructures, together with their long-term effects on the complex dynamics of remembrance and forgetting. A related phenomenon is the disappearance of the clear distinction between private and public memories attributed by the rise of online platforms and increased interconnectedness between producers and consumers of memory (Hoskins, 2009; Van Dijck, 2011), which is amplified by new possibilities for engaging with mnemonic narratives through platform affordances (e.g. liking or sharing; Hood and Reid, 2018).

The majority of studies look at memory infrastructures from the point of view of human actors who employ those infrastructures to engage with the past. However, we argue that some infrastructures become memory actors themselves, considering their high degree of autonomy (e.g. in the form of the ability for automated decision-making). An example of such infrastructures are algorithmic information retrieval systems, which have substantial implications for the ways individuals and societies interact with the past (Esposito, 2017). By filtering and ranking online content, these systems become key information intermediaries (Wallace, 2018) that shape how their users perceive social reality (Noble, 2018). Search engines, in particular, play an important role in this process by prioritizing specific information sources and interpretations which define ontologies of present as well as historical phenomena (see, for instance, Zavadski and Toepfl, 2019).

Despite the growing importance of search engines as actors shaping the perception of the past (Makhortykh et al., 2021b), their impact on collective memories currently remains understudied. The few existing empirical studies usually focus on how search engines select sources when dealing with history-related queries. For instance, in the case of memories about Shanghai’s Jewish community, Jakubowicz (2009) found that search engines prioritize authoritative sources (e.g. governmental or educational websites). Similarly, Reading (2011) observed that search results prioritized Wikipedia, Facebook, and news sources while limiting the visibility of other mnemonic sources in relation to the protests in Iran.

Besides the tendency to prioritize specific groups of information sources, the way search engines circulate information about the past is subject to the lack of transparency (Paßmann and Boersma, 2017) and can inherit the distorted representations of social reality (Noble, 2018). These issues are highlighted by several studies discussing different forms of history-related search malperformance, ranging from the disproportionate visibility of the certain categories of mass atrocity victims (Makhortykh et al., 2021b) to the prioritization of interpretations supporting fractured memory regimes (Zavadski and Toepfl, 2019).

So far, however, there are no studies looking in detail at how search engines deal with ontologically contested memories. Research dealing with other contested subjects, such as the representation of race in the context of technological innovation (Makhortykh et al., 2021a) or of gender in the context of professional vocation (Kay et al., 2015) or face-ism (Ulloa et al., 2022a), suggests that the possibility of distorted treatment (e.g. by reinforcing stereotypes) of such subjects by search engines is high. In the case of memory wars, such treatment can result in the reiteration of existing misconceptions (e.g. denialist claims) and subjugation of alternative interpretations that can turn search engines into memory warriors actively taking sides in ontological confrontations.

Methodology

Data collection

To examine the role of search algorithms in the context of Holodomor-related memory wars, we conducted an agent-based audit of four search engines: Google, Yandex, Bing, and DuckDuckGo. This auditing approach uses virtual agents (i.e. software simulating user behavior, such as scrolling webpages) to generate system inputs and then record the resulting outputs (Ulloa et al., 2022b). In the context of search engine research, it allows controlling for personalization (Hannak et al., 2013) and randomization (Makhortykh et al., 2020) factors. In our study, we controlled for these factors by synchronizing agent activity to isolate the effect of time at which the inputs were generated, deploying agents in a controlled environment (i.e. virtual machines with the same operating systems made from scratch) to minimize personalization, and making the large number of agents simultaneously enter the same query to account for randomization-caused variation in output.

To implement the study, we built a network of CentOS virtual machines in the Frankfurt region of the Amazon Elastic Compute Cloud. Practically, it meant that the agents had German IP addresses similar to human agents from this part of Germany. On each machine, we deployed two agents: one in the Chrome browser and one in the Firefox one. Each agent consisted of two browser extensions: a bot and a tracker. The bot emulated human behavior by (1) visiting a search engine page, (2) entering the query, (3) scrolling down to capture top 10 outputs, (4) removing data accessible by the browser and the engine’s JavaScript to prevent personalization (e.g. earlier searches affecting the subsequent ones). The tracker collected the HTML of all pages visited by the bot and sent it to a storage server.

We conducted two rounds of data collection: on 27 February 2020 and 8 May 2021. We used 64 and 20 agents for the respective rounds (16 and 5 agents per each engine). The ‘.com’ version of each engine was used to make outputs more comparable (i.e. to avoid differences caused by different engine versions). While some engines, such as Google or Yandex, have regional versions (e.g. google.ru), others like DuckDuckGo do not or they redirect to the default .com version (as in the case of Bing).

Each agent searched first for the term ‘holodomor’ and then ‘голодомор’ (‘holodomor’ in Cyrillic script; the same spelling in Ukrainian and Russian). After collecting data, we extracted the top 10 organic search results (i.e. the ones not related to ads or interface panels) and aggregated them across all agents per engine. The resulting data set consisted of 208 search results for both search queries: the exact number of unique results per engine is shown in Table 1.

Table 1.

Number of unique links in the top 10 results aggregated across agents per search engine by the script of the query.

	Query script	Bing	DuckDuckGo	Google	Yandex
February 2020	Latin	17	11	10	12
February 2020	Cyrillic	16	15	12	15
May 2021	Latin	14	12	12	11
May 2021	Cyrillic	16	11	11	13

Data analysis

After extracting all unique links in the top 10 results across all agents per engine,³ we used qualitative content analysis to examine how they circulate information about the Holodomor. Specifically, we focused on the four features of the linked content: (1) the type of the source, (2) the interpretation of the Holodomor, (3) the number of victims and (4) the comparisons with other atrocities. With the exception of the first feature, the selection of features was informed by the existing research on the Holodomor-related memory wars. The specific options for the four features are listed below; all of them have emerged through the data analysis as part of the inductive coding.

The type of the source defines what web resource the linked content is coming from: (1) activist: websites associated with activist and non-governmental organizations; (2) alternative: non-mainstream and niche political websites (e.g. anonymous blogs); (3) commerce: business-related websites (e.g. online shops); (4) educational: websites of educational institutions (e.g. museums and university departments); (5) entertainment: websites dealing with popular culture and entertainment (e.g. IMBD); (6) journalistic: websites of journalistic media organizations (e.g. CNN); (7) non-available: websites that are no longer available and were not retrievable via Internet Archive; (8) political: websites of political parties or governments; (9) reference: online encyclopedias or dictionaries (e.g. Wikipedia); (10) social media: social media platforms (e.g. YouTube).

The interpretation of the Holomodor determines how the linked content interprets the event: (1) crime: the criminal nature of the Holodomor is acknowledged but the event is not treated as a genocide; (2) denial: the criminal nature of the Holodomor is denied (e.g. by attributing it to natural causes or claiming that it did not happen); (3) genocide: the Holodomor is recognized as a genocide; (4) mixed: several interpretations of the Holodomor are noted without giving a clear priority to one of them; (5) none: no clear interpretation of the Holodomor is given (e.g. the event is mentioned, but is not explained).

The number of victims indicates the upper limit of the number of victims who perished during the Holodomor. Our decision to use the upper limit is attributed to the linked content often reporting a range of numbers (e.g. 3.5–7 millions) or multiple estimates. Hence, to standardize the coding, we relied on the highest number of victims (in millions) reported as a valid estimate (i.e. not used exclusively for the debunking purposes): (1) unspecified: the exact number of victims is not given (e.g. the result refers to ‘millions of victims’); (2) below 1: the reported number of victims is below 1 million; (3) 1–5: the reported number of victims is between 1 and 5 million; (4) 5–10: the reported number of victims is between 5 and 10 million; (5) above 10: the reported number of victims is above 10 million.

Finally, the comparison with other atrocities indicates whether the linked content draws parallels between the Holodomor and other atrocities (e.g. to facilitate its interpretation) and which ones: (1) Armenian genocide: the Holodomor is compared with the genocide of Armenians in 1915–1917; (2) Holocaust: the Holodomor is compared with the Holocaust; (3) mixed: the Holodomor is compared with multiple instances of mass atrocities (e.g. the Holocaust and the Stalinist repressions); (4) none: there are no parallels drawn between the Holodomor and other atrocities; (5) Soviet famines: the Holodomor is compared with other famines in the Soviet Union (e.g. the famine of 1921–1922 in the Volga region or the famine of 1932–1933 in Kazakhstan).

To measure the coding reliability, 53 links (i.e. 25% of the data) were coded by two coders, both of whom had working knowledge of English, Russian, Ukrainian and German. Based on this subsample, we calculated the Kripperndorf’s alpha values⁴ for each of the features. The results showed a high level of reliability for most of the features: 0.97 (type of the source), 0.97 (interpretation of the Holodomor), 0.83 (the number of victims) and 0.69 (comparison with other atrocities). The lower level of reliability for the last feature can be attributed to the disagreement between coders on how detailed the comparison shall be to count: the final consensus was that even non-detailed comparisons (e.g. the statement that some Ukrainian politicians refer to the Holodomor as the Ukrainian Holocaust) shall count. Following the reliability assessment, the identified disagreements were consensus coded.

Findings

Types of sources

We started our analysis by examining the types of sources prioritized by search algorithms. Figure 1 shows that most common sources of information about the Holodomor were journalistic media (e.g. BBC or Ukrainska Pravda) and reference websites (e.g. Wikipedia or Britannica). These observations align with earlier findings (Zavadski and Toepfl, 2019) on the representation of historical phenomena by the search engines, with the exception of the small presence of personal websites in the case of the Holodomor. Journalistic media and reference websites are also frequently prioritized sources in the case of non-historical queries (e.g. ones related to current politics; Urman et al., 2021).

Figure 1.

Types of information sources on the Holodomor across engines, time periods and queries.

Google was the most consistent in prioritizing journalistic/reference sources across both Latin and Cyrillic queries. For other engines (e.g. Yandex and DuckDuckGo), we observed substantial differences: for Latin queries, both engines put more emphasis on educational sources (e.g. Holodomor Research and Education Consortium), whereas for Cyrillic queries, such sources were replaced with reference (DuckDuckGo) and journalistic outlets (Yandex). Furthermore, in the case of Yandex, alternative websites (e.g. RationalWiki or holodomor.info) appeared more frequently for Latin queries (i.e. 2 results in February 2020 and 3 results in May 2021, as contrasted by 1 and 1 results for the Cyrillic queries).

Finally, some categories of sources appeared relatively infrequently among all search engines. Such categories included social media (e.g. YouTube videos and LiveJournal blogs), entertainment websites (predominantly discussing Holodomor-themed movies), and commercial websites (e.g. Amazon pages selling Holodomor-themed items).

Interpretation of the Holodomor

Figure 2 demonstrates substantial variation in the interpretations of the Holodomor prioritized by the search engines. The two most prevalent interpretations are of the Holodomor being a genocide (Yandex and DuckDuckGo for Latin queries) and of its status being debated (Google for Latin queries and Bing for Latin/Cyrillic queries). The number of search outputs interpreting the Holodomor as a crime of the Soviet leadership, but not a genocide, was relatively low across all engines.

Figure 2.

Interpretations of the Holodomor across engines, time periods and queries.

In addition to the outputs discussing, in one way or another, the responsibility of the Soviet regime for the Holodomor, we observed outputs denying not only the interpretation of the Holodomor as a genocide, but also its criminal nature. Such denialist outputs were particularly visible in the case of Yandex, specifically for Cyrillic queries (i.e. 5 and 4 results for February 2020 and May 2021, respectively), and usually came from journalistic outlets and social media platforms.

Furthermore, some search outputs claiming that the Holodomor was a genocide did so in a rather concerning manner. A few results, primarily coming from Yandex and Bing, linked to anti-Semitic materials stressing the connection between Bolshevism and Jews and blaming Jews for orchestrating a genocide against other ethnicities. An illustrative example of such a stance is the holodomor.info website titled ‘The Jewish Ethnic Cleansing Of Europeans’ (currently not functioning, but still available via the Internet Archive).

The number of Holodomor victims

Following our examination of the Holodomor’s interpretations, we looked at what numbers of victims are prioritized by the search engines. Figure 3 demonstrates that for most engines (except DuckDuckGo for Cyrillic queries, where the range from 5 to 10 million was prioritized) the prevalent number is between 1 and 5 million, which also aligns with the numbers prevalent in the academic scholarship. While some results also reported extremely low numbers (i.e. below 1 million), their presence was marginal.

Figure 3.

Reported number of victims of the Holodomor across engines, time periods and queries.

The high number of outputs reporting the numbers above 10 million, in particular, on Google and Bing, can be attributed to these search engines linking more frequently to the resources of political institutions (e.g. Ukrainian embassies), together with the German version of Wikipedia, which uses a higher estimate of victims (in contrast, for instance, to the English version with lower estimates). While such estimates of victims are criticized by scholars (e.g. Gadzins’ka, 2021) for being inflated, they still appear in the statements of the Ukrainian officials and the journalistic materials reporting on the ongoing debates. By prioritizing these estimates as part of their tendency to promote journalistic/political sources, search engines contribute to normalization of the higher estimates of the Holodomor victims (at least in response to the Latin queries).

Comparability of the Holodomor with other mass atrocities

Figure 4 shows that comparisons between the Holodomor and other atrocities vary depending on the query. For Latin queries, the majority of search outputs either abstain from drawing parallels between the Holodomor and other atrocities or discuss a number of possible comparisons (e.g. the Holocaust together with other Soviet famines). In some cases (e.g. Google), the Holodomor was compared only with the Holocaust and no other parallels were drawn, albeit the frequency of such comparisons was not too high.

Figure 4.

Comparisons between the Holodomor and other mass atrocities across engines, time periods and queries.

For Cyrillic queries, the distribution of comparisons turned out to be different. The number of comparisons to the Soviet famines substantially increased when using Yandex and DuckDuckGo (as well as Google for 2020). The prevalence of such comparisons, which are common for the Russian historiography of the Holodomor, tends to downplay the importance of the Holodomor as a genocide targeting specifically Ukrainians and normalizes it by drawing parallels with other famines in the Soviet Union.

Conclusion

In this study, we investigated how search algorithms circulate information about the Holodomor-related memory wars. Our observations demonstrate that search engines tend to prioritize journalistic media and reference websites as the main sources of information about the Holodomor. Such prioritization is attributed to several reasons: one of them is that some engines (e.g. Google; Google, n.d.) take into consideration the number of links to a specific website from other websites, together with interaction data. This benefits reference websites (e.g. Wikipedia), which are often extensively linked and interacted with. Another reason is the tendency of engines like Google to prioritize sources that demonstrate ‘authoritativeness and expertise on a topic’ (Google, 2020), which includes journalistic and government-related outlets. Furthermore, news organizations are known to be active in using search engine optimization to promote their content (e.g. Giomelakis and Veglis, 2016), which also contributes to the visibility of their websites.

The prevalence of journalistic media and reference websites has implications not only for how search algorithms represent the Holodomor, but also for how they deal with related ontological conflicts. The prevalence of journalistic commentaries translates into a focus on recent developments, including, for instance, public events around Holodomor anniversaries, but also provocative comments made by public figures in relation to the Holodomor. Together with sometimes fragmentary reporting on the historical nuances (e.g. the debates on the number of Holodomor victims) and the common presence of (self-)censorship in relation to the contested past, particularly in the case of the Eastern European fractured memory regimes, this results in journalistic outlets often fuelling memory wars. Similarly, reference websites, particularly crowd-sourced ones such as Wikipedia, often become memory battlegrounds where different mnemonic communities try to promote their preferred versions of the past (often aligning with official memory politics; Kaprāns, 2016; Makhortykh, 2017) and suppress alternative views.

The potential for search algorithms to reiterate memory wars by prioritizing sources that fuel these conflicts is amplified by the major ontological differences in the interpretation of the Holodomor depending on the engine and the query. Specifically, in response to Cyrillic queries, we observed the higher presence of results denying the criminal nature of the Holodomor (in particular, on Yandex) together with the larger number of outputs focusing on the comparison between the Holodomor and other Soviet famines. By contrast, Latin queries for most of the search engines (except Google) returned results offering a high estimate of the number of victims (i.e. more than 10 million), thus prioritizing interpretations that are criticized as the form of instrumentalization of the past (Gadzins’ka, 2021).

There are several reasons that can explain the abovementioned differences. The first of them is the unequal distribution of content with specific features (e.g. Holodomor denialism), which can be retrieved via Cyrillic and Latin queries. It is reasonable to expect that contextual factors (e.g. memory politics in Russia) might result in more Cyrillic content criticizing the interpretation of the Holodomor as genocide. At the same time, the very existence of such content does not imply that it has to be prioritized by search algorithms, thus suggesting that the differences we observe can be attributed to the different logic behind individual algorithms.

One example of this logic is the degree to which the search algorithm takes user engagement into consideration when prioritizing sources for a specific topic. While it can be assumed that the engagement is an important factor for most search engines, Yandex is assumed to pay more attention to it than its Western competitors (Niechai, 2020), which might result in a self-reinforcing loop of outlets that offer more engaging (but not necessarily high quality) content (e.g. tabloids) being prioritized and consequently engaged with more by Russian users. Another example is the varying notions of authoritative sources: while both Google and Yandex may prioritize sources of the same type (e.g. journalistic outlets), the exact choice of sources varies substantially with Google prioritizing more independent media, which are often critical of the Kremlin, and Yandex often giving priority to pro-Kremlin media (Makhortykh et al., 2022).

These observations highlight once more that non-human actors, such as search engines, are neither objective nor neutral in their treatment of collective memories. Similar to less complex analogue-focused information infrastructures (Bowker and Star, 2000), search algorithms shape social reality under a certain angle, and, by doing so, become memory warriors actively engaged in the ontological contestation of the past. While examined engines do not offer exclusive support to a specific interpretation of the Holodomor, the substantial variation in the interpretations’ visibility leads to the users’ awareness about the event being shaped rather differently depending on whether they search for it on Google or Yandex. Such differences not only lead to information inequalities, but also contribute to the reiteration of memory wars by exposing users to ideologically charged narratives.

The potential detrimental effect of search engines becoming memory warriors also stresses the importance of thinking about the ways of preventing them from fuelling memory wars. One approach could be to limit the presence of journalistic and reference sources, and instead put a larger emphasis on educational sources in order to decrease the visibility of one-sided interpretations of the past that often reiterate narratives associated with official memory politics. While educational sources can also be part of the memory wars, our observations suggest that they may offer a more nuanced treatment of the ontologically contested subjects as well as offering more scrutiny of new ontological claims.

Another possible solution could be to make users more aware about the ontological contestation of the subject which they are searching for (e.g. by including a disclaimer when retrieving search results). While such a disclaimer can possibly nurture a more critical treatment of the retrieved information, the practical implementation of such a solution will require a list of ontologically contested subjects, which itself may be ontologically contested. Establishing such a list, however, might be easier than finding a universally recognized baseline for Holodomor-related information (or for other subjects of memory wars), which would be required for the consistent prevention of prioritizing undesirable content (e.g. denialist claims).

Finally, it is important to note several limitations of the conducted study. First, it relies on a single search term (‘Holodomor’) written in Latin and Cyrillic script, whereas there are other general (e.g. ‘Ukrainian famine’) as well as more specific (e.g. ‘Holodomor Memorial Day’) terms that can be used to retrieve information about the Holodomor. Future research will benefit from expanding the selection of queries and using them in specific languages (e.g. Ukrainian) to conduct a more systematic audit of the role of search engines in the context of memory wars. Second, the current study looks at performance of search algorithms in a single location (i.e. Frankfurt), while earlier studies (e.g. Kliman-Silver et al., 2015) suggest that location plays a significant role in search personalization. Thus, future studies can examine potential variation in the role of search engines as memory warriors depending on where the user is located.

Footnotes

Acknowledgements

We would like to thank the anonymous reviewers and the guest editors of the special issue for their valuable feedback that helped us improve the manuscript.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Data collections were sponsored from the SNF (100001CL_182630/1) and DFG (MA 2244/9-1) grant for the project ‘Reciprocal relations between populist radical-right attitudes and political information behaviour: A longitudinal study of attitude development in high-choice information environments’ led by Silke Adam (University of Bern) and Michaela Maier (University of Koblenz-Landau) and FKMB (the Friends of the Institute of Communication and Media Science at the University of Bern) grant ‘Algorithmic curation of (political) information and its biases’ awarded to Mykola Makhortykh and Aleksandra Urman.

ORCID iDs

Mykola Makhortykh

Aleksandra Urman

Roberto Ulloa

Notes

Author biographies

Mykola Makhortykh is a postdoctoral researcher at the Institute of Communication and Media Research at the University of Bern. In his research, Mykola examines politics- and history-centred information behaviour in online environments and how it is affected by the algorithmic systems, such as search engines and recommender systems.

Aleksandra Urman is a postdoctoral researcher at Social Computing Group, University of Zurich. In her research, Aleksandra employs computational methods to examine various aspects of information distribution via online platforms. She is particularly interested in political communication on social media and algorithm audits of online platforms.

Roberto Ulloa is a postdoctoral researcher in the Computational Social Science department at GESIS – Leibniz Institute for the Social Sciences. His interests include online behaviour and the role of digital platforms, as forms of institutions, in shaping societies.

References

Applebaum

(2017) Red Famine: Stalin’s War on Ukraine. Oxford: Signal.

Bernhard

Kubik

(2016) Twenty Years after Communism: The Politics of Memory and Commemoration. Oxford: Oxford University Press.

Bosch

(2017) Twitter activism and youth in South Africa: the case of #RhodesMustFall. Information, Communication & Society 20(2): 221–232.

Bowker

Star

(2000) Sorting Things Out: Classification and Its Consequences. Cambridge: Massachusetts Institute of Technology Press.

Dyck

(2021) Holodomor and Holocaust memory in competition and cooperation. In: Cox

Khoury

Minslow

(eds) Denial: The Final Stage of Genocide? London: Routledge, pp. 29–44.

Esposito

(2017) Algorithmic memory and the right to be forgotten on the web. Big Data & Society 4(1): 1–11.

Fedor

(2017) Memory, kinship, and the mobilization of the dead: the Russian state and the ‘immortal regiment’ movement. In: Fedor

Kangaspuro

Lassila

, et al. (eds) War and Memory in Russia, Ukraine and Belarus. Cham: Palgrave Macmillan, pp. 307–345.

Gadzins’ka

(2021) Desiat’ milioniv piatsot tysh. I krapka. Ukraintsiam nav’iazyit’ superechlivi dani pro 10,5 mil’ioniv zhertv Golodomory. Istorycha Pravda, 26 November. Available at: https://www.istpravda.com.ua/articles/2021/11/26/160564/ (accessed 30 March 2022).

Gaufman

(2015) World war II 2.0: digital memory of fascism in Russia in the aftermath of Euromaidan in Ukraine. Journal of Regional Security 10(1): 17–35.

10.

Giomelakis

Veglis

(2016) Investigating search engine optimization factors in media websites: the case of Greece. Digital Journalism 4(3): 379–400.

11.

Google (n.d.) How search works: ranking results. Available at: https://www.google.com/search/howsearchworks/how-search-works/ranking-results/ (accessed 29 June 2022).

12.

Google (2020) How Google delivers reliable information in search. Available at: https://blog.google/products/search/how-google-delivers-reliable-information-search/ (accessed 29 June 2022).

13.

Gudz’

(2019) Istoriografiia Golodomoru 1932–1933 rokiv v Ukraini. Melitopol’: Odnorog.

14.

Hannak

Sapiezynski

Kakhki

, et al. (2013) Measuring personalization of web search. In: WWW ’13: proceedings of the 22nd international conference on World Wide Web, Rio de Janeiro, Brazil, 13–17 June, pp. 527–538. New York: ACM.

15.

Hood

Reid

(2018) Social media as a vehicle for user engagement with local history: a case study in the north east of Scotland. Journal of Documentation 74(4): 741–762.

16.

Hoskins

(2009) The mediatisation of memory. In: Garde-Hansen

Hoskins

Reading

(eds) Save as. . . Digital Memories. London: Palgrave Macmillan, pp. 27–43.

17.

Jakubowicz

(2009) Remembering and recovering Shanghai: seven Jewish families [re]-connect in cyberspace. In: Garde-Hansen

Hoskins

Reading

(eds) Save as. . . Digital Memories. London: Palgrave Macmillan, pp. 96–114.

18.

Jilge

(2006) Zmagannia zhertv. Krytyka 5(103): 14–17.

19.

Jonassohn

(1998) Genocide and Gross Human Rights Violations: In Comparative Perspective. New Brunswick, NJ: Transaction Publishers.

20.

Kasianov

(2008) Revisiting the great famine of 1932–1933: politics of memory and public consciousness (Ukraine after 1991). In: Kopeček

(ed.) Past in the Making: Historical Revisionism in Central Europe After 1989. Budapest: Central European University Press, pp. 197–219.

21.

Kaprāns

(2016) Hegemonic representations of the past and digital agency: Giving meaning to “The Soviet Story” on social networking sites. Memory Studies 9(2): 156–172.

22.

Kay

Matuszek

Munson

(2015) Unequal representation and gender stereotypes in image search results for occupations. In: CHI ’:15 Proceedings of the 33rd annual ACM conference on human factors in computing systems, Seoul, Republic of Korea, 18–23 April 2015, pp. 3819–3828. New York: ACM.

23.

Kliman-Silver

Hannak

Lazer

, et al. (2015) Location, location, location: the impact of geolocation on web search personalization. In: IMC ’15: proceedings of the 2015 internet measurement conference, Tokyo, Japan, 28–30 October 2015, pp. 121–127. New York: ACM.

24.

Klymenko

(2016) The Holodomor law and national trauma construction in Ukraine. Canadian Slavonic Papers 58(4): 341–361.

25.

Koposov

(2017) Memory Laws, Memory Wars: The Politics of the Past in Europe and Russia. Cambridge: Cambridge University Press.

26.

Kravets

Toepfl

(2021) Gauging reference and source bias over time: how Russia’s partially state-controlled search engine Yandex mediated an anti-regime protest event. Information, Communication & Society. Epub ahead of print 7 June 2021. DOI: 10.1080/1369118X.2021.1933563.

27.

Krippendorff

(2011) Computing Krippendorff’s Alpha-Reliability. Penn Libraries. Available at: https://repository.upenn.edu/asc_papers/43 (accessed 29 October 2022).

28.

Makhortykh

(2017) Framing the Holocaust online: memory of the Babi Yar massacres on Wikipedia. Studies in Russian, Eurasian and Central European New Media 18: 67–94.

29.

Makhortykh

(2018) #NoKievNazi: social media, historical memory and securitization in the Ukraine crisis. In: Apryshchenko

Strukov

(eds) Memory and Securitization in Contemporary Europe. London: Palgrave Macmillan, pp. 219–247.

30.

Makhortykh

(2019) Nurturing the pain: audiovisual tributes to the Holocaust on YouTube. Holocaust Studies 25(4): 441–466.

31.

Makhortykh

(2021) Memoriae ex machina: how algorithms make us remember and forget. Georgetown Journal of International Affairs 22(2): 180–185.

32.

Makhortykh

Sydorova

(2022) Animating the subjugated past: digital greeting cards as a form of counter-memory. Visual Communication 21(1): 28–52.

33.

Makhortykh

Urman

Ulloa

(2020) How search engines disseminate information about COVID-19 and why they should do better. Harvard Kennedy School Misinformation Review 1(3): 1–12.

34.

Makhortykh

Urman

Ulloa

(2021a) Detecting race and gender bias in visual representation of AI on web search engines. In: Boratto

Faralli

Marras

, et al. (eds) Advances in Bias and Fairness in Information Retrieval. Cham: Springer, pp. 36–50.

35.

Makhortykh

Urman

Ulloa

(2021b) Hey, Google, is it what the Holocaust looked like? First Monday 26(10). Available at: https://firstmonday.org/ojs/index.php/fm/article/view/11562

36.

Makhortykh

Urman

Wijermars

(2022) A story of (non)compliance, bias, and conspiracies: how Google and Yandex represented smart voting during the 2021 parliamentary elections in Russia. Harvard Kennedy School Misinformation Review 3(2): 1–16.

37.

Mälksoo

(2015) ‘Memory must be defended’: beyond the politics of mnemonical security. Security Dialogue 46(3): 221–237.

38.

Marchukov

(2008) ‘Golodomor’ v ideologicheskoi sisteme ukrainstva. Vestnik Iugo-Zapadnoi Rusi 2: 81–95.

39.

Marochko

(2017) Statystyka zhertv Golodomoru: antropologichno-demografichnyi dyskurs. Ukraiins’kyi istorychnyi zhurnal 5: 112–132.

40.

Mayer-Schönberger

(2011) Delete: The Virtue of Forgetting in the Digital Age. Princeton, NJ: Princeton University Press.

41.

Nevedov

(2018) Politizirovannaia istoriografiia. Vyzovy vremeni i sotsialno-istoricheskoe znanie 7: 16–18.

42.

Niechai

(2020) 9 SEO tips on how to optimize for Yandex. SEO PowerSuite. Available at: https://www.link-assistant.com/news/yandex-seo.html (accessed 29 October 2022).

43.

Noble

(2018) Algorithms of Oppression. New York: New York University Press.

44.

Paßmann

Boersma

(2017) Unknowing algorithms. On transparency and unopenable black boxes. In: Schäfer

Van Es

(eds) The Datafied Society: Studying Culture Through Data. Amsterdam: Amsterdam University Press, pp. 139–146.

45.

Pethig

Kroenung

(2022) Biased humans, (un)biased algorithms? Journal of Business Ethics. Epub ahead of print 28 February. DOI: 10.1007/s10551-022-05071-8.

46.

Petryshyn

Gerasimenko

Stasiuk

(2021) Genotsid ukraiintsiv 1932–1933 za materialamy dosydovyh rozslidyvan’. Kyiv: Vydavnitstvo Marka Mel’nika.

47.

Prudnikova

Chigirin

(2013) Mifologiia ‘golodomora ’. Moscow: OLMA Press.

48.

Rambler (2017) Zapadnaia fabrika lzhi: SSHA pridumali golodomor chtoby sdelat’ iz ukraintsev rusofobov. Available at: https://news.rambler.ru/world/36076780-zapadnaya-fabrika-lzhi-ssha-pridumali-golodomor-chtoby-sdelat-iz-ukraintsev-rusofobov/ (accessed 30 March 2022).

49.

Reading

(2011) Memory and digital media: six dynamics of the global memory field. In: Neiger

Meyers

Zandberg

(eds) On Media Memory. London: Palgrave Macmillan, pp. 241–252.

50.

Rudnytskyi

Levchuk

Wolowyna

, et al. (2015) Demography of a man-made human catastrophe: the case of massive famine in Ukraine 1932-1933. Canadian Studies in Population 42(1–2): 53–80.

51.

Rutten

Fedor

Zvereva

(2013) Memory, Conflict and New Media: Web Wars in Post-socialist States. London: Routledge.

52.

Shanapinda

(2018) Protecting our digital heritage in the age of cyber threats. The Conversation, 5 December. Available at: https://theconversation.com/protecting-our-digital-heritage-in-the-age-of-cyber-threats-108252 (accessed 30 March 2022)

53.

Shubin

(2008) 10 mifov Sovetskoi strany. Pravda o Velikoi epohe. Moscow: Iauza.

54.

Ulloa

Makhortykh

Urman

(2022a) Scaling up search engine audits: practical insights for algorithm auditing. Journal of Information Science. Epub ahead of print 2 May. DOI: 10.1177/01655515221093029.

55.

Ulloa

Richter

Makhortykh

, et al. (2022b) Representativeness and face-ism: gender bias in image search. New Media & Society. Epub ahead of print 19 June. DOI: 10.1177/14614448221100699.

56.

Urman

Makhortykh

Ulloa

(2021) The matter of chance: auditing web search results related to the 2020 U.S. presidential primary elections across six search engines. Social Science Computer Review 40(5): 1323–1339.

57.

Van Dijck

(2011) Flickr and the culture of connectivity: sharing views, experiences, memories. Memory Studies 4(4): 401–415.

58.

Van House

Churchill

(2008) Technologies of memory: key issues and critical perspectives. Memory Studies 1(3): 295–310.

59.

Wallace

(2018) Modelling contemporary gatekeeping: the rise of individuals, algorithms and platforms in digital news dissemination. Digital Journalism 6(3): 274–293.

60.

Zavadski

Toepfl

(2019) Querying the Internet as a mnemonic practice: how search engines mediate four types of past events in Russia. Media, Culture & Society 41(1): 21–37.

Memory,counter-memory and denialism: How search engines circulate information about the Holodomor-related memory wars

Abstract

Keywords

Introduction

The Holodomor in the context of memory wars

The algorithmic turn in memory wars

Methodology

Data collection

Data analysis

Findings

Types of sources

Interpretation of the Holodomor

The number of Holodomor victims

Comparability of the Holodomor with other mass atrocities

Conclusion

Footnotes

Acknowledgements

Funding

ORCID iDs

Notes

Author biographies

References