Sage Journals: Discover world-class research

Abstract

Search engines play an important role in the spread of disinformation and conspiracy theories, accentuating the power of global platform companies such as Google to contribute to the digital (information) divide by providing search results of lesser quality in certain countries. We investigated this phenomenon by asking what kind of results users see when they search for information on eleven popular conspiracy theories (CTs) via Google. We analysed links from Google search results (N = 1259) in 12 Western and non-Western countries and 10 languages. Overall, users are more likely to encounter neutral or debunking content when using Google to search for prominent CTs. However, for some CTs, strong country differences in the quality of search results emerge, showing clear correlations between categorical inequalities and unequal access to reliable information. In countries where journalists enjoy less freedom, people enjoy fewer democratic rights and are less able to rely on social elites, Google also provides less enlightening content on CTs than in developed and prosperous democracies. The countries thus disadvantaged are precisely those countries where there is a high propensity to believe in CTs according to comparative survey research. However, in countries where a global language is spoken, for example, English or Portuguese, there is no correlation between structural, country-specific factors and the quality of search results. In this sense, structurally disadvantaged countries seem to benefit from belonging to a larger language community.

Keywords

Comparative research conspiracy theories digital divide Google information inequality search engine

Introduction

In recent years, social media platforms have taken most of the blame for their role in spreading disinformation, including conspiracy content (cf. Allington et al., 2021; Mahl et al., 2021; Xiao et al., 2021). Yet, the spread of conspiracy theories often begins with an active search for information (boyd, 2018). This, in turn, creates opportunities for actors to strategically place disinformation. However, the role of search engines as a disinformation vector has not received much attention in academic research, despite growing anecdotal evidence of its importance not only in wealthy Western countries, but also in economically weaker countries, transitional democracies and countries of the Global South. It is the purpose of this article to explore how these opportunity structures for news hackers (von Nordheim and Kleinen-von Königslöw, 2021) vary across countries with different structural conditions.

An optimistic view of the development of the past few years (cf. Napoli, 2021, for example) might conclude that in the wake of Trump and Covid, social media and search engine companies are becoming increasingly aware of their social responsibility, due to public pressure and – as profit-oriented platform companies – because they are weighing their own economic interests. Ultimately, initiatives against hate speech and disinformation are first and foremost an investment to protect them against the wrath of advertising clients and regulators.

But what happens in countries that are not considered relevant advertising markets and which pose no threat of political-regulatory pressure? What about publics in the periphery of the West and in the Global South that are not under the same constant critical public scrutiny? It seems likely that companies such as Google will develop their services differently in these countries. Furthermore, news hackers do not always target the search algorithm itself, but rather ‘the information landscape that the search engine depends on’ (Golebiewski and boyd 2018: 13). Hence, for countries with weak media systems and a low overall quality of available information (and corresponding low levels of public trust in media information) the question of how responsibly search engines companies address the problem of conspiracy content in their search results is even more important.

To this moment, there exists little empirical research on the question of a potential global divide in the quality of information provided by search engines. The aim of this article is to gather further evidence through a comparative study in 12 countries (Australia, Brazil, Czech Republic, Germany, Ghana, Hungary, Italy, Poland, Romania, Serbia, Switzerland and Ukraine) on conspiracy theories (CTs), or more precisely: on search results displayed to users in different countries when they search for specific CTs. In this respect, the countries we studied differ in their level of democratic development and prosperity and thus the quality of their general information landscape, as well as their expected conspiracy-prone mindset – meaning that they also differ in their demand for CT-affirmative content, which might be reflected in the search results.

Theoretical framework and state of research

The relevance of Google in the context of conspiracy theories

Google’s search engine has immense influence on our daily lives; an analysis of browser histories showed that in one in five sessions, people begin their news consumption with an online search (Bentley et al., 2019). According to Toff and Nielsen (2018: 636), many people are convinced that ‘the information is out there’, that is, that it is easy to find – this widespread ‘folk theory’ often goes hand in hand with a lack of trust in established news media and a high self-assessment, which mostly does not correspond with their actual information literacy. Against this backdrop, it seems very logical to examine the role of Google in the dissemination of CTs; for here too, distrust of mainstream media sources (Barkun, 2003), lack of media literacy (Craft et al., 2017) and an urge to seek the truth independently (Ballatore, 2015) are crucial catalysts.

Conspiracy theories are defined as explanations of ‘important events as secret plots by government or powerful individuals’ (Kapantai et al., 2021: 1325) that ‘contradict the general consensus among epistemic authorities’ (Brotherton and Eser, 2015: 1). They are thus answers to questions that people ask themselves in times of uncertainty. Whether they are accepted or not will depend significantly on whether people are confronted with content that confirms or debunks a particular CT (Warner and Neville-Shepard, 2014).

Hence, a wide range of actors competes to be listed among the first search results for CTs: Professional news media (often public service media) who inform about and debunk CTs, sometimes with dedicated fact-checkers, as do some government websites and academics. Yet, there are also actors who actively promote CTs suiting their political or financial goals: Prominent Republican politicians embracing QAnon in the US, news media sacrificing professional norms for more clicks and revenue, as well as other economic actors striving for financial gains.

These ‘conspiracy entrepreneurs’ (Campion-Vincent, 2015) or ‘news hackers’ (von Nordheim and Kleinen-von Königslöw, 2021) occupy specific search words (‘key word squatting’, Donovan and Friedberg, 2019: 37) and fill ‘data voids’ (Golebiewski and boyd, 2018) with their content, that is, in subject areas for which more mainstream media have not (yet) provided any content. They optimise their content for search engines, and reciprocal links within the Alternative Media Ecosystem (Starbird, 2017) signal relevance to search engine algorithms. Furthermore, the presence of confirmatory information about CTs in Google search results can encourage users to communicate their own conspiracy beliefs, triggering a ‘“snowball effect” of conspiracy theory’ (Mahl et al., 2021: 2).

However, while the prevalence of CTs on social media has been studied extensively in recent years (cf. Allington et al., 2021; Mahl et al., 2021; Xiao et al., 2021), we still lack corresponding studies for search engines. An exception is Ballatore’s (2015) analysis of the visibility of 15 well-known CTs in different search engines (Google and Bing). He concluded that a total of 48.4% of all Google result pages were conspiratorial, while only 14.1% were neutral, and 33.1% offered debunking content, showing that ‘fringe and non-mainstream views are heavily represented by search engines’ – the results thus representing an ‘imperfect, and yet useful mirror of society’ (Ballatore, 2015).

Google and the digital information divide

However, we should expect this mirror image to differ across countries, reflecting differences in the quality of information available to citizens. This has long been discussed as an ‘information divide’ in particular between industrialised and developing countries, and more recently with regard to transitional democracies in Central and Eastern Europe (CEE) and beyond (Ragnedda and Kreitern, 2018). In the mid-1990s, it was recast as ‘digital divide’, accompanied by the hope that the Internet would level out power inequalities. Unfortunately, this hope has not been fulfilled. According to Van Dijk’s ‘resources and appropriation theory’ (2017: 3), the self-reinforcing relationship between categorical inequalities (such as belonging to a developed/developing nation), unequal distribution of resources and unequal access to digital technologies persist. Under-resourced people are thus denied the ‘benefits of access’, for example, finding job opportunities, buying cheap products, but also forming an opinion on political topics or educating themselves about health issues. And in many countries, US platform companies have become the main Internet and information access point for citizens (Newman, 2021).

Still, there is little research on how this has impacted the digital divide and whether platform-based disparities in web content across countries actually enhance it: In a cross-national study, Scherr et al. (2019) found that Google advantages users in certain countries in suicide prevention (displaying supportive, educational information when certain search terms are entered). In his study of Google News and Google Earth, Segev (2010) also detected a bias ‘which provides an obvious informational and political advantage for the USA and some other capitalist countries’ (xxix).

Regarding CTs, Google search results are likely to reflect the more general country differences in information opportunities and information quality due to the respective media systems (Castro et al., 2017). As Humprecht et al. (2020) outline in their conceptualisation of country resilience to online disinformation, in countries with strong public service media (PSM) there should be more debunking information available in general, especially if these media enjoy the trust of the public. Similarly, stable, non-polarised democracies with low levels of populist communication should have fewer political actors exploiting CTs for their own gain and thus less pro-conspiracy information floating around. By contrast, in highly politicised or transitional media systems (Dobek-Ostrowska, 2019), journalists face substantial pressure by political and economic actors which may depress journalistic professionalism and information quality, providing more opportunities for the spread of CTs.

At the same time, the search engine is known to actively shape and even at times moderate its search results (Grind et al., 2019). And yet, the search engine’s business strategy will generate country differences – for two reasons.

First, the search engine is a mirror of its users, with different search results reflecting different demand (Trielli and Diakopoulos, 2020). Google has been repeatedly criticised for prioritising site popularity over content quality criteria (Diaz, 2008). Thus, in countries prone to conspiracy thinking, more users will express their demand for CT content and send corresponding relevance signals to curating algorithms (Cordonier et al., 2021). This way, positive feedback effects could arise between search engine algorithms and a strong conspiracy-prone mindset among a certain population.

Secondly, the economic incentive for Google to invest in algorithm improvement or content moderation will depend on the potential profits, that is, the volume of the respective advertising market – which, in turn, directly correlates with a country’s economic power. Afterall, companies do not develop algorithms ‘to be imbalanced’ (Scherr et al., 2019: 566), it is mainly a lack of incentive that is causing them to become imbalanced. These business strategies following the logic of advertising platforms are in constant conflict with Google’s infrastructural role which is associated with an influence that “goes way beyond markets, affecting entire societal sectors, democratic processes, online social traffic, and national institutions” (van Dijck et al., 2019: 9). The “infrastructural power” (van Dijck et al., 2019: 12) resulting from the synergetic combination of data flows for different applications and business models brings with it a social responsibility comparable to other highly regulated sectors (e.g. in the field of critical infrastructures such as transport, energy or communications). Google has responded to societal backlash regarding the abuse of its monopoly position by repeatedly emphasizing the neutrality of its search algorithms (Rieder and Sire, 2014) – thus largely avoiding regulatory intervention in its business model. However, as van Dijck et al. (2019: 12) point out, there is a lack of case studies that test the seriousness of these assurances.

According to a recent comparative study on the prevalence of conspiracy-prone mindsets in 22 Western and non-Western countries (Cordonier et al., 2021), low levels of democracy, high levels of corruption as well as high economic insecurity also make it more likely for conspiracy theories to thrive: When people feel politically and economically powerless, they appear to be more receptive to conspiracy theories. In line with the commercial, demand-driven logic of the search engine, the algorithm’s ‘user input bias’ (Trielli and Diakopoulos, 2020: 3) and the dynamics of the ‘snowball effect’ of conspiracy theory’ (Mahl et al., 2021: 2) suggest that in these less stable countries with a high prevalence of conspiracy thinking (and thus demand for conspiracy promoting results), the quality of search results should be lower. At the same time, an unstable economic would make this market less attractive for Google, leading it to invest less in curating its algorithms.

Research questions

Building on comparative research on the quality of search results conducted by Scherr et al. (2019), we ask how search results on CTs differ between countries. Given the well-documented consequences of conspiracy belief – for example, it lowers people’s willingness to engage in protective health behaviours (e.g. Freeman et al., 2020) or pro-environmental behaviours (Van der Linden, 2015), while leading to an increase in hatred and violence towards minorities (Pasek et al., 2015) – we consider the quality of search results as higher the less CT confirming content they contain compared to debunking or neutral content.

RQ1

What is the quality of search results on CTs in different countries, that is, what is the share of conspiracy confirming content?

As Mahl et al. (2021) emphasise, certain CTs may gain more traction in certain communities due to ideological and geographic proximity, we will thus also explore how these country differences are reflected on the level of different CTs (RQ 1a). According to Ballatore (2015), differences in the share of confirming content between certain CTs may be explained by the type of sources shown in the search results, with social media/blog content being particularly prone to confirming CTs. Given the differences in the general information offer to be expected due to differences in the countries’ media systems, we will also explore the role of source types in the quality of search results on CTs in different countries (RQ1b).

With the second research question, we want to probe whether the identified digital divides run parallel to socio-economic fault lines, that is, whether those who suffer less from categorical inequalities profit more from the benefits of access. According to Humprecht et al. (2020) and Dobek-Ostrowska (2019) media related categories such as trust in news media and journalistic freedom should be particularly relevant for the prevalence of disinformation, as well as political factors such as societal polarisation and overall democratic quality. Following Cordonier et al. (2021), economic factors such as economic stability of the country and the perceived corruption should also be important in relation to conspiracy theories.

Accordingly, we ask:

RQ2

How does the quality of search results on CTs correlate with structural characteristics (regarding media, politics and economy) in the respective country?

Method

Case selection

This research can be characterized as an exploratory case study, serving as an “empirical inquiry that investigates a contemporary phenomenon (the‘case’) in depth and within its real-world context” (Yin, 2014: 16). Specifically, we examine the cases of different countries, which are primarily distinguished by their socioeconomic and political landscapes, as well as their provisions for free media. Notably, our selection of countries has paid special attention to ‘flawed democracies’ (The Economist Intelligence Unit, 2021: 57). While such states hold free elections and respect basic civil liberties, other aspects of democracy present significant weaknesses, such as governance issues, an underdeveloped political culture and low levels of political participation: In our sample, this category includes Ghana and Brazil as countries of the Global South, and Serbia, the Czech Republic, Hungary, Romania and Poland as representatives of (transitional) democracies in Central and Eastern Europe. Ukraine is another former Eastern bloc country, but it is classified as a ‘hybrid regime’ with widespread corruption, weak rule of law and civil society (The Economist Intelligence Unit, 2021: 57). The results from these countries are then contrasted with those from ‘full democracies’ in the Global North: Germany, Italy, Switzerland and Australia.

Structural characteristics

For our explorative comparative analysis, we include three sets of contextual factors which might explain the quality of information available via search results (Table 1). Regarding media, we look at ratings of the Press Freedom Index to assess journalists’ freedom to publish high quality information (Dobek-Ostrowska, 2019) as well as trust in most used (public service) media as main factors influencing a country’s information resilience (Humprecht et al., 2020). Following again Humprecht et al. (2020), we include societal polarisation as well as the Democracy Index, which is based on five categories: electoral process and pluralism, civil liberties, the functioning of government, political participation and political culture. For economic factors, we look at the GDP per capita to assess the economic stability of a country as well as the Corruption Perception Index (CPI, Cordonier et al., 2021).

Table 1.

Explanatory factors per country.

	AU	BR	CZ	DE	GH	HU	IT	PL	RO	RS	CH	UA
Trust in (PS)M^a	70	67	60	70	55	40	67	36	69	56	77	40
Press Freedom^b	3	3	3	4	3	2	3	3	3	2	4	2
Societal polarisation^c	1.37	0.06	1.83	1.56	0.3	0.02	1	0.05	1.21	0.37	1.23	0.66
Democracy Index^d	8.96	6.92	7.67	8.67	6.5	6.56	7.74	6.85	6.4	6.22	8.83	5.81
CPI^e	77	38	54	80	43	44	53	56	44	38	85	33
GDP per capita^f	62,720	7010	25,730	51,860	2370	18,080	35,000	16,930	14,970	8750	94,700	3980

^aFor Ghana (Zupork Dome et al., 2020) ‘how much do you trust information from public media institutions’; for Serbia (Ipsos, 2021) ‘how much trust do you have in Public Service Media (RTS (Radio televizija Srbije)), when it comes to reporting the news fully, accurately, and fairly’; for Ukraine (Internews, 2021: 4) ‘how much do you trust the following media?’: national TV. For all other countries (Newman, 2021) ‘how trustworthy would you say news from the following brands is?’: trust = % scored 6–10 on 10-point scale for the most used (public) news media brand.

^bRanges from 1 ‘not free’ to 4 ‘free’ (Freedom House, 2022).

^cRanges from 0 ‘Serious polarisation. There are serious differences in opinions in society on almost all key’ to 4 ‘No polarisation. There are differences in opinions but there is a general agreement on the direction for key political issues’ (Coppedge et al., 2019).

^dThe Economist Intelligence Unit’s index of democracy is based on the ratings for 60 indicators (combination of a dichotomous and a three-point scoring system, based on experts’ assessments and public-opinion surveys) grouped into five categories: electoral process and pluralism, civil liberties, the functioning of government, political participation and political culture. Scale from 0 to 10; the overall index is the simple average of the five category indexes (The Economist Intelligence Unit, 2021).

^eThe CPI (Transparency International, 2020) is based on aggregated and rescaled data from 13 data sources: 1. African Development Bank Country Policy and Institutional Assessment 2018, 2. Bertelsmann Stiftung Sustainable Governance Indicators 2020, 3. Bertelsmann Stiftung, Transformation Index 2020, 4. Economist Intelligence Unit Country Risk Service 2020, 5. Freedom House Nations in Transit 2020, 6. Global Insight Country Risk Ratings 2019, 7. IMD World Competitiveness Centre World Competitiveness Yearbook Executive Opinion Survey 2020, 8. Political and Economic Risk Consultancy Asian Intelligence 2020, 9. The PRS Group International Country Risk Guide 2020, 10. World Bank Country Policy and Institutional Assessment 2019, 11. World Economic Forum Executive Opinion Survey 2019, 12. World Justice Project Rule of Law Index Expert Survey 2020, 13. Varieties of Democracy (V-Dem v. 10) 2020.

^fGDP per capita, current prices/U.S. dollars per capita (IMF, 2021).

Conspiracy theories

Mahl et al. (2021) identified the 10 most shared CTs on Twitter (based on a dataset of 106,807 tweets published over 6 weeks from 2018 to 2019): Agenda 21 (or Depopulation Agenda), Anti-Vaccination (or Vaccine Damage), Chemtrails, Climate Change Denial/Hoax, Directed Energy Weapons, Flat Earth, Illuminati, Pizzagate, Reptilians and 9/11 (Inside Job). All CTs have overlaps in their topoi, some are primarily directed against elites and stoke fears of a (worldwide) conspiracy of secret networks (Illuminati, Pizzagate, Reptilians); some primarily challenge official explanations for events or scientific facts (Vaccine Damage, Flat Earth, Climate Change Hoax); some focus on reinforcing anti-‘globalist’, right-wing ideologies (Depopulation Agenda, Directed Energy Weapons, Mahl et al., 2021: 9–10). The ten CTs are not specifically associated with any country in the sample. Due to their international spread, they are a suitable starting point for our investigation (supplemented by Qanon, which Mahl et al. (2021:4) left out for methodological reasons).

Gathering search results

We searched for the eleven conspiracy theories in the respective national language (see supplement for search terms) – the search terms were translated (English for Ghana and Australia, German for Germany and Switzerland, Hungarian, Polish, Czech, Portuguese, Romanian, Ukrainian, Serbian and Italian) by native speaker researchers. The search words were localised for common language use, which is why some search terms were adapted to better reflect national language use patterns.

To avoid personalisation effects, we made sure that users were not logged into Google when they conducted their search; that the local language was selected in the search settings; and that search customisation was disabled (a default option which draws on past Google searches to personalise results). With these settings in place, we searched for the eleven CTs and manually saved the URLs of the organic search results (including results that appeared in a video box). We limited our collection of search results to the first Google results page, since users rarely take note of hits on the second page (Jansen and Spink, 2006).

To reduce the temporal bias¹ that distorts the representativeness of search results (Ballatore, 2015), we conducted our searches on 10 consecutive days² in August/September 2021. Our final sample only included URLs that appeared on the first results page on at least 5 days. We took no measures to reduce spatial bias (e.g. via Tor network), since geographical personalisation is of particular interest to us to identify country differences and possible digital divides. We conducted our searches from desktop computers located in each of the countries studied.³

Content analysis and conspiracy index

We analysed temporally stable URLs (N = 1259) in two categories in keeping with the setup of Ballatore’s study (2015) (see codebook in the supplement): First, we determined the source type of each link (Krippendorff’s alpha mean= 0.92, min = 0.83, see supplement for an overview of all values), extending Ballatore’s categories by additional source types (e.g. social media, government sites, e-commerce platforms or alternative media⁴). Second, we recorded the content of the pages was conspiratorial, debunking or neutral (alpha mean= 0.88, min = 0.82). Neutral content could a) either describe the CT without confirming or denying it, b) make fun of the CT without debunking or confirming it, c) depict a topic related to the CT, but not the CT itself (e.g. climate change, but not the CT that climate change is a hoax) or d) depict an issue completely unrelated to the CT and its topic.

Results

RQ1: What is the quality of search results on CTs in different countries, that is, what is the share of conspiracy confirming content?

If users search for conspiracy theories, the share of CT confirming content in their search result varies significantly, depending on their country. It is lowest in Australia, Brazil, Germany and Switzerland (see Table 2), with shares below 10%, followed by Ghana and Poland at 12%. In the Czech Republic, Hungary, Italy and Ukraine almost every fifth search result links to CT confirming content. Most problematic are the search results in the CEE countries Serbia and Romania where search results propagate CT messages in one out of four (Romania) or almost one out of three search results (Serbia).

Table 2.

Overview of ideological bias of search results by country (share in %).

Country (n)	Confirming	Debunking	Neutral				Unclear
Country (n)	Confirming	Debunking	Desc	Satirical	Related	Unrelated	Unclear
Australia (104)	5	38	26	6	20	2	3
Brazil (71)	7	52	13	0	0	28	0
Czech Rep. (115)	17	17	37	3	14	5	5
Germany (113)	8	32	28	0	30	2	2
Ghana (103)	12	38	40	0	5	1	4
Hungary (115)	17	44	7	9	15	6	2
Italy (101)	19	47	16	1	3	12	2
Poland (111)	12	27	21	4	11	14	11
Romania (107)	26	20	15	5	13	19	2
Serbia (103)	30	19	26	4	13	8	0
Switzerland (109)	5	41	37	3	7	5	2
Ukraine (107)	19	21	29	3	22	7	0
N	187	413	311	40	167	106	35

RQ1a: Differences in terms of CTs

The eleven CTs in our analysis can be separated into three groups: The first group consists of six CTs for which the search engine yields CT confirming content only in very few countries (QAnon, Illuminati, Pizzagate, Vaccination Damage, Chemtrails), though in these countries the share of CT confirming links can be quite high. Given its strong popularity in the US, it is somewhat surprising that QAnon does not appear to gain any traction elsewhere: Google searches for QAnon only linked to debunking or neutral content in our countries. Pizzagate (8% confirmatory search results) also does not appear to resonate culturally in our countries with the exception of Serbia, where 40% of all search results link to websites confirming that US Democrats operated a paedophile ring from a Washington Pizza restaurant.

Following searches for Vaccination Damage, CT confirming content is extremely rare, only users in the Czech Republic, Poland, Ukraine or Switzerland will find 10% CT confirming results, mirroring Ballatore’s (2015) study which showed the least CT confirming search results for CTs on vaccines and autism. Here, it seems likely that the high political relevance of this CT has led to strong moderation efforts on part of the search engine, but also by media and political actors within the country.

For CTs in the second group, confirming content can be found in more than half of the countries in our sample, though on average it is rather rare (below 20%): This is the case for CTs on Climate Change Hoax, Directed Energy Weapons, Flat Earth and 9/11 (Inside Job). Search results confirming that the Earth is flat can be found in seven countries (mostly CEE countries, but also in Germany and Ghana), but their share among all search results is still comparatively low (at most 25% CT confirming results in Ghana). CTs concerning Climate Change are confirmed by search results in seven countries at a share of up to 38% (in Serbia). But this time, not only CEE countries are affected, Australia, Brazil and Italy also show at least every fifth search results supporting the CT – which could be related to the fact that in both Australia and Brazil political elites are denying climate change to ensure economic profits. For CTs concerning 9/11, only users in two countries of our sample, Brazil and Ghana, are shown no confirming search results. In most countries, their share is rather low, with the exception of Czech Republic and Hungary where users will be confronted with 36 and 44% results confirming the CT, respectively. Search results confirming CTs on Directed Energy Weapons are present in eight countries, but their salience varies strongly, from 10% in Germany and Hungary to more than half in Ukraine and Italy.

By contrast, users will find search results confirming the CTs of the third group (Reptilians and Depopulation Agenda) in almost all countries, except for Australia or Switzerland (Reptilians) and Ghana (Depopulation Agenda). More importantly, the share of CT confirming content is much higher than for the previous CTs: For Reptilians, it reaches up to 73% in the Czech Republic and Romania (on average: 30%) and for the Depopulation Agenda, the share is more than half in Romania, Serbia and Poland (on average: 28%) (see Figure 1 and Table S4 in the supplement). As the existence of Reptilians is by far the most implausible of the CTs in our sample, it may be that the search engine simply does not bother to moderate the corresponding search results and the responsible communicating actors (such as traditional media or fact-checkers) do not debunk this CT. By contrast, the strong support for the Depopulation Agenda is most likely linked to the fact that this CT is in fact an important talking point for populist political actors in many countries (Mahl et al., 2021).

Figure 1.

Box plot of share of CT confirming results by conspiracy theories (CTs).

Still, the search results for our CTs mostly return neutral or educational content debunking the CTs. Only for two CTs (Reptilians and Depopulation Agenda) the share of confirming content reaches more than a quarter of all results. However, country differences are rather strong, leaving users in selected countries, particularly in CEE, with high chances of encountering confirmatory content if they search for specific CTs.

RQ1b: Differences in terms of source types

A first look at the different proportions of source types aggregated by country (see supplement) reveals that mainstream media clearly dominate the search results everywhere. In each country, more than one fifth of Google’s first-page links stem from large, established media outlets (33% on average, 45% maximum in Brazil, 23% minimum in Ukraine and Ghana). Other frequent source types are social media (9% on average) and cyclopedias (11%, almost exclusively Wikipedia). It is striking that the occurrence of links to these two source types correlates negatively (r = −0.63) – in Romania, Hungary and Serbia, for example, the results contain fewer links to Wikipedia than in other countries, but an above-average proportion of search results for CTs refer to social media sites. These social media links often point to blog accounts (in Romania) or alternative media (in Serbia and Hungary).

For most countries, the frequent links provided by mainstream media contain little to no conspiracy confirming content (5% on average). Only in Ukraine and Serbia, mainstream media provide up to 12% CT confirming links. The main channel for CTs in Google search results in most countries is social media: More than a third of social media links confirm the CTs, only in Austria, Germany and Ghana this share is at a low 12%; in Serbia, Romania and Italy more than half of all social media links confirm the CT. Though the share of CT confirming content is on average even higher for alternative media (64%!), these links are significantly less frequent than links to social media in most countries (except Serbia).

In a similar manner, e-commerce contents (e.g. books or merchandise on CTs offered on online marketplaces such as Amazon) can contain up to 36% CT confirming content, but only make up a very small share of Google search results. By contrast, the links provided by public service media contain no CT confirming links at all, and fact-checking websites, academic websites, as well as government websites only do this in very few countries – in the latter case this only happens in Ukraine, Ghana and Italy (see Figure 2).

Figure 2.

Box plot of share of CT confirming results by source types.

To compare the relative importance of our possible predictors (CT topics, countries or source types) for the chance of a search result confirming the CT, we conducted a dominance analysis of the binary logistic regression model (with CT confirming links as our dichotomous dependent variable). According to this analysis, the source type explains 67% of the Pseudo R² (0.37), followed by CT topic (21%) and country (12%). No matter which country or CT topic, it is thus the type of source the search results link to which predicts most accurately whether the CT is confirmed or not (see supplement for full model specification). This would imply that for content moderation, privileging certain reliable source types might already be an effective strategy.

RQ2: How does the quality of search results on CTs correlate with structural characteristics in the respective country?

To explore the potential role of structural characteristics in affecting the quality of search results, we first analysed the correlations (see supplement and Figure 3): Of these, only the Democracy Index (r = −0.68, p = .015), the Corruption Perception Index (r = −0.65, p = .021) and the Press Freedom Index (r = −0.64, p = .025) are statistically significant, implying that the more a country is classified as democratic, the less it is perceived as corrupt, the more freedom its journalists enjoy, the lower is the share of CT confirming content its citizens encounter in search results. The correlations with other factors are not strong enough to cross the threshold for significance, but a higher GDP per capita (r = −0.42, p = .18) does appear to lead to less CT confirming content, as does higher trust in most used (public service) media (r = −0.3, p = .35) and less societal polarisation (r = −0.12, p = .72).

Figure 3.

Correlations between share of CT confirming content and contextual factors.

However, there are also rather strong correlations between the different structural characteristics themselves (see supplement), making it impossible to pinpoint the impact of each factor. Instead, we used the factors to classify our countries using hierarchical clustering into different groups. This then allows us to explore whether countries in clusters characterised by structural deficits are also afflicted by high shares of conspiracy content in search results.

The cluster analysis identifies four country clusters, each consisting of three countries. Starting with the cluster containing the most structurally privileged countries, that is, Australia, Germany and Switzerland, it becomes clear that their users also benefit from being shown the lowest share of CT confirming content in their search results (Figure 4).

Figure 4.

Hierarchical clusters of countries based on structural characteristics (dendrogram) and their share of CT confirming search results (barplot with 95% confidence intervals).

For the remaining clusters of somewhat less privileged countries, the patterns are not as clear. There is a second cluster of countries benefitting economically from their membership in the EU, with trusted public service media, and rather little societal polarisation, that is, Italy, Romania and the Czech Republic. But in this cluster the perceived corruption is substantially higher than in the first cluster, and so is the share of CT confirming links, particularly in Romania. The third cluster combines Hungary, Serbia and Ukraine, countries plagued by high societal polarisation, high perceived corruption, with the lowest press freedom as well as the lowest trust in media. And yet, the share of CT confirming links is comparable to the second cluster, with the notable exception of Serbia which stands out with a share of 30%. By contrast the final cluster of countries (Ghana, Brazil and Poland) is also characterised by a rather low GDP per capita, high societal polarisation and high perceived corruption, but the share of CT confirming links is comparatively low. In other words, even though the countries in each of these three clusters share structural deficits with each other, there is no consistent pattern in the share of CT confirming links. In particular, Brazil and Ghana appear to contradict the assumption of the digital divide as these two countries with strong structural deficits still have a rather low share of CT confirming links. One possible explanation might be that in both countries the primary language is a world language (Portuguese and English). They thus might benefit from the fact that a) in general there is more quality (i.e. CT debunking) content available in their language from more structurally privileged countries sharing the respective language and b) that the search engine may invest more in content moderation given the other strong markets in that language.

Discussion

Based on our results, the initial question of our paper – does Google reinforce the digital divide? – can be answered in the affirmative, we ascertained a connection between categorical inequalities and unequal access to quality information (Van Dijk, 2017): In countries where people enjoy fewer democratic rights, are more likely to live in poverty, less able to rely on social elites, and lack trustworthy, free media, Google provides less enlightening content on conspiracy theories than in developed and prosperous democracies. As countries thus disadvantaged are also precisely those where there is a high propensity to believe in CTs (Cordonier et al., 2021), Google is contributing to a vicious cycle by catering to the increased demand for CT-affirming content in countries with a high proclivity towards conspiracy belief.

And yet, looking more closely at these less advantaged countries, the impact of contextual factors on the quality of search results is less consistent than the framework on disinformation resilience by Humprecht et al. (2020) suggested: High trust in media does not preclude a high share of CT confirming search results (e.g. in Romania), neither does low trust necessarily lead to low quality information (e.g. in Poland). Instead, the journalists’ freedom to report independent of political and economic influences emerges as a better predictor of information quality. In a similar manner, countries with similar levels of (high) societal polarisation or (low) democratic quality show a wide range of CT confirming link shares. Only the perceived corruption index suggested by Cordonier et al. (2021) consistently predicts link quality, whereas the economic stability does not.

There are two possible explanations for these inconsistencies: First, CEE countries such as Serbia, the Czech Republic, Romania and Ukraine stand out with high shares of CT confirming content, but also by their high proportion of conspiracy believers (e.g. Hajdu et al., 2021). This peculiarity has been linked to the tradition of anti-Western, anti-liberal CTs used as propaganda against Western countries during the Communist era (Plenta, 2020) as well as Russian digital information warfare targeting CEE countries (Štětka et al., 2021; Mujanović, 2019). The lingering aftereffects of historic geopolitical factors may thus be obscuring the impact of other factors.

Second, language seems to be a strong mitigating factor: Users in countries whose official language is common in many other countries have a better chance of finding quality content when they search for CTs, as, for example, people searching for CTs in English in Ghana. There is no evidence that the digital divide is being exacerbated by Google here, either because selection algorithms work more reliably the larger their data base or because language data models are better developed than for smaller (low-resource) languages (Caswell and Liang, 2020).

Our analysis of dominance has clearly identified source type as the most important predictor for the prevalence of CTs: Social media, alternative media and e-commerce sites in particular tend to convey conspiracy content whereas traditional media sites and PSM, government and scientific organisations are the ones that tend to educate about CTs. Especially social media appear to represent an important opportunity structure for ‘news hackers’ or ‘conspiracy entrepreneurs’, spreading CTs for their own political or economic goals. However, we also found much fewer social media sources and blogs in the search results for CTs than Ballatore (2015) which may explain why his results for English-language keywords contained significantly more conspiratorial content (for Google, on average 48%) than our English-language results (Australia 5%, Ghana 12%). This may indicate a positive development with regard to conspiracy content, that is, that Google has adjusted its algorithm for search result optimisation to prioritise large websites, though not all countries in our sample have profited equally.

And finally, we identified striking differences between different CTs: Whereas for QAnon no confirmatory results could be found in any of our countries, for Depopulation Agenda (and Reptilians) users saw confirmatory links in almost all countries, pointing towards the problematic consequences of right-wing populists promoting a conspiracy theory in the public arena.

The study is limited in several respects: The study by Mahl et al. (2021), which we used as a starting point for our selection of CTs, was based on a purposive sample of English-language Twitter accounts. It is unclear whether the selected CTs limit the comparability of our results, as they may be better known in some (particularly English-speaking) countries than in others. And yet, it is striking that even a search for a very country-specific CT such as Pizzagate yields search results in all countries (and languages!). At the same time, we cannot rule out that appropriate queries on CTs were partially lost in translation into different language search terms. Future studies could choose strategies to circumvent this potential bias, such as formulating specific questions rather than using individual search terms.

In addition, we examined only Google – which in some countries has notable competitors – and only Google’s organic search results, not suggested videos or news. It would also be interesting to explore the role of features such as autocomplete or suggested searches. In addition, other platforms, such as YouTube, play an important role in the dissemination of CTs.

Unfortunately, the structural inequalities at the heart of the digital divide and van Dijk ‘resources and appropriation theory’ (2017) are also mirrored in the research literature and made it difficult to derive clear theoretical assumptions about all countries in our sample, let alone access the data on structural characteristics necessary to systematically test them. Our study was thus restricted to an exploratory analysis of the potential impact of media, political and economic factors. And yet, we believe it was worthwhile exploring the world of Google beyond those countries traditionally in the focus of Western communication science research and showing, once again, that promising conceptual frameworks (such as Humprecht et al., 2020) come to their limits as soon as you move a few kilometres to the East or South. Based on our analysis, we would argue that future research on online disinformation and conspiracy theories needs to systematically include measures of press freedom (and not just quality of public service media), language, culture as well as geopolitical ties as factors contributing to the quality of information available to users.

In view of global crises that can only be addressed by supranational coordinated action – such as climate change or the COVID-19 pandemic – it is in the international interest to counter epistemic fragmentation along national and linguistic borders. Globally operating information intermediaries have a key role to play, which has received little attention to date. Our paper thus contributes to the question of whether platforms add a new layer to the digital divide, as they provide different countries with different access to information, reinforcing structural differences. Using the example of Google search results for well-known conspiracy theories, we were able to show that this danger is real. In most of the countries in our study where we suspected more strongly conspiracy-prone mindsets, the likelihood of finding CT content was particularly high: In these countries, users looking to find information about CTs on Google are more likely to be led down a ‘rabbit hole’. It is striking that Google does not live up to its role as a “societal infrastructure” (van Dijck et al., 2019: 10) especially in these countries, which are not in the (West-centric) focus of a regulatory debate. It is hoped that our findings will reinvigorate the debate on search engine regulation and an algorithmic “firewalling” of Google Search (van Dijck et al., 2019: 10) with a more international perspective.

Supplemental Material

Supplemental Material - The different worlds of Google – A comparison of search results on conspiracy theories in 12 countries

Supplemental Material for The different worlds of Google – A comparison of search results on conspiracy theories in 12 countries by Gerret von Nordheim, Tina Bettels-Schwabbauer, Katharina Kleinen-von Königslöw, Paulina Barczyszyn-Madziarz, Halyna Budivska, Philip Di Salvo, Filip Dingerkus, Liziane Soares Guazina, Kwaku Krobea Asante, Michał Kuś, Sandra Lábová, Antonia Matei, Norbert Merkovity, Fernando Oliveira Paulino, László Petrovszki-Oláh, Michael Yao Wodui Serwornoo, Jonas Valente, Alexandra Wake, Viktória Zakinszky Toma in Convergence.

Footnotes

Acknowledgements

We would like to thank Gordon Farrer at RMIT University for his assistance with double coding the Australian data.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Brazilian part of this study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de NivelSuperior—Brasil (CAPES) under the project “Communication and democracy: mediaaccountability, public service media, internet access and the right to informationin Germany and Brazil”, supported by the Probral, Brazilian-German academiccooperation program CAPES/DAAD (Grant 88887.371422/2019-00)

ORCID iDs

Gerret von Nordheim

Tina Bettels-Schwabbauer

Katharina Kleinen-von Königslöw

Paulina Barczyszyn-Madziarz

Philip Di Salvo

Liziane Soares Guazina

Michał Kuś

Sandra Lábová

Norbert Merkovity

László Petrovszki-Oláh

Michael Yao Wodui Serwornoo

Alexandra Wake

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Allington

Duffy

Wessely

, et al. (2021) Health-protective behaviour, social media usage and conspiracy belief during the COVID-19 public health emergency. Psychological Medicine 51(10): 1763–1769. DOI: 10.1017/S003329172000224X.

Ballatore

(2015) Google chemtrails: a methodology to analyze topic representation in search engine results. First Monday 20(7):1–20. DOI: 10.5210/fm.v20i7.5597.

Barkun

(2003) A Culture of Conspiracy: Apocalyptic Visions in Contemporary America. 1st edition. CA, USA: University of California Press. Available at: https://www.jstor.org/stable/10.1525/j.ctt1pnjvz (accessed 11 September 2021).

Bentley

Quehl

Wirfs-Brock

, et al. (2019) Understanding online news behaviors. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow Scotland Uk, 2 May 2019, pp. 1–11. ACM. DOI: 10.1145/3290605.3300820.

boyd

(2018) You Think You Want Media Literacy… Do You? Available at: https://points.datasociety.net/you-think-you-want-media-literacy-do-you-7cad6af18ec2

Brotherton

Eser

(2015) Bored to fears: boredom proneness, paranoia, and conspiracy theories. Personality and Individual Differences 80: 1–5. DOI: 10.1016/j.paid.2015.02.011.

Campion-Vincent

(2015) Remarks on conspiracy theory entrepreneurs. Diogenes 62(3-4): 64–70. DOI: 10.1177/0392192120945606.

Castro Herrero

Humprecht

Engesser

, et al. (2017) Rethinking hallin and mancini beyond the West: an analysis of media systems in central and eastern Europe. International Journal of Communication 11: 4797–4823.

Caswell

Liang

(2020) Recent advances in Google translate In: Google AI Blog. Available at: https://ai.googleblog.com/2020/06/recent-advances-in-google-translate.html

10.

Coppedge

Gerring

Knutsen

, et al. (2019) V-dem codebook v9. Varieties of Democracy (V-dem) Project. Available at: https://www.v-dem.net/en/data/archive/previous-reference-materials/reference-materials-v9/

11.

Cordonier

Cafiero

Bronner

(2021) Why are conspiracy theories more successful in some countries than in others? An exploratory study on Internet users from 22 Western and non-Western countries. Social Science Information 60(3): 436–456. DOI: 10.1177/05390184211018961.

12.

Craft

Ashley

Maksl

(2017) News media literacy and conspiracy theory endorsement. Communication and the Public SAGE Publications 2(4): 388–401. DOI: 10.1177/2057047317725539.

13.

Diaz

(2008) Through the Google goggles: sociopolitical bias in search engine design. In: Spink

Zimmer

(eds), Web Search. Information Science and Knowledge Management. Berlin, Heidelberg: Springer, 11–34. DOI: 10.1007/978-3-540-75829-7_2.

14.

Dobek-Ostrowska

(2019) How the media systems work in central and eastern Europe. In: Polonska-Kimunguyi

Beckett

(eds), Public Service Broadcasting and Media Systems in Troubled European Democracies. London, UK. Palgrave Macmillan, 259–278. DOI: 10.1007/978-3-030-02710-0_12.

15.

Donovan

Friedberg

(2019). Source Hacking-Media Manipulation in Practice. Data & Society’s Media Manipulation research initiative, 4 September. Available at: https://datasociety.net/wp-content/uploads/2019/09/Source-Hacking_Hi-res.pdf (accessed 29 June 2020).

16.

Freedom House (2022) Freedom in the World 2022. Online Report. Available at: https://freedomhouse.org/sites/default/files/2022-02/FIW_2022_PDF_Booklet_Digital_Final_Web.pdf

17.

Freeman

Waite

Rosebrock

, et al. (2020) Coronavirus conspiracy beliefs, mistrust, and compliance with government guidelines in England. Psychological Medicine 52(2): 251–263. DOI: 10.1017/S0033291720001890.

18.

Golebiewski

boyd

(2018) Data voids: where missing data can easily Be exploited. Data & Society. Available at: https://datasociety.net/output/data-voids-where-missing-data-can-easily-be-exploited/ (accessed 11 February 2019).

19.

Grind

Schechner

McMillan

, et al. (2019) How Google Interferes With Its Search Algorithms and Changes Your Results. Wall Street Journal. Available at: https://www.wsj.com/articles/how-google-interferes-with-its-search-algorithms-and-changes-your-results-11573823753 (accessed 11 September 2021).

20.

Hajdu

Klingová

Sawiris

, et al. (2021) GLOBSEC Trends 2021: Central and Eastern Europe One Year into the Pandemic. GLOBSEC. Available at: https://www.globsec.org/wp-content/uploads/2021/06/GLOBSEC-Trends-2021_final.pdf

21.

Holt

Ustad Figenschou

Frischlich

(2019) Key dimensions of alternative news media. Digital Journalism 7(7): 860–869. DOI: 10.1080/21670811.2019.1625715.

22.

Humprecht

Esser

Van Aelst

(2020) Resilience to online disinformation: a framework for cross-national comparative research. The International Journal of Press/Politics 25(3): 493–516. DOI: 10.1177/1940161219900126.

23.

IMF (2021) GDP per Capita, Current Prices. Washington DC, USA: International Monetary Fund. Available at: https://www.imf.org/external/datamapper/NGDPDPC@WEO/OEMDC/ADVEC/WEOWORLD

24.

Internews (2021) USAID-internews 2021 Media Consumption Survey Available at: https://imi.org.ua/upload/media/2021/11/17/61951144dc1fc-usaid-internews-media-report-2021-ukr.pdf

25.

IPSOS (2021) Trust in the Media - Serbia. Paris, France: Ipsos strategic marketing. Available at: https://seenpm.org/wp-content/uploads/2021/09/Trust-in-the-media-Serbia_technical-report.pdf

26.

Jansen

Spink

(2006) How are we searching the World Wide Web? A comparison of nine search engine transaction logs. Information Processing & Management 42(1): 248–263. DOI: 10.1016/j.ipm.2004.10.007.

27.

Kapantai

Christopoulou

Berberidis

, et al. (2021) A systematic literature review on disinformation: toward a unified taxonomical framework. New Media & Society 23(5): 1301–1326. DOI: 10.1177/1461444820959296.

28.

Mahl

Zeng

Schäfer

(2021) From “nasa lies” to “reptilian eyes”: mapping communication about 10 conspiracy theories, their communities, and main propagators on twitter. Social Media + Society 7(2): 205630512110174. DOI: 10.1177/20563051211017482.

29.

Mujanović

(2019) Russia’s toxification of civil society in the western balkans. Sudosteuropa Mitteilungen 02: 49–56. https://www.ceeol.com/search/article-detail?id=791303

30.

Napoli

(2021) The platform beat: algorithmic watchdogs in the disinformation age. European Journal of Communication 36(4): 376–390. DOI: 10.1177/02673231211028359.

31.

Newman

(2021) Reuters Institute Digital News Report 2021. Oxford: Reuters Institute for the Study of Journalism. Available at: http://www.digitalnewsreport.org/

32.

Pasek

Stark

Krosnick

, et al. (2015) What motivates a conspiracy theory? Birther beliefs, partisanship, liberal-conservative ideology, and anti-Black attitudes. Electoral Studies 40: 482–489. DOI: 10.1016/j.electstud.2014.09.009.

33.

Plenta

(2020) Conspiracy theories as a political instrument: utilization of anti-Soros narratives in Central Europe. Contemporary Politics 26(5): 512–530. DOI: 10.1080/13569775.2020.1781332.

34.

Ragnedda

Kreitem

(2018) The three levels of digital divide in East EU countries. World of Media. Journal of Russian Media and Journalism Studies 1(4): 5–26. DOI: 10.30547/worldofmedia.4.2018.1.

35.

Rieder

Sire

(2014) Conflicts of Interest and Incentives to Bias: a microeconomic critique of Google’s tangled position on the Web. New Media & Society 16(2): 195–211. DOI: 10.1177/1461444813481195.

36.

Scherr

Haim

Arendt

(2019) Equal access to online information? Google’s suicide-prevention disparities may amplify a global digital divide. New Media & Society 21(3): 562–582. DOI: 10.1177/1461444818801010.

37.

Segev

(2010) Google and the Digital Divide: The Bias of Online Knowledge Chandos Internet Series. Oxford: Chandos Pub.

38.

Starbird

(2017) Examining the alternative media ecosystem through the production of alternative narratives of mass shooting events on Twitter. Proceedings of the International AAAI Conference on Web and Social Media 11(1): 230–239.

39.

Štětka

Mazák

Vochocová

(2021) “Nobody tells us what to write about”: the disinformation media ecosystem and its consumers in the Czech republic. Javnost/The Public 28(1): 90–109. DOI: 10.1080/13183222.2020.1841381.

40.

The Economist Intelligence Unit (2021) Democracy Index 2020. In sickness and in health? Available at: https://www.eiu.com/n/campaigns/democracy-index-2020/

41.

Toff

Nielsen

(2018) “I just Google it”: folk theories of distributed discovery. Journal of Communication 68(3): 636–657. DOI: 10.1093/joc/jqy009.

42.

Transparency International (2020) Corruption perception index 2020. Available at: https://images.transparencycdn.org/images/CPI2020_Report_EN_0802-WEB-1_2021-02-08-103053.pdf

43.

Trielli

Diakopoulos

(2020) Partisan Search Behavior and Google Results in the 2018 U.S. Midterm Elections. Information, Communication & Society, 1–17. DOI: 10.1080/1369118X.2020.1764605.

44.

Van der Linden

(2015) The conspiracy-effect: Exposure to conspiracy theories (about global warming) decreases pro-social behavior and science acceptance. Personality and Individual Differences 87: 171–173. DOI: 10.1016/j.paid.2015.07.045.

45.

Van Dijck

Nieborg

Poell

(2019) Reframing platform power. Internet Policy Review 8(2): 1–18. DOI: 10.14763/2019.2.1414.

46.

Van Dijk

JAGM

(2017) Digital divide: impact of access. In: Rössler

Hoffner

Zoonen

(eds), The International Encyclopedia of Media Effects. 1st edition. Hoboken, NJ, USA: Wiley, 1–11. DOI: 10.1002/9781118783764.

47.

von Nordheim

Kleinen-von Königslöw

(2021) Uninvited dinner guests: a theoretical perspective on the antagonists of journalism based on serres’ parasite. Media and Communication 9(1): 88–98. DOI: 10.17645/mac.v9i1.3419.

48.

Warner

Neville-Shepard

(2014) Echoes of a conspiracy: birthers, truthers, and the cultivation of extremism. Communication Quarterly 62(1): 1–17. DOI: 10.1080/01463373.2013.822407.

49.

Xiao

Borah

(2021) The dangers of blind trust: examining the interplay among social media news use, misinformation identification, and news trust on conspiracy beliefs. Public Understanding of Science 30: 096366252199802. DOI: 10.1177/0963662521998025.

50.

Yin

(2014) Case Study Research: Design and Methods. Thousand Oaks: Sage Publications.

51.

Zupork Dome

Adu Duayeden

Armah-Attoh

(2020) Afrobarometer Round 8. Survey in Ghana, 2019. Available at: https://www.afrobarometer.org/publication/summary-results-afrobarometer-round-8-survey-ghana-2019-0/

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

3.46 MB