The spread of COVID-19 vaccine information in Arabic on YouTube: A network exposure study

Abstract

Objective

The Arabic-speaking world had the lowest vaccine rates worldwide. The region's increasing reliance on social media as a source of COVID-19 information coupled with the increasing popularity of YouTube in the Middle East and North Africa region begs the question of what COVID-19 vaccine content is available in Arabic on YouTube. Given the platform's reputation for being a hotbed for vaccine-related misinformation in English, this study explored the COVID-19 vaccine-related content an individual is likely to be exposed to on YouTube when using keyword-based search or redirected to YouTube from another platform from an anti-vaccine seed video in Arabic.

Methods

Only using the Arabic language, four networks of videos based on YouTube's recommendations were created in April 2021. Two search networks were created based on Arabic pro-vaccine and anti-vaccine keywords, and two seed networks were created from conspiracy theory and anti-vaccine expert seed videos. The network exposure model was used to examine the video contents and network structures.

Results

Results show that users had a low chance of being exposed to anti-vaccine content in Arabic compared to the results of a previous study of YouTube content in English. Of the four networks, only the anti-vaccine expert network had a significant likelihood of exposing the user to more anti-vaccine videos. Implications were discussed.

Conclusion

YouTube deserves credit for its efforts to clean up and limit anti-vaccine content exposure in Arabic on its platform, but continuous evaluations of the algorithm functionality are warranted.

Keywords

YouTube Arabic vaccine hesitancy vaccine acceptance MENA network exposure analysis

The outcome of the current global effort to curb the spread of the COVID-19 pandemic depends not only on the vaccines’ efficacy and safety but also on the vaccines’ acceptance among the general public. Vaccine hesitancy, defined as “a delay in acceptance, or refusal of vaccines, despite their availability”(¹, p. 4163) was among the top 10 threats to global health identified by the World Health Organization (WHO) in 2019.² Although vaccine hesitancy is not a new phenomenon, the novelty of the new Coronavirus combined with the spread of large amounts of false and misleading information (e.g. COVID-19 infodemic) contributed to the flourishing of anti-vaccine movements,^3,4 further hindering the global vaccination efforts to end the pandemic.^5–6

Studies examining the public's vaccine acceptance rates indicated that the Arabic-speaking countries in the Middle East and North Africa (MENA) region had some of the lowest reported COVID-19 vaccine acceptance rates worldwide.^7,8 Given the Arab world's unique characteristics—the unequal distribution of wealth among the region's countries, the lack of sociopolitical stability in a number of countries, and the absence of media transparency revolving around COVID-19 infection and death reporting—there have been uneven responses to the COVID-19 pandemic and different levels of vaccine hesitancy among its public.^9,10

Such variations in vaccine hesitancy are further highlighted by the Arab countries’ vaccination coverage.¹¹ In August 2021, only 12% of the region's population received at least one dose of the vaccine.¹² Although wealthier Gulf countries were more successful in their vaccination efforts (e.g. Qatar: 62%, the UAE: 75%, and Bahrain: 63%), and Jordan and Morocco had respective vaccination rates of 25% and 27%, no other country was able to administer at least one dose of the vaccine to more than 15% of its population in 2021.¹² A cross-sectional study¹³ found that by the end of October 2022, the mean vaccination coverage rate (i.e. the percentage of people who had completed the initial two doses) was 43.8% in the Arab countries. By November 2022, Gulf countries topped the list of Arab countries in terms of vaccination coverage (e.g. Qatar: 105.7%, UAE:103.7%, and Bahrain: 83.3%)¹. In Tunisia and Morocco, more than 50% of their population had been fully vaccinated. Yet, other countries (e.g. Libya, Iraq, Algeria, Sudan, Syria, and Yemen) had less than 20% of their population vaccinated.¹³ The increase in vaccination rates in many Arab countries is largely due to the vaccine mandates imposed by the governments once the COVID-19 vaccines became available in their country in an effort to raise the vaccination rate and reach herd immunity.¹⁴ See Appendix 1 for 2023 vaccination rates.

Factors contributing to vaccine hesitancy have been extensively studied in different regions,^15–16 yet, less attention has been given to this topic in the Arab world^8,17–18 until recently.^19,20 Research on vaccine hesitancy has often shown that the exposure to misleading information on social media can increase hesitancy^21,22 and that social media is disproportionately successful in promoting vaccine hesitancy as opposed to vaccine uptake.²³ During the COVID-19 pandemic, a noticeable shift in the Arab countries’ media consumption patterns was observed, indicating a heavier reliance on social media as the primary COVID-19 and COVID-19 vaccines source of information rather than traditional media.^24–26 Surveys gauging the Arab public's understanding of COVID-19 indicated differences in the trust and confidence of the information on social media, which differs based on the platform.²⁷ However, such studies do not directly examine the type of COVID-19 content that is deemed untrustworthy by individuals or the likelihood of exposure to misleading information about COVID-19.

Although non-English COVID-19 disinformation and misinformation research on different social media platforms has started to gain momentum, research on the Arabic language continues to lag behind. Specifically, researchers have yet to explore COVID-19 information dissemination on YouTube in Arabic. YouTube—one of the main sources of English-language vaccine misinformation^28,29—has recently gained popularity in the Arab world, upstaging the region's long-time leader of social media platforms—Facebook.³⁰ In fact, Saudi Arabia has the largest number of YouTube users per capita worldwide.³¹ Arabs indicated that they lacked confidence in YouTube, with 67.03% of respondents to a survey believing that YouTube somewhat spreads COVID-19-related rumors.²⁷ Yet very little is known about the role the platform's recommendation algorithm plays in the dissemination and spread of content that might have contributed to vaccine hesitancy and propagated vaccine mistrust among the Arab public.

Thus, given the current global pandemic, the Arab public's increased dependency on social media during the COVID-19 pandemic, and YouTube's anti-vaccine reputation,³² this study aims to understand how YouTube is being utilized to disseminate COVID-19 vaccine-related content in Arabic. More specifically, we examine the extent to which viewers are exposed to COVID-19 vaccine-related information on YouTube and investigate YouTube's search and recommendation algorithm in Arabic.

Vaccine information on social media

Earlier studies highlighted that Internet-based searches were likely to yield anti-vaccine messages³³ and that social media platforms often host anti-vaccine information,²¹ which are sometimes used by hostile entities to strategically spread health and vaccine misinformation.³⁴ Specific platforms, such as Twitter, were weaponized to promote anti-vaccine messages using bots and trolls.³⁵ During the pandemic, the #FilmYourHospital was used to spread a conspiracy theory that the pandemic was a hoax and that hospitals were empty. The hashtag gained international reach, spreading the conspiracy worldwide (not only restricted to English or Portuguese engagement), with Arabic being the second largest non-English cluster of tweets.³⁶ Focusing more closely on Twitter discussions in Arabic, one study³⁷ found that Arabic tweets captured over the first 12 weeks since the COVID-19 outbreak were likely to evoke conspiracist ideation about the spread of COVID-19, especially when China and the USA were mentioned. Another content analysis study of tweets originating from Saudi Arabia identified three main topics: conspiracy theories, civil liberties, and the violation of freedoms during the pandemic—particularly due to the vaccine mandates.³⁸ Other studies have also used Twitter data to gauge the Arab public's attitude toward COVID-19-related topics.^39–40

Similarly, Facebook was considered another rich platform for seeking and sharing health-related information that facilitates sustained anti-vaccination movements with global reach.⁴¹ More specifically, a report by the Institute of Strategic Dialogue revealed that Arabic-language COVID-19 misinformation communities on Facebook have grown, identifying Facebook pages responsible for amplifying COVID-19 misinformation coming from the West.⁴² Furthermore, the report highlighted an imbalance in the effectiveness of health disinformation content moderation between English and Arabic content; videos that have been flagged for fact checks in the West have escaped such scrutiny among Arabic-language Facebook communities.⁴² It was concluded that the exposure to misleading Arabic COVID-19 information on Facebook was easy and that Facebook's moderation of Arabic misinformation is not as effective as it is in English.⁴³

The availability of vaccine misinformation on social media can not only mislead the public but also impact the individual's ability to believe in scientific facts and trust experts.⁴⁴ Given the impact of social media, studies often conclude that exposure to such information about vaccination negatively impacts people's vaccine attitude and acceptance.^45,46 In fact, findings suggested that uncertainty about the safety and effectiveness of vaccines, misbeliefs, and conspiracy theories about vaccines—all factors contributing to vaccine hesitancy among Arabs—were exacerbated by exposure to disinformation on social media.^47,48 Particularly, active Arab social media users were at least two times more likely to be reluctant to receive the COVID-19 vaccine compared to passive users.⁴⁹ In addition, vaccine information shared by an individual's online social network/ contacts (e.g. family, friends, etc.) impacts their trust and attitudes in favor of COVID-19 vaccination.⁴⁹ Thus, the recommendation provided by such studies is to thoroughly monitor and control the spread of misinformation on social media.⁴⁸

COVID-19 vaccine information on YouTube

Although YouTube is the most popular online video platform worldwide with an estimated 2.1 billion users,⁵⁰ it is also one of the leading sources of health-related misinformation, particularly misinformation about vaccines.³² Early research focused on the platform's vaccination content revealed that approximately half of the vaccine-related videos on YouTube were not explicitly supportive of vaccines.⁵¹ More recently, of the 87 most viewed videos related to vaccines using the keywords “vaccine safety” and “vaccine and children,” the majority of videos (65.5%) discouraged vaccination.²⁸ Similarly, more than two-thirds of the YouTube videos obtained using vaccine-related keyword searches on the platform—133 of the 196 videos examined in the study—contained some type of unreliable information regarding vaccine safety and effectiveness.⁵²

There is evidence that anti-vaccine content on YouTube has decreased in recent years due to YouTube's content regulation. Of the 20 most-watched videos on YouTube with vaccine content,⁵³ found that six (30%) were pro-vaccine, nine (45%) were vaccine-supportive, three (15%) were vaccine-skeptical, and two (10%) were anti-vaccine. Moreover, Abul-Fottouh et al.⁵⁴ found that there were more pro-vaccine videos (64.75%) than anti-vaccine videos (19.98%), and the rest were neutral in sentiment. While studies suggested that YouTube has been taking firm steps to contain the viewer's exposure to anti-vaccine videos and health misinformation,⁵⁴ such studies were solely conducted in English, and few, if any is known about how YouTube's recommendation algorithm works when searches are done in languages other than English. Thus, it remains unclear whether the measure to control the spread of vaccine misinformation is reflected in other languages, particularly in Arabic.

YouTube’s recommendation algorithm

The recommendation algorithms can be considered a mechanism through which information is managed and disseminated to individuals on social media. Given the unprecedented amount of information, an individual can access on social media platforms; the recommendation algorithms determine what information is presented to individuals and entire communities.⁵⁵ It is mainly driven by the personalization of information; the algorithm aggregates data from a user with data from his/her network of friends/family to suggest content based on shared interests.⁵⁶ Filtering information by prioritizing or hiding certain content optimizes the experience of the individual on the platform⁵⁷; however, it can also facilitate both the exposure to problematic content⁵⁸ and the development of echo chambers and filter bubbles.^59–60

There are two ways someone can be exposed to information on YouTube: they can directly seek information by searching the platform using keywords (i.e. goal-oriented browsing), or alternatively, users can be taken to a video on YouTube from another platform (i.e. direct navigation^20,46,47). Regardless of how someone reaches a YouTube video, the platform will automatically recommend a set of videos based on the initial video the individual selected. Prior to COVID-19 pandemic, Tang et al.²⁹ conducted a network analysis examining the type of vaccine content users were recommended and the exposure likelihood to vaccine misinformation based on English anti- and pro-vaccine search terms (e.g. direct browsing on YouTube) as well as seed networks (e.g. when users are redirected to YouTube from another platform and recommended more videos). Their findings indicated that even if English-language users were to watch a pro-vaccine video, they still had a relatively high chance of being recommended an anti-vaccine video. Furthermore, anti-vaccine videos were much more likely to lead to more anti-vaccine videos contributing to the creation of filter bubbles.²⁹ To explore the patterns of exposure to anti-vaccine information in the Arabic language on a macro level, we propose our first research question (RQ1):

RQ1: When YouTube users start their viewing with keyword-based search (pro-vaccine and anti-vaccine) or anti-vaccine seed videos (conspiracy theories and anti-vaccine experts) in the Arabic language, to what extent will they be exposed to pro-vaccine, anti-vaccine, neutral vaccine, or mixed vaccine content?

Moreover, to explore these patterns on a micro-level, the network exposure model, which is based on the diffusion of innovation theory,⁶¹ can be used to measure the degree to which a node (in this case, a video) in the network is exposed to other nodes with a certain attribute.^62–63 To compute the exposure of a node to a certain attribute, the average edge from a specific node to other nodes that exhibit the attribute is measured. A visual explanation of this computation can be seen in Figure 1.

Figure 1.

Network exposure model computation example. Note. The green videos (A and Z) are the targeted nodes. The grey videos are the unexposed nodes. The blue videos are the exposed nodes. These examples show two nodes (A and Z) exposed to attributes based on their ties. Figure adapted from⁶⁴. NEM: network exposure model.

In the current study, network exposure is an indicator of how likely one type of video (pro-vaccine, anti-vaccine, neutral, or mixed) leads to different types of additional videos with a particular attribute (pro-vaccine, anti-vaccine, neutral, or mixed content). Since we were interested in the exposure to anti-vaccine videos, we asked RQ2:

RQ2: What is the degree of exposure of pro-vaccine, anti-vaccine, mixed, neutral videos to anti-vaccine videos?

To address these research questions, a network analysis as well as network exposure analysis of both search networks (goal-oriented browsing using pro- and anti-vaccine search terms) and seed networks (direct navigation from conspiracy theories and anti-vaccine experts) were conducted.

Methods

Data collection

To collect the YouTube videos for the search networks, keywords and short phrases were selected based on the most popular Arabic searches on Google Trends as well as the popular Twitter hashtags during April 2021. For this study, four pro-vaccine key phrases [“اللقاح ينقذ” (vaccine saves), “اللقاحات حصانة لنا” (vaccines protects us), “اللقاحات فعالة” (vaccines work), “اللقاح أمن” (vaccine is safe)], and four anti-vaccine phrases [“اللقاح يقتل” (vaccine kills), “اللقاح فاشل “ (vaccines are a failure), “اللقاح كذبة “ (vaccines are a lie), “شرعاً اللقاح حرام” (vaccines are religiously forbidden)] were used. To collect the videos for the seed networks (both the conspiracy theory network and anti-vaccine expert seed network), a snowball sampling was conducted, and videos were collected from social media accounts circulating anti-vaccine content and from community members who received such videos.

The recommendation networks from YouTube videos were retrieved and created using CAS²T.⁶⁵ This open-sourced program is linked to the YouTube application programming interface,⁶⁶ which allows it to retrieve each video's recommended videos without it being affected by the user's watch history or region. Thus, videos collected through CAS²T are only retrieved based on YouTube's recommendation algorithm. Once the videos were retrieved, CAS²T automatically created a file with the video ID and its recommended videos, as well as the metadata for each video (number of likes, number of views, channel information, etc.). The URLs of the first six videos for each of the pro- and anti-vaccine search terms were captured and then collected four depth levels of related videos. Based on Tang et al.,²⁹ the decision to select six videos for four layers was based on both considerations of the typical screen size factor (computer and phone screens show a limited number of videos) and the assumption that individuals will not look for more than the first few results.

Annotation

The videos were annotated based on (a) whether the video was vaccine-related, (b) if it was vaccine-related, whether it was pro-vaccine, anti-vaccine, contained mixed messages, or neutral vaccine content. Videos were coded as pro-vaccine when the content exclusively and explicitly encouraged the viewer to get the vaccine and provided no doubt about the vaccine's efficacy. Videos were coded neutral when the content simply focused on the scientific background of the vaccine without any input or encouragement to take the vaccine (simply how vaccines work). Videos were coded mixed when the contradicting content was detected or when different viewpoints were presented (both encouraging and discouraging vaccines). Videos were coded as anti-vaccine when the content explicitly discouraged viewers from taking the vaccine or presented doubts about the vaccine efficacy. If the videos were not vaccine-related, they were coded on whether (a) they were health-related or (b) whether they were autism-related. In addition, all videos were also coded for (a) whether they were COVID-19 related or not, (b) who uploaded the video (personal, professional, or celebrity), (c) who made the video based on Briones et al.'s⁶⁷ code which included “governmental agencies,” “nonprofit/academic organizations,” “pharmaceutical/for-profit companies,” “consumer-generated content,” “news sources,” “ advocacy groups,” “medical centers/hospitals, professional associations,” and “other,” and (d) the focus of the video (media reporting, religious, political/economy, scientific/research/informative, or advertisement/entertainment).

For the search network, the pro-vaccine search terms retrieved a total of 319 videos and the anti-vaccine search terms retrieved a total of 290 videos. In terms of the seed networks, a total of 214 videos were retrieved from the conspiracy theory seed videos, and a total of 144 videos were retrieved from the anti-vaccine expert seed videos. Live videos, unavailable videos (e.g. private or removed by YouTube), as well as videos in languages other than Arabic were excluded from the analysis and labeled as invalid. The videos included in this study were all coded by five coders who are fluent in Arabic. To establish intercoder reliability, 10% of videos in each network were selected through random systematic sampling to be annotated. After resolving some inconsistencies with the coding among the coders, the achieved intercoder reliability was good; Krippendorff α = 0.812. The videos for all networks were then split between the five coders and annotated independently.

Data analysis

CAS²T automatically creates a node list, which contains each video (its unique ID) and its relevant metadata. It also creates an edge list, which provides each video's ID with its recommended video. Both the node list and the edge list were used to generate the four directed networks that were later used for network analysis and network exposure analysis. Gephi was used to compute the network statistics and to generate the visualizations of the two networks.⁶⁸ The four networks were analyzed using a few network metrics to recognize the relationship among the different types of videos within each network.

For each of the four networks, we examined how likely non-vaccine videos, vaccine-related videos (pro-vaccine, neutral vaccine content, mixed vaccine content, and anti-vaccine videos), health-related videos (unrelated to vaccines), and COVID-19-related videos are to be exposed to anti-vaccine videos. NET-EXPO, a Gephi plugin, was used to calculate network exposure.⁶⁹ Other network statistics, such as frequencies and odds ratio, were computed using Stata17.

Results

Four directed networks were generated. Two search networks were based on either pro-vaccine or anti-vaccine search terms. Two seed networks were based on seed videos (conspiracy theory videos or anti-vaccine experts). Each node in the network represented a video and each edge in the network represented a link (a recommendation) relationship between nodes (see Table 1 for the descriptive statistics of the networks). The node size ranged from 144 to 315, and the number of edges ranged from 188 to 570. The average degree was slightly higher in the anti-vaccine search network (1.814) than in the pro-vaccine search network (1.810). For the seed networks, the average degree was much lower (1.479 for the conspiracy theory seed network and 1.306 for the anti-vaccine expert seed network). The average clustering coefficient, which indicated how well connected the “neighborhood” (or clustering) of the node in the network,^70,71 was 0.042 for the pro-vaccine network and 0.041 for the anti-vaccine network. Among the seed networks, it was 0.032 for the conspiracy theory network and 0.017 for the anti-vaccine expert network. This means that nodes in the two seed networks were relatively less connected to other nodes in their respective networks and the two seed networks were also less clustered on a local level.

Table 1.

Descriptive statistics of four networks.

Characteristic	Search networks		Seed networks
Characteristic	Pro-vaccine keyword network	Anti-vaccine keyword network	Conspiracy videos network	Expert anti-vaccine videos network
Network characteristics
Nodes (n)	315	290	214	144
Edges (n)	570	526	318	188
Average degree	1.81	1.814	1.479	1.306
Network diameter (directed)	12	11	8	5
Average clustering coefficient (directed)	0.042	0.041	0.032	0.017
Video type, n (%)^a
Non-vaccine related	101 (60.6)	181 (64.2)	134 (72.4)	95 (70.4)
Pro-vaccine	59 (18.7)	33 (11.7)	10 (5.4)	19 (14.1)
Anti-vaccine	12 (3.8)	5 (1.0)	5 (2.7)	14 (10.4)
Mixed messages	9 (2.9)	9 (3.2)	4 (2.2)	4 (3.0)
Neutral	30 (9.5)	54 (19.1)	32 (17.3)	3 (2.2)
Source of videos, n (%)^b
News media	55 (49.5)	52 (52.0)	27 (52.9)	21 (53.8)
Consumer-generated	50 (45.0)	44 (44.0)	24 (47.1)	18 (46.2)
Government agencies	2 (1.8)	3 (3.0)	0 (0)	0 (0)
Nonprofit/academic organizations	2 (1.8)	0 (0)	0 (0)	0 (0)
Other	2 (1.8)	1 (1.0)	0 (0)	0 (0)
Video context, n (%)^b
Media reporting	37 (33.3)	48 (48.0)	25 (49.0)	7 (17.9)
Religion	1 (0.9)	1 (1.0)	1 (2.0)	1 (2.6)
Political	7 (6.3)	1 (1.0)	(0.0)	3 (7.7)
Research/ scientific	59 (53.2)	45 (45.0)	23 (45.1)	27 (69.2)
Entertainment	7 (6.3)	5 (5.0)	2 (3.9)	1 (0)
Covid-19-related videos, n (%)^a	103 (92.8)	93 (93.0)	70 (37.8)	51 (37.8)

Note. ^aindicates the numbers and percentages that are based on valid videos.

^bindicates the number and percentages that are based on vaccine-related videos.

RQ1 asked whether starting with a pro-vaccine and an anti-vaccine search term or starting from an anti-vaccine seed video (conspiracy theory or anti-vaccine expert) would lead YouTube users to pro-, anti-, mixed, or neutral vaccine contents in Arabic. (Table 1 contains the descriptive statistics of the four networks and Figures 2–5 visualize the networks generated). Overall, around 30–40% of the videos in each of the four networks were related to vaccines. We observed a higher proportion of pro-vaccine videos and a lower percentage of anti-vaccine videos in the two search networks, compared to the two seed networks. In the pro-vaccine search network, the majority of vaccine-related videos were pro-vaccine (18.7%), followed by videos containing neutral vaccine content (9.5%). Only 3.8% of the videos (n = 9) in this network were clearly anti-vaccine. In the anti-vaccine search network, the majority of retrieved videos related to vaccines (19.1%) were neutral, followed by videos that were pro-vaccine (11.7%). Only five videos (1%) were clearly anti-vaccine (See Table 1). In the conspiracy theory seed network, the majority of the retrieved videos were neutral (17.3%), and in the anti-vaccine expert seed network, the majority of retrieved videos were pro-vaccine (14.1%). Anti-vaccine videos made up 5% and 14% of these two seed networks.

Figure 2.

Network visualization of the anti-vaccine search. Note. Each node represents a video and each edge represents a link between videos (recommendation). Dark Green represents pro-vaccine videos. Light Green represents neutral videos. Red represents anti-vaccine videos. Pink represents mixed views on vaccines. Gray represents non-vaccine-related videos. The size of nodes is based on the exposure.

Figure 3.

Network visualization of the pro-vaccine search. Note. Each node represents a video, and each edge represents a link between videos (recommendation). Dark green represents pro-vaccine videos. Light green represents neutral videos. Red represents anti-vaccine videos. Pink represents mixed views on vaccines. Gray represents non-vaccine-related videos. The size of nodes is based on the exposure.

Figure 4.

Network visualization of the conspiracy seed network. Note. Each node represents a video, and each edge represents a link between videos (recommendation). Dark green represents pro-vaccine videos. Light green represents neutral videos. Red represents anti-vaccine videos. Pink represents mixed views on vaccines. Gray represents non-vaccine-related videos. The size of nodes is based on the exposure.

Figure 5.

Network visualization of anti-vaccine expert seed network. Note. Each node represents a video, and each edge represents a link between videos (recommendation). Dark green represents pro-vaccine videos. Light green represents neutral videos. Red represents anti-vaccine videos. Pink represents mixed views on vaccines. Gray represents non-vaccine-related videos. The size of nodes is based on the exposure.

In terms of the video sources, in all four networks, the most common source was the news media: pro-vaccine search network (49.5%), anti-vaccine search network (52.0%), conspiracy theory seed network (52.9%), and anti-vaccine expert seed network (53.8%). The second largest group of videos was consumer generated: 45.0% in the pro-vaccine keyword network, 44.0% in the anti-vaccine keyword network, 47.1% in the conspiracy theory seed network, and 46.2% in the anti-vaccine expert seed network. Other sources were not significantly present in YouTube Arabic (See Table 1).

In terms of the video topic, media reporting and research/science were the two most dominant topics. In the pro-vaccine search network, the most common topic was research/science (53.2%), followed by media reporting (33.3%). In the anti-vaccine search network, media reporting (48.0%) was the most common topic, followed by research/science content (45.0%). In the seed networks, the most common topic in the conspiracy theory seed network was also media reporting (49.0%), followed by research/science (45.1%). In the anti-vaccine expert seed network, research/science was the most common topic (69.2%) (See Table 1).

RQ2 examined the degree of exposure of different types of videos (pro-vaccine, anti-vaccine, mixed vaccine content, neutral vaccine content, non-vaccine-related content, health-related content, and COVID-19-related content) to additional anti-vaccine information. Table 2 reports the control-case odds ratios for exposure to anti-vaccine content and Figures 2–5 provide visualization of the networks.

Table 2.

Exposure statistics of four networks.

Statistics	Search networks		Seed networks
Statistics	Pro-vaccine search terms	Anti-vaccine search terms	Conspiracy theory seed network	Anti-vaccine expert seed network
Mean (SD)	0.035 (0.18)	0.048 (0.215)	0.037 (0.190)	0.083 (0.277)
Range	0.67	0.33	0.17	0.33
Nodes exposed, n (%)	11 (3.5)	14 (4.8)	8 (3.7)	12 (8.3)
Nodes unexposed, n (%)	304 (96.5)	276 (95.2)	206 (96.3)	132 (91.7)
Odds ratio (95% CI)
Non-vaccine video	0.30 (0.01- 2.11)	3.71 (0.42–44.98)	4.90 (0.54–59.63)	8.37 (2.23–34.22)^b
Vaccine video	0.32 (0.01–2.28)	4.60 (0.51–55.83)	6.75 (0.74–82.34)	3.69 (0.86–13.96)
Pro-vaccine video	0.36 (0.01–2.58)	1.23 (0.25–12.82)	2.82 (0.54–30.52)	0.64 (0.01–4.92)
Anti-vaccine video	0.00 (0–8.94)^a	0.00 (0.00–15.88)^a	7.21 (0.13–85.49)	6.10 (1.12–27.63)^b
Mixed vaccine messages video	1.28 (0.28–9.71)	0.00 (0.00–18.83)^a	6.28 (0.11–72.90)	0.00 (0–12.50)^a
Neutral vaccine video	0.45 (0.01–3.22)	6.27 (0.69–76.10)	7.46 (0.81–91.03)	9.92 (0.12–784.56)
Health related	0.56 (0.01–4.08)	5.29 (0.59–64.08)	1.93 (0.04–20.48)	8.86 (2.32–34.42)^b
COVID related	0.27 (.01–1.93)	4.36 (0.49–52.78)	6.54 (0.72–79.74)	5.80 (1.53–21.69)^b

Note. ^aindicates Cornfield adjustment, ^bindicates significant odds ratio findings based on the confidence intervals.

Health-related videos refer to videos that tackle any health topics (e.g. nutrition, high blood pressure, etc.), COVID Related videos refer to videos that tackle COVID-19-related topics, excluding COVID-19 vaccines (e.g. masks, protective measures against COVID-19).

In both pro-vaccine and anti-vaccine search networks, non-vaccine-related videos had a low likelihood of being exposed to anti-vaccine videos while vaccine-related videos had a slightly higher chance of being exposed to anti-vaccine videos. When examining the vaccine-related videos, videos with mixed vaccine messages had 1.5 times the odds of being exposed to anti-vaccine videos in the pro-vaccine search terms network. In the anti-vaccine search term network, neutral videos had 6.5 times the odds of being exposed to anti-vaccine videos. However, the confidence intervals indicate that none of the odds ratios calculated were statistically significant in both search networks. This suggests that it is uncertain whether the exposure to anti-vaccine, pro-vaccine, mixed, or neutral videos either increases or decreases the odds of being exposed to anti-vaccine videos in these search networks.

The seed networks—conspiracy theory network and anti-vaccine expert network—differed. In the conspiracy theory seed network, vaccine-related videos had a higher likelihood of being exposed to anti-vaccine videos. In terms of the network's vaccine-related content, anti-vaccine videos and neutral vaccine videos had 7 and 7.5 times the odds of being exposed to anti-vaccine videos, respectively. Mixed vaccine message videos also had a high likelihood of being exposed to anti-vaccine videos. However, none of the odds ratios in this network was statistically significant.

In the anti-vaccine expert seed network, contrary to all other networks, non-vaccine-related videos had a higher likelihood of being exposed to anti-vaccine videos. Among vaccine-related ideas, neutral vaccine videos had nine times the odds of being exposed to anti-vaccine videos. Similarly, anti-vaccine videos had six times the odds of being exposed to more anti-vaccine videos. The confidence intervals of the odds ratio indicated that non-vaccine videos, anti-vaccine videos, health-related videos, and COVID-19 videos significantly predicted the exposure to anti-vaccine videos in this network.

Discussion

This study examined the characteristics of YouTube's search, and recommendation algorithms by exploring the information users are likely to be exposed to when they search YouTube directly or are directed to YouTube from another platform about COVID-19 vaccine information in Arabic. Four networks—pro-vaccine search network, anti-vaccine search network, conspiracy theory seed network, and anti-vaccine expert seed network—were examined in terms of both video content and network structure. This study replicates and extends on the work of Tang et al.²⁹ and focused on the Arabic language, expanding on the health misinformation on social media literature and providing insights into the language-based algorithmic information disparities and the language-based digital divides that is often unseen given the focus on English language.⁷²

Our analysis revealed that whether the user searched YouTube (using pro- or anti-vaccine Arabic term) or was redirected to YouTube from a seed video in Arabic, the user was likely to be recommended news media content. This finding differs from Tang et al.,²⁹ who found that English-language users searching for vaccine information reached credible sources such as government agencies and hospitals. This difference could be partially due to the studied period as²⁹ examined vaccine information on YouTube prior to the pandemic; thus, the study did not capture the novelty of the virus and the constantly evolving knowledge about it. Furthermore, while news media may not be considered the most credible or scientific source for health information, media in the MENA region is often under state control and typically functions as a mouthpiece for the governments.^19,73 Thus, the media content most likely reflects the governments’ position on vaccination, which tended to encourage and slowly normalize COVID-19 vaccines due to the government's vaccination agenda (e.g. vaccine mandates).⁴⁸

Our analysis indicated that when users used pro- or anti-vaccine search terms on YouTube in Arabic, they had a very high chance of reaching COVID-19 vaccine-related content. More significantly, users had a relatively low chance of being recommended an anti-vaccine video from either search networks. This suggests that YouTube has taken solid measures to not only limit the anti-vaccine content in general when searches are done directly on the platform in Arabic, but it has also pushed the COVID-19 agenda to the forefront of its platform to address the current global pandemic. This finding suggests that unlike Facebook, which has failed to effectively tackle viral COVID-19 misinformation among Arabic-language communities on its platform,⁴² YouTube's recommendation algorithm lowered the visibility of anti-vaccine videos and suggested content that encouraged vaccination.

Our findings also indicate that in the seed networks (where users were redirected to YouTube from another platform), users had roughly 2–10 times the chance of being recommended an anti-vaccine video depending on the network. However, such a pattern was only statistically significant in the anti-vaccine expert seed network in which four types of videos (non-vaccine related, anti-vaccine, health related, and COVID-19 related) were likely to recommend more anti-vaccine videos to users. In this study as well as Tang et al.,²⁹ the anti-vaccine expert seed networks were likely to recommended more anti-vaccine videos. This finding is particularly alarming to the MENA region given the fact that regardless of nationality, gender, age, and level of education, a video that features a doctor (or those perceived as experts) were found to be the most consumed and trusted content among Arabs on social media.²⁵ In addition, the region's heavy reliance on social media still lacks good awareness of the drawbacks of social media in terms of fake news and rumors.²⁵ Thus, although the YouTube recommendation algorithm seems to work best when searches are conducted on the platform's homepage, the algorithm's recommendation of more anti-vaccine in the expert seed network arguably causes greater damage to the user of this region.

The unique findings in the conspiracy theory seed network suggest that there may be a cultural dimension at play. Vaccine conspiracy theories in Western culture have been excessively studied^74,75 and are often connected to libertarian views. COVID-19 vaccine conspiracy theories are caused by a distrust of experts and other authority figures, ideological division, and partisan politics.⁷⁶ The MENA region is culturally and politically different, and although conspiracy theories are also prominent and embedded in the way Middle Eastern think,⁷⁷ these theories are not vaccine specific. These cultural and political differences may be why conspiracy videos in Arabic were not likely to lead to anti-vaccine videos, unlike English-language conspiracy videos.

Limitations and future directions

This study has its limitations. Our sampling method was purely based on YouTube's recommendation algorithm. In reality, YouTube's recommendation also takes into consideration a viewer's profile and viewing history. Thus, our sampling method can only allow us to estimate information exposure on YouTube to a certain extent. Secondly, our data provide a snapshot of the content on YouTube at a specific time (April 2021), which could also be one reason why this study got different results from.²⁹ YouTube has since made more changes to control the spread of vaccine misinformation, thus recent studies might have different findings.⁷⁸ A natural next step of the study is to focus on COVID-19 vaccine discussions on YouTube and track its changes over time in various languages.

Conclusion

This study is among the first to examine Arabic vaccine content on YouTube in the English-language literature, diversifying the empirical evidence and providing insights into the nature and spread of misleading and harmful content on social media outside of the Western world. We explored the vaccine-related information YouTube users are recommended in Arabic using network exposure analysis. Our findings indicate that users searching for vaccine information directly on YouTube were unlikely to be recommended anti-vaccine content; however, they were likely to be recommended more anti-vaccine videos if they were redirected back to YouTube from an anti-vaccine expert video. While YouTube deserves credit for its efforts to clean up and limit anti-vaccine content exposure in Arabic on its platform, continuous evaluations of the algorithm functionality are warranted.

Footnotes

Acknowledgement

We want to extend our deepest gratitude to our coders who played an essential role in making this study possible. To Iman Knadil, Ismail Mostafa, Omar Mostafa, and Mohamed Zeid.

Contributorship

NZ, LT, and MTA conceptualized the study. NZ collected the data. NZ and MTA conducted data analysis. All three authors contributed to the writing of the manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

This study is a review of published YouTube videos and does not constitute human subject research.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was partially supported by funding to the corresponding author from Texas A&M Institute of Data Science, and from the National Institute of Health, US (grant No: 3U01AG070112-02S2.)

Guarantor

LT takes full responsibility for this article.

Informed consent

No patient consent is necessary as the study does not involve human subjects.

Qatar and the UAE exceed 100% vaccination coverage due to the vaccination of non-residents.

ORCID iDs

Nour Zeid

Lu Tang

Appendix

Appendix 1.

COVID-19 vaccination statistics for Arab countries for the year 2023 (sorted according to the most recent update)

Date of latest reported vaccination rates	Country	Percentage of vaccinated population
April 2023	Qatar^a	105.8%
April 2023	Kuwait	78.4%
April 2023	Morocco	62.8%
April 2023	Egypt	38.0%
April 2023	Mauritania	32.5%
April 2023	Djibouti	31.4%
April 2023	Sudan	28.6%
April 2023	Yemen	2.3%
April 2023	Tunisia	51.8%
April 2023	Syria	10.7%
April 2023	Saudi Arabia	69.9%
April 2023	Somalia	40.0%
January 2023	Libya	18.2%
January 2023	Iraq	17.9%
December 2022	Bahrain	83.3%
December 2022	Lebanon	44.0%
October 2022	Comoros	47.5%
October 2022	Oman	66.6%
October 2022	Palestine	33.9%
September 2022	Algeria	14.4%
August 2022	Jordan	40.4%
June 2022	United Arab Emirates (UAE)^a	103.7%

Note. The reported percentages represent the total number of people who received all doses prescribed by the initial vaccination protocol (2 doses) divided by the total population of the country.

The data was collected from.¹¹

^aQatar and the UAE exceed 100% due to the vaccination of non-residents.

No information was available for the year 2023 for Bahrain, Lebanon, Comoros, Oman, Palestine, Algeria, Jordan, and the United Arab Emirates (UAE).

References

MacDonald

. Vaccine hesitancy: definition, scope and determinants. Vaccine 2015; 33: 4161–4164.

Ten health issues WHO will tackle this year [Internet]. 2019 [cited 2021 Apr 3]. Available from: https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019

Infodemic [Internet]. [cited 2023 Jul 30]. Available from: https://www.who.int/health-topics/infodemic

Naeem

Bhatti

Khan

. An exploration of how fake news is taking over social media and putting public health at risk. Health Info Libr J 2021; 38: 143–149.

Ahmad

Murad

. The impact of social media on panic during the COVID-19 pandemic in Iraqi Kurdistan: online questionnaire study. J Med Internet Res 2020; 22: e19556.

Tasnim

Hossain

Mazumder

. Impact of rumors and misinformation on COVID-19 in social media. J Prev Med Public Health 2020; 53: 171–174.

Sallam

. COVID-19 vaccine hesitancy worldwide: a concise systematic review of vaccine acceptance rates. Vaccines 2021; 9(2): 160.

Kaadan

Abdulkarim

Chaar

, et al. Determinants of COVID-19 vaccine acceptance in the Arab world: a cross-sectional study. Glob Health Res Policy 2021; 6: 23.

dw.com [Internet]. [cited 2023 Jul 30]. Arab countries struggle with vaccine hesitancy – DW – 04/16/2021. Available from: https://www.dw.com/en/middle-east-covid-vaccine-rollout-hesitancy/a-57227395

10.

Karamouzian

Madani

. COVID-19 response in the Middle East and North Africa: challenges and paths forward. Lancet Glob Health 2020; 8: e886–e887.

11.

Coronavirus (COVID-19) Vaccinations - Our World in Data [Internet]. [cited 2023 Aug 3]. Available from: https://ourworldindata.org/covid-vaccinations

12.

Dyer

Schaider

Letzkus

.Infographic: COVID-19 vaccination efforts in the Middle East and North Africa. [Cited 2023 Sep 27] Brookings 2021 Available from: https://www.brookings.edu/articles/covid-19-vaccination-efforts-in-the-middle-east-and-north-africa/

13.

Alrasheedi

. COVID-19 statistics in the Arab world by the end of October 2022: a cross-sectional study. Cureus 2022; 14(12): e32670.

14.

Rashwan

. Egypt imposes a vaccine mandate for all government workers and university students. The New York Times [Internet]. 2021 Oct 20 [cited 2021 Dec 13]; Available from: https://www.nytimes.com/2021/10/20/world/egypt-covid-vaccine-mandate.html.

15.

Dubé

Gagnon

Nickels

, et al. Mapping vaccine hesitancy–Country-specific characteristics of a global phenomenon. Vaccine 2014; 32: 6649–6654.

16.

Kukreti

Strong

Chen

, et al. The association of care burden with motivation of vaccine acceptance among caregivers of stroke patients during the COVID-19 pandemic: mediating roles of problematic social media use, worry, and fear. BMC Psychol 2023; 11(1): 157.

17.

Al-Qerem

Jarab

. COVID-19 vaccination acceptance and its associated factors among a Middle Eastern population. Front Public Health 2021; 9: 632914.

18.

Qunaibi

Basheti

Soudy

, et al. Sultan

. Hesitancy of Arab healthcare workers towards COVID-19 vaccination: a large-scale multinational study. Vaccines 2021; 9(5): 446.

19.

Zeid

Tang

. Egyptian newspapers coverage of COVID-19 vaccines: a theoretically driven content analysis. J Health Commun 2022; 27: 727–736.

20.

AlAshry

. A comparative analysis of the reportage of COVID-19 during the first and second wave: a case of the Egyptian and Jordanian newspapers. Int J Media Inf Literacy 2021: 17–33.

21.

Wang

McKee

Torbica

, et al. Systematic literature review on the spread of health-related misinformation on social media. Soc Sci Med 2019; 240: 112552.

22.

Suarez-Lledo

Alvarez-Galvez

. Prevalence of health misinformation on social media: systematic review. J Med Internet Res 2021; 23: e17187.

23.

Puri

Coomes

Haghbayan

, et al. Social media and vaccine hesitancy: new updates for the era of COVID-19 and globalized infectious diseases. Hum Vaccin Immunother 2020; 16: 2586–2593.

24.

Abdelhafiz

Mohammed

Ibrahim

, et al. Knowledge, perceptions, and attitude of Egyptians towards the novel coronavirus disease (COVID-19). J Community Health 2020; 45: 881–890.

25.

Kalliny

. The impact of social media on Arab health risk perception during COVID-19. Arab Media Soc 2021; Winter/Spring (31): 1–24.

26.

Sallam

Dababseh

Yaseen

, et al. Conspiracy beliefs are associated with lower knowledge and higher anxiety levels regarding COVID-19 among students at the university of Jordan. Int J Environ Res Public Health 2020; 17: 4915.

27.

Khalifa

Mohsen Al-Absy

Badran

, et al. COVID-19 pandemic and diffusion of fake news through social media in the Arab world. Arab Media Soc 2020: 43–58.

28.

Basch

Zybert

Reeves

, et al.

What do popular YouTubeTM videos say about vaccines?

Child Care Health Dev 2017; 43: 499–503.

29.

Tang

Fujimoto

Amith

, et al. Down the rabbit hole” of vaccine misinformation on YouTube: network exposure study. J Med Internet Res 2021; 23: e23262.

30.

Khalife

. StepFeed. 2019 [cited 2021 Apr 12]. The world’s most dedicated YouTube viewers are in Saudi Arabia. Available from: https://stepfeed.com/the-world-s-most-dedicated-youtube-viewers-are-in-saudi-arabia-2354.

31.

Radcliffe

Abuhmaid

. Social Media in the Middle East: 2019 in Review [Internet]. Rochester, NY: Social Science Research Network, 2020, Report No.: ID 3517916. [Cited 2023, Sep 27]. Available from: https://papers.ssrn.com/abstract=3517916.

32.

Madathil

Rivera-Rodriguez

Greenstein

, et al. Healthcare information on YouTube: a systematic review. Health Informatics J 2015; 21: 173–194.

33.

Davies

Chapman

Leask

. Antivaccination activists on the world wide web. Arch Dis Child 2002; 87: 22–25.

34.

Wilson

Wiysonge

. Social media and vaccine hesitancy. BMJ Glob Health 2020; 5: e004206.

35.

Broniatowski

Jamison

, et al. Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. Am J Public Health 2018; 108(10): 1378–1384.

36.

Gruzd

Mai

. Going viral: how a single tweet spawned a COVID-19 conspiracy theory on Twitter. Big Data Soc 2020; 7: 2053951720938405.

37.

Essam

Abdo

. How do Arab tweeters perceive the COVID-19 pandemic? J Psycholinguist Res 2021; 50: 507–521.

38.

Al-Rawi

Fakida

Grounds

. Investigation of COVID-19 misinformation in Arabic on Twitter: content analysis. JMIR Infodemiol 2022; 2: e37007.

39.

Mahlous

Al-Laith

. Fake news detection in Arabic tweets during the COVID-19 pandemic. Int J Adv Comput Sci Appl 2021; 12(6): 776-785.

40.

Albalawi

Nikolov

Buckley

. Trustworthy health-related tweets on social media in Saudi Arabia: tweet metadata analysis. J Med Internet Res 2019; 21(10): e14731.

41.

Smith

Graham

. Mapping the anti-vaccination movement on Facebook. Inf Commun Soc 2019; 22: 1310–1327.

42.

O’Connor

Ayad

. MENA Monitor: Arabic COVID-19 Vaccine Misinformation Online [Internet]. [Cited 2023 Sep 27]. Institute for Strategic Dialogue; 2021. Available from: https://www.isdglobal.org/isd-publications/mena-monitor-arabic-covid-19-vaccine-misinformation-online/.

43.

New Vaccine Conspiracy Theories Are Going Viral in Arabic | WIRED [Internet]. [cited 2023 Aug 1]. Available from: https://www.wired.com/story/new-vaccine-conspiracy-theories-going-viral-arabic/

44.

Swire-Thompson

Lazer

. Public health and online misinformation: challenges and recommendations. Annu Rev Public Health 2020; 41: 433–451.

45.

Dunn

Surian

Leask

, et al. Mapping information exposure on social media to explain differences in HPV vaccine coverage in the United States. Vaccine 2017; 35: 3033–3040.

46.

Featherstone

Zhang

. Feeling angry: the effects of vaccine misinformation and refutational messages on negative emotions and vaccination attitude. J Health Commun 2020; 25: 692–702.

47.

Dadras

Seyedalinaghi

Karimi

, et al. Public acceptability of COVID-19 vaccines and its predictors in Middle Eastern/North African (MENA) countries: a systematic review. Hum Vaccines Immunother 2022; 18(5): 2043719.

48.

Elbarazi

Yacoub

Reyad

, et al. Exploring enablers and barriers toward COVID-19 vaccine acceptance among Arabs: a qualitative study. Int J Disaster Risk Reduct 2022; 82: 103304.

49.

Biswas

Ali

, et al. Influences of social media usage on public attitudes and behavior toward COVID-19 vaccine in the Arab world. Hum Vaccin Immunother 2022; 18: 2074205.

50.

Ceci. Statista. 2021 [cited 2021 Nov 2]. YouTube - Statistics & Facts. Available from: https://www.statista.com/topics/2019/youtube/

51.

Keelan

Pavri-Garcia

Tomlinson

, et al. YouTube as a source of information on immunization: a content analysis. J Am Med Assoc 2007; 298: 2482–2484.

52.

Murphy

Nanadiego

McCavera

, et al. Assessing the validity and accuracy of online videos on vaccine health risks. Clin Pediatr (Phila) 2020; 59: 458–466.

53.

Elkin

Pullon

SRH

Stubbe

. Should I vaccinate my child?’ Comparing the displayed stances of vaccine information retrieved from Google, Facebook and YouTube. Vaccine 2020; 38: 2771–2778.

54.

Abul-Fottouh

Song

Gruzd

. Examining algorithmic biases in YouTube’s recommendations of vaccine videos. Int J Med Inf 2020; 140: 104175.

55.

Beer

. The social power of algorithms. Inf Commun Soc 2017; 20: 1–13.

56.

Bozdag

. Bias in algorithmic filtering and personalization. Ethics Inf Technol 2013; 15: 209–227.

57.

Caplan

. Algorithmic filtering. In: Mediated communication [Internet]. De Gruyter Mouton, 2018. pp. 561–584. [Cited 2023 Sep 27]. Available from: https://www.degruyter.com/document/doi/10.1515/9783110481129-030/html

58.

Yesilada

Lewandowsky

. Systematic review: YouTube recommendations and problematic content. Internet Policy Rev 2022; 11(1): 1652.

59.

Bessi

Zollo

Del Vicario

, et al. Users polarization on Facebook and YouTube. PLoS One 2016; 11: e0159641.

60.

Roth

Mazières

Menezes

. Tubes and bubbles topological confinement of YouTube recommendations. PLoS One 2020; 15(4).e0231703.

61.

Rogers

. Diffusion of innovations. New York: Free Press, 2003.

62.

Burt

. Social contagion and innovation: cohesion versus structural equivalence. AJS 1987; 92: 1287–1335.

63.

Valente

Vega Yon

. Diffusion/contagion processes on social networks. Health Educ Behav 2020; 47: 235–248.

64.

Valente

. Network models and methods for studying the diffusion of innovations. In: Carrington

Wasserman

(eds) Models and methods in social network analysis. Cambridge: Cambridge University Press, 2005, pp. 98–116.

65.

Amiriparian

Pugachevskiy

Cummins

, et al. CAST a database: Rapid targeted large-scale big data acquisition via small-world modelling of social media platforms. In: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), 2017, pp. 340–345.

66.

Google Developers [Internet]. [cited 2021 May 2]. YouTube Data API. Available from: https://developers.google.com/youtube/v3

67.

Briones

Nan

Madden

, et al. When vaccines go viral: an analysis of HPV vaccine coverage on YouTube. Health Commun 2012; 27: 478–485.

68.

Bastian

Heymann

Jacomy

. Gephi: an open source software for exploring and manipulating networks. Proc Int AAAI Conf Web Soc Media 2009; 3: 361–362.

69.

Amith

Fujimoto

Tao

. NET-EXPO: a gephi plugin towards social network analysis of network exposure for unipartite and bipartite graphs. In: Stephanidis

(eds) HCI International 2019 - posters. Cham: Springer International Publishing, 2019, pp. 3–12. (Communications in Computer and Information Science).

70.

Latapy

. Main-memory triangle computations for very large (sparse (power-law)) graphs. Theor Comput Sci 2008; 407: 458–473.

71.

Watts

Strogatz

. Collective dynamics of ‘small-world’ networks. Nature 1998; 393: 440–442.

72.

Scherr

Arendt

Haim

. Algorithms without frontiers? How language-based algorithmic information disparities for suicide crisis information sustain digital divides over time in 17 countries. Inf Commun Soc 2022. Epub ahead of print.

73.

Dragomir

. Control the money, control the media: how government uses funding to keep media in line. Journalism 2018; 19: 1131–1148.

74.

Goertzel

. Belief in conspiracy theories. Polit Psychol 1994; 15: 731–742.

75.

Goreis

Voracek

. A systematic review and meta-analysis of psychological research on conspiracy beliefs: field characteristics, measurement instruments, and associations with personality traits. Front Psychol 2019; 10: 205.

76.

Uscinski

Enders

Klofstad

, et al. Why do people believe COVID-19 conspiracy theories? Harvard Kennedy School Misinformation Review [Internet]. 2020 Apr 28 [cited 2022 Feb 14];1(3). Available from: https://misinforeview.hks.harvard.edu/article/why-do-people-believe-covid-19-conspiracy-theories/.

77.

Zonis

Joseph

. Conspiracy thinking in the Middle East. Polit Psychol 1994; 15: 443–459.

78.

YouTube is cracking down on anti-vaccine misinformation - CNN [Internet]. 2021 [cited 2022 Feb 14]. Available from: https://www.cnn.com/2021/09/29/tech/youtube-vaccine-misinformation/index.html