The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study

Abstract

Background

The detection rate of thyroid nodules has witnessed a significant surge in recent years, triggering heightened public apprehension. Short video platforms such as TikTok and BiliBili have showed tremendous potential in the dissemination of health information. There is a plethora of videos about thyroid nodules on TikTok and BiliBili, but the quality and reliability of videos concerning thyroid nodules remains unknown.

Methods

On December 3^rd, 2023, the top 100 short videos related to thyroid nodules on BiliBili and TikTok were collected through a comprehensive search in Chinese. After extracting the basic information, the quality and reliability of each video was assessed by using the global quality score (GQS) and DISCERN score. Further, Spearman correlation analyses were applied to examine the correlation among video variables, GQS and DISCERN score.

Results

Compared to BiliBili, TikTok exhibits greater popularity, as evidenced by higher counts of likes (P = 0.021), comments (P = 0.008) and shares (P = 0.017). The median (interquartile range) scores of GQS and DISCERN score were 3 (2–3) on TikTok while 2 (2–3) on BiliBili. Both reviewers exhibited good consistency in GQS and DISCERN score. Moreover, it was observed that the videos shared by thyroid specialists demonstrated higher scores both in GQS (P = 0.014) and DISCERN score (P = 0.022) than others on TikTok. Spearman correlation analysis revealed no significant correlation between video variables and the scores of GQS and DISCERN score.

Conclusions

The quality and reliability of thyroid nodules videos on BiliBili and TikTok were unsatisfactory. Notably, videos shared by thyroid specialists are more likely to exhibit superior quality and reliability. People should exercise caution when perusing short videos.

Keywords

Thyroid nodules short videos global quality score DISCERN score quality reliability

Introduction

Over the past three decades, there has been a significant rise in the prevalence of thyroid nodules with around 60% adults have one or more thyroid nodules, which may be attributed by various factors including advancements in testing technologies, lifestyle and environmental influences.^1–4 The prevalence of thyroid nodules spans all age groups and can occur at any stage of life, including children.⁵ The incidence of thyroid nodules increases with age.⁶ Women are more likely to develop thyroid nodules than men.⁷ The prevalence is associated with factors such as iodine deficiency and metabolic syndrome.^8,9 While most thyroid nodules are asymptomatic, there still 7% to 15% of thyroid nodules are malignant, leading to thyroid cancer and endangering the health and life of patients.^2,10 Given the growing awareness of health issues among the general population, there is an increasing concern and anxiety regarding the possibility of malignant thyroid nodules. Therefore, individuals are seeking not only medical advice but also turning to online platforms, such as social media, in search of information, treatment recommendations and preventive measures for thyroid nodules.

Recently, as a new type of social media, short video platforms such as BiliBili and TikTok have gradually emerged on the Internet due to their ease of use for producing and sharing short videos that incorporate elements like animations and various visual effects.¹¹ Compared to traditional text-based information, short videos are more attractive and easier to be absorbed and remembered. Several studies have demonstrated an increased trend in the dissemination of health information through short video platforms over the past decades.^12,13 Public is available to health-related short videos from the BiliBili and TikTok. However, the videos on these two platforms showed a wide variation of quality and reliability due to the lack of a censorship system and peer review process on these platforms. In fact, a majority of videos demonstrate poor quality and reliability, even contain deceptive and misleading information, posing significant challenges to patient healthcare.^14,15 Therefore, it is essential to assess the quality and reliability of short videos available on BiliBili and TikTok. Previous studies have analyzed the quality of short videos on TikTok or BiliBili regarding liver cancers,¹⁶ thyroid cancers,¹¹ chronic obstructive pulmonary diseases,¹⁷ inflammatory bowel diseases,¹⁸ gallstone diseases¹⁹ and so on. Importantly, there are a large number of short videos about thyroid nodules with uncertain quality and reliability due to the increased attention paid to thyroid nodules. Therefore, this study aimed to investigate the quality and reliability of thyroid nodules videos on BiliBili and TikTok platforms.

Methods

Search strategy and data extraction

We employed the “甲状腺结节” (“thyroid nodules” in Chinese) as the keyword to retrieve the top 100 videos in comprehensive ranking from BiliBili and TikTok on December 3^rd, 2023. A new account was created and logged into on each short video platform in order to minimize the bias introduced by the personal recommendation algorithms. Non-Chinese videos, repeated videos, videos without identifiable authors, and unrelated videos were excluded until the videos were collected in the top 100 (Figure 1). Limiting our analysis to the top 100 videos served two purposes. Firstly, both TikTok and BiliBili employ search algorithms that prioritize topic relevance, ensuring that the majority of videos related to thyroid nodules appear at the top of the search results. When the results exceed 100, some videos may have little or no relevance to the topic. Secondly, most video viewers tend to focus on the top search results rather than scrolling through the entire list.^17,20,21 Basic information on the included videos was extracted, including the sources, contents and formats of the videos, the duration (in seconds), the number of likes, comments, shares, and saves, as well as the number of days since publication.

Figure 1.

Search strategy and video filtering program.

Videos assessments

The reliability and quality of the videos were assessed by the DISCERN score and the global quality score (GQS). The DISCERN score has been extensively validated and used to evaluate health-related contents on video sharing platforms.^22,23 The first part of the DISCERN score listed in Supplementary Table S1 was used to assess video reliability, as previously studied.¹⁶ Higher video score indicates better reliability, and to facilitate statistics and comparison, the points were categorized into five levels based on the evaluation, which are: unreliable, less reliable, medium reliable, relatively reliable, and reliable, as presented in Supplementary Table S2. The GQS was applied to assess the quality of the videos, which is a commonly used 5-point Likert scale, which are: poor quality, generally poor quality, moderate quality, good quality, and excellent quality, as shown in Supplementary Table S3. The GQS and DISCERN score were chosen due to their widespread use in assessing information quality on various video-based platforms or diseases.^17,24 Two qualified doctors specialized in thyroid disease scored the videos. Before scoring, the two raters reviewed the instructions of the GQS and DISCERN score and resolved ambiguous details to reduce bias. Cohen κ coefficients were calculated to quantify the agreement between the two raters. According to the criteria proposed by Landis and Koch, a value of κ > 0.8 indicates excellent consistency and reliability.²⁵ Disagreements were resolved by an arbitrator and finally all authors reached a consensus on all ratings.

Statistical analysis

Considering that our data is nonparametrically distributed, the data will be expressed by the median and interquartile ranges (IQR). Mann–Whitney test was used for comparison of two groups, and Kruskal–Wallis H test was used for comparison of three groups and above. Cohen κ coefficients were calculated to assess the agreement between the two raters. Spearman correlation analysis was used to reveal the correlation between different video variables and the relationship between variables and scores, considering the non-normal distribution of our data. P < 0.05 was considered statistically significant. IBM SPSS Statistics version 26.0 for Windows was used for statistical analysis and GraphPad Prism version 8.0.1 for Windows for data visualization.

Results

Video characteristics

We analyzed 200 videos from BiliBili and TikTok, finding that BiliBili videos had a longer duration and fewer likes, comments, and shares compared to TikTok videos, but there was no significant difference in the saves of videos. General features of videos are presented in Table 1. Among the video sources, non-thyroid specialists uploaded most videos on BiliBili (59/100, 59%) and TikTok (62/100, 62%), followed by individual science communicators on BiliBili (18/100, 18%) and thyroid specialists on TikTok (26/100, 26%). The thyroid specialists referred to thyroid surgeons and endocrinologists. Videos from news agencies and social organizations were rare, with no videos from hospitals. Surprisingly, videos from thyroid specialists received less likes, comments, saves, and shares compared to those from news agencies and social organizations. BiliBili had the most videos on disease treatments (49%) and TikTok had the most videos on disease knowledge (45%). In terms of video contents, BiliBili had the most videos on disease treatments (49/100, 49%) and TikTok had the most videos on disease knowledge (45/100, 45%). Disease prevention videos were the least common on both platforms. Meanwhile, videos about disease treatments received more likes, comments, saves, and shares on BiliBili, whereas disease prevention received more engagement on TikTok. In addition, 84% (84/100) of videos on BiliBili were live videos, while 16% (16/100) were animations. On TikTok, 96% (96/100) were live videos and 4% (4/100) were animations. Despite live videos being more common, animations were preferred on both platforms. Figure 2 and Tables 2 and 3 show the breakdown of video characteristics on BiliBili and TikTok.

Figure 2.

Percentage of videos according to video sources, video contents, and video formats on TikTok and BiliBili. (A) Video sources on BiliBili. (B) Video sources on TikTok. (C) Video contents on BiliBili. (D) Video contents on TikTok. (E) Video formats on BiliBili. (F) Video formats on TikTok.

Table 1.

Characteristics of the videos on TikTok and BiliBili.

Variable	BiliBili, median (IQR)	TikTok, median (IQR)	P-Value
Days since published	259 (62–669)	654 (184–862)	<0.001
Duration (seconds)	133 (67–257)	58 (38–89)	<0.001
Likes (counts)	152 (16–758)	2028 (350–7266)	0.021
Comments (counts)	13 (2–97)	142 (32–713)	0.008
Saves (counts)	89 (18–763)	216 (53–823)	0.051
Shares (counts)	57 (7–543)	431 (69–2140)	0.017

Table 2.

Characteristics of the videos in accordance with sources, contents and formats on BiliBili.

Variable	Days since published, median (IQR)	Duration (seconds), median (IQR)	Likes (counts), median (IQR)	Comments (counts), median (IQR)	Saves (counts), median (IQR)	Shares (counts), median (IQR)
Video sources (n = 100)
Thyroid specialists (n = 14)	351 (44–766)	61 (41–129)	24 (9–116)	4 (2–41)	33 (4–77)	12 (3–124)
Non-thyroid specialists (n = 59)	213 (45–594)	133 (66–263)	161 (25–1067)	9 (1–107)	194 (19–1369)	126 (11–652)
Patients (n = 5)	1042 (449–1261)	157 (130–511)	47 (22–258)	38 (6–224)	28 (16–68)	27 (6–47)
Individual science communicators (n = 18)	180 (85–529)	182 (92–382)	274 (12–783)	24 (2–87)	136 (26–588)	105 (7–604)
Social organizations (n = 3)	1010 (5–1014)	162 (150–246)	349 (0–395)	18 (0–25）	1143 (0–1290)	520 (0–703)
News agencies (n = 1)	868	122	13,000	607	1359	3149
Video content (n = 100)
Disease knowledge (n = 42)	238 (70–692)	157 (64–278)	132 (10–783)	9 (0–110)	76 (8–300)	48 (6–207)
Disease treatments (n = 49)	286 (53–678)	128 (79–245)	253 (36–850)	25 (3–105)	288 (44–1348)	248 (18–683)
Disease prevention (n = 3)	25 (5–80)	133 (90–313)	25 (16–1054)	17 (1–20)	21 (14–301)	15 (2–140)
Case reports and news (n = 6)	226 (37–730)	90 (49–230)	49 (14–81)	5 (1–66)	23 (10–45)	7 (3–29)
Video formats (n = 100)
Live Videos (n = 84)	273 (46–683)	123 (65–244)	132 (16–843)	16 (2–106)	88 (15–941)	61 (7–578)
Animations (n = 16)	180 (91–598)	253 (147–667)	169 (11–294)	11 (1–43)	136 (33–467)	57 (7–260)

Table 3.

Characteristics of the videos in accordance with sources, contents and formats on TikTok.

Variable	Days since published, median (IQR)	Duration (seconds), median (IQR)	Likes (counts), median (IQR)	Comments (counts), median (IQR)	Saves (counts), median (IQR)	Shares (counts), median (IQR)
Video sources (n = 100)
Thyroid specialists (n = 26)	654 (1–744)	59 (52–104)	1149 (249–3301)	88 (21–328)	216 (53–509)	385 (63–1105)
Non-thyroid specialists (n = 62)	631 (98–883)	54 (32–77)	2003 (300–4912)	108 (29–666)	157 (40–752)	272 (31–1921)
Patients (n = 3)	714 (1–773)	185 (101–224)	4620 (3148–43,000)	1302 (821–7188)	258 (164–6253)	810 (479–62,000)
Individual science communicators (n = 2)	618 (524–711)	42 (22–61)	16,002 (2004–30,000)	1289 (552–2025)	1569 (352–2785)	3661 (1703–5619)
Social organizations (n = 2)	992 (720–1264)	107 (87–126)	502,033 (65–1,004,000)	21,001 (1–42,000)	31,009 (18–62,000)	72,007 (14–144,000)
News agencies (n = 5)	1159 (715–1773)	50 (27–94)	66,000 (19,854–392,000)	5727 (274–12,533)	2284 (1229–21,626)	12,000 (2905–122,000)
Video contents (n = 100)
Disease knowledge (n = 45)	720 (446–922)	70 (50–116)	3078 (1103–16,500)	310 (74–1081)	294 (124–2012)	1076 (286–2926)
Disease treatments (n = 29)	711 (1–917)	51 (31–77)	1578 (169–3303)	82 (10–369)	161 (25–572)	329 (15–1474)
Disease prevention (n = 4)	286 (1–683)	131 (31–219)	22,356 (685–136,000)	2647 (42–6694)	3261 (146–33,813)	26,604 (162–59,750)
Case reports and news (n = 22)	623 (1–760)	47 (22–62)	891 (147–3266)	56 (24–396)	91 (17–327)	82 (17–418)
Video formats (n = 100)
Live Videos (n = 96)	654 (196–857)	58 (39–89)	1979 (350–5898)	129 (32–667)	203 (53–727)	386 (69–1944)
Animations (n = 4)	601 (74–1175)	52 (12–114)	34,539 (774–769,500)	4106 (38–33,516)	2492 (541–47,205)	2627 (284–109,029)

Video quality and reliability assessments

The quality of video was assessed by using the GQS and reliability was evaluated by the DISCERN score. There was a good agreement with a κ value of 0.81. The TikTok videos demonstrated moderate quality and medium reliability with a median (IQR) GQS of 3 (2–3) and a median (IQR) DISCERN score of 3 (2–3). While the median (IQR) GQS and DISCERN score were 2 (2–3) on BiliBili videos, indicating that the BiliBili videos were of generally poor quality and less reliable. Statistical analysis showed that TikTok videos exhibited better quality and reliability compared to BiliBili videos, with significant differences in GQS and DISCERN score (Figure 3).

Figure 3.

The GQS and DISCERN score of videos related to thyroid nodules on TikTok and BiliBili. (A) Comparison of GQS between TikTok and BiliBili videos. (B) Comparison of DISCERN score between TikTok and BiliBili videos. (C) Ridge plot showing the overall distribution of GQS. (D) Ridge plot showing the overall distribution of DISCERN score. “*” means P < 0.05; “**” means P < 0.01.

Furthermore, we statistically analyzed the GQS and DISCERN score based on different video sources, contents and formats. The specific values are presented in Table 4 and Figure 4. In terms of video sources, the videos shared by thyroid specialists and non-thyroid specialists showed higher GQS than patients on BiliBili. Notably, it was observed that the GQS scores of the videos uploaded by thyroid specialists were significantly higher than non-thyroid specialists, patients and news agencies on TikTok. Regarding video reliability, the videos from thyroid specialists tended to have higher DISCERN score than those from patients and news agencies. In conclusion, among the videos from different sources, the videos shared by thyroid specialists seemed to have better quality and reliability than others. When comparing the video contents on TikTok, videos focusing on disease treatments and disease knowledge obtained higher GQS and DISCERN score than case reports and news. Meanwhile, the BiliBili videos about disease knowledge had better quality and reliability than disease treatments. This suggests a preference for disease treatment videos among BiliBili users. In addition, the case reports and news scored lower than the other three in GQS on BiliBili, suggesting poor video quality. All in all, in terms of the contents of the videos, the videos related to disease knowledge seemed to exhibit the highest quality and reliability, while case reports showed the lowest. Furthermore, despite live videos being more prevalent than animations, no significant difference was found in terms of quality and reliability.

Figure 4.

The GQS and DISCERN score of videos related to thyroid nodules on BiliBili and TikTok. (A) The GQS and DISCERN score from different sources on BiliBili and TikTok. (B) The GQS and DISCERN score from different formats on BiliBili and TikTok. (C) The GQS and DISCERN score from different sources on BiliBili and TikTok. “*” means P < 0.05; “**” means P < 0.01; “***” means P < 0.001.

Table 4.

The GQS and DISCERN score based on different video sources, contents and formats about thyroid nodules on TikTok and BiliBili.

Variable	BiliBili		TikTok
Variable	GQS, Median (IQR)	DISCERN score, Median (IQR)	GQS, Median (IQR)	DISCERN score, Median (IQR)
Overall score	3.00 (2.00–3.00)	3.00 (2.00–3.00)	2.00 (2.00–3.00)	2.00 (2.00–3.00)
Video sources
Thyroid specialists	2.00 (2.00–3.25)	2.00 (2.00–3.25)	3.00 (3.00–4.00)	3.00 (3.00–3.00)
Non-thyroid specialists	2.00 (2.00–3.00)	2.00 (2.00–3.00)	3.00 (2.00–3.00)	3.00 (2.00–3.00)
Patients	1.00 (1.00–2.00)	2.00 (1.00–2.00)	2.00 (1.00–2.00)	2.00 (1.00–2.00)
Individual science communicators	2.00 (1.00–4.00)	2.00 (2.00–3.25)	2.50 (2.00–3.00)	2.50 (2.00–3.00)
Social organizations	2.00 (1.00–2.00)	2.00 (2.00–3.00)	2.50 (2.00–3.00)	2.50 (2.00–3.00)
News agencies	3.00 (3.00–3.00)	3.00 (3.00–3.00)	3.00 (1.00–3.00)	2.00 (1.00–2.50)
Video contents
Disease knowledge	3.00 (2.00–4.00)	3.00 (2.00–3.00)	3.00 (3.00–4.00)	3.00 (2.50–3.00)
Disease treatments	2.00 (2.00–3.00)	2.00 (2.00–3.00)	3.00 (2.00–3.00)	3.00 (3.00–3.00)
Disease prevention	2.00 (2.00–2.00)	2.00 (2.00–2.00)	2.50 (2.00–4.50)	2.50 (2.00–3.75)
Case reports and news	1.00 (1.00–1.25)	2.00 (1.00–2.00)	2.00 (1.00–2.00)	2.00 (1.00–2.00)
Video formats
Live Videos	2.00 (2.00–3.00)	2.00 (2.00–3.00)	3.00 (2.00–3.00)	3.00 (2.00–3.00)
Animations	2.00 (1.25–4.00)	3.00 (2.00–3.75)	3.00 (3.00–4.50)	3.00 (2.25–3.75)

Spearman correlation analysis

Spearman correlation analysis was used to explore the correlation among different video variables, the GQS and DISCERN score in thyroid nodules related videos. There were strongly positive correlations among likes, comments, shares and saves. Moreover, the days since publication had a moderate positive with likes, comments, saves, and shares. Unfortunately, our analysis found both in GQS and DISCERN score had no correlations with video variables such as duration, the days since published, likes, comments, saves, and shares. Besides, no correlation was observed between the animations and video variables. The specific correlation coefficients are shown in Figure 5.

Figure 5.

Spearman correlation analysis among different video variables, GQS and DISCERN score concerning thyroid nodules videos. “*” means P < 0.05; “**” means P < 0.01; “***” means P < 0.001.

Discussion

Thyroid nodules are a prevalent condition that can manifest at any age. It is estimated that over 60% of adults may have thyroid nodules.^1–4 Moreover, diagnosis of thyroid nodules in younger individuals is associated with an elevated risk of malignant pathology in comparison to older age groups.²⁶ However, public awareness and understanding of this condition remain relatively limited. The extensive reach and popularity of social media platforms (such as TikTok) among younger age groups make them an ideal vehicle for disseminating accurate health education. Accordingly, this study aims to assess the necessity and efficacy of utilizing social media for the dissemination of health knowledge, thereby underscoring its practical significance in addressing this public health challenge.

TikTok and BiliBili as health information sources

Social media plays an increasingly important role in healthcare, with approximately 80% of Internet users seeking health information online.^27,28 Among the various social media platforms, short video-based platforms have gained significant popularity. As a representative example, TikTok boasts over 100 million users worldwide across over 150 countries.¹⁹ Besides, TikTok ranked the first position on the global mobile app download list in December 2021 in China.¹¹ Likewise, BiliBili, another popular short video platform, has millions of monthly active users.¹⁶ Both TikTok and BiliBili have emerged as influential platforms for generating and disseminating health-related videos. Some evidences indicated that TikTok had vast communication potential during the COVID-19 pandemic.^13,29 In addition, BiliBili and TikTok contain numerous popular videos about diabetes, inflammatory bowel disease, obstructive pulmonary disease, liver cancer and other diseases, which have gained a plenty number of likes, comments, and so on.^16–18,30 In our study, we found a similar trend in the abundance of videos related to thyroid nodules, capturing the interests of viewers. The 200 videos surveyed in our study have received approximately 3.2 million likes and 152,000 comments since they were published. Most of the videos achieved high popularity and engagement. TikTok and BiliBili may be promising platforms for disseminating health information including the thyroid nodules. Short videos offer a valuable opportunity to disseminate health information, but they also pose potential risks to human health and have some negative effects. Due to the diversity of uploaders and the lack of monitoring and censorship mechanisms on short video platforms lead to frequent display of poor quality and unreliable videos,³¹ or even spread of deceptive and misleading contents to the public.^14,32 A previous study on heart failure videos on TikTok found that the video contents lacked comprehensiveness and the reliability and quality of the videos were questionable.³³ Similar analyses have revealed that there are poor information quality in short videos about anorexia, idiopathic pulmonary fibrosis and other fields.^34,35 More seriously, several studies have shown the potential for short video platforms to be misleading in disseminating health information.^{14,15,34–36} Simon et al. evaluated the atopic eczema-related videos on short video platform and surprisingly found that up to 34.0% of the videos contained potentially harmful information.¹⁴ Alarmingly, a study evaluating the quality of psoriasis-related videos found that over half of the videos spread misleading information and about 10% even provide dangerous recommendations.³⁷ Compounding this issue, misinformation tends to spread faster and wider than verified information.¹¹ Patients relying on such inaccurate information from short videos may make misguided health decisions, putting themselves at risk. To ensure the dissemination of accurate health knowledge, it is crucial for video platforms to strengthen their monitoring and censorship mechanisms. This will help guarantee the high quality and reliability of the videos and protect users from potentially harmful or misleading information.

Quality and reliability of the short videos about thyroid nodules

To the best of our knowledge, the quality and reliability of online information pertaining to thyroid nodules, specifically in short video platforms are still unclear. As the first study to fill this gap, the results of this study indicate that the majority of these videos fail to meet satisfactory standards. Only 8.0% (16/200) and 14.0% (28/200) of the videos were of excellent and good quality, offering scientifically accurate information and being beneficial to patients. Meantime, only 3.0% (6/200) and 9.5% (19/200) of the videos were deemed as reliable or more reliable, having more objective sources of information and convincing power. Previous studies have indicated that the overall quality of health education videos varied according to the scores.^30,38 The videos uploaded by professionals and professional institutions seemed better than that of videos uploaded by others.¹⁶ Supporting that, our results showed that the videos from medical professionals, especially the thyroid specialists, possess comparatively higher guidance value, as they exhibited superior quality and reliability. This may be attributed to their deeper understanding of professional knowledge, clinical guidelines, and up-to-date research about thyroid nodules, whereas non-medical professionals, such as patients and individual science communicators, rely more on their own experience and personal insights, leading to biases in sharing videos.³⁹ This indicates that the accumulation of expertise is crucial for healthcare information videos.⁴⁰ Unfortunately, our study found that thyroid specialists contributed only a limited proportion, accounting for merely 20% (40/200), while others shared a higher proportion of videos, thereby reducing the quality of thyroid nodules videos on short video platforms. Consequently, thyroid specialists and medical institutions should be encouraged to provide higher-quality videos about science popularization of thyroid nodules, increasing the public correct understanding of thyroid nodules and leveraging the potential of social media channel to promote public health.

Correlation of video quality and reliability with video variables

Studies have shown that video variables, such as likes and comments, may have a certain relationship with the quality and reliably of the videos.^14,40 A previous analysis of the videos quality of gallstone on TikTok found a negative correlation between likes and DISCERN score.¹⁹ Unfortunately, we found there was no correlation in video quality and reliability with the video variables such as likes, comments, shares and saves, which is consistent with the findings of Zheng et al.¹⁶ Nevertheless, we observed some interesting trends across videos from different sources and contents. Surprisingly, videos published by thyroid specialists, despite have higher quality, received less attention in terms of likes, comments, saves, and shares. Similarly, higher quality of the videos about disease knowledge attracted less attention than lower quality video about disease treatments on BiliBili. These findings are consistent with Simon's study, which highlighted that patients preferred lower quality videos when watching videos.³⁷ Similar trends favoring useless videos over useful ones have been reported in studies on hemodialysis and vaccination, which were very confusing and puzzling.^36,41 The reason for the above may be that high-quality videos often employ rigorous language and complex professional vocabularies, which is less attractive to nonprofessionals.^42,43 In addition, these trends might also be influenced by the recommendation mechanisms on short video platforms.¹⁹ Specifically, videos with more likes and interactions are more likely to be recommended, thereby widening the gap between video quality and popularity. The use of animations will be more vivid and interesting, which could attract more attention of people. Animations were reported to be correlated positively with popularity, measured by likes and views.^11,44 However, this relationship was not found in our correlation analysis, which may be related to the limited number of animations. Among the few animations, there exhibited high likes, comments, shares and saves, especially on TikTok. Taking all these factors into consideration, it prompts us that animations may have an advantage in sharing healthcare videos. As a consequence, it is recommended for thyroid specialists to consider people's needs and preferences when uploading videos, utilizing simpler language and incorporating visuals, animations and other forms. Furthermore, platforms should set up a video screening mechanism, prioritize professional and high-quality videos in search results to ensure the dissemination of accurate knowledge.

Correlation analysis

Prior research has demonstrated a correlation between the number of likes, comments, and saves on social media.⁴⁵ Our findings indicate a strong positive correlation (above 0.78) between likes, comments, saves, and shares. This implies that content with a high number of likes is also likely to receive more comments, be saved by more users, and be shared. This high correlation suggests that positive user feedback tends to manifest in multiple forms of interaction.

Advantages and significance

The popularity of online social media, particularly short video platforms, highlights the importance of examining the quality of health-related content. The Chinese government has recently issued guidelines on the publication and dissemination of health science knowledge through various media platforms.¹⁶ Therefore, it is crucial to evaluate the quality and reliability of existing health videos. Our research serves as the first study to evaluate the quality and reliability about thyroid nodules videos. Meanwhile, we analyzed the two largest short video platforms in China: BiliBili and TikTok, avoiding the deviations brought by a single platform, making the results more representative and reliable. Besides, we analyzed the information of videos from two perspectives through the GQS and DISCERN score, which will lead to more comprehensive and complete results. In addition, our research provides valuable insights for promoting the dissemination of correct and high-level health knowledge. The first of all, short video platforms should strengthen their video monitoring and censorship mechanisms to ensure the quality of videos. Meantime, prioritizing professional, high-quality videos in search results will help ensure that accurate knowledge is available to the public efficiently. Furthermore, specialists and medical institutions should be encouraged to provide higher quality videos combining that cater the needs and preferences of the masses and appropriately leverage the power of this social media channel to promote public health. Finally, patients should be cautious and selective when browsing short videos, and avoid making medical decisions solely based on the content of health videos. Raising awareness among viewers about critically evaluating health information is crucial. In conclusion, the dissemination of high-quality health videos requires collaborative efforts to improve public health knowledge. By implementing the recommendations outlined in this study, we can contribute to the availability of accurate health information on short video platforms.

Limitations

However, our study does have certain limitations that should be acknowledged. Firstly, the videos on TikTok and BiliBili are constantly updated, making it difficult to assess their quality dynamically. Evaluations were done at a specific time and subjective differences in assessment may still exist, suggesting the need for a broader range of expert evaluations in future studies. Secondly, our assessment was limited to videos uploaded on Chinese short video platforms. However, these findings may not be generalizable to other language platforms, emphasizing the need for cross-language research subsequently.

Conclusions

This study assessed the quality of 200 videos on thyroid nodules on TikTok and BiliBili using GQS and DISCERN score. Overall, the videos on these platforms were deemed unsatisfactory in quality and reliability. TikTok videos on thyroid nodules were slightly better than those on BiliBili. Videos shared by thyroid specialists were found to be of higher quality and reliability, providing more valuable information for viewers. Therefore, it is crucial for healthcare professionals and institutions to acknowledge the rise of short videos and provide high-quality content on thyroid nodules. Additionally, short video platforms should strengthen their monitoring and censorship mechanisms, while patients should exercise caution and selectivity when browsing short videos on BiliBili and TikTok.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241288831 - Supplemental material for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study

Supplemental material, sj-docx-1-dhj-10.1177_20552076241288831 for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study by Yuying Chen, Qianyu Wang, Xuanwei Huang, Yu Zhang, Yuanhui Li, Tianyu Ni, Gang Pan, Dingcun Luo and Yeqin Ni in DIGITAL HEALTH

Supplemental Material

sj-doc-2-dhj-10.1177_20552076241288831 - Supplemental material for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study

Supplemental material, sj-doc-2-dhj-10.1177_20552076241288831 for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study by Yuying Chen, Qianyu Wang, Xuanwei Huang, Yu Zhang, Yuanhui Li, Tianyu Ni, Gang Pan, Dingcun Luo and Yeqin Ni in DIGITAL HEALTH

Supplemental Material

sj-doc-3-dhj-10.1177_20552076241288831 - Supplemental material for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study

Supplemental material, sj-doc-3-dhj-10.1177_20552076241288831 for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study by Yuying Chen, Qianyu Wang, Xuanwei Huang, Yu Zhang, Yuanhui Li, Tianyu Ni, Gang Pan, Dingcun Luo and Yeqin Ni in DIGITAL HEALTH

Supplemental Material

sj-doc-4-dhj-10.1177_20552076241288831 - Supplemental material for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study

Supplemental material, sj-doc-4-dhj-10.1177_20552076241288831 for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study by Yuying Chen, Qianyu Wang, Xuanwei Huang, Yu Zhang, Yuanhui Li, Tianyu Ni, Gang Pan, Dingcun Luo and Yeqin Ni in DIGITAL HEALTH

Supplemental Material

sj-doc-5-dhj-10.1177_20552076241288831 - Supplemental material for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study

Supplemental material, sj-doc-5-dhj-10.1177_20552076241288831 for The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: Cross-sectional study by Yuying Chen, Qianyu Wang, Xuanwei Huang, Yu Zhang, Yuanhui Li, Tianyu Ni, Gang Pan, Dingcun Luo and Yeqin Ni in DIGITAL HEALTH

Footnotes

Acknowledgements

The authors would like to express their gratitude to the participants who participated in the study.

Contributors

Yuying Chen did conceptualization and writing the original draft. Qianyu Wang did conceptualization, formal analysis, and validation. Xuanwei Huang performed data curation and formal analysis. Yu Zhang did data curation. Yuanhui Li did funding acquisition and data curation. Tianyu Ni was responsible for the software and visualization. Gang pan did funding acquisition, writing the review and editing. Dingcun Luo participated in writing, reviewing, editing, and funding acquisition. Yeqin Ni did writing, reviewing, editing, and project administration. All authors contributed to the article and approved the submitted version.

Data availability

Raw data were generated at Hangzhou First People's Hospital. Derived data supporting the findings of this study are available from the corresponding authors on request.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

No clinical data, human specimens, or laboratory animals were used in this study. All information was obtained from publicly released TikTok and BiliBili videos, and none of the data involved personal privacy concerns. In addition, this study did not involve any interaction with users; therefore, no ethics review was needed.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Zhejiang Medical and Health Science and Technology Plan Project, Medical and Health Technology Projects of Hangzhou, Zhejiang Basic Public Welfare Research Plan Project (grant numbers 2023KY179, 2023RC218 and 2022KY939, Z20210025, LQ24C120004 and LGF22H160082).

Guarantor

Dingcun Luo; Yeqin Ni.

ORCID iDs

Xuanwei Huang

Yeqin Ni

Supplemental material

Supplemental material for this article is available online.

References

Lim

Devesa

Sosa

, et al. Trends in thyroid cancer incidence and mortality in the United States, 1974–2013. JAMA 2017; 317: 1338–1348. [Medline:28362912].

Wong

Farrell

Grossmann

. Thyroid nodules: diagnosis and management. Med J Aust 2018; 209: 92–98. [Medline:29996756].

Alexander

Doherty

Barletta

. Management of thyroid nodules. Lancet Diabetes Endocrinol 2022; 10: 540–548. [Medline:35752201].

Grani

Sponziello

Pecce

, et al. Contemporary thyroid nodule evaluation and management. J Clin Endocrinol Metab 2020; 105: 2869–2883. [Medline:32491169].

Cherella

Feldman

Hollowell

, et al. Natural history and outcomes of cytologically benign thyroid nodules in children. J Clin Endocrinol Metab 2018; 103: 3557–3565.

Grani

Sponziello

Filetti

, et al. Thyroid nodules: diagnosis and management. Nat Rev Endocrinol. Epub ahead of print 2024.

Kobaly

Kim

Mandel

. Contemporary management of thyroid nodules. Annu Rev Med 2022; 73: 517–528.

Tian

, et al. Iodine nutrition and the prevalence status of thyroid nodules in the population: a cross-sectional survey in Heilongjiang Province, China. Biol Trace Elem Res 2021; 199: 3181–3189.

Zhang

Gao

Han

, et al. Correlation between thyroid nodules and metabolic syndrome: a systematic review and meta-analysis. Front Endocrinol (Lausanne) 2021; 12: 730279.

10.

Paschou

Vryonidou

Goulis

. Thyroid nodules: a guide to assessment, treatment and follow-up. Maturitas 2017; 96: 1–9. [Medline:28041586].

11.

Wang

, et al. A quality analysis of thyroid cancer videos available on TikTok. Front Public Health 2023; 11: 1049728. [Medline:37033054].

12.

Zhu

Zhang

, et al. How health communication via Tik Tok makes a difference: a content analysis of Tik Tok accounts run by Chinese provincial health committees. Int J Environ Res Public Health 2020; 17: 192. [Medline:31892122].

13.

Basch

Fera

Pierce

, et al. Promoting mask use on TikTok: descriptive, cross-sectional study. JMIR Public Health Surveill 2021; 7: e26392. [Medline:33523823].

14.

Mueller

Hongler

VNS

Jungo

, et al. Fiction, falsehoods, and few facts: cross-sectional study on the content-related quality of atopic eczema-related videos on YouTube. J Med Internet Res 2020; 22: e15599. [Medline:32329744].

15.

Sampige

Rodgers

Huang

, et al. Education and misinformation: exploring ophthalmology content on TikTok. Ophthalmol Ther 2024; 13: 97–112. [Medline:37875699].

16.

Zheng

Tong

Wan

, et al. Quality and reliability of liver cancer-related short Chinese videos on TikTok and BiliBili: cross-sectional content analysis study. J Med Internet Res 2023; 25: e47210. [Medline:37405825].

17.

Song

Xue

Zhao

, et al. Short-video apps as a health information source for chronic obstructive pulmonary disease: information quality assessment of TikTok videos. J Med Internet Res 2021; 23: e28318. [Medline:34931996].

18.

Wang

Song

, et al. The reliability and quality of short videos as a source of dietary guidance for inflammatory bowel disease: cross-sectional study. J Med Internet Res 2023; 25: e41518. [Medline:36757757].

19.

Sun

Zheng

. Quality of information in gallstone disease videos on TikTok: cross-sectional study. J Med Internet Res 2023; 25: e39162. [Medline:36753307].

20.

Ferhatoglu

Kartal

Ekici

, et al. Evaluation of the reliability, utility, and quality of the information in sleeve gastrectomy videos shared on open access video sharing platform YouTube. Obes Surg 2019; 29: 1477–1484. [Medline:30706318].

21.

Kaplan

Solak

. Evaluation of YouTube videos on hepatocellular carcinoma. J Korean Med Sci 2023; 38: e50. [Medline:36808545].

22.

Langille

Bernard

Rodgers

, et al. Systematic review of the quality of patient information on the internet regarding inflammatory bowel disease treatments. Clin Gastroenterol Hepatol 2010; 8: 322–328. [Medline:20060070].

23.

Singh

. YouTube for information on rheumatoid arthritis—a wakeup call. J Rheumatol 2012; 39: 899–903. [Medline:22467934].

24.

Sun

Yang

Zheng

. Evaluation of the liver disease information in Baidu encyclopedia and Wikipedia: longitudinal study. J Med Internet Res 2021; 23: e17680. [Medline:33459597].

25.

Landis

Koch

. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. [Medline:843571].

26.

Kwong

Medici

Angell

, et al. The influence of patient age on thyroid nodule formation, multinodularity, and thyroid cancer risk. J Clin Endocrinol Metab 2015; 100: 4434–4440.

27.

Grajales III

Sheps

, et al. Social media: a review and tutorial of applications in medicine and health care. J Med Internet Res 2014; 16: e13. [Medline:24518354].

28.

Madathil

Rivera-Rodriguez

Greenstein

, et al. Healthcare information on YouTube: a systematic review. Health Informatics J 2015; 21: 173–194. [Medline:24670899].

29.

Comp

Dyer

Gottlieb

. Is TikTok the next social media frontier for medicine. AEM Educ Train 2021; 5. DOI: https://doi.org/10.1002/aet2.10532[Medline:34095694].

30.

Kong

Song

Zhao

, et al. TikTok as a health information source: assessment of the quality of information in diabetes-related videos. J Med Internet Res 2021; 23: e30409. [Medline:34468327].

31.

Lai

Liao

, et al. The status quo of short videos as a health information source of Helicobacter pylori: a cross-sectional study. Front Public Health 2024; 11: 1344212. [Medline:38259733].

32.

Zimba

Gasparyan

. Social media platforms: a primer for researchers. Reumatologia 2021; 59: 68–72. [Medline:33976459].

33.

Gong

Dong

, et al. TikTok video as a health education source of information on heart failure in China: a content analysis. Front Public Health 2023; 11: 1315393. [Medline:38146471].

34.

Syed-Abdul

Fernandez-Luque

Jian

W-S

, et al. Misleading health-related information promoted through video-based social media: anorexia on YouTube. J Med Internet Res 2013; 15: e30. [Medline:23406655].

35.

Goobie

Guler

Johannson

, et al. YouTube videos as a source of misinformation on idiopathic pulmonary fibrosis. Ann Am Thorac Soc 2019; 16: 572–579. [Medline:30608877].

36.

Donzelli

Palomba

Federigi

, et al. Misinformation on vaccination: a quantitative analysis of YouTube videos. Hum Vaccin Immunother 2018; 14: 1654–1659. [Medline:29553872].

37.

Mueller

Jungo

Cajacob

, et al. The absence of evidence is evidence of non-sense: cross-sectional study on the quality of psoriasis-related videos on YouTube and their reception by health seekers. J Med Internet Res 2019; 21: e11935. [Medline:30664460].

38.

ReFaey

Tripathi

Yoon

, et al. The reliability of YouTube videos in patients education for glioblastoma treatment. J Clin Neurosci 2018; 55: 1–4. [Medline:30075980].

39.

Berland

Elliott

Morales

, et al. Health information on the internet: accessibility, quality, and readability in English and Spanish. JAMA 2001; 285: 2612–2621. [Medline:11368735].

40.

Yilmaz Ferhatoglu

Kudsioglu

. Evaluation of the reliability, utility, and quality of the information in cardiopulmonary resuscitation videos shared on Open access video sharing platform YouTube. Australas Emerg Care 2020; 23: 211–216. [Medline:32563667].

41.

Garg

Venkatraman

Pandey

, et al. YouTube as a source of information on dialysis: a content analysis. Nephrology (Carlton) 2015; 20: 315–320. [Medline:25641264].

42.

Reddy

Kearns

Alvarez-Arango

, et al. YouTube and food allergy: an appraisal of the educational quality of information. Pediatr Allergy Immunol 2018; 29: 410–416. [Medline:29512839].

43.

Xue

Yang

, et al. TikTok as an information hodgepodge: evaluation of the quality and reliability of genitourinary cancers related content. Front Oncol 2022; 12: 789956. [Medline:35242704].

44.

Cassidy

Fitzgerald

Cassidy

, et al. YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surg Sports Traumatol Arthrosc 2018; 26: 840–845. [Medline:28314888].

45.

Mao

Zhao

Jiang

, et al. Evaluation of TikTok videos on acute pancreatitis: content quality and reliability analysis. BMC Public Health 2024; 24: 1216.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.01 MB

0.02 MB

0.03 MB

0.01 MB