Sage Journals: Discover world-class research

Abstract

Objective

This study was designed to systematically evaluate the reliability and quality of content related to sleep disorders on four leading Chinese short-form video platforms (TikTok, Bilibili, Kwai, and Xiaohongshu) to inform strategies to improve the dissemination of accurate health information in China.

Methods

A cross-sectional analysis was conducted in March 2025, in which 400 short videos related to sleep disorders were identified and included. These videos were published between 2022 and 2025 on four major short-form video platforms. Video quality was assessed using the Global Quality Scale (GQS), JAMA benchmark criteria (JAMA), and a modified version of the DISCERN tool. Influencing factors were examined using Spearman correlation analysis and Poisson regression, with all multiple comparisons adjusted using the Bonferroni correction.

Results

Video popularity metrics and quality scores (GQS, JAMA, modified DISCERN) were significantly higher on TikTok than on other platforms (Adjusted P < .001). The quality of videos produced by physicians (64%) was better than that of laypersons (Adjusted P < .005), with the quality of videos produced by Traditional Chinese Medicine (TCM) practitioners significantly higher than that of Clinical medicine practitioners (Adjusted P < .05). The highest quality scores were found for videos presented as expert commentary (52%). After Bonferroni correction, quality differences among specific “disease knowledge” categories were no longer statistically significant (Adjusted P > .05). Although user interaction metrics showed a statistically significant correlation with quality scores (Adjusted P < .001), the Spearman correlation coefficients were weak to moderate (r = .18–.33), indicating the practical association was limited.

Conclusions

Information on sleep disorders on Chinese short-video platforms is of low quality. These findings are specific to the Chinese digital ecosystem and may not be generalizable to global platforms. Content posted by medical professionals (especially TCM practitioners) was associated with higher reliability scores, suggesting a potential role for professional oversight in improving the quality of sleep disorder information on short video platforms.

Keywords

Sleep disorders short videos health information quality and reliability assessment

Highlights

This study represents the first systematic evaluation of the quality and reliability of sleep disorder-related content across China's four main short-form video platforms. An innovative combination of three assessment tools—GQS, the JAMA Benchmarks, and a modified version of DISCERN—enables a multidimensional evaluation of videos. A unique insight highlights the benefits of doctor-led content, especially videos created by traditional Chinese medicine practitioners, in improving content credibility and quality. Platforms should adopt specific strategies—such as implementing stricter censorship procedures and enhancing video quality—to improve the accuracy of sleep disorder information.

Introduction

Sleep disorders constitute a widespread public health concern that adversely affects quality of life and overall well-being.¹ The most prevalent forms include insomnia and obstructive sleep apnoea (OSA).² Statistical analyses indicate that approximately 30% of adults worldwide are affected by sleep disorders.^3,4 These conditions are associated with a range of adverse effects depending on their type, including excessive daytime sleepiness, which increases the risk of accidents; impaired physical and mental health, leading to reduced quality of life; and an elevated susceptibility to cardiovascular diseases and premature mortality.^5–7 Early identification and treatment of sleep disorders are crucial for reducing their adverse impacts. Lifestyle modifications can also play an effective role in preventing both the onset and recurrence of these conditions.^8,9 Hence, improving public health literacy and providing self-management education are essential.

The proliferation of internet technology has led to the gradual replacement of traditional paper-based media with electronic information. As a result, an increasing number of people are turning to the internet to obtain detailed health-related information.^10–12 In recent years, short-form video platforms have significantly expanded the dissemination of health-related video content.^13,14 However, the lack of effective censorship mechanisms on these platforms allows users to frequently upload videos containing false or misleading information. This, in turn, contributes to the generally low quality and credibility of health-related videos.^15,16 Research into medical content on global platforms such as YouTube and the international TikTok consistently indicates that similar issues of widespread misinformation and substandard educational materials are prevalent.^17–19 Consequently, users are more likely to be exposed to erroneous health information, which may lead them to make health decisions based on inaccuracies.^20,21 While numerous studies have evaluated the quality of disease-related videos on established platforms such as YouTube,^22,23 the content on emerging short-form video applications remains underexplored. In 2021, TikTok (owned by ByteDance) reported over 1.6 billion monthly active users, while Bilibili attracted millions through its diverse content offerings.^24,25 Similarly, Xiaohongshu and Kwai have captured substantial market shares with their large user bases and distinctive content ecosystems.^26,27 Despite the ease of accessing health information on these platforms, users often struggle to assess its quality and accuracy.^28,29 Therefore, there is a growing need for the public to critically evaluate the quality, content, and credibility of health-related videos.

Previous studies have examined health information on conditions such as hypertension, pancreatitis, and lung cancer on video platforms.^30–32 However, systematic evaluation of sleep disorder-related content on short-form video platforms—particularly in China—remains lacking. Existing analyses are also constrained using a single assessment tool. This study is the first to employ three validated instruments (GQS, JAMA, and modified DISCERN) to comprehensively evaluate the quality and reliability of sleep disorder-related videos across four major Chinese platforms (TikTok, Bilibili, Kwai, and Xiaohongshu) from multiple dimensions. Based on prior research,^33,34 we hypothesized that: first, videos created by medical professionals would receive significantly higher quality scores than those from non-professional sources; second, content related to TCM would demonstrate higher reliability than clinical medicine content across standardized metrics; and third, user engagement indicators (e.g., likes, saves) would positively correlate with video quality scores. This study is essential given the high prevalence of sleep disorders and the public's growing reliance on short videos for health information.

Methods

STROBE statement compliance

This cross-sectional study adhered to the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines. The completed STROBE checklist is provided in Supplementary Table 4.

Ethical considerations

This study did not require ethical approval as it solely analyzed publicly accessible short-form videos and did not involve human participants. No personally identifiable or user identity data was collected.

Search strategy and data collection

This cross-sectional study searched the Chinese versions of TikTok, Bilibili, Kwai, and Xiaohongshu on March 22, 2025, using the keywords“睡眠障碍” (sleep disorders), “失眠” (insomnia), “入睡困难” (difficulty sleeping), and “睡眠呼吸暂停” (sleep apnea) . Data were collected between 8:00 and 20:00 on the same day. To minimize algorithmic bias, all searches were performed from the same IP address using newly created accounts in Chrome Incognito mode (v121.0). Default sorting settings (“comprehensive ranking” on all platforms) were applied, and no user accounts were logged in to prevent personalized content interference. The top 100 videos by default sorting from 2022 to 2025 on each platform, were selected for analysis. To minimize bias from personalized recommendation algorithms, new user accounts were created and accessed before data collection. Clear inclusion and exclusion criteria were defined. Included videos were required to: (1) focus explicitly on sleep disorders, and (2) be in Chinese. Exclusion criteria were: (1) duplicates; (2) promotional or advertising content; (3) videos unrelated to sleep disorders; and (4) content without a clear author (Figure 1).To ensure consistent application of these criteria, the following steps were taken: duplicates were identified by comparing video titles and core messages; relevance was independently assessed by two researchers based on thematic focus; and advertising material was excluded based on commercial intent. All exclusions were reviewed, and any disagreements were resolved by a third researcher (Z-DR) to reach consensus. The selection of the top 100 videos from each platform was based on two key considerations. First, the search algorithms of TikTok, Bilibili, Kwai, and Xiaohongshu prioritize relevance, with results beyond the top 100 showing markedly decreased association with sleep disorders. Second, user behavior data indicate that most viewers engage primarily with the most popular results, making the top 100 videos both representative and relevant to typical audience exposure.^35,36 Although this approach might overlook some less popular yet high-quality content, it offers an ecologically valid reflection of the information typically encountered by users. We extracted and analyzed the following characteristics of included videos: number of likes, comments, saves, shares, days since publication, video duration, source of video, uploader's follower count, presentation format, video content theme, as well as related diseases and medical expertise involved. Video data were extracted within 24 hours after the search to minimize variations in engagement metrics over time. All data were recorded using Microsoft Excel.

Figure 1.

Search strategies for short videos on sleep disorders.

Classification of videos

Videos were categorized according to five criteria: source, content type, disease topic, presentation format, and medical specialty. The sources of the videos were categorized as follows: (1) physicians, (2) hospitals, (3) news agencies, and (4) independent users. Video content was categorized into three types: (1) disease knowledge, (2) outpatient scenarios, and (3) personal experiences. The knowledge of different diseases was classified into seven categories: (1) treatment, (2) prevention, (3) incidence, (4) symptom, (5) definition, (6) posttreatment caveats, and (7) reexamination. Video presentation formats were classified as follows: (1) expert monologues, (2) dialogues, (3) visual pictures and literature, and (4) vlogs of patients. The different medical specialties were classified into two categories: (1) Clinical medicine and (2) TCM.

Video quality and reliability assessments

The JAMA Benchmark Criteria, a well-established tool for assessing the reliability of health-related websites, comprises four dimensions: authorship, attribution, disclosure, and currency. The GQS is a 5-point Likert scale used to subjectively rate the overall quality of videos based on their flow and usability, ranging from 1 (poor quality) to 5 (excellent flow and high quality). The modified DISCERN instrument, widely endorsed for evaluating the reliability and quality of online health information, includes five questions, each of which is scored one point. This study employed the JAMA, GQS, and modified DISCERN tools to evaluate video content. Detailed scoring criteria for each tool are provided in Supplementary Tables 1, 2, and 3.

Table 1.

General characteristics and scoring of videos related to sleep disorders.

Variable	Total(n = 100)	TikTok(n = 100)	Bilibili(n = 100)	Kwai(n = 100)	Xiaohongshu(n = 100)	P value
Likes, median (IQR)	2120 (349–10,309)	8677 (3443–48,500)	3020 (1172–9174)	1390 (63–27,300)	126 (34–543)	<.001
Comments, median (IQR)	118 (20–599)	522 (132–1666)	175 (73–434)	162 (10–924)	7 (2–30)	<.001
Saves, median (IQR)	1175 (152–5437)	4068 (1179–22,433)	1736 (852–6138)	726 (22–10,320)	83 (26–353)	<.001
Shares, median (IQR)	506 (82–2724)	2353 (834–23,562)	587 (254–1659)	564 (25–5071)	44 (13–187)	<.001
Days since published, median (IQR)	120 (30–317)	52 (25–212)	355 (120–542)	82 (15–219)	130 (29–265)	<.001
Duration (seconds), median (IQR)	80 (43–167)	61 (44–111)	217 (117–562)	51 (14–84)	72 (39–131)	<.001
Fans, median (IQR)	1,17,000 (24,000–5,87,634)	3,09,000 (1,01,582–28,14,250)	1,31,500 (26,000–5,73,500)	2,80,100 (90,000–6,27,825)	12,000 (2853–50,000)	<.001
Year, n (%)						<.001
2022	2 (0%)	0 (0%)	0 (0%)	1 (1%)	1 (1%)
2023	58 (14%)	9 (9%)	36 (36%)	1 (1%)	12 (12%)
2024	181 (45%)	37 (37%)	47 (47%)	49 (49%)	48 (48%)
2025	159 (40%)	54 (54%)	17 (17%)	49 (49%)	39 (39%)
GQS score, n (%)						<.001
1	2 (0%)	0 (0%)	0 (0%)	2 (2%)	0 (0%)
2	83 (21%)	17 (17%)	26 (26%)	15 (15%)	26 (26%)
3	229 (57%)	47 (47%)	64 (64%)	62 (62%)	56 (56%)
4	74 (18%)	31 (31%)	7 (7%)	20 (20%)	15 (15%)
5	12 (3%)	5 (5%)	3 (3%)	1 (1%)	3 (3%)
JAMA score, n (%)						<.001
1	22 (6%)	3 (3%)	3 (3%)	7 (7%)	10 (10%)
2	235 (59%)	45 (45%)	79 (79%)	57 (57%)	55 (55%)
3	133 (33%)	48 (48%)	16 (16%)	34 (34%)	33 (33%)
4	10 (2%)	4 (4%)	2 (2%)	2 (2%)	2 (2%)
Modified DISCERN score, n (%)						.014
1	7 (2%)	0 (0%)	1 (1%)	4 (4%)	2 (2%)
2	91 (23%)	14 (14%)	33 (33%)	19 (19%)	26 (26%)
3	189 (47%)	46 (46%)	48 (48%)	46 (46%)	48 (48%)
4	104 (26%)	37 (37%)	15 (15%)	29 (29%)	23 (23%)
5	9 (2%)	3 (3%)	3 (3%)	2 (2%)	1 (1%)

Abbreviations: JAMA: Journal of American Medical Association; GQS: Global Quality Scale.

Table 2.

Sources and content of videos related to sleep disorders.

Variable	Total(n = 400), n (%)	TikTok (n = 100), n (%)	Bilibili(n = 100), n (%)	Kwai(n = 100), n (%)	Xiaohongshu(n = 100),n (%)	P value
Video Source						<.001
Physicians	255 (64%)	74 (74%)	57 (57%)	74 (74%)	50 (50%)
Hospital	13 (3%)	3 (3%)	2 (2%)	3 (3%)	5 (5%)
News agencies	31 (8%)	11 (11%)	5 (5%)	10 (10%)	5 (5%)
Independent users	101 (25%)	12 (12%)	36 (36%)	13 (13%)	40 (40%)
Different medical specialties						<.001
Clinical medicine practitioner	220 (55%)	58 (58%)	68 (68%)	36 (36%)	58 (58%)
TCM practitioner	161 (40%)	37 (37%)	27 (27%)	60 (60%)	37 (37%)
Others	19 (5%)	5 (5%)	5 (5%)	4 (4%)	5 (5%)
Video content						<.001
Disease knowledge	334 (84%)	84 (84%)	88 (88%)	84 (84%)	78 (78%)
Outpatient scenarios	31 (8%)	5 (5%)	6 (6%)	12 (12%)	8 (8%)
Personal experience	35 (9%)	11 (11%)	6 (6%)	4 (4%)	14 (14%)
Different disease knowledge						<.001
Treatment	212 (53%)	71 (71%)	50 (50%)	50 (50%)	41 (41%)
Prevention	54 (14%)	13 (13%)	8 (8%)	13 (13%)	20 (20%)
Incidence	39 (10%)	2 (2%)	27 (27%)	3 (3%)	7 (7%)
Symptom	61 (15%)	7 (7%)	7 (7%)	24 (24%)	23 (23%)
Definition	17 (4%)	3 (3%)	4 (4%)	5 (5%)	5 (5%)
Posttreatment caveats	10 (2%)	2 (2%)	2 (2%)	3 (3%)	3 (3%)
Reexamination	7 (2%)	2 (2%)	2 (2%)	2 (2%)	1 (1%)
Video presentation form						<.001
Expert monologue	209 (52%)	63 (63%)	50 (50%)	48 (48%)	48 (48%)
Dialogue	63 (16%)	7 (7%)	7 (7%)	17 (17%)	11 (11%)
Visual pictures and literature	98 (24%)	37 (37%)	37 (37%)	30 (30%)	24 (24%)
Vlog of patients	30 (8%)	6 (6%)	6 (6%)	5 (5%)	17 (17%)

Abbreviation: TCM, traditional Chinese medicine.

All videos were collected and downloaded by a single researcher (M-DM). The video order was randomized to minimize rating bias. Two psychiatrists (M-SH and W-QL) from different departments of the institution independently performed professional evaluations and scoring of the videos. The two raters operated at different times and locations to ensure independence. Before formal scoring, both raters received systematic training on the application of the GQS, JAMA, and modified DISCERN tools, including trial scoring of 10 non-study videos. This process helped calibrate scoring standards and improve consistency. Disagreements between raters were resolved by a third arbitrator (Z-DR), with final scores determined by consensus among all authors. Inter-rater reliability, assessed using Cohen's κ, was .81 (P < .001), indicating excellent agreement.

Statistical analyses

The data in this study were nonparametrically distributed; therefore, descriptive statistics were performed using the median and interquartile range (IQR). Between-group differences were assessed using the Kruskal-Wallis test. To strictly control for Type I errors across multiple pairwise comparisons, post hoc analyses were performed using Dunn's test with the Bonferroni adjustment. Relationships between quantitative variables were evaluated using Spearman correlation analysis with Bonferroni-adjusted significance thresholds. Given the large sample size, which can yield statistical significance even for trivial associations, we prioritized the magnitude of the effect size (e.g., the correlation coefficient r) over the P-value alone when assessing clinical relevance. All analyses were performed using R version 4.4.2 (http://www.R-project.org, The R Foundation) and Empower software (www.empowerstats.com; X&Y Solutions, Inc., Boston, MA); a two-tailed P < .05 was considered statistically significant.

Results

Video characteristics

As shown in Table 1, the videos display diverse characteristics. The included sleep disorder-related videos were uploaded between 2022 and March 22, 2025. Videos from TikTok and Kwai were predominantly uploaded in 2025, constituting 54% and 49% of their respective samples, whereas those from Bilibili and Xiaohongshu were primarily from 2024, accounting for 47% and 48%, respectively. Significant differences in user interactivity and content quality were observed across platforms (P < .05). TikTok videos exhibited significantly higher user engagement compared to those on Bilibili, Kwai, and Xiaohongshu, with median values of 8677 likes, 522 comments, 2353 shares, and 4068 saves (all P < .001). In terms of video duration, Bilibili videos were significantly longer (median: 355 seconds) than those on Kwai (median: 51 seconds; P < .001). Regarding time since publication, Bilibili videos also had a significantly higher median age (355 days) than TikTok videos (52 days; P < .001). Quality assessments revealed that TikTok videos outperformed those from the other three platforms in GQS, JAMA, and modified DISCERN scores (all P < .05). Specifically, 36% of TikTok videos received a GQS score ≥4, 52% achieved a JAMA score ≥3, and 40% attained a modified DISCERN score ≥4. Analysis of the author's regional distribution showed that Beijing contributed the most videos (124), followed by Guangdong (47) and Henan (35) (Figure 2).

Figure 2.

The distribution of video authors in China.

Video source and content

As shown in Table 2 and Figure 3, Video uploaders were mainly physicians (255/400, 64%) and clinical medicine practitioners (220/400, 55%), and the content of the videos mostly included disease knowledge (334/400, 84%). Disease knowledge in the videos focused on treatment (212/400, 53%), symptoms (61/400, 15%), and prevention (54/400, 14%). Video presentations mainly include expert monologues (209/400, 52%) and visual pictures and literature (98/400, 24%).

Figure 3.

The number and percentage of sleep disorder videos from various sources and content types on TikTok, Bilibili, Kwai, and Xiaohongshu: (A) sources of the videos; (B) content types.

Video quality and reliability assessments

As shown in Figure 4, no significant difference was observed in GQS scores between TikTok and Kwai (P > .05); however, TikTok scored significantly higher than both Bilibili and Xiaohongshu (all P < .05). Similarly, JAMA scores differed significantly across all platforms (all P < .05). Modified DISCERN results were largely consistent with GQS scores, showing no notable difference between TikTok and Kwai (P > .05), but significant differences between TikTok and both Bilibili and Xiaohongshu (all P < .05). Although Kwai demonstrated certain strengths, its overall scores remained lower than those of TikTok. Additionally, many videos on Bilibili and Xiaohongshu demonstrated lower quality and reliability.

Figure 4.

GQS scores, JAMA scores, and modified DISCERN scores of short videos on sleep disorders on different platforms (TikTok, Bilibili, Kwai, and Xiaohongshu). Pairwise comparisons were performed using Dunn's test with Bonferroni correction for multiple comparisons. *P < .05, **P < .01, ***P < .001. ns: not significant at P ≥ .05.

The quality and popularity of videos from different sources, content, and presentation forms

Videos uploaded by independent users received significantly lower GQS, JAMA, and modified DISCERN scores compared to those from physicians, hospitals, and news agencies (all P < .005; Figure 5). Among physician-generated content, videos created by TCM practitioners scored significantly higher across all three metrics than those by clinical medicine practitioners (all P < .05; Figure 6). Furthermore, videos focusing on disease knowledge and outpatient scenarios achieved markedly higher scores than those based on personal experiences (all P < .01; Figure 7).

Figure 5.

GQS score, JAMA score, and modified DISCERN score from different sources of videos related to sleep disorders. Pairwise comparisons were performed using Dunn's test with Bonferroni correction for multiple comparisons. *P < .05, **P < .01, ***P < .001. ns: not significant at P ≥ .05.

Figure 6.

GQS, JAMA, and modified DISCERN scores for sleep disorder-related videos from different medical specialties. Pairwise comparisons were performed using Dunn's test with Bonferroni correction for multiple comparisons. *P < .05, **P < .01, ***P < .001. ns: not significant at P ≥ .05.

Figure 7.

GQS score, JAMA score, and modified DISCERN score for sleep disorder-related videos with different content. Pairwise comparisons were performed using Dunn's test with Bonferroni correction for multiple comparisons. **P < .01, ***P < .001. *ns: not significant at P ≥ .05.

In terms of disease knowledge content, no significant quality differences were observed among videos of different categories (all P > .05) (Figure 8). Regarding the presentation format, the patient video log consistently received the lowest score among all three assessment tools. In contrast, the scores of GQS, JAMA, and modified DISCERN obtained from expert monologues and dialogues were significantly higher than those from visual graphics/documents and patient video logs (all P<.01) (Figure 9).

Figure 8.

GQS scores, JAMA scores, and modified DISCERN scores for sleep disorder-related videos for different disease knowledge. Pairwise comparisons were performed using Dunn's test with Bonferroni correction for multiple comparisons. *ns: not significant at P ≥ .05.

Figure 9.

GQS, JAMA, and modified DISCERN scores for sleep disorder-related videos for different presentation forms. Pairwise comparisons were performed using Dunn's test with Bonferroni correction for multiple comparisons. *P < .05, **P < .01, ***P < .001. *ns: not significant at P ≥ .05.

This study also compared the popularity—measured by likes, comments, saves, and shares—of videos from different sources, content types, and presentation formats (Table 3). Among video sources, those from hospitals received significantly more shares (P < .01). Regarding content, videos providing disease knowledge—particularly those focused on treatment—garnered more comments, saves, and shares (P < .05), while those sharing personal experiences received more likes (P < .001).

Table 3.

The popularity of videos from different sources with different content and presentation forms.

Variables	Likes	Comments	Saves	Shares
Video source
Physicians, median (IQR)	1933 (172–9716)	123 (13–486)	1090 (110–5742)	591 (75–2767)
Hospital, median (IQR)	5311 (928–45,000)	412 (35–4179)	1743 (519–23,000)	2194 (315–17,700)
News agencies, median (IQR)	4308 (999–39,113)	323 (32–1520)	1832 (256–13,050)	906 (240–21,100)
Independent users, median (IQR)	2461 (380–8015)	89 (22–429)	994 (195–4233)	343 (71–1272)
P value	.086	.279	.297	.005
Different medical specialties
Clinical medicine practitioner, median (IQR)	2025 (251–9602)	116 (21–553)	900 (110–4988)	426 (72–2919)
TCM practitioner, median (IQR)	2120 (407–13,000)	112 (10–570)	1366 (189–6756)	749 (84–2711)
Others, median (IQR)	2951 (1511–10,801)	339 (58–892)	1532 (574–2649)	501 (184–1336)
P value	.618	.181	.16	.588
Video content
Disease knowledge, median (IQR)	2426 (410–13,000)	152 (25–647)	1352 (198–6824)	620 (96–4229)
Outpatient scenarios, median (IQR)	126 (32–1174)	23 (3–106)	65 (10–433)	83 (5–318)
Personal experience, median (IQR)	3013 (686–8012)	93 (42–590)	1262 (261–4234)	540 (74–1702)
P value	<.001	.008	<.001	<.001
Different disease knowledge
Treatment, median (IQR)	2996 (574–11,000)	160 (30–724)	1488 (214–7852)	855 (132–4446)
Prevention, median (IQR)	1962 (478–9083)	118 (27–420)	1347 (264–5082)	586 (95–2333)
Incidence, median (IQR)	2468 (950–8734)	226 (40–542)	1305 (328–4058)	481 (116–1102)
Symptom, median (IQR)	1086 (97–4805)	83 (5–494)	313 (44–2525)	258 (16–1166)
Definition, median (IQR)	162 (31–1981)	30 (4–142)	69 (6–803)	33 (12–104)
Posttreatment caveats, median (IQR)	3020 (92–6522)	199 (10–573)	1416 (65–3711)	464 (43–972)
Reexamination, median (IQR)	1092 (170–8663)	58 (27–592)	260 (133–2049)	185 (140–1426)
P value	.052	.288	.01	.024
Video presentation form
Expert monologue, median (IQR)	2099 (417–10,079)	117 (25–628)	1304 (164–6565)	615 (84–3067)
Dialogue, median (IQR)	1504 (58–9976)	133 (6–675)	581 (22–3684)	376 (17–3430)
Visual pictures and literature, median (IQR)	2370 (566–15,250)	131 (26–468)	1483 (217–6226)	492 (104–3259)
Vlog of patients, median (IQR)	1254 (387–4689)	55 (21–307)	574 (192–3840)	346 (74–926)
P value	.603	.824	.137	.463

Abbreviation: TCM: traditional Chinese medicine.

Correlation analysis

Given the non-normal distribution of the data, Spearman's correlation analysis was performed to evaluate relationships between video variables. To ensure statistical rigor and minimize the risk of Type I errors, a Bonferroni correction was strictly applied for multiple comparisons. The results initially indicated significant positive intercorrelations among all user interaction metrics (likes, comments, saves, shares, and fans) (all adjusted P < .001; Table 4; Figure 10). Furthermore, these metrics showed statistically significant positive correlations with video quality scores (GQS, JAMA, and modified DISCERN) (all adjusted P < .001; Table 5; Figure 10). However, it is important to note that these correlations were weak to moderate in strength, with coefficients ranging from .18 to .33. This suggests that while “popularity” and “information quality” are statistically associated, their practical predictive value is limited. In contrast, after Bonferroni correction, neither video duration nor publication date demonstrated any statistically significant correlation with quality metrics (all adjusted P > .05), indicating that content length and recency are not reliable indicators of quality.

Figure 10.

Spearman correlation analysis between video variables and quality scores. P-values were adjusted using the Bonferroni correction. Significance levels: *Adjusted P < .05, **Adjusted P < .01, ***Adjusted P < .001. Color intensity represents the strength of the correlation coefficient (r).

Table 4.

The correlation analysis between the video variables.

Variable		Likes	Comments	Saves	Shares	Days since published	Duration
Likes
	r	1	.91	.94	.92	.11	.14
	P value	—	<.001	<.001	<.001	1	.35
Comments
	r	.91	1	.83	.86	.11	.18
	P value	<.001	—	<.001	<.001	1	<.05
Saves
	r	.94	.83	1	.93	.13	.16
	P value	<.001	<.001	—	<.001	.77	.17
Shares
	r	.91	.86	.93	1	.11	.09
	P value	<.001	<.001	<.001	—	1	1
Days since published
	r	.1	.11	.13	.11	1	.28
	P value	1	1	.77	1	—	<.001
Duration
	r	.14	.18	.16	.09	.28	1
	P value	.36	<.05	.17	1	<.001	—
Fans
	r	.56	.54	.46	.48	−.06	.09
	P value	<.001	<.001	<.001	<.001	1	1

Table 5.

The correlation analysis between video variables and video quality.

Variable		GQS score	JAMA score	Modified DISCERN score
Likes
	r	.31	.28	.21
	P value	<.001	<.001	<.001
Comments
	r	.29	.27	.20
	P value	<.001	<.001	<.001
Saves
	r	.31	.26	.18
	P value	<.001	<.001	<.001
Shares
	r	.33	.32	.25
	P value	<.001	<.001	<.001
Days since published
	r	−.02	.03	−.02
	P value	1	1	1
Duration
	r	−.02	−.07	−.06
	P value	1	1	1
Fans
	r	.24	.2032	.2031
	P value	<.001	<.001	<.001

Abbreviations: JAMA: Journal of the American Medical Association; GQS: Global Quality Scale. Values presented are the Spearman correlation coefficient (r) and the corresponding Bonferroni-adjusted P-value.

Values presented are the Spearman correlation coefficient (r) and the corresponding Bonferroni-adjusted P-value.

Discussion

Principal findings

This study is the first to use a multi-method approach to assess the quality and reliability of sleep disorder-related videos on four major Chinese short-form platforms: TikTok, Bilibili, Kwai, and Xiaohongshu. The findings reveal that although public engagement is high, the overall quality of videos remains inadequate. Most videos were created by clinicians, with notable geographic concentration in Beijing, Guangdong, and Henan. Content primarily delivered disease knowledge through expert narration, mixed image-text formats, and dialogues, focusing mainly on treatment, symptoms, and prevention. Videos from medical professionals—particularly TCM practitioners, hospitals, and news organizations showed higher quality. This study addresses a critical gap in sleep medicine-related online health information and provides insights that may inform future platform oversight and professional health communication strategies.

Factors correlated with the popularity of videos

Previous studies have shown that metrics such as likes, comments, saves, and shares can reflect the popularity of a video.^37,38 The median number of likes, comments, saves, and shares on the video was 2,120, 118, 1,175, and 506, respectively, which suggests that sleep disorders have received widespread attention. In this study, the median values for these indicators were 2120 likes, 118 comments, 1175 saves, and 506 shares, indicating considerable public interest in sleep disorders. Although more videos were uploaded by Western medicine practitioners than by TCM practitioners, both types of content achieved similar levels of popularity. This may suggest growing public recognition and acceptance of TCM. This study found that most short-form videos primarily disseminate knowledge about sleep disorders, with treatment-related content receiving more comments, saves, and shares. Previous studies have shown that individuals with sleep disorders are 4.5 times more likely to experience accidents and have a higher risk of psychiatric disorders—such as generalized anxiety disorder and depression—compared to control groups.^39,40 Therefore, a deeper understanding of sleep disorder treatment is essential to mitigate adverse outcomes. However, many patients underestimate its clinical importance. Additionally, a positive correlation was observed between follower count and engagement metrics (likes, comments, saves, shares), suggesting that higher engagement increases the likelihood of a video being saved and shared. Furthermore, follower count significantly influenced video popularity, indicating that creators with larger followings have greater influence in disseminating health messages.

Factors related to video quality

Although physicians upload most sleep disorder-related videos and primarily focus on disease knowledge, the overall reliability and quality of these videos remain suboptimal. This finding aligns with previous studies in other medical domains. For instance, Aktas et al.⁴¹ reported that the quality of YouTube videos about prostatitis is generally low and fails to accurately convey health information. Similarly, our analysis indicates that the overall quality of videos addressing sleep disorders tends to be low, a finding consistent with multiple international studies. For instance, videos on medical topics such as diabetes and cancer on YouTube typically score lower on standardized quality metrics.^42–45 This study also identified instances where uploaders falsely claimed to be healthcare professionals—or even impersonated physicians—to gain trust and increase viewership. This issue likely stems from insufficient platform oversight, which undermines the overall credibility of sleep disorder content. Moreover, some creators may disseminate inaccurate or exaggerated information to attract traffic, potentially leading patients to make misinformed health decisions. Therefore, viewers should exercise caution when consulting online health information.

Videos uploaded by medical professionals, hospitals, and news agencies demonstrated higher quality than those from independent users, a finding consistent with prior studies evaluating health information videos.^46,47 This discrepancy may be attributed to the use of stringent vetting processes and authoritative sources by institutional uploaders to ensure informational accuracy—standards often absent among independent creators. A notable new finding of this study is that videos on sleep disorders produced by TCM practitioners were of higher quality than those created by Western clinicians. We hypothesize that this discrepancy may be attributed to the distinctive features of Traditional Chinese Medicine knowledge. For example, TCM theoretical systems are highly systematized and transmissible, often characterized by a logical structure and narrative coherence that may translate well to short-video formats. Furthermore, TCM practitioners may place greater emphasis on maintaining the integrity and authority of classical theories during knowledge dissemination. However, we emphasize that these interpretations are preliminary and speculative based on the current cross-sectional data. The specific mechanisms driving these quality differences remain to be fully understood. Future research should prioritize qualitative methodologies, such as in-depth interviews with creators and content framework analysis, to verify these hypotheses. Additionally, cross-cultural comparative analyses are needed to determine whether this phenomenon is unique to the Chinese digital ecosystem or if similar patterns exist in traditional medicine practices globally. Regarding the relationship between popularity and quality, correlations were identified between user engagement metrics (likes, comments, saves, shares, and fans) and video quality scores. This study identified statistically significant correlations (all P < .001) between these interaction indicators and objective quality measures (GQS, JAMA, modified DISCERN). Poisson regression, adjusted with the Bonferroni correction, further confirmed that certain metrics—such as likes predicting GQS and saves predicting modified DISCERN scores—served as significant predictors of specific quality dimensions (adjusted P < .05). However, we emphasize prioritizing effect size over statistical significance to interpret these findings. Although the P-values were highly significant (likely due to the large sample size, n = 400), the effect sizes were limited: Spearman correlation coefficients ranged only from .18 to .33, indicating weak-to-moderate associations. This suggests that the relationship between user engagement metrics and quality scores is negligible in practical terms. Thus, user interaction data should not be used as a reliable indicator of information quality, as highly engaged videos are not necessarily superior in clinical or practical value. Furthermore, although numerous statistically significant differences were identified (e.g., across platforms), most exhibited small effect sizes. The most clinically meaningful finding—supported by a consistently large effect size— was that videos created by healthcare professionals were significantly higher in quality than those by independent users.

Recommendations based on our results

Short videos can enhance public comprehension of complex sleep disorders, yet low-quality content poses a risk of misinformation. Therefore, the use of short videos should be recognized as a vital tool for future health education initiatives. Based on our findings, we suggest that platforms could consider promoting content from verified medical professionals, exploring enhanced identity verification processes, and developing content screening mechanisms to improve the accuracy of health information. Further research is needed to evaluate the effectiveness of such interventions.

Practical significance

Public awareness of sleep disorders is essential for timely diagnosis and prevention. Today, short-form online videos have become a major source of health information. However, the quality of such videos—including those related to sleep medicine—varies widely. This study is the first to use standardized instruments (JAMA, GQS, and modified DISCERN) to comprehensively evaluate the quality and reliability of short videos on sleep disorders. The results indicate that the overall quality of sleep disorder-related content on platforms including TikTok, Bilibili, Kwai, and Xiaohongshu is inadequate. Therefore, improving the quality of these videos is recommended to better support public education and awareness regarding sleep disorders.

Strengths and limitations

This study has several notable strengths. First, it focuses on the four major short-form video platforms in China, providing more reliable and robust results by avoiding the limitations of single-platform analyses. Second, a multidimensional approach was adopted to evaluate video content, incorporating both quality assessments (using GQS and JAMA scores) and reliability evaluations (with the modified DISCERN tool). To our knowledge, this is the first study to systematically examine the quality of short videos related to sleep disorders across multiple major social media platforms in China.

This study is subject to several limitations. First, as the analysis was confined to Chinese-language content on domestic platforms, the findings may not be generalizable to other cultural or linguistic contexts. Second, our sampling strategy introduces potential selection bias. By focusing on the top 100 videos to reflect real-world user exposure, we relied on platform algorithms that prioritize popularity. Consequently, high-quality content that failed to gain algorithmic traction—potentially due to varying platform rules or moderation mechanisms—may have been excluded. Third, while evaluators underwent systematic training and demonstrated high inter-rater reliability, the assessment tools remain subjective. Despite efforts to minimize bias through randomization and arbitration, the use of raters from a single institution without full blinding may have introduced subtle institutional bias. Fourth, the study lacks external validation; we did not evaluate the impact of video quality on actual patient outcomes or validate scores against an independent external panel. Finally, we did not control for confounding variables such as production quality or topic complexity, and statistical analyses were conducted without adjustment for multiple comparisons, potentially increasing the risk of Type I errors.

Conclusion

This study suggests that the overall quality of sleep disorder-related information on Chinese short-form video platforms remains generally low; however, content produced by medical professionals—particularly TCM practitioners—was associated with higher reliability scores. To potentially improve the quality of health information, platforms could consider implementing a tiered review mechanism that integrates manual screening with professional credential verification and might prioritize the distribution of videos created by licensed physicians. Healthcare organizations, especially those in high-activity regions such as Beijing, Guangdong, and Henan, could be encouraged to produce standardized health education content focused on disease knowledge, ideally delivered in the form of expert commentary. Furthermore, health authorities might develop and promote content guidelines for health-related short videos to help reduce public health risks associated with misleading information.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076261415944 - Supplemental material for Video in Chinese short video sharing platforms as a source of information on sleep disorders: A cross-sectional content analysis study

Supplemental material, sj-docx-1-dhj-10.1177_20552076261415944 for Video in Chinese short video sharing platforms as a source of information on sleep disorders: A cross-sectional content analysis study by Qilong Wang, Siheng Ma, Dongmei Ma, Chen Wang, Xin Qi, Sha Liu, Lei Zhang, Runwu Xiang and Dongrong Zhao in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076261415944 - Supplemental material for Video in Chinese short video sharing platforms as a source of information on sleep disorders: A cross-sectional content analysis study

Supplemental material, sj-docx-2-dhj-10.1177_20552076261415944 for Video in Chinese short video sharing platforms as a source of information on sleep disorders: A cross-sectional content analysis study by Qilong Wang, Siheng Ma, Dongmei Ma, Chen Wang, Xin Qi, Sha Liu, Lei Zhang, Runwu Xiang and Dongrong Zhao in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076261415944 - Supplemental material for Video in Chinese short video sharing platforms as a source of information on sleep disorders: A cross-sectional content analysis study

Supplemental material, sj-docx-3-dhj-10.1177_20552076261415944 for Video in Chinese short video sharing platforms as a source of information on sleep disorders: A cross-sectional content analysis study by Qilong Wang, Siheng Ma, Dongmei Ma, Chen Wang, Xin Qi, Sha Liu, Lei Zhang, Runwu Xiang and Dongrong Zhao in DIGITAL HEALTH

Supplemental Material

sj-docx-4-dhj-10.1177_20552076261415944 - Supplemental material for Video in Chinese short video sharing platforms as a source of information on sleep disorders: A cross-sectional content analysis study

Supplemental material, sj-docx-4-dhj-10.1177_20552076261415944 for Video in Chinese short video sharing platforms as a source of information on sleep disorders: A cross-sectional content analysis study by Qilong Wang, Siheng Ma, Dongmei Ma, Chen Wang, Xin Qi, Sha Liu, Lei Zhang, Runwu Xiang and Dongrong Zhao in DIGITAL HEALTH

Footnotes

Acknowledgments

The authors would like to thank all participants in this study for their assistance.

ORCID iDs

Qilong Wang

Xin Qi

Sha Liu

Dongrong Zhao

Contributorship

Each of the authors made significant contributions to this paper. Qilong Wang, Siheng Ma, and Dongmei Ma collected the data, and Dongrong Zhao formulated the research hypotheses. Qilong Wang and Siheng Ma analyzed the data, and Qilong Wang drafted the manuscript. Dongrong Zhao critically revised the manuscript for important intellectual content and supervised its writing. All authors approved the final version of the manuscript.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Gansu Provincial Disease Prevention and Control Research Project (Grant No. GSJKKY2025-26), the In-hospital Research Fund of Gansu Provincial People's Hospital (Grant No. 23GSSYD-24), and the Graduate Student Innovation and Entrepreneurship Fund of Gansu University of Chinese Medicine (Grant No. 2026CXCY-112).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

The data used in this study were from TikTok, Bilibili, Kwai, and Xiaohongshu platforms. All data were publicly available on the platform at the time of collection, and no individual consent was required for access and use. Data access and use are strictly in accordance with the terms of service of each platform.

Supplemental material

Supplemental material for this article is available online.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

References

Iranzo

Cochen De Cock

Fantini

, et al. Sleep and sleep disorders in people with Parkinson's disease. Lancet Neurol 2024; 23: 925–937.

Thorpy

. Classification of sleep disorders. Neurotherapeutics 2012; 9: 687–701.

Freeman

Sheaves

Waite

, et al. Sleep disturbance and psychiatric disorders. Lancet Psychiatry 2020; 7: 628–637.

Zhang

Wang

, et al. The association between dietary vitamin C and sleep disorders: a cohort study based on UK Biobank. Nutrients 2024; 16: 3661.

Wen

, et al. Association between sleep disorder and depression in stroke in the national health and nutrition examination surveys (NHANES) 2005 to 2014. Sleep Med 2024; 124: 201–208.

Gong

Zhao

. Association between Body Roundness Index and sleep disorder: the mediating role of depression. BMC Psychiatry 2025; 25: 12.

Lappas

Glarou

Polyzopoulou

, et al. Pharmacotherapy for sleep disturbances in post-traumatic stress disorder (PTSD): a network meta-analysis. Sleep Med 2024; 119: 467–479.

Peersmann

SHM

Grootenhuis

van Straten

, et al. Prevalence of sleep disorders, risk factors and sleep treatment needs of adolescents and young adult childhood cancer patients in follow-up after treatment. Cancers (Basel) 2022; 14: 926.

Chen

Huang

, et al. Predicting sleep quality among college students during COVID-19 lockdown using a LASSO-based neural network model. BMC Public Health 2025; 25: 21.

10.

Swire-Thompson

Lazer

. Public health and online misinformation: challenges and recommendations. Annu Rev Public Health 2020; 41: 433–451.

11.

Yeung

AWK

Hammerle

Behrens

, et al. Online information about Side effects and safety concerns of semaglutide: mixed methods study of YouTube videos. JMIR Infodemiology 2025; 5: e59767.

12.

Yang

Chen

, et al. How to select high-quality health educational short videos on social media? Insights from YouTube and douyin. BMC Med Educ 2025; 25: 1418.

13.

Gaş

Zincir

Bozkurt

. Are YouTube videos useful for patients interested in Botulinum toxin for bruxism? J Oral Maxillofac Surg 2019; 77: 1776–1783.

14.

Kang

Chen

, et al. YouTube as a source of information on preventing the use of valproic acid in women during pregnancy. BMC Public Health 2023; 23: 1225.

15.

Yan

Lyu

, et al. Assessing quality and reliability of online videos on tachycardia: a YouTube video-based study. BMC Public Health 2024; 24: 2620.

16.

Chen

, et al. The quality and reliability of online video resources concerning patient education for migraine treatment in China mainland: an electronic media-based study. BMC Public Health 2025; 25: 56.

17.

Wang

Jia

, et al. Videos on YouTube, bilibili, TikTok as sources of medical information on hashimoto's thyroiditis. Front Public Health 2025; 13: 1611087.

18.

Selvaraj

Sennimalai

Parija

, et al. Content quality and reliability of YouTube videos on oral appliance therapy for obstructive sleep apnea: a systematic analysis. Spec Care Dentist 2024; 44: 1307–1316.

19.

Thang

Garate

Thang

, et al. Short-Form medical Media: a multi-platform analysis of acne treatment information in TikTok videos, Instagram reels, and YouTube shorts. JMIR Dermatol 2023; 6: e48140.

20.

Ming

Han

Yao

, et al. Myopia information on TikTok: analysis factors that impact video quality and audience engagement. BMC Public Health 2024; 24: 1194.

21.

Mao

Zhao

Jiang

, et al. Evaluation of TikTok videos on acute pancreatitis: content quality and reliability analysis. BMC Public Health 2024; 24: 1216.

22.

Perry

Kelly

Levy

. Editorial commentary: surgical videos on YouTube are not peer reviewed and have low educational value. Arthroscopy 2024; 40: 2244–2245.

23.

Buschulte

El-Hadi

Höger

, et al.

Is YouTube a sufficient source of information on sarcoidosis?

Respir Res 2024; 25: 34.

24.

Guan

Xia

Zhao

, et al. Videos in short-video sharing platforms as sources of information on colorectal polyps: cross-sectional content analysis study. J Med Internet Res 2024; 26: e51655.

25.

Song

Xue

Zhao

, et al. Short-Video apps as a health information source for chronic obstructive pulmonary disease: information quality assessment of TikTok videos. J Med Internet Res 2021; 23: e28318.

26.

Zhao

Yao

Sui

, et al. Current status of short video as a source of information on lung cancer: a cross-sectional content analysis study. Front Oncol 2024; 14: 1420976.

27.

Lang

Chen

, et al. Refusing appearance anxiety": a content analysis of body positive posts on Chinese social media. Body Image 2023; 45: 414–419.

28.

Kwak

Park

Won

, et al. Quality and reliability evaluation of online videos on carpal tunnel syndrome: a YouTube video-based study. BMJ Open 2022; 12: e059239.

29.

Zhang

Yuan

Zhang

, et al. Short video platforms as sources of health information about cervical cancer: a content and quality analysis. PLoS One 2024; 19: e0300180.

30.

Zhu

Gao

, et al. YouTube and TikTok as sources of information on acute pancreatitis: a content and quality analysis. BMC Public Health 2025; 25: 1446.

31.

Che

, et al. The quality and reliability of short videos about hypertension on TikTok: a cross-sectional study. Sci Rep 2025; 15: 25042.

32.

Zeng

Zhang

Wang

, et al. Douyin and bilibili as sources of information on lung cancer in China through assessment and analysis of the content and quality. Sci Rep 2024; 14: 20604.

33.

Wang

Yang

Shu

, et al. Tiktok as a potential patient educational tool for thyroid-associated ophthalmopathy: a cross-sectional study. Digit Health 2024; 10: 20552076241304594.

34.

Zhang

Jie

, et al. Analyzing dissemination, quality, and reliability of Chinese brain tumor-related short videos on TikTok and bilibili: a cross-sectional study. Front Neurol 2024; 15: 1404038.

35.

Ren

, et al. The quality and reliability of short videos about premature ovarian failure on bilibili and TikTok: cross-sectional study. Digit Health 2025; 11: 20552076251351077.

36.

Chen

Wang

Huang

, et al. The quality and reliability of short videos about thyroid nodules on BiliBili and TikTok: cross-sectional study. Digit Health 2024; 10: 20552076241288831.

37.

Shi

El Haddad

Cai

, et al. Mpox (monkeypox) information on TikTok: analysis of quality and audience engagement. BMJ Glob Health 2023; 8: e011138.

38.

Pan

Cheng

Zhang

, et al. Professional content analysis and quality assessment of cardiopulmonary resuscitation educational videos on social media platforms: a comparative study of YouTube, BiliBili, and TikTok. Front Public Health 2025; 13: 1657233.

39.

Roth

. Insomnia: definition, prevalence, etiology, and consequences. J Clin Sleep Med 2007; 3: S7–10.

40.

Robbins

Quan

. Sleep disorders. NEJM Evid 2024; 3: EVIDra2400096.

41.

Aktas

Demirel

Celikkaleli

, et al. YouTube™ as a source of information on prostatitis: a quality and reliability analysis. Int J Impot Res 2024; 36: 242–247.

42.

Birch

Leziak

Jackson

, et al. Content quality of YouTube videos about gestational diabetes: systematic evaluation. JMIR Diabetes 2022; 7: e30156.

43.

Liu

Chen

Lin

, et al. YouTube/ bilibili/ TikTok videos as sources of medical information on laryngeal carcinoma: cross-sectional content analysis study. BMC Public Health 2024; 24: 1594.

44.

Zhao

. Analysis of the quality of vulvar cancer-related videos on YouTube. Transl Cancer Res 2025; 14: 102–111.

45.

Duran

Kizilkan

. Quality analysis of testicular cancer videos on YouTube. Andrologia 2021; 53: e14118.

46.

Wang

Yao

Wang

, et al. Bilibili, TikTok, and YouTube as sources of information on gastric cancer: assessment and analysis of the content and quality. BMC Public Health 2024; 24: 57.

47.

Niu

Hao

Yang

, et al. Quality of pancreatic neuroendocrine tumor videos available on TikTok and bilibili: content analysis. JMIR Form Res 2024; 8: e60033–e60033.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

0.03 MB