Abstract
Objectives
Douyin (TikTok) has gradually emerged as an important channel through which the public obtains health information. This study aimed to evaluate the content, quality, and reliability of syphilis-related short videos on Douyin.
Methods
We conducted two rounds of data collection on Douyin, extracting video duration, engagement metrics, uploader identity, and video content for syphilis-related videos. Video quality and reliability were assessed using the Global Quality Score (GQS) and the modified DISCERN (mDISCERN) tool. Correlation analyses were performed between video metrics and quality scores.
Results
A total of 81 and 95 videos were included in the first and second rounds, respectively. Clinical manifestations were the most frequently discussed topic, whereas key information on diagnosis and prognosis was insufficiently covered. In the first round, the median GQS was 2.00 (IQR: 2.00–3.00), and the median mDISCERN score was 3.00 (IQR: 3.00–3.00). In the second round, the median GQS was 2.00 (IQR: 2.00–2.00), and the median mDISCERN score was 2.00 (IQR: 2.00–2.00). In both analyses, videos uploaded by healthcare professionals had higher GQS and mDISCERN scores than those uploaded by individual users (p < 0.05). No significant correlations were found between video engagement metrics and either GQS or mDISCERN scores in either round (p > 0.05).
Conclusions
The overall quality and reliability of syphilis-related videos on Douyin are suboptimal, and current content may not adequately meet public health information needs. Strengthening the involvement of healthcare professionals and improving content structure are essential to enhance the health education value of short video platforms.
Introduction
Syphilis is an infectious disease caused by Treponema pallidum.1,2 It presents with diverse clinical manifestations, including painless ulcers and rashes in the early stages, and may involve the cardiovascular and nervous systems in later stages.3,4 Although effective diagnostic methods and antibiotic therapies are available, the global disease burden of syphilis remains significant. 5 In recent years, its incidence has increased in many countries, including China,6–8 particularly among young adults, older individuals, and men who have sex with men.9,10 Limited public awareness, delayed medical consultation, and the often asymptomatic nature of early infection have contributed to increased transmission risk and serious complications such as neurosyphilis, cardiovascular syphilis, and ocular syphilis.11–14 Furthermore, congenital syphilis is a significant public health issue, as this form of syphilis has a profound impact on child mortality, particularly stillbirths.15,16 Improving public access to scientific and reliable information about syphilis is beneficial for early diagnosis and timely treatment, which helps reduce the transmission risk and associated complications, including the incidence of congenital syphilis.
With the rapid development of digital media, social media platforms have become an important source of health information for the public.17–19 Among them, short video platforms such as TikTok are increasingly used for health communication due to their wide reach, powerful recommendation algorithms, and low production thresholds, especially among younger audiences.20–22 However, the lack of professional content review mechanisms raises concerns about misinformation, inaccuracy, and potential health risks.23–25 Previous studies have shown that the quality of health information on short video platforms varies significantly, with the overall quality and reliability of videos being suboptimal.26,27 A quality assessment study of TikTok videos about hypertension reported moderate content quality and emphasized the need for strengthened review to improve video reliability. 28 Another study on premature ovarian insufficiency found that video quality and reliability were inadequate, suggesting the need for oversight and regulation of health content on such platforms. 29 Although short video platforms have great potential in health education, studies specifically focusing on the quality of syphilis-related content remain scarce. In light of the rising incidence of syphilis and increasing public demand for accurate health information,30,31 a systematic evaluation of syphilis-related TikTok videos is important for identifying informational deficiencies and improving future public health communication strategies.
This study aims to systematically evaluate the content themes, quality, and reliability of syphilis-related short videos on TikTok and to explore the relationship between video quality and user engagement. The findings are intended to inform improvements in digital health education and the scientific dissemination of syphilis-related information.
Methods
Study design, data sources, and data search
This was a cross-sectional study aimed at evaluating the content characteristics, quality, and user engagement of syphilis-related short videos on TikTok. On July 23, 2025 and November 9, 2025, we searched the Douyin platform (TikTok in China) using the Chinese keyword “梅毒” and did not use any English search terms or hashtags. To minimize the potential impact of platform algorithms on search results, we cleared the browser history and used a new Douyin account on Douyin (version 36.6.0) for the search within China. This study included the top 100 videos based on comprehensive ranking, as multiple studies32–34 support that this sample selection is representative and that including more than 100 videos does not significantly affect the analysis.
Inclusion and exclusion criteria
Inclusion criteria: (1) videos related to syphilis, (2) videos presented in Chinese, (3) videos publicly accessible at the time of data collection. Exclusion criteria: (1) non-syphilis videos, (2) duplicate videos, (3) advertisement or promotional videos, (4) news videos. The flowchart of this study is presented in Figure 1A and B.
Flowchart of video selection and analysis process. (A) Flowchart of the screening and selection of 81 syphilis-related videos in the first round (July); (B) Flowchart of the screening and selection of 95 syphilis-related videos in the second round (November).
Video coding and variable definitions
For each included video, the following variables were extracted: video duration (in seconds), number of likes, collections, comments, and shares. Uploaders were categorized as dermatologists, gynecologists, urologists, traditional Chinese medicine doctors, reproductive medicine specialists, general surgeons, infectious disease specialists, internal medicine doctors, nephrologists, neurologists, pharmacists, critical care physicians, emergency physicians, andrologists, proctologists, radiologists, ophthalmologists, rehabilitation physicians, and individual users (IUs). All healthcare workers were collectively categorized as healthcare professionals (HPs). Among them, “other HPs” refers to a group of doctors excluding dermatologists and gynecologists. Content themes were classified into seven categories: epidemiology, etiology, transmission, clinical manifestations, diagnosis, treatment, and prognosis.
Quality and reliability assessment
Two trained reviewers independently assessed the quality and reliability of each video using the Global Quality Score (GQS) and the modified DISCERN (mDISCERN) tool. The GQS is a 5-point scale used to evaluate the overall quality of the video, with higher scores indicating better quality. The mDISCERN tool consists of five binary (yes/no) items assessing the reliability of the content. The full scoring criteria are shown in Tables 1 and 2. For both instruments, scores of 1–2 indicate a low level, a score of 3 indicates a moderate level, and scores of 4–5 indicate a high level. Before formal scoring, three dermatologists received standardized training, which included: (1) standardized training on the GQS and mDISCERN instruments, and (2) a pilot calibration with 10 videos to reduce evaluator bias. The GQS and mDISCERN assessment of all included videos was conducted independently by two dermatologists (Junlan Yang and Kaidi Zhao). In case of any discrepancies in the ratings, the final score was determined by the senior dermatologist (Jiashu Liu). The GQS and mDISCERN evaluations were conducted from July 24 to 26, 2025, and from November 9 to 15, 2025, respectively.
The Global Quality Score (GQS) quality criteria.
The modified DISCERN (mDISCERN) quality criteria.
Statistical analysis
Descriptive statistics were used to summarize the characteristics of the videos. Non-normally distributed variables were reported as medians with interquartile ranges (IQR). The Mann–Whitney U test was used to compare video quality scores between different uploader groups. Spearman's correlation coefficients were used to analyze the relationships between engagement metrics (likes, collections, comments, shares), video duration, and quality scores (GQS and mDISCERN). All statistical analyses were performed using R software (version 4.3.2), and a p-value <0.05 was considered statistically significant.
Results
Baseline characteristics
This study collected 81 syphilis-related videos from the TikTok platform during the first round in July. The median video duration was 45.00 seconds (IQR: 36.00–61.00 seconds). For video engagement metrics, the median number of likes was 417.00 (IQR: 93.00–4043.00), favorites 117.00 (IQR: 21.00–1122.00), comments 38.00 (IQR: 14.00–152.00), and shares 208.00 (IQR: 46.00–2672.00). In the second round of data collection, 95 syphilis-related videos were included in November. Compared to the videos included in the first round, the videos in the second round had longer duration (p < 0.05), higher engagement metrics (p < 0.05), and lower GQS and mDISCERN scores (p < 0.05). The median video duration for the second round was 58.00 seconds (IQR: 39.00–88.50 seconds), with a median of 2543.00 likes (IQR: 514.50–14732.50), 499.00 favorites (IQR: 136.00–3256.50). Detailed parameters for other baseline characteristics are shown in Table 3.
General information, quality, and reliability scores of syphilis-related videos on TikTok.
Video uploaders
In the first round of data collection, the top three uploader categories for syphilis-related videos were dermatologists, gynecologists, and other HPs, as shown in Figure 2A. Figure 2B shows the proportion of uploader types in the second round of data collection. The specific parameters for the distribution of video uploaders in both rounds of data collection are presented in Table 4.
Distribution of video uploaders by category. (A) Distribution of uploaders of the 81 syphilis-related videos in the first round (July); (B) Distribution of uploaders of the 95 syphilis-related videos in the second round (November).
Uploader of short videos.
“–” indicates that this type of uploader does not exist.
Video content
Figure 3A shows the distribution of video content in the first round of data collection in July. The most frequently mentioned topics were clinical manifestations (71.60%) and transmission routes (34.57%), while epidemiology (9.88%), treatment (19.75%), and prevention (18.52%) were mentioned less frequently. Figure 3B shows the distribution of video content in the second round of data collection in November. Clinical manifestations (74.74%) and transmission routes (41.05%) remained the most frequently mentioned topics, while epidemiology (4.21%), treatment (18.95%), and prevention (13.68%) were less frequently discussed. There were no significant differences in video content between the two time periods. The specific parameters for the video content collected in both periods are shown in Table 5.
Distribution of content themes in syphilis-related videos. (A) Distribution of the content of the 81 syphilis-related videos in the first round (July); (B) Distribution of the content of the 95 syphilis-related videos in the second round (November).
Differences in video characteristics and quality between personal users and healthcare professionals.
Bold values indicate p < 0.05.
Video quality and reliability by uploader identity
The median GQS score of the first-round syphilis videos was 2.00 (IQR: 2.00–3.00), and the median mDISCERN score was 3.00 (IQR: 3.00–3.00) (Table 3). Videos uploaded by HPs had significantly higher GQS (p < 0.05, Figure 4A) and mDISCERN (p < 0.05, Figure 4B) scores compared to those uploaded by IUs. Detailed data on the GQS and mDISCERN scores of the uploaders are provided in Table 5. The median GQS score of the syphilis videos included in the second round was 2.00 (IQR: 2.00–2.00), and the median mDISCERN score was 2.00 (IQR: 2.00–2.00) (Table 4). Videos uploaded by HPs still had higher GQS scores (p < 0.05, Figure 4C) and mDISCERN scores (p < 0.05, Figure 4D) than those uploaded by IUs, with detailed parameters of the ratings provided in Table 5.
Comparison of GQS and mDISCERN scores between individual users and healthcare professionals. (A) Differences in GQS among different uploaders of the 81 syphilis-related videos in the first round (July); (B) Differences in mDISCERN among different uploaders of the 81 syphilis-related videos in the first round (July); (C) Differences in GQS among different uploaders of the 95 syphilis-related videos in the second round (November); (D) Differences in mDISCERN among different uploaders of the 95 syphilis-related videos in the second round (November). Note: The numbers represent correlation coefficients. *p < 0.05, **p < 0.01, ***p < 0.001, ****p < 0.0001.
Correlation between interaction metrics and video quality and reliability
The analysis of videos collected in the first round revealed a positive correlation between video duration and GQS scores (ρ = 0.25, p < 0.05). Significant positive correlations were observed among the interaction metrics. Specifically, likes were positively correlated with comments (ρ = 0.98, p < 0.05), shares (ρ = 0.87, p < 0.05), and favorites (ρ = 0.90, p < 0.05). Comments were positively correlated with shares (ρ = 0.82, p < 0.05) and favorites (ρ = 0.84, p < 0.05). Similarly, shares were positively correlated with favorites (ρ = 0.90, p < 0.05). No significant correlation was found between the interaction metrics and either GQS or mDISCERN scores (p > 0.05) (Figure 5A).
Heatmap of correlation coefficients among video characteristics. (A) Correlation between engagement metrics and GQS and mDISCERN scores for the 81 syphilis-related videos in the first round (July); (B) Correlation between engagement metrics and GQS and mDISCERN scores for the 95 syphilis-related videos in the second round (November).
In the second round of data collection, video duration showed no correlation with GQS (p > 0.05). Likes were positively correlated with favorites (ρ = 0.96, p < 0.05), comments (ρ = 0.91, p < 0.05), and shares (ρ = 0.94, p < 0.05). Comments were positively correlated with shares (ρ = 0.86, p < 0.05) and favorites (ρ = 0.86, p < 0.05). Shares were positively correlated with favorites (ρ = 0.91, p < 0.05). No correlation was observed between the interaction metrics and either GQS or mDISCERN scores (p > 0.05) (Figure 5B).
Overall, the results from both time periods indicate that interaction metrics are positively correlated with each other, while no correlation is found between interaction metrics and GQS or mDISCERN scores. The video duration in the first round showed a weak correlation with GQS, while in the second round, no correlation was observed between video duration and GQS.
Discussion
This study evaluated the content, quality, and reliability of syphilis-related videos collected in two rounds, in July and November 2025. The content of the syphilis-related videos in both rounds primarily focused on clinical manifestations and transmission routes, with limited discussion on epidemiology and etiology. The median GQS for the first round of syphilis-related videos was 2, and the median mDISCERN score was 3. The median GQS and mDISCERN scores for the second round of syphilis-related videos were both 2. The overall quality and reliability of the syphilis-related videos from both rounds were suboptimal. The quality and reliability of videos uploaded by HPs showed consistency across both rounds, with higher quality and reliability compared to those uploaded by IUs. Furthermore, there was a positive correlation between engagement metrics in the videos across both rounds, but engagement metrics were not associated with video quality or reliability. Future efforts should focus on strengthening platform supervision of videos, emphasizing the importance of epidemiological and etiological content in health education. Encouraging greater participation by HPs in uploading syphilis-related videos and optimizing health communication strategies can help promote the dissemination of high-quality videos.
The median GQS score for the first round of syphilis-related videos was 2, and the median mDISCERN score was 3. The median GQS and mDISCERN scores for the syphilis-related videos collected in the second round were both 2. Overall, the quality and reliability of syphilis-related videos on Douyin were at or below the moderate level, indicating that the quality and reliability of these videos are suboptimal. A study by Cai et al. 35 on gastroesophageal reflux disease also found that video quality and reliability on TikTok were relatively low. Similarly, Gong et al. 36 reported consistent findings in their evaluation of short videos related to coronary artery disease. This may be related to the limited breadth and depth of content covered due to the short duration of videos on these platforms. In the future, platform content supervision and review should be strengthened to improve the quality and reliability of videos.
In the videos collected across both rounds, this study found that videos uploaded by HPs scored significantly higher on both the GQS and mDISCERN scales compared to those created by IUs. This trend aligns with findings from other studies on health topics in social media, where professionals are more likely to deliver structured and reliable information.37,38 For instance, an analysis of brain tumor-related short videos showed that content uploaded by professionals exhibited higher quality and reliability. 39 Similarly, a study evaluating the quality and reliability of TikTok videos about nonalcoholic fatty liver disease in China and the United States found that videos uploaded by HPs had significantly higher GQS and mDISCERN scores than those by non-HPs. 40 However, even videos created by HPs in the present study only reached moderate quality levels. This suggests that while medical expertise contributes to content accuracy, it does not automatically translate into effective communication. HPs may require training in digital communication to enhance their ability to convey complex medical concepts through short video platforms.
In terms of video content, the first round of the study conducted in July found that “clinical manifestations” and “transmission routes” were the most frequently addressed themes, while “prevention” and “epidemiology” were less frequently covered. The second round of the study, conducted in November, further supported the findings from the first round, with video content again predominantly focusing on “clinical manifestations” and “transmission routes,” with limited coverage of “prevention” and “epidemiology.” Video uploaders appear to favor topics that evoke emotional resonance or curiosity, while overlooking critical elements essential for disease prevention and control. Although syphilis is a curable disease, delayed treatment can lead to severe outcomes such as neurosyphilis, cardiovascular syphilis, and gummatous syphilis.1,41–43 The absence of epidemiological information may lead the public to underestimate the likelihood of infection, while the lack of preventive information undermines the overall effectiveness of health communication. The absence of preventive information may result in poor public awareness of and access to prevention services. Therefore, future health communication efforts should incorporate structured content frameworks that ensure coverage of the entire disease continuum, thereby enhancing public understanding and awareness of syphilis.
The results from both rounds of this study indicate that there is no significant association between interaction metrics (likes, comments, favorites, shares) and video quality scores (GQS and mDISCERN). Similar findings were reported by Wang et al., 44 who found no significant association between user engagement and content quality scores. Another study on premature ovarian insufficiency also confirmed the absence of a meaningful relationship between interaction metrics and quality scores. 29 These findings suggest that videos with high dissemination potential do not necessarily possess high educational value. Instead, such popularity may be driven by factors like compelling titles, visual effects, or emotionally charged language. Current recommendation algorithms on short video platforms tend to favor emotionally engaging and entertaining content, which inadvertently amplifies the visibility of non-professional creators. 45 Furthermore, the results from both rounds of the study consistently indicate a positive correlation between video interaction metrics. Previous studies on videos related to melanoma, 46 gonorrhea, 27 and melasma 47 support the findings of this study. The positive correlation between interaction metrics facilitates the rapid dissemination of videos on the platform. However, it may also lead to the rapid spread of videos that contain misinformation, falsehoods, or emotional content. In the future, we should optimize health information dissemination strategies to increase the exposure of high-quality videos and leverage the rapid dissemination features of these platforms to promote the efficient spread of high-quality videos.
In summary, the findings of this study indicate that while syphilis-related videos on TikTok exhibit high levels of user engagement, there remains considerable room for improvement in terms of informational quality, content structure, and scientific rigor. The lack of structural coherence, insufficient professional involvement, and incomplete thematic coverage represent major shortcomings in the current state of short video-based health communication.48–50 To enhance the quality of medical communication on short video platforms, we propose the following strategies: First, a “professionally driven” approach should be established by encouraging HPs to participate in content creation, supported by targeted training in digital communication skills. Second, content governance mechanisms on platforms should be strengthened by implementing medical verification badges, expert review channels, and integrated quality rating systems to guide the algorithmic promotion of credible content. Third, standardized content frameworks should be promoted, encouraging creators to structure information in the following order: epidemiology and etiology, symptoms, diagnosis, treatment, and prevention. Content should also advise individuals to seek medical consultation after unprotected sexual intercourse or at the onset of symptoms. When necessary, modular or serialized formats may be adopted to enable the systematic delivery of complex medical knowledge within time-constrained formats. Improving the educational effectiveness of health communication via short videos will require coordinated efforts from content creators, medical professionals, and platform stakeholders to promote the wide dissemination of structured, accurate, and user-friendly medical information.
This study also has several limitations. First, the video data were collected solely from the TikTok platform, and the findings may not be generalizable to other short video platforms such as Bilibili or YouTube, given the differences in user demographics, content regulations, and algorithmic mechanisms. Second, although GQS and mDISCERN are widely used tools for evaluating health information, their scoring process relies on subjective judgment and may be influenced by individual evaluator bias. Third, this study focused exclusively on video content and did not examine comment sections, user profiles, or viewer feedback, which limits the ability to assess the actual impact of the videos on public understanding or health-related behaviors. In addition, the sample size in this study was relatively limited (81 and 95 videos in the two datasets, respectively), whereas the number of candidate predictors was relatively large, including uploader type, video content categories, and multiple engagement metrics with unbalanced distributions. Therefore, multivariable logistic or ordinal regression models were not further applied to evaluate the independent associations between specific video characteristics and higher GQS and mDISCERN scores. The present findings are mainly based on group comparisons and correlation analyses, and future studies with larger samples are warranted to apply multivariable modeling approaches to systematically identify key factors influencing the quality and reliability of online videos. Furthermore, to reduce heterogeneity in content intent, we excluded paid advertisements and promotional materials. This decision may have led to an underrepresentation of commercially driven health information on Douyin and may have altered the distribution of uploader types. Consequently, our estimates of video quality and reliability may not be fully generalizable to sponsored content. Lastly, due to the rapidly evolving nature of short video content, the findings represent only a snapshot at the time of data collection. Future studies should incorporate dynamic monitoring and longitudinal follow-up.
Conclusion
This study, based on two rounds of data collection, found that the quality and reliability of syphilis-related videos on Douyin were suboptimal, with limited coverage of topics such as diagnosis, treatment, and prevention. Videos uploaded by HPs demonstrated higher quality and reliability. Video engagement metrics were not associated with video quality and reliability. These findings suggest that the current video content does not effectively meet the public's need for syphilis-related knowledge. Although the overall quality and reliability of syphilis-related videos on Douyin are suboptimal, short videos can quickly raise public awareness when the content is accurate and comprehensible. Such content helps promote preventive measures, encourages help-seeking behavior, and guides audiences towards testing, counseling, and early treatment. In the future, platforms should strengthen supervision of video quality and reliability, encourage more HPs to upload videos, enhance awareness of syphilis diagnosis, treatment, and prevention, and optimize health communication strategies to improve the quality and reliability of syphilis-related videos.
Footnotes
Acknowledgements
The authors express their gratitude to the Dermatology Department of the Second Affiliated Hospital of Xi'an Jiaotong University and its leadership for their support.
Ethical approval
This study did not involve human participants, clinical data, experimental animals, or histological investigations. All data analyzed in this study were obtained from publicly available Douyin videos, and the data collection process fully complied with the Terms of Service of Douyin. No private information or personally identifiable information was collected or processed, and no interaction of any kind was conducted with platform users. The use of these data did not require individual informed consent. Therefore, this study did not require formal ethical approval.
Patient and public involvement
Patients and/or the public were not involved in the design, conduct, reporting, or dissemination plans of this research.
Patient consent for publication
Not applicable.
Contributorship
Junlan Yang and Kaidi Zhao jointly conceptualized the study. Kaidi Zhao was responsible for methodology design, data analysis, and drafting the original manuscript. Junlan Yang and Jiashu Liu contributed to the review and revision of the manuscript. Jiashu Liu supervised the overall study and managed the project. All authors have read and approved the final version of the manuscript. The authors declare no conflict of interest.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data availability statement
The data supporting the findings of this study are available from the corresponding author upon reasonable request.
Guarantor
Liu Jiashu is the guarantor of this article. She takes full responsibility for the integrity of the research and data, has full access to all data, and had the final decision-making authority regarding publication.