Abstract
Objective
Thyroid-associated ophthalmopathy (TAO) is a prevalent orbital disease significantly affecting patients’ daily lives. Nowadays, TikTok acts as a novel tool for healthcare but involved videos need to be further assessed. Many individuals search for disease information on TikTok before professional consultation. This study aimed to assess the quality of TAO-related TikTok videos and correlations between video variables and quality.
Methods
The top 150 TikTok videos were collected using the keyword TAO. Duplicate, too-short, irrelevant videos and similar videos from the same source were excluded. Two raters evaluated the included videos’ overall quality, reliability, understandability, and actionability on different sources and content focuses.
Results
Ninety videos had received nearly 15,000 likes and 2000 shares. Ophthalmologists and treatment focus were two primary parts among categorized groups, whose quality scores were much higher than others. The average Patient Education Materials Assessment Tool for Audiovisual Materials scores, Global Quality Scores, and DISCERN scores indicated that these videos were easy to understand (87.6%), actionable (74.5%), and fair in quality (44.97). The number of added hashtags was an essential variable positively correlated with video understandability. Additionally, popularity showed negative correlations with the overall quality, while video length positively correlated with its reliability and negatively correlated with the uploaded days.
Conclusion
Certified healthcare professionals uploaded most TAO videos, resulting in acceptable quality with minimal misinformation. To serve as a qualified source of patient educational materials, TikTok is supposed to promote longer disease-related videos and enhance reliability and understandability simultaneously.
Keywords
Introduction
Thyroid-associated ophthalmopathy (TAO), also known as thyroid eye disease (TED), is an autoimmune disorder characterized by eyelid retraction, exophthalmos, and diplopia. 1 Due to factors as the optic nerve's compression at the apex, approximately 5%–8% of TAO develop dysthyroid optic neuropathy. 2 Studies have demonstrated that despite an increasing trend in the mild phase 3 and a decreasing trend in Asia, approximately 5% of patients experience late reactivation,4,5 and nearly 20% of Indian patients report visual morbidity features with moderate proptosis. 6
Nowadays, patients with TAO still frequently encounter difficulties in achieving correct diagnosis and adequate treatment, 7 which possibly causes an increased risk of suicide. 8 It is such a chronic condition that profoundly impacts patients’ daily lives due to changes in eye appearance, often necessitating multiple medical therapies 7 and placing a significant burden on healthcare systems.
Graves’ disease patients with orbitopathy were revealed not to have correct conceptions of disease diagnosis, management, and treatment, 9 indicating the deficiencies in patients’ disease knowledge and received education. 10 Comprehensive educational materials and counseling are essential for helping patients effectively cope with illness rather than relying on folk remedies found elsewhere. Surprisingly, at an endocrinology clinic, the percentage of patients having any basic knowledge of TED increased significantly across the control, poster, and pamphlet phases. 11
TikTok, a novel social application established to be characterized by short videos, has quickly become one of the most tremendously popular applications worldwide.12,13 Online social media offers a new medium for the public, patients, and healthcare professionals to interact. 14 In particular, TikTok shows potential for public health management, especially in exchanging medical information and providing social and emotional support. 15 Before seeking treatment, many patients search TikTok to learn about the disease. However, various online information can easily influence TAO patients’ expectations regarding surgical outcomes. 16 Similar to other public applications, the evaluation process of TikTok was not completely strict, some inaccurate videos may be widely spread, leading to something adverse. 17 Over the last 5 years, studies have investigated the reliability of TikTok videos concerning several diseases, but the findings were not as expected.17–19
As interest in the quality of materials on social media grows, this study aimed to assess the information quality of TAO-related TikTok videos on different sources and content focuses. Whether TikTok could serve as a qualified source of patient educational materials and the correlations between video variables and quality scores were also analyzed.
Materials and methods
Search strategy and methods
In this cross-sectional study, videos were searched by TikTok 3.4.0 for China (updated on 30 January 2024) on 2 February 2024. Using “甲状腺相关眼病” (“Thyroid-Associated Ophthalmopathy” in Chinese) as the keyword, the top 150 videos recommended by TikTok's default sorting algorithm were then retrieved. This algorithm sorts videos without a time limit and is preferred by most users. 12 Based on TikTok's characteristic as a platform for short videos, the screening criteria in this study were adjusted in comparison to previous research.16,20 Only relatively long videos directly related to TAO were included, while unrelated videos (n = 13), advertisements (n = 2), and videos lasting for less than ten seconds without vocal explanation (n = 12) were initially excluded. For duplicates (n = 1) or similar videos from the same source (n = 32), only the earliest ones were included. After two screenings, 90 videos were obtained for further analysis (Figure 1). This study is a digital media-based, cross-sectional study.

The search and extraction process for videos related to thyroid-associated ophthalmopathy (TAO).
According to TikTok's default recommendation order, videos were initially numbered from 1 to 150 and further renumbered from 1 to 90. The basic characteristics of videos, including video captions (added hashtags), duration, the number of likes, shares, and comments received, days since upload (the duration between the video upload date and the extraction date), and a brief introduction of contents were all recorded. In addition, the names, followers, and platform authentication information of uploaders’ accounts were collected. These data were saved in Microsoft Excel 2019.
Quality assessment
After learning specific scoring guidelines, two resident ophthalmologists (YW and SY), who were both proficient in the knowledge of TAO, with experiences assisting in diagnosis and independently managing patients with TAO, viewed and independently rated the videos under strict supervision. During the scoring process, all basic information was hidden except the URL and video sorting numbers. Apart from the commonly used DISCERN instrument, the Global Quality Scores (GQS) and the Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT-A/V) were also applied for the quality assessment of TAO-related TikTok videos.
According to the DISCERN total score, the overall quality is divided into five levels: very poor (16–26), poor (27–38), fair (39–50), good (51–62), and excellent (63–80). 19 GQS is an instrument that evaluates the overall reliability and comprehensiveness of the video content on a 5-point scale (Supplementary Table S1). On this basis, videos are classified as useful, misleading, or patient views. 21 PEMAT-A/V assesses the quality of audiovisual patient educational materials from two aspects: understandability and actionability (13 items, four items, respectively; Supplementary Table S2). 18 After rating each item, the final percentage score (%) is calculated as the total points/total possible points (excluding N/A items). As a quality assessment tool numbered according to the items for printable materials, questions 2, 6, 7, 15, 16, 17, 23, and 24 are missing in PEMAT-A/V.
The PEMAT1, assessment of understandability mainly focuses on the logicality and structure of video, including whether it contains visual cues, illustrations, photographs, etc. However, the PEMAT2, assessment of actionability focuses on whether video provides actionable advice and detailed medical guidance. Videos with higher PEMAT1 and PEMAT2 scores are considered easier to understand and more actionable. When scoring actionability, most videos marked N/A on question 25, but videos lacking action recommendations marked 0 on all four questions. In the absence of advice on specific departments and examinations, question 22 was marked 0 (Supplementary Table S3). Thus, videos only rated 0%, 67%, or 100% on actionability in this study.
Statistical analysis
The intraclass correlation coefficients were calculated using SPSS 21.0 statistical software, and all other analyses were performed using GraphPad Prism 10.2.0 for Windows. Data were presented as frequency, percentage, mean, standard deviation (SD), median, or interquartile range (IQR) as appropriate. The Shapiro–Wilk test, Anderson–Darling test, Kruskal–Wallis H test, Dunn's multiple comparisons test, and nonparametric Spearman’s correlation analysis were used. P < .05 was considered significant.
Ethical approval
Included videos returned by TikTok search were all publicly released and neither patients nor animals were involved in this cross-sectional study. Therefore, there was no need for ethical approval, the same as other TikTok and YouTube studies did.
Result
Video variables
The included 90 TikTok videos were categorized into personal and organizational accounts. Further, personal accounts were then divided into several Western Medicine clinicians (abbreviated as clinicians, including ophthalmologists and other clinicians) and Traditional Chinese Medicine (TCM) practitioners (Table 1). Sorting from high to low, ophthalmologists uploaded 54 videos (60%), followed by other clinicians (18.9%), TCM practitioners (11.1%), and organizations (10%). As shown in Table 2, videos were also categorized based on the caption and duration of each section, including diagnosis (21.1%), symptoms (26.7%), consultant advice (7.8%), health management (13.3%), and treatment (31.1%).
Descriptions of various video sources.
Descriptions of different video content focuses.
The 90 videos collected by the TikTok search received 14,724 likes, 2058 comments, and 4259 shares in total. The middle video duration was 54.5 s, with the longest lasting about 1 h, and the shortest only 17 s. Moreover, the median of uploaded days (days since upload) was 235 days. The earliest video of this study had been uploaded to TikTok for 1682 days, while the latest was released only three days before the data extraction date.
Additionally, before uploading a video, users can optionally choose several relevant hashtags, allowing it to get more popularity and form communities. 22 The five most popular hashtags were #medical science, #thyroid associated ophthalmopathy, #hyperthyroidism exophthalmos, #thyroid, and #hyperthyroidism. For example, in a video containing a maximum of 11 hashtags, not all added hashtags were relevant to the content, some were too broad or unrelated to TAO.
When categorized by video sources and content focuses, video variables were determined and are presented in Table 3, where statistical features are described in the median form.
Video of continuous variables on various classifications (median numbers).
Length: the length of the video (seconds).
Uploaded days: days since upload (days).
TCM: Traditional Chinese Medicine.
Management: health management advice.
Video quality assessment
The video quality of different sources and content focuses was assessed by using three scoring tools. The intraclass correlation coefficient of the DISCERN total scores was 0.982, while for GQS, PEMAT1, and PEMAT2 scores, it was 0.856, 0.893, and 0.953, respectively.
The mean scores for understandability (PEMAT1 scores), actionability (PEMAT2 scores), GQS, and DISCERN total scores of the 90 videos were 87.6% (SD = 0.10), 74.5% (SD = 0.38), 3.86 (SD = 0.82), and 44.97 (SD = 7.5), respectively.
As shown in Figure 2, the video understandability and actionability did not significantly differ across sources while those significantly differed across diverse content focuses. In detail, the PEMAT1 scores for diagnosis and health management were higher than those for treatment (p = .03 and p = .02, respectively). Furthermore, treatment received the highest actionability scores, significantly higher than symptoms and diagnosis (p = .0001 and p = .002, respectively), and PEMAT2 scores for health management were higher than for diagnosis and symptoms (p = .04 and p = .009, respectively, Table 4).

Comparison of video quality, understandability, and actionability between source and content types. (a and d) Global Quality Scores (GQS), (b and e) PEMAT1, (c and f) PEMAT2. *p < .05; **p < .01; ***p < .001.
Kruskal–Wallis H test of the DISCERN scores and PEMAT-A/V scores on different video sources (median and IQR form).
In GQS analysis, a large proportion rated above three, with 46 videos rated as good (four out of five, 51.1%) and 18 as excellent (five out of five, 20%). GQS was significantly different on different sources and contents. Videos uploaded by ophthalmologists received the highest GQS, higher than those uploaded by other clinicians (p = .04). Besides, the QGS for health management and treatment were higher than those for consultant advice (p = .04, p = .04, Figure 2).
Among the classification groups, the DISCERN total scores were significant (Figure 3). Videos uploaded by ophthalmologists had higher DISCERN total scores than TCM practitioners (p = .04). Videos focusing on treatment had significantly higher DISCERN total scores than symptoms and diagnosis (p = .01 and p = .001, respectively). In detail, according to the DISCERN section 1 scores, videos uploaded by ophthalmologists were more reliable than those uploaded by other clinicians and TCM practitioners (p = .03 and p = .008, respectively). No differences between sources were found in the reliability on treatment choice and overall quality (DISCERN section 2 and 3 scores). However, treatment contained significantly higher-quality information than symptoms and diagnosis (p = .003 and p < .0001, respectively), and the same was observed between health management and diagnosis videos (p = .03, Table 4).

Comparison DISCERN sectional scores and DISCERN total scores of different sources and contents. (a and b) Different video sources; (c and d) different content focuses; *p < .05; **p < .01; ***p < .001; ****p < .0001.
To sum up, ophthalmologists were the best source type of high-quality videos under comprehensive assessments. Although relatively difficult to understand, treatment was the most excellent content focus, whose actionability, reliability on treatment choice, and overall quality were significantly better than other focuses.
Nonparametric Spearman’s correlation analysis
Regarding the correlations between various video variables, there were prominently positive correlations between likes, comments, and shares (all r > .70, all p < .0001). The days since upload positively correlated with likes (r = .36, p = .0005), comments (r = .30, p = .0038), and shares (r = .41, p < .0001). Interestingly, there was a negative correlation between the number of hashtags and video duration (r = −.36, p = .0004).
In the case of different criteria, the correlations between three quality scoring tools, DISCERN, PEMAT, and GQS were also calculated. Understandability was positively correlated with the reliability (r = .35, p = .0008). Additionally, actionability had positive correlations with DISCERN sectional scores and its total scores (all r ≥ .30, all p < .005), as well as GQS (all r > .50, all p < .0001).
When calculating the correlations between video variables and quality scores: Negative correlations were found between video popularity and GQS, while positive correlations were found between video duration, days since upload, and GQS and DISCERN total scores. Among video variables, only the number of hashtags positively correlated with the understandability, and none correlated with the actionability (Table 5). In general, video variables correlate with each other as well as various quality scores, and video duration and days since upload are two variables that significantly correlate with quality scores.
Spearman’s correlation analysis between video variables and the global quality scores (GQS), the DISCERN total scores (DISCERN), and the PEMAT-A/V scores.
PEMAT1: understandability measured by PEMAT-A/V.
PEMAT2: actionability measured by PEMAT-A/V.
p < .05.
p < .001.
Discussion
Principal findings
As social media acts as an essential component of daily life, the abundant information on these platforms assists individuals in obtaining and exchanging knowledge as well as entertainment for relaxing.17,23,24 In this study, for the first time, the quality of TAO-related TikTok videos was assessed to be acceptable using different scoring tools, and some novel correlations between various video variables and quality scores were also calculated. With the involvement of several healthcare professionals, TikTok was identified as a potential patient-educational tool for TAO.
Compared to a similar cross-sectional study on YouTube TED-related videos, 16 no collected TikTok videos were uploaded by patients. Patient's diagnosis and treatment experience may provide valuable advice for others and assist surgeons in patient education and pre-procedural counselling. 25
Remarkably constituted of ophthalmologists, healthcare professionals uploaded the vast majority (90%) of the videos. Other videos featured specialists or records of popular health programs. Among the studied videos, only four lacked authentication, and 75 were uploaded by chief physicians or associate chief physicians. It's concluded that videos uploaded by TCM practitioners were significantly more popular than those of ophthalmologists and organizations (Supplementary Figure S1).
Despite the most commonly used scoring systems (DISCERN and GQS), a new tool, PEMAT-A/V was also used in this study. Regarding the DISCERN scores, the maximum score of the 90 videos was 61, which is still considered good but not excellent on a 5-point scale (16–26, 1, very poor. 27–38, 2, poor. 39–50, 3, fair. 51–62, 4, good. 63–80, 5, excellent). 19 Compared to YouTube TED videos, TAO-related videos on TikTok had lower DISCERN total scores (44.97 ± 7.5 vs. 55.27 ± 16.57) but higher GQS (3.86 ± 0.82 vs. 3.44 ± 0.93), indicating that TAO-related videos on TikTok may be more reliable than those on YouTube but with higher overall quality. 16 The main reasons for the significant quality score differences were the video duration (2.34 ± 8.57 vs. 13.25 ± 19.13 min), major components (ophthalmologists and other clinicians vs physicians and private health institutions), and the professional background between TikTok and YouTube. More than half of 90 videos focused on diagnosis and symptoms, while others focused on disease management and treatment were not detailed and only superficially presented. In addition, video content was deficient in information sources, balance, and generation, leading to the DISCERN scores not being high enough.
In PEMAT analysis, only 19 videos rated 100% in understandability, providing relevant information to facilitate patient comprehension. Although no videos involved an action table or diagram, 55 videos rated 100% on actionability and provided detailed guidance on how to take action. The top three questions scores at 0 were questions 4, 12, and 22, indicating that many videos lacked definitions of professional medical terms, visual cues, and explicit action steps.
It is well-known that the number of likes, comments, and shares reflects the video's popularity and the viewers’ preference.20,26 The likes, comments, and share times of TikTok TAO videos, presented statistically positive correlations with each other. There was also a remarkable correlation between the days since upload and video popularity, indicating that TikTok might also recommend relatively old videos, making them more likely to be liked, commented on, and shared. A new variable was added to the analysis, the number of hashtags in the captions, which was inversely proportional to video duration, suggesting that TikTok users preferred to add fewer hashtags to longer videos.
Several recent studies have shown that variables are not only the video components but also correlate with video quality.13,27 In this study, video length positively correlated with GQS and DISCERN total scores, indicating that longer videos could contain more details, making them more reliable and of higher quality. Astonishingly, a negative correlation was found between popularity and GQS, and a positive correlation was found between the number of hashtags and video understandability.
Social media has a significant impact on disease knowledge's creation and dissemination, indicating patients might absorb accurate and inaccurate information.28,29 The probable reason why social media plays a vital role in the clinical process is that patients may not benefit much from short-term treatment, leading some individuals to search for novel treatment instead of traditional face-to-face consultations or paper publications. Although the information lacked theoretical support and verification, many viewers liked, shared, and commented on the videos, highlighting the shortcomings of social media as a platform for medical knowledge popularization.
This study included social platforms targeting non-English users, the Chinese version of TikTok, and further evaluated the usage and information dissemination to provide a broader global perspective. 30 An important question for future studies is to compare the quality of TAO-related videos across different social media platforms.
Assessment of three quantitative scoring tools
As a commonly used standard for assessing the quality of paper publications, DISCERN may not be fully applicable to video materials.12,31 To address this, QGS and PEMAT-A/V, designed to evaluate the video quality in understandability and actionability, 32 were included as evaluation standards for TikTok TAO videos for the first time.
In particular, positive correlations were also found between various scoring systems. 16 QGS positively correlated with each sectional score of DISCERN and its total scores. Video understandability was observed to positively correlate with reliability, while actionability positively correlated with overall quality scores and DISCERN total scores. As mentioned above, GQS and PEMAT-A/V could be potential quality scoring tools for video-formed disease educational materials in the future.
Limitations
This study has limitations. Without logging any accounts on the computer downloaded TikTok for the first time, it effectively avoided the influence of various personal characteristics (age, gender, etc.) and previous browsing habits, 22 but the included videos might be different from those pushed to others. “甲状腺相关眼病”, not a specific hashtag was chosen as a keyword for the search on TikTok. Some videos captured hashtags with similar meanings, such as #thyroid function and #hyperthyroidism exophthalmos.
Another inherent shortcoming was the uncertainty of the time sequence. Instead of establishing a causal relationship, only the correlation between the two variables was judged. Our findings are based on a single platform, TikTok 3.4.0 for China, using the default sorting process, which resulted in an imbalance in the number of videos included across different categories. This limitation should be carefully addressed in future research. The encouraging results of this study should be validated through more extensive research, comparing the effectiveness of TAO-related videos across other language versions of TikTok and on other social media platforms. Additionally, future studies should explore whether educational videos influence patients’ diagnosis and treatment decisions.
Future prospects
Social media, especially TikTok, can facilitate health education, knowledge translation, and even public health management. However, misinformation on these platforms may lead to negative health behaviors and adverse health outcomes. 33 If the information quality and authority are too high, the benefits of gaining social and emotional support and audience engagement are probably weakened. 34 Therefore, it's necessary to balance between video accuracy and understandability.
Popularizing accurate health knowledge and reducing misinformation requires the joint effort of the platform and medical practitioners. 33 For public health, TikTok is supposed to supervise the reliability of video content, avoid the phenomenon that giving excessive traffic to videos with misinformation, and give warnings to these accounts. While continuing to promote the entry of healthcare professionals and give them certification, TikTok should also give more traffic and optimize the recommendation mechanism of hashtags. There is an urgent need to expand the influence of videos with accurate and professional knowledge, provide live broadcasts for academic conferences, and encourage users to upload wonderful content clips.
At the same time, TikTok users need more changes, for example, healthcare professionals should appropriately extend the video duration to give more detailed advice and use editing software to produce more attractive videos. Assembling videos of relevant content and creating collections is also excellent advice. Besides uploading videos to popularize accurate health information, they can also prepare and disseminate after verifying related posts. 35
Conclusion
In this study, we evaluated whether TikTok videos offered a novel approach for patients with TAO and relative personnel to acquire disease knowledge. The 90 videos analyzed were primarily uploaded by ophthalmologists and focused on the disease's treatment and symptoms. Two essential variables, hashtags in the captions and video duration, were found to correlate significantly with the video's popularity and reliability. As a prominent social media platform, TikTok demonstrated that the overall quality of most TAO-related videos was acceptable, suggesting its potential as an effective patient educational tool for public health. However, there remains plenty of room for improvement.
Supplemental Material
sj-docx-1-dhj-10.1177_20552076241304594 - Supplemental material for TikTok as a potential patient educational tool for thyroid-associated ophthalmopathy: A cross-sectional study
Supplemental material, sj-docx-1-dhj-10.1177_20552076241304594 for TikTok as a potential patient educational tool for thyroid-associated ophthalmopathy: A cross-sectional study by Yue Wang, Shuo Yang, Xingyi Shu, Yuqing Chen and Ruili Wei in DIGITAL HEALTH
Footnotes
Contributorship
Conceptualization or design by YW and YQC. Curation and formal analysis of data by SY and YW. Funding acquisition by RLW. Original draft writing by YW and SY. Manuscript editing by XYS. Supervision and draft review by XYS, YQC, and RLW. All authors contributed to the article and approved the final manuscript.
Data availability
The original contributions generated in the study are included in the article/Supplementary material; further inquiries can be directed to the corresponding authors.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical approval
This cross-sectional study involved no clinical data, participants (patients), or laboratory animals. All information was obtained from publicly released TikTok videos and users’ homepages, and none of the data involved personal privacy. No ethics committee approval was required.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: RLW is supported by the Funds from the National Natural Science Foundation of China (Grant No. 81770959 and 81570885) and the Major Clinical Research Project of Shanghai Shenkang Hospital Development Center (Grant No. SHDC12018103). The funder had no role in the design of the study or collection, analysis and interpretation of data or in writing the manuscript.
Guarantor
RLW.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
