Abstract
Background:
Social media platforms, like TikTok, have become popular options for the distribution of health care information. Because of the lack of scientific oversight, the quality of health care–related videos has become a focus of the current literature. However, orthopaedic surgery has lagged behind other fields in acknowledging the widespread utilization of TikTok videos for medical information consumption. This study aims to assess the quality and educational benefits of ankle sprain–related TikTok videos.
Methods:
TikTok was queried using the hashtag “#anklesprainexercises.” One hundred videos were included after applying the exclusion criteria. The number of views, likes, shares, comments, and favorites was recorded. The content was graded using DISCERN (a well-validated informational analysis tool) and ASEES (a self-designed tool for exercise evaluation). We hypothesized that information on TikTok related to ankle sprain exercises would be poor in quality.
Results:
The total number of views of the 100 videos was 6 483 412, with a median of 5377.5 (IQR = 1074-20 275). The videos collectively received 385 847 likes, 3642 comments, 55 574 favorites, and 14 918 shares with a median of 267.5 (IQR = 41.5-1678.0), 4.0 (IQR = 0.0-23.0), 42.0 (IQR = 4.8-264.5), and 13.0 (IQR = 1.8-67.8), respectively. General users had a higher percentage of their videos graded as “very poor” (61.8%) in comparison to the number of videos uploaded by health care professionals deemed “very poor” (34.4%). Neither general user nor health care professionals had videos graded as “good” or “excellent.” There were significant differences between the 2 groups for DISCERN 1, 3, and ASSES scores.
Conclusion:
Although TikTok is a powerful tool for information distribution, the educational value of the videos related to ankle sprain injury exercises was poor. With only 2% of videos receiving a grade of “fair,” and no videos reaching a score of “good” or “excellent,” health care professionals should be aware of the low-quality content easily accessible on TikTok.
Level of Evidence:
Level III, cross-sectional study.
Keywords
Introduction
Ankle sprains involving the lateral ligamentous complex account for 80% of all ankle injuries and usually occur when the ankle is in an inverted and plantar flexed position. With regard to etiology, Vuurberg et al 39 determined that roughly 40% of all traumatic ankle injuries occur during athletic participation. Doherty et al 12 reported that up to 70% of patients reported persistent residual symptoms and injury recurrence after ankle sprain. It is estimated that proper proprioceptive rehabilitation reduces the risk of recurrent ankle sprain by 36%.23,30 Proprioceptive training can vary widely, but examples include exercises such as single leg stand with the eyes closed, balancing on a wobble board, and balancing on a single leg while catching a ball. 30 The negative implications of inadequate proprioceptive rehabilitation may include repetitive injury to the ankle joint and the development of chronic ankle instability. Therefore, successful treatment of ankle sprains involves an accurate diagnosis and rigorous, proprioceptive rehabilitation. 11
During the last decade, social media has risen in use to become a chief form of information dissemination. TikTok is a social media platform that was released for use in 2017. Users can share and watch short-form videos that range from 15 seconds to 10 minutes long. Users are also able to “like” and “comment” on videos, enhancing the ability of the platform’s algorithms to create unique content based on each user’s interests. TikTok reaches a massive audience, and the platform had more than 1.6 billion users and 2.6 billion downloads worldwide at the end of 2021. 16
The use of online platforms for dissemination of medical information drastically increased in response to the COVID-19 pandemic as a surrogate for person-to-person visits.13,27 For example, prior to the pandemic, only 2% of physical therapy providers in the United States reported providing telehealth services. By July 2020, this number increased to 47%. 14 TikTok has also emerged as a distribution platform for medical information, and contains videos pertaining to a wide variety of other medical conditions.6,21,33 Highly prolific users with large followings known as “influencers” can have a tremendous impact on the information transmitted through these platforms, and their videos can reach a huge audience. 36 However, many health care professionals question the quality of information presented on TikTok,2,37,41 and others note that health care–related posts on TikTok often contain no references and lack oversight from recognized health authorities. 13
The orthopaedic surgery literature lags behind other medical fields in acknowledging and analyzing the growing body of TikTok videos that contain medical information. Other areas of medicine including endocrinology, 21 obstetrics and gynecology, 40 and urology 41 have addressed the growing body of information available on TikTok related to their specialties. Especially for common orthopaedic conditions that are relevant to younger populations such as ankle sprain, TikTok may be consulted prior to or in lieu of seeking expert consultation in an orthopaedic clinic. 4 Given the increased incidence of ankle sprain in younger populations and the desire for information dissemination quickly within this population, assessing the efficacy of social media platforms such as TikTok in broadcasting medical advice takes on unprecedented importance. Therefore, this study aims to assess the quality and educational suitability of the information presented in videos on ankle sprain rehabilitation exercises on TikTok. We hypothesize that the information contained on TikTok related to ankle sprain would be of poor quality.
Materials and Methods
Search Strategy and Data Collection
The social media platform TikTok (https://www.tiktok.com/en/) was queried to find videos related to ankle sprain exercises on August 13, 2022. The search was conducted by using the hashtag “#anklesprainexercises” without any search filters. The search term “#anklesprainexercises” was cross checked with various other search terms including “#anklesprainrehab,” “anklesprainadvice,” “#anklespraininfo,” and significant overlap with regard to pulled videos was found with these other search terms. Thus, we decided to focus solely on the search term “#anklesprainexercises.”
Our search yielded a large number of total videos (n = 390). To add an element of randomization to the selection, we elected to implement a form of block randomization and chose to analyze every third video for inclusion in the study (n = 130). 20 We then performed an initial screening of the videos and eliminated those that were (1) exercises that target different muscle groups (n = 11), (2) not directly related to ankle sprain exercises (n = 5), (3) duplicates (n = 1), (4) private page (n = 1), and (5) not in English (n = 1). After full screening, 100 videos remained (Figure 1).
Flowchart of the search process for videos related to ankle sprain exercises.
For each of the videos included in the analysis, we recorded the creator of the video’s data (username, video identification number, and if they were a health care provider), the number of views, likes, shares, comments, and favorites. Video creators were deemed as a “health care provider” if they listed a degree (MD, DC, DPT, etc) in the video description (which is easily visible to the viewer) or obviously were affiliated with an active health care provision practice. In the 3 cases where uncertainty existed, the uploader’s homepage was searched and reviewed for more information regarding their background as a health care provider. The uploader’s homepage contained more details regarding their background, education, and affiliated degrees, and allowed for determination of status as a health care provider. This study did not require any human participants or animals; thus, ethics committee approval was not required.
Scoring System
Two separate scoring systems were used to evaluate the quality and educational value of the videos: the DISCERN, a previously validated tool used for evaluating reliability and quality of a treatment approach, and the Ankle Sprain Exercise Education Score (ASEES) to assess the education suitability of the information in each video. The ASEES score was modified from a similar scale created for evaluating scoliosis exercise video quality from previously published work by Jang et al. 17
DISCERN for quality assessment
The DISCERN is a questionnaire that provides researchers with a reliable and accurate way of assessing the quality of information on treatment choices for a health problem. The tool is well validated, 7 has been used since the late 1990s, and is composed of 15 questions in addition to a final section in which an assessment of the overall quality of the information is made (Figure 2).
DISCERN instrument.
The first set of 8 questions assess the reliability of the publication (DISCERN 1) and the next set of 7 questions reviews the quality of the author’s source base (DISCERN 2), and the final question then rates the publication as a whole in terms of its quality as a source of information (DISCERN 3). Although initially designed as a tool for written information, DISCERN has been successfully applied as a scoring test for grading the quality of videos in prior research.1,7 DISCERN scores are categorized as follows: excellent is denoted by scores of 63 to 75 points, good is denoted by scores of 51 to 62 points, fair is denoted by scores of 39 to 50 points, poor is denoted by scores of 27 to 38 points, and very poor is denoted by scores of 16 to 26 points.
ASEES for educational suitability assessment
To grade the educational value of the videos, we developed ASEES as a modification from previously published work by Jang et al. 17 Although neither this test nor the tests from which the ASEES was derived have been formally validated, in conjunction with the DISCERN, metrics of this kind have been used to assess whether or not viewers can properly understand and follow exercises after watching a TikTok video. 17
The ASEES has 5 grading marks: “Exercise cycle (Does the video describe the exercise cycle?),” “Target (Does the video describe the target area of the exercise?),” “Effect (Does the video describe the expected effect of the exercise?),” “Safety (Does the video describe the precautions and safety components of the exercise?),” and “Rationale (Does the video explain the rationale of the exercise?).” Each grading mark is scored between 0 and 5, with increasing scores demonstrating higher quality. The sum of all 5 grading marks is the final ASEES score (0-25, with scores of 0 representing the lowest possible quality and scores of 25 representing the highest possible quality).
Assessment
The videos were collected by 2 reviewers and independently evaluated by our orthopaedic research team. Once the data regarding video distribution metrics was collected for each video, the content of the videos was graded using the DISCERN and ASEES tools. Each video was graded separately by 2 medical students who had received training on ankle sprain proprioceptive exercise. Any points of discrepancy between the 2 reviewers were resolved by a third author, an orthopaedic surgeon.
After scoring the videos anonymously, the videos were placed into groups based on the uploader’s background into 3 categories: general users, health care providers, and health organizations. The health care provider category included users that described themselves as health care professionals such as chiropractors, physicians, physical therapists, and nurses. The health organization group includes clinics, hospitals, and treatment centers. In this study, there were no videos from users that were in the health organization group.
Statistical analysis
Scoring and characteristic data are presented as the mean (SD), median (interquartile range [IQR]), and percentage. A 2-sample t test was used to compare the 2 types of uploaders by using the mean, SD, and sample size of each continuous and categorical variable. Statistical significance was set at P <.05, for comparisons other than interrater reliability. For interobserver reliability, the class 3 model intraclass correlation coefficient (ICC) was calculated for both the DISCERN and ASEES grading scales with a 95% CI using a 2-factor analysis of variation without replication model. The benchmarking of ICC values was adapted from previous studies as follows: <0.50, poor; 0.50 to 0.75, moderate; 0.76 to 0.90, good; and >0.9: excellent reliability.9,22 All analyses were performed using Microsoft Excel (Redmond, WA).
Results
Basic Characteristics
In total, 130 videos were pulled after searching “#anklesprainexercises.” Of the 130 videos, 30 videos were excluded after applying exclusion criteria and randomization protocol, and 100 videos were included in the final analysis. Table 1 presents the basic characteristics of the videos analyzed.
Characteristics of Included Videos.
Abbreviations: ASEES, Ankle Sprain Exercise Education Score; IQR, interquartile range.
Type of Uploaders
General users uploaded more videos (68%) when compared to health care professionals (32%). However, the dissemination statistics (ie, number of views, likes, comments, favorites, and shares) between the general users and health care professionals were all statistically insignificant (Table 2). The tabulated scores for DISCERN 1, DISCERN 3, Total DISCERN, and ASEES between general users and health care professionals were all statistically significant (P = .015, P = .026, P = .012, and P = .030, respectfully) (Table 2), indicating much lower quality ratings in videos uploaded by general users than those uploaded by health care professionals.
Characteristics of the Videos Across the Type of Uploaders.
Abbreviations: ASEES, Ankle Sprain Exercise Education Score; IQR, interquartile range.
The DISCERN grading results are recorded in Table 3. DISCERN scores are categorized as follows: “excellent” is denoted by scores of 63 to 75 points, “good” by scores of 51 to 62 points, “fair” by scores of 39 to 50 points, “poor” by scores of 27 to 38 points, and “very poor” by scores of 16 to 26 points. These labels provide an overall summary and quality of the video. Generally, “poor” videos are those that fail to clearly specify the aims of the video, provide a balanced and unbiased overview of the topic, document source information, and describe risks associated with the presented exercise. General users had a higher percentage of their videos graded as “very poor” (61.8%) in comparison to the number of videos uploaded by health care professionals deemed “very poor” (34.4%). Health care professionals had a higher percentage of their videos graded as “poor” (65.6%) in comparison to the videos uploaded by general users (35.3%). In addition, general users had a higher percentage of their videos graded as “very poor” (2.9%) in comparison to the number of videos uploaded by health care professionals deemed “very poor” (0.0%). Neither general user nor health care professionals had videos graded as “good” or “excellent.”
Percentage of DISCERN Grades Across the 2 Types of Uploaders.
Interobserver Reliability
The estimates for interobserver reliability for the DISCERN and ASEES were 0.87 (95% CI, 0.81-0.91) and 0.59 (95% CI, 0.44-0.70), respectively. According to prior studies, these results correlate to “good” and “moderate” reliability for each of those scoring systems, respectively.9,22
Discussion
Inclusion of medical information on social media platforms has provided patients more ways to obtain medical advice without having to physically interact with a clinician or therapist.34,38 By expanding the availability of freely accessible information, patients can avoid in-person visits and save money, time, and potential exposure to various communicable diseases, especially in light of the recent pandemic.5,19,26,42 Moreover, patients in medically underserved areas may require only internet service to gain access to a vast amount of medical information. For these reasons, health care professionals have started to utilize social media as a means to reach a wider audience. 31 Despite these advantages, many of these media platforms do not regulate their content based on quality and reliability of information. Thus, the potential for dissemination of biased, incomplete, or even potentially damaging information exsists.3,15
Although the availability of the internet has increased access to informational resources, no overall improvement in health literacy has followed. 29 Especially among youth (a large proportion of TikTok’s userbase) in the United States, widespread poor health literacy has been documented to and found to be associated with adverse health outcomes.24,32 Despite poor health literacy, younger individuals paradoxically demonstrated higher competency than older populations in using social media to find information related to search topics of interest, such a health-related material. 28 Thus, the growth of TikTok’s younger audience has allowed for the dissemination of a massive quantity of medical information on topics ranging from COVID-19 to scoliosis.17,21,27,35 Although other specialties have evaluated the quality of the widely viewed information on TikTok pertaining to their respective field, orthopaedics has lagged behind in assessing orthopaedic-related content.21,27,35 With just more than 6 million total views of the videos in our analysis (which represent only a portion of the videos related to ankle sprain on the TikTok), this platform has the potential to have a wide impact on patient perception of injury and treatment options.
Our intent was to analyze the videos that a TikTok user would be most likely to encounter if searching the platform for exercises related to rehabilitation of ankle sprain. However, our data analysis revealed that videos of ankle sprain exercises found on TikTok were found to be low in quality. Concerningly, there were no videos that were categorized as “good” or “excellent” with an average DISCERN score of 28.00. Fifty-three percent of the videos were graded as “very poor,” and 45.0% of the videos were graded as “poor.” More specifically, the DISCERN 1, DISCERN 2, and DISCERN 3 were all calculated to be less than 50% of the maximum score for the DISCERN tool. Although there was a statistically significant difference between the DISCERN scores for general users and health care professionals, these results are not likely clinically relevant, as this difference is less than 1 point and the range for considering a “poor” or “very poor” score is >10 points. In addition, the ASEES scores were low with the total average score around 35% of the maximal value for the ASEES. These results further indicate that the videos did not accurately describe components of the exercise, safety considerations, rationale, and/or targeted effects.
There are a few potential explanations for these concerning results, which do not reflect on the quality or background of the uploader. For example, the short video playtime does not allow for appropriate citation of sources or in-depth discussion of risks and safety features, all important factors included in the DISCERN metric. Similarly, although TikTok may provide visual demonstrations of various exercises, users do not have access to direct professional feedback as would be available during a formal physical therapy visit. Evidence suggests that repeated engagement in exercise with incorrect technique may worsen instability, increase muscular imbalance, or exacerbate prior injuries, especially if eccentric loading is improperly performed.10,18 Overall, our findings corroborate previous studies evaluating the quality of TikTok videos for dissemination of medical information.8,21,25
A recent study by Jang et al 17 looked at the quality and reliability of TikTok for exercises on scoliosis. This study used the DISCERN and SEES scores to assess the videos. Similar to our findings, this study did not rate any of the videos as excellent or good according to the DISCERN, and their mean SEES score was 2.02 out of 5. The authors concluded that the overall educational suitability of videos about scoliosis exercise in TikTok were low and advised against using TikTok as a source for scoliosis exercise information.
This study also found that the quality of ankle sprain rehabilitation exercises differed between general users and health care professionals. Despite general users trending toward more social attraction across all categories from likes to shares, they had significantly lower scores for both the DISCERN and ASEES tools when compared to those uploaded by health care professionals. Although health care professionals did produce higher quality videos, the overall quality of the majority of the instructional videos was still graded as being “poor” or “very poor.” Thus, these videos may be inadequate to act as a stand-in for formal physical therapy or clinical visits. TikTok is an objectively inferior avenue for information dissemination compared to appropriate medical consultation from a board-certified physician. However, orthopaedic surgeons should be aware of the widespread use of this platform as a delivery mechanism for ankle sprain–related content and may consider advocating for improved scientific regulation and content creation on TikTok. Moreover, some orthopaedic surgeons who are inclined toward engagement with social media may view the expanding use of TikTok as an opportunity for distribution of more accurate medical information. This study highlights the extraordinary view counts that TikTok videos can amass. Among other use cases, TikTok may be an excellent platform to prescreen patients for clinical appointments (videos such as: “Do I need to see my doctor for my ankle sprain?”) or provide detailed exercise demonstrations with links to research-based therapeutic guidelines.
Future research should aim to evaluate if patients with ankle sprain use TikTok as an adjunct to advice from an orthopaedic provider, or as the primary means of treatment-related information. Additionally, as short-form information sharing platforms such as TikTok expand in use, future research should aim to determine if ankle sprain recurrence and rates of progression to chronic instability are impacted by reliance on web-based platforms in lieu of formal rehabilitative intervention for proprioceptive-type exercises.
Limitations
There are several limitations in our assessment of the educational quality of the videos related to ankle sprain exercises posted to TikTok. The potential for selection bias exists with the search term used. In an attempt to simulate a search term that a majority of users interested in this topic may utilize, we picked a general search term, “ankle sprain exercises.” However, users who are familiar with their injury pattern may favor more specific search terms. Similarly, because we elected to incorporate a randomization protocol to select every third video for inclusion, we may have excluded some of the more popular and potentially higher-quality videos near the top of the algorithm. Furthermore, the process of grading videos contains inherent subjectivity and observer bias. We attempted to mitigate this bias by including the use of a well-validated tool, the DISCERN, in addition to the use of the ASEES. Each video was independently reviewed by 2 separate reviewers, and a third reviewer was used where there was any significant discrepancy between grading scores. The ASEES and other scoring systems of its kind have not been formally validated and the DISCERN is a tool developed to assess the quality of written information. Thus, the use of these and similar instruments for analyzing the video quality of TikTok may be inadequate. Therefore, this study highlights the need for future research investigating the optimal scoring metrics for short-form informational videos as social media use grows worldwide.
Another major concern is that there is no way to definitively confirm whether an account is a true health care professional or not. Finally, it is possible that the short-form video format of TikTok could lead to production of poorly cited content. Although TikTok videos are generally under a minute long, videos up to 10 minutes long can be uploaded onto the platform. Thus, videos with low scores could possibly be attributed to short video playtime.
Conclusion
In summary, the increasing utilization of social media platforms such as TikTok among younger demographics has led to wide dissemination of short-form videos on a variety of health-related topics, including ankle sprain. These videos are of questionable quality and completeness. Notably, there was significant improvement in video quality across multiple scoring systems from health care professionals in comparison to general users. However, the overall educational value for both groups remained poor. Although readily accessible health care information via online resources can provide access for underserved populations, providers should remain aware of the broad dissemination of this material while also raising awareness of the deficiencies of the platform as a medium for distribution of orthopaedic-related information. The orthopaedic surgery community, in failing to acknowledge inaccurate and poorly cited information on social media and produce easily accessible yet high-quality content, is falling behind other medical subspecialties in adapting to the modern world of information dissemination.
Footnotes
Ethical Approval
No ethics approval was needed for this study. No patients were included in this work. Online, publicly available videos were analyzed.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. ICMJE forms for all authors are available online.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
