Abstract
Background:
Videos relating to rehabilitation exercises for common injuries relevant to younger populations such as anterior cruciate ligament (ACL) tear receive high view counts on social media platforms such as TikTok.
Purpose/Hypothesis:
The purpose of this study was to analyze the quality, reliability, and educational value of TikTok videos among the patient population for ACL injury. It was hypothesized that TikTok videos related to ACL rehabilitation exercises would lack quality, reliability, and educational value.
Study Design:
Cross-sectional study.
Methods:
TikTok was queried for relevant videos using the hashtags “#ACLrehab” and “#ACLexercises.” For each included video, the type of uploader (general user or health care professional) was identified. In addition, the number of views, likes, shares, comments, and favorites for each included video was recorded, and the content of each video was graded using the DISCERN (a well-validated informational analysis tool) and the ACL exercise education score (ACLEES - a custom-designed tool for the evaluation of ACL-related exercises).
Results:
A total of 111 videos with 5,520,660 cumulative views were included; the median number of views per video was 9801.0 (interquartile range [IQR], 3583.0-28,000.0). Of these videos, 84 and 27 were created by the general public and health care professionals, respectively. The videos collectively received 335,577 likes, 2969 comments, 22,856 favorites, and 6142 shares, with a median of 439 (IQR, 111-1374), 10 (IQR, 2.5-25.5), 54 (IQR, 18-172.5), and 12 (IQR, 2-36), respectively. The tabulated scores for the DISCERN and ACLEES between general users and health care professionals were all statistically nonsignificant. Health care professionals had a higher percentage of videos with a “very poor” DISCERN score in comparison with the general public (66.67% vs 53.57%, respectively).
Conclusion:
The overall educational value of the TikTok videos related to ACL rehabilitation exercises was very poor. Health care professionals should be aware of the broad distribution of ACL rehabilitation exercise videos that are accessible on TikTok and raise awareness of the deficiencies of the platform as a medium for educational medical-related information.
Rupture of the anterior cruciate ligament (ACL) is a common and debilitating injury that may result from a noncontact event, such as a sudden twisting motion due to changing direction at high speeds. ACL injury has a median incidence of 0.03% per person per year overall and up to 3.7% in some groups of professional athletes. 23 Operative management of a ruptured ACL has been recommended due to the concern that recurrent knee instability and biomechanical alteration may lead to further meniscal or articular damage, exacerbating the risk of osteoarthritis, especially in younger patients. 10 Thus, ACL reconstruction is commonly performed in younger, adolescent patients globally.1,13,37
Over the past decade, a sharp increase in information dissemination through various social media platforms has occurred. Over 90% of the American teenage population utilizes some form of social media. 7 The COVID-19 pandemic drastically accelerated the use of social media platforms as a key medium for dissemination medical information.8,16,24 In the wake of the COVID-19 pandemic, patients have begun to use these platforms to discuss their medical conditions and experiences and to explore health-related information in an easily accessible manner from the comfort of their homes. 18 As many as 42% of Americans have reported utilizing social media platforms, such as Facebook, Twitter, and YouTube, to access health information. 12
The distribution of medical information on various social media platforms has provided patients with limited health care access the ability to seek and obtain medical advice without having to interact with a medical provider. 33 With this understanding, health care professionals have started to utilize social media platforms to reach underserved communities and further their impact beyond in-person interactions. 29 Health care professionals, along with general users, have begun to disseminate information pertaining to medical conditions such as diabetes, breast cancer, and concussion.4,19,32 In 2020, Sierro et al 31 described the emergence of social media “influencers” with large followings and their ability to distribute medical information to vast audiences. However, literature suggests that only a small number of medically related social media posts are created by board-certified professionals. 27 Due to this shift in unregulated spread of medical information, which bypasses the typical peer-review process, many health care professionals question the accuracy and quality of the information being presented on applications such as TikTok.3,36
While other medical fields, such as internal medicine, 34 endocrinology, 19 oncology,32,36 and neurology, 4 have begun to evaluate the quality of the information on TikTok, orthopaedics has failed to do so adequately. Studies evaluating the quality of medical information on YouTube have been published but, given the recent rise of this newer platform, TikTok remains understudied.11,25,26 TikTok was launched in 2016 and has become one of the fastest growing social media mobile applications to date. 17 Its audience consists mainly of a younger user base and at the end of 2021 it had over 1.6 billion users and 2.6 billion downloads worldwide. 14 TikTok allows users to create, watch, and share videos that vary in length from 15 seconds to 10 minutes. The platform's advanced algorithm utilizes the user's engagement and watch time on each video to tailor content to their unique interests. Due to the large number of young TikTok users, videos relating to rehabilitation exercises for common injuries relevant to younger populations such as an ACL tear receive high view counts and are widely distributed (ie, “shared”).
In this study, we aimed to assess the quality, reliability, and educational value of ACL rehabilitation exercises on TikTok. We hypothesized that TikTok videos related to ACL rehabilitation exercises would lack quality, reliability, and educational value.
Methods
Search Strategy and Data Collection
Ethics committee approval for this study was not required. The social media platform TikTok (https://www.tiktok.com/en/) was searched for videos related to ACL injury exercises on October 25, 2022. We intended to analyze the videos that a TikTok user would most likely encounter if searching the platform for exercises related to the rehabilitation of ACL injuries. To appropriately choose search terms, we queried the TikTok search platform with a variety of phrases related to ACL reconstruction rehabilitation, such as “#ACLrehab,”“#ACLexercises,”“ACLreconstructionrehab,” and “ACLphysicaltherapy.” After doing so, significant overlap between videos was noted. Thus, the simple search terms “#ACLrehab” and “#ACLexercises” were chosen.
The search yielded a large number of videos (N = 150). We excluded videos that did not directly pertain to ACL exercises or rehabilitation (n = 33), duplicates (n = 4), and advertisements (n = 2) from the results obtained through the 2 search terms. After the screening, 25 videos remained for the keyword “#ACLrehab” (Figure 1A), and 86 remained for the keyword “#ACLexercises” (Figure 1B), for a total of 111 remaining videos.
Flowcharts of the search process for videos related to ACL injury exercises for search terms (A) #ACLrehab” and (B) “#ACLexercises.” ACL, anterior cruciate ligament.
For each video included in the analysis, we recorded data related to the uploader of the video (video link, username, video identification number, and whether the uploader was a health care provider), the number of views, likes, shares, comments, and favorites. Video uploaders were identified as health care providers if they included a relevant degree (eg, MD, DC, DPT) in their profile or video description or if they were associated with an active health care provision practice, such as an institutional or practice-wide TikTok account.
Video Scoring
Two separate scoring systems were utilized to evaluate the quality and educational value of the videos: DISCERN - a previously validated tool used for evaluating the reliability and quality of a publication describing a treatment approach 6 ; and a custom-designed tool, the ACL exercise education score (ACLEES), used to assess the education suitability of the information in each video. The ACLEES score was modified from a similar scale created for evaluating scoliosis exercise video quality from previously published work by Jang et al. 15 While neither the ACLEES nor the tests from which the ACLEES was derived have been formally validated in conjunction with the DISCERN, metrics of this kind have been utilized in the literature to assess whether viewers can properly understand and follow exercises after watching a TikTok video. 15
DISCERN for Reliability and Quality Assessment
The DISCERN questionnaire provides researchers with a reliable and accurate way of assessing the quality of information on treatment choices for a health problem. The tool, which comprises 16 questions, is well validated and has been used since the late 1990s (Table 1).
DISCERN Questions and Grading a
Scoring for questions 1 to 15 is from 1 (lowest) to 5 (highest). Scoring for question 16 is from 1 to 5, with 1 = serious or extensive shortcomings, 3 = potentially important but not serious shortcomings, and 5 = minimal shortcomings.
The first set of 8 questions (DISCERN 1) assesses the reliability of the publication and the next set of 7 questions (DISCERN 2) reviews the quality of the author's source base. The final question (DISCERN 3) rates the publication in terms of its overall quality as a source of information. Although initially designed as a tool for written information, DISCERN has been applied as a scoring test for grading the quality of videos in previous research.2,6,34,36 DISCERN scores are interpreted as excellent (63-75), good (51-62), fair (39-50), poor (27-38), and very poor (16-26).
ACLEES for Educational Suitability Assessment
This test evaluates whether viewers can adequately understand and follow exercises after watching a video. The ACLEES has 5 grading categories: exercise cycle, target, effect, safety, and rationale (Table 2). Each category is scored between 0 and 5, with higher scores demonstrating higher quality. The sum of all 5 grading categories is the final ACLEES score (range, 0-25).
ACLEES Categories and Grading a
Scoring for each category ranges from 0 to 5, with 0 = serious or extensive shortcomings; 3 = potentially important but not serious shortcomings; 5 = minimal shortcomings. ACLEES, anterior cruciate ligament exercise education score.
Assessment
All videos were evaluated independently by our orthopaedic research team. Once data regarding video distribution metrics were collected for each video, the content of the videos was graded in a blinded fashion (ie, without reviewing uploader information) using the DISCERN and ACLEES tools. Each video was graded separately by 2 reviewers (M.A.B. and K.A.-K.) trained carefully by a resident and attending orthopaedic surgeon after a thorough literature review of evidence-based post-ACL rehabilitative exercises. Any points of discrepancy between the 2 reviewers were resolved by a third reviewer (A.T.A.).
After scoring, each video was placed into 1 of 3 groups based on the uploader's background: general users, health care professionals, or health organizations. Health care professionals included users who described themselves as chiropractors, physicians, physical therapists, and nurses. The health organization group included clinics, hospitals, and treatment centers and associated TikTok accounts. In this study, there were no videos from users in the health organization group.
Statistical Analysis
Video scores and other data are presented as means and standard deviations, median (interquartile range [IQR]), or count and percentage. A 2-sample t-test was used to compare the 2 types of uploaders (general users vs health care professionals). The interobserver reliability of the DISCERN and ACLEES grading scales was determined using the class 3 model intraclass correlation coefficient (ICC). A 2-factor analysis of variance without replication model was employed to calculate the ICC along with a 95% confidence interval (CI). The benchmarking of ICC values was based on previous research, 20 with values <0.50 indicating poor reliability, 0.50 to 0.75 indicating moderate reliability, 0.76 to 0.90 indicating good reliability, and >0.90 indicating excellent reliability. Statistical significance was set at P < .05 for all comparisons. All analyses were performed using Microsoft Excel.
Results
Basic Characteristics
The total number of views of the 111 included videos included was 5,520,660. The videos collectively received 335,577 likes, 2969 comments, 22,856 favorites, and 6142 shares. Median values of these characteristics per video as well as DISCERN and ACLEES scores are shown in Table 3.
Characteristics of Included Videos (N = 111) a
Data are reported as median (IQR) or mean ± SD. ACLEES, anterior cruciate ligament exercise education score; IQR, interquartile range.
Types of Uploaders
General users uploaded more videos (76%) than health care professionals (24%). However, the dissemination statistics (including views, likes, comments, favorites, and shares) were similar between general users and health care professionals (Table 4). Scores tabulated for DISCERN 1, DISCERN 2, DISCERN 3, total DISCERN, and ACLEES demonstrated no statistically significant differences between general users and health care professionals, as indicated in Table 4.
Characteristics of the Videos According to Type of Uploader a
Data are reported as median (IQR) or mean ± SD. ACLEES, anterior cruciate ligament exercise education score; IQR, interquartile range.
The DISCERN grading results are recorded in Table 5. Both general users and health care professionals displayed overall low-quality ratings in their uploaded videos. Health care professionals had a higher percentage of their videos graded as “very poor” (66.67%) in comparison with the percentage of videos uploaded by general users deemed “very poor” (53.57%). Conversely, general users had a higher percentage of their videos graded as “poor” (46.43%) in comparison with the videos uploaded by health care professionals (33.33%). Neither general users nor health care professionals had videos graded as “fair,”“good,” or “excellent.”
Distribution of DISCERN Grades According to Type of Uploader a
Data are reported as n (%).
Interobserver Reliability
The estimates for interobserver reliability for the DISCERN and ACLEES were 0.74 (95% CI, 0.62-0.82) and 0.79 (95% CI, 0.69-0.86), respectively. These results indicated moderate and good reliability for those scoring systems, respectively.
Discussion
Our data analysis revealed that videos of ACL rehabilitation exercises found on TikTok had overall low reliability (DISCERN 1), quality (DISCERN 2, DISCERN 3), and educational value (ACLEES). There were no videos that were graded as “excellent,”“good,” or “fair,” with an average DISCERN score of 25.91. In total, 56.76% of the videos were graded as “very poor” and 43.24% as “poor.” DISCERN 1, DISCERN 2, and DISCERN 3 were all calculated to be <50% of the maximum score for the DISCERN tool. The tabulated ACLEES scores were low, with the total average score around 13% of the maximal value for the ACLEES tool. These results further indicate that the videos did not accurately describe the components of the exercise, rationale, targeted effects, or safety considerations.
These results demonstrate even more concerning findings than those presented by Jang et al, 15 where scoliosis exercises were found to have a total DISCERN score of 33.60. In direct contrast, other studies have found TikTok to be an adequate source of medical information for other pathologies in fields outside of orthopaedics: both chronic obstructive pulmonary disease (total DISCERN score of 56-67) and diabetes (total DISCERN score of 40-51) demonstrate relatively high DISCERN scores when compared with our results.19,34 However, the present study reflects the findings of other studies evaluating YouTube as a source of information pertaining to ACL, meniscus, kyphosis, and adhesive capsulitis.5,9,21,35
While the content uploaders should assume responsibility for the quality of their videos, the poor quality of medically related TikTok videos could stem from other factors. TikTok relies on short video playtime and engagement times that may not allow for appropriate citation of sources or thorough discussion of rationale, risks, benefits, and safety precautions. While TikTok allows for video demonstration of exercises, the platform lacks the immediate, tailored feedback that can be found in a clinic that functions to ensure that patients are performing the exercise as intended and not exposing themselves to injury due to improper form. In addition, TikTok lacks the necessary scientific oversight from experts in the field of interest required to peer-review the quality of health care information being posted and shared. Moreover, this study found that the quality of ACL rehabilitation exercises did not differ between general users and health care professionals, further indicating that TikTok may simply be a poor avenue for dissemination of rehabilitative and orthopaedic information.
Although some aspects of content creation are outside of the uploader's control, there were clear deficiencies in both the DISCERN and ACLEES subsection scores that could be easily addressed to increase both the quality and educational value of the videos related to ACL rehabilitation on TikTok. As a first point of emphasis, proper citation of peer-reviewed, evidence-based information is critical, even in short-form video format. However, the vast majority of videos analyzed did not provide a clear indication of where the information was obtained (eg, PubMed ID, weblink, or citation), thus limiting credibility. Moreover, videos could aim to support a shared decision-making model, with proper discussion of risks and benefits of the demonstrated exercise. Finally, all of the videos analyzed failed to describe safety precautions for any of the exercises demonstrated. This could potentially be dangerous for certain exercises that involve complex movements, especially in the setting of ACL reconstruction and the lack of professional supervision. If uploaders work to fix these deficiencies, many of the videos could be greatly improved, and TikTok could actually serve as a reasonable repository for evidence-based, short-form videos related to ACL rehabilitation.
Implementation of a peer-reviewed system for health care–related TikTok videos is unlikely. However, health care providers should strive to improve the information present on TikTok. Initial efforts could center primarily on the correction of misinformation. This can be accomplished through “commenting” on videos and providing evidence-based information and links to peer-reviewed sources. Health care providers may consider creating their own TikTok videos to share accurate information and reach a wider audience. Specifically, orthopaedic surgeons may find the short-form video format perfect to answer questions such as “What should I do if my wound is draining after surgery?” or “How do I know if my wound is infected?” Finally, health care providers should engage with their patients to encourage critical thinking when using social media sources such as TikTok. Emphasis on proper citation of peer-reviewed sources and recognition of overstated claims can help patients interpret the enormous amount of information available online. By taking these steps, health care providers can help to improve the accuracy of health-related information presented on TikTok and help to ensure that the platform is a valuable source of information.
Videos related to ACL rehabilitation can reach a large audience on TikTok; there were over 5.5 million views of the videos included in our analysis, representing only a portion of videos related to ACL tear on TikTok. If the informational content on TikTok is of low quality and is unsafe, the impact of this viewership may be exacerbated given the documented lack of health literacy among young people in the United States.22,30 Despite this finding, younger age groups were found to be more competent than older age groups in utilizing YouTube and Facebook to access relevant information quickly. 28 Thus, it is reasonable to assume that the user base accessing TikTok for sports-related rehabilitation exercises may continue to grow, and this group may lack the requisite health literacy to properly screen content for safe and efficacious use.
Limitations
There are several limitations in our assessment of the educational quality of the videos related to ACL rehabilitation exercises posted to TikTok. The potential for selection bias exists with the search terms used. We used general search terms such as “ACLexercises” and “ACLrehab” to attempt to simulate a search term that the majority of users interested in this topic might utilize to obtain ACL rehabilitation videos. However, it could be possible that users favor alternative search terms. In addition, the process of grading videos contains an inherent subjectivity in the assessment of the quality of the video based on previous knowledge of the reviewer. We attempted to address this bias by utilizing a well-validated tool, the DISCERN, in addition to using the ACLEES. nother limitation is that the ACLEES test has not been formally validated. In addition, each video was independently reviewed by two separate reviewers, and a third reviewer was consulted where there was any significant discrepancy. Further research should aim to validate this and other tools that can be utilized to assess short-form informational videos in an era of social media expansion.
Conclusion
The overall educational value of the TikTok videos related to ACL rehabilitation exercises was very poor. Health care professionals should be aware of the broad distribution of ACL rehabilitation exercise videos accessible on TikTok and raise awareness of the deficiencies of the platform as a medium for educational medical-related information.
Footnotes
Final revision submitted May 17, 2023; accepted July 11, 2023.
One or more of the authors has declared the following potential conflict of interest or source of funding: B.C.L. has received grants from Zimmer Biomet Holdings and DJO, education payments from SouthTech Orthopedics and Smith & Nephew, and hospitality payments from Stryker and Wright Medical Technology. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval was not sought for the present study.