Sage Journals: Discover world-class research

Abstract

Background:

Ulnar collateral ligament (UCL) reconstruction has received a unique level of attention in the press and social media. There has also been an increasing use of the internet by patients to seek medical information. Concern exists regarding the quality and comprehensibility of online information when used for patient education.

Purpose:

To evaluate the quality and comprehensibility of the most-viewed YouTube videos related to the diagnosis and management of UCL injuries. Based on our new evidence-based scoring rubrics, we hypothesized that the quality and comprehensibility of these videos would be poor.

Study Design:

Cross-sectional study.

Methods:

The YouTube platform was searched on September 7, 2021, with the terms “UCL injury,” “ulnar collateral ligament injury,” “UCL surgery,” “ulnar collateral ligament surgery,” and “Tommy John surgery,” and the 50 most-viewed videos from each search were compiled, yielding 250 videos. After removal of duplicates and application of exclusion criteria, the 100 most-viewed videos remained. Basic attributes, including duration of video and number of views, were recorded. Each video was then analyzed by 2 independent reviewers and evaluated for 4 key parameters (quality of diagnostic content [QAR-D], quality of treatment content [QAR-T], presence of inaccurate information, and comprehensibility) and graded on a novel scale from 1 to 4 (4 being the most appropriate for patient education).

Results:

The mean QAR-D was 4.83 ± 3.41 (fair quality), and the mean QAR-T was 2.76 ± 3.26 (poor quality). Physician-led educational videos had both the highest mean QAR-D (6.37) and the highest mean QAR-T (4.34). No correlation was observed between video quality and views/likes. A total of 12 videos included ≥1 inaccuracy. The mean comprehensibility score was 2.66 ± 1.12, with 39 videos falling below the acceptable comprehensibility threshold (score <3).

Conclusion:

The overall quality of UCL injury–related YouTube content was low. In addition, the absence of correlation between video quality and views/likes suggests that patients are not preferentially utilizing the limited high-quality content that does exist on the YouTube platform. In addition, inaccurate videos were prevalent (12%), and almost half of all videos were deemed inappropriate for patient education in terms of comprehensibility, as defined by our comprehensibility parameter.

Keywords

Tommy John UCL injury UCL reconstruction ulnar collateral ligament YouTube

The ulnar collateral ligament (UCL) is a 3-part ligamentous complex in the medial elbow that serves as the primary restraint to valgus stress of the elbow.^25,32 Consequently, the integrity of this ligament is of particular importance to overhead athletes whose motion relies on a stable complex to throw a baseball, javelin, or discus, for example. Injury to the UCL can occur acutely as a traumatic rupture from an excessive valgus load or gradually over time from repetitive valgus stress resulting in attenuation of the ligamentous complex, as is seen in proportionally larger numbers in overhead athletes.⁴³ Once considered a relatively uncommon injury, UCL tears have become increasingly prevalent over the past few decades.^16,23,27 This phenomenon is likely the result of numerous factors including an overall rise in the popularity of throwing sports, early sports specialization, increased participation on club teams, and year-round play, as well as greater throwing velocity and higher pitch counts observed among young (age, 6-18 years) baseball players between youth and high school levels.^{5,19,24,38,39,43} A recent epidemiological study conducted in New York state, for instance, found that UCL reconstruction (UCLR) volume increased 193% between 2002 and 2011.²³ Among professional baseball pitchers, the prevalence of UCLR has been shown to be as high as 20%, further exemplifying the significant burden of UCL injury.^11,27

As the prevalence of UCL injuries has increased, UCLR has received a unique level of attention in the press and social media, often referred to as “Tommy John surgery” in reference to the first baseball player to undergo the procedure.^1,12 Between August 2016 and August 2019, 3118 posts related to UCL injury and UCLR were made on the Instagram social media platform.⁵⁴ The posts came from a wide variety of sources including patients, physicians, news outlets, professional organizations, and sports fans, and the number of posts per year increased by 39% over the 3-year study period. Unfortunately, the increased social awareness of UCL injuries has been accompanied by widespread misconceptions regarding UCLR, with up to 25% of media professionals believing performance enhancement was the primary indication for UCLR.¹²

The utilization of social media platforms to discuss medical conditions is representative of a larger phenomenon in which patients are increasingly utilizing medical information available on the internet to learn about and make decisions regarding their health. An analysis from 2012 found that 59% of all US adults had searched the internet for medical information in the previous year alone.³⁷ Furthermore, 35% of all US adults attempted to make a medical diagnosis or treatment decision using information obtained on the internet. While there are certainly many potential benefits that accompany the widespread availability and accessibility of medical information on the internet, including patient empowerment and increased shared decision making, the overall utility of medical information on the internet is largely contingent upon the quality of information.^21,46 Interestingly, a survey of 1050 US physicians revealed widespread concern over the prospect of patients accessing and utilizing inaccurate medical information on the internet.³³

In addition to the accuracy of patient-oriented medical information on the internet, another important attribute to consider is comprehensibility.⁵³ With regard to orthopaedic conditions, several studies have shown that, even when online information is deemed accurate, the readability often exceeds the recommended sixth-grade level.^{14,31,41,42,53} This observation may explain, at least in part, the increasing availability and utilization of online videos as a source of medical information for patients.²⁹ However, the potential for increased comprehensibility offered by videos does not preclude inaccuracy. A recent systematic review³⁰ assessing the accuracy of medical information on YouTube, the largest online video repository platform in the world, found that upward of 20% to 30% of videos analyzed in their included studies contained misleading information and also demonstrated relatively low variation between the mean view counts of accurate and inaccurate videos.⁴⁴

Given the widespread use of YouTube and other online video platforms by patients to obtain medical information, it is critical for providers to be aware of the overall quality of information currently available on such platforms. This is especially true for a condition such as UCL injury, which is increasing in prevalence, has received significant attention in the media, and is associated with several widely held misconceptions. Accordingly, the purpose of this review was to evaluate the quality and comprehensibility of the most-viewed YouTube videos related to the diagnosis and management of UCL injuries. We hypothesized that, based on our new evidence-based scoring rubrics, the quality and comprehensibility of the most-viewed YouTube videos related to UCL injuries would be poor.

Methods

Search Methodology

The YouTube platform was searched on September 7, 2021, to compile a list of the most-viewed videos related to UCL injury and management uploaded in the previous 10 years. Five independent search queries were performed using the following terms: “UCL injury,” “ulnar collateral ligament injury,” “UCL surgery,” “ulnar collateral ligament surgery,” and “Tommy John surgery.” The following exclusion criteria were utilized: videos not related to UCL injury of the elbow (or <50% of the video duration related to UCL injury), videos containing only footage of actual UCL injuries occurring, duplicate videos, videos without audio or explanatory text, videos in non-English languages, and videos posted before September 7, 2011. For each of the 5 queries, the 50 most-viewed videos were compiled. From this list of 250 videos, 123 duplicates were excluded, resulting in a list of the 127 videos. Of these, the 100 videos with the most views were identified for analysis and the remainder were added to a reserve list. An additional 5 videos (3 unrelated to UCL injury, 2 with no audio or text) were excluded during the video analysis phase; thus, the next 5 most-viewed videos from the reserve list were included for the final analysis (Figure 1).

Figure 1.

Search methodology flowchart. UCL, ulnar collateral ligament.

Of note, videos deemed to be from a series were analyzed as a single video. Videos from a series were either in a YouTube playlist, were from the same producer and had identical titles followed by a colon with the video subtitle, were directly linked by the producer on each video page, or were clearly titled (part 1, part 2, etc). Videos could be from the same producer and not be considered part of a series if these qualifications were not met.

Video Characteristics

The 100 videos that met inclusion criteria were analyzed independently by 2 authors (C.R.C., A.G.). Basic attributes were recorded for each video, including duration, date uploaded, number of views, number of “likes,” number of “dislikes,” and video source. Average views per day, likes per 1000 views, average likes per day, and likes:dislikes ratio were calculated based on these data.

The 6 possible video source categories were “physician-led educational,” “other medical practitioner–led educational,” “nonmedical practitioner–led educational,” “surgical technique,” “patient testimonial,” and “news.” Physician-led educational videos were educational videos presented by a physician (MD or DO) and/or endorsed by a medical institution. Videos categorized as other medical practitioner–led educational were from a nonphysician medical practitioner, including but not limited to physical therapists, occupational therapists, athletic trainers, chiropractors, massage specialists, physician assistants, and nurse practitioners. Nonmedical practitioner–led educational videos were from nonmedical personnel such as students who did not meet the above qualifications, coaches, and sports fans. Surgical technique videos involved recordings of UCLR or repair procedures; these were often paired with audio of the operating surgeon. Patient testimonials involved patients describing their experiences with UCL injuries. If the patient was a medical professional or if a medical professional also provided information in the video, the video was classified in the appropriate educational category. Finally, news videos were uploaded by an official news or sports broadcast agency and did not include a physician interview.

Quality Assessment

The primary outcome measures in this analysis were quality assessment ratings (QARs) for diagnostic (QAR-D) and treatment (QAR-T) content. Scoring rubrics were created by the authors to evaluate quality of content in regard to thoroughness, accuracy, and appropriateness for patient education, in similar format to previous studies.^13,29 Details of the QAR-D and QAR-T scoring rubrics can be found in Table 1 and Table 2, respectively. Videos were assigned separate categorical scores for both diagnostic and treatment content: excellent (QAR = 13-16), high (10-12), moderate (7-9), fair (4-6), and poor (0-3) quality. Reviewer disagreement on quality scoring was reconciled via discussion and joint review of videos as needed.

Table 1

QAR for Evaluation of UCL Injury-Related Videos: Diagnostic Content ^a

Video Characteristic	Points	Requirements and Appropriate Answers ^b
Explanation of Injury
UCL defined	1	Any mention of structure, function, or location of the UCL^25,32
UCL injury defined	1	Any mention or description of what UCL injury entails (including but not limited to partial/complete tear or sprain of the ligament)⁴³
Reference to “Tommy John surgery”	1	Any mention or relation of UCL injury to the phrase “Tommy John Surgery” or “Tommy John ligament”; inclusion in video title considered sufficient^1,12
Differential diagnosis	1	Inclusion of ≥1 of the following: medial epicondylitis, elbow fracture, elbow arthritis, flexor-pronator mass strain, valgus extension overload⁴³
History: Mechanism of Injury
Acute vs chronic	1	Any mention that UCL injury can be acute (including but not limited to single event, “giving way,” “pop”) or chronic^10,43
High-risk activities	1	Inclusion of ≥2 of the following high-risk activities: any throwing sport such as baseball or javelin, tennis, wrestling, gymnastics, arm wrestling⁴³
Valgus load	1	Video or illustration demonstrating valgus loading of the elbow; need not explicitly use the word “valgus”
History: Symptoms
Pain in medial elbow	1	Any mention of pain or discomfort in medial elbow⁴³
Decreased athletic performance	1	Any mention of decreased throwing performance or decreased athletic performance related to affected elbow or pain with throwing⁴³
Additional symptoms	1	Inclusion of ≥1 of the following symptoms: mechanical symptoms (locking or catching in the elbow), paresthesias (any mention of loss of sensation in ulnar distribution)⁴³
History: Risk Factors
Risk factors discussed	1	Inclusion of ≥1 risk factor (including, but not limited to, increased throwing volume, increased throwing velocity, shoulder/core weakness, and stiffness of upper extremity)^{5,19,24,39,43}
Physical Examination
Tenderness at medial elbow	1	Tenderness to palpation at the location of the UCL⁴³
Valgus instability testing	1	Medial joint line opening or basic valgus instability discussed or graphically depicted⁵⁰
Advanced maneuvers	1	Milking maneuver or moving valgus test discussed or depicted graphically^36,50
Imaging
Common modalities used	1	≥1 relevant imaging modality discussed (radiograph, radiograph with valgus stress, CT, MRI, MRA, dynamic ultrasound)⁷
Findings on imaging	1	Basic findings discussed (ligament partial or complete tear with or without edema, avulsion fracture, loose bodies)^7,34
Total QAR-D score ^c	16

^a CT, computed tomography; MRA, magnetic resonance arthrogram; MRI, magnetic resonance imaging; QAR, quality assessment rating; QAR-D, QAR for diagnostic content; UCL, ulnar collateral ligament.

^b If any medically inaccurate statement or scientifically unproven data were included in a specific category, all points in that category were forfeited.

^c Grading: 13-16, excellent quality; 10-12, high quality; 7-9, moderate quality; 4-6, fair quality; 0-3, poor quality.

Table 2

QAR for Evaluation of UCL Injury-Related Videos: Treatment Content ^a

Video Characteristic	Points	Requirements and Appropriate Answers ^b
General Information
Multifactorial nature of treatment	1	Any mention of multifactorial nature of treatment decisions and/or that risks and benefits should be discussed with a clinician
Treatment Options
Nonoperative management	1	Any mention of nonoperative management including rest, activity modification, anti-inflammatory medications, or physical therapy⁴⁵
UCL repair	1	Any mention and description of UCL repair^47,48
UCL reconstruction	1	Any mention and description of UCL reconstruction including discussion of graft selection^22,45,48,51
Indications
Indications for nonoperative management	1	≥1 indication mentioned (including, but not limited to, partial UCL tears, UCL tears in nonathletes, UCL tears in those unwilling to undergo extensive rehabilitation)⁴⁵
Indications for UCL repair	1	≥1 indication mentioned (including but not limited to acute avulsion-type ruptures, proximal or distal tears with good tissue quality)^47,48
Indications for UCL reconstruction	2	Up to 2 points awarded for appropriate indications (including, but not limited to, acute ruptures in high-level throwing athletes, significant chronic instability, insufficient tissue for primary repair, recurrent pain after physical therapy)^22,45,48,51
Postoperative Rehabilitation
Immobilization	1	Immobilization for 7-14 days²⁸
Early range of motion	1	Any mention of active wrist, elbow, or shoulder range of motion after initial immobilization²⁸
Strengthening	1	Any mention of strengthening exercises beginning 4-8 weeks after surgery²⁸
Avoidance of valgus stress	1	Any mention of avoiding valgus stress until ≥12 weeks after surgery using description or graphic illustration (appropriate to refer to previous valgus stress illustration if provided earlier in video)²⁸
Sport-specific rehabilitation	1	Any statement explaining that additional sport-specific rehabilitation protocols exist²⁸
Natural Course and Prognosis
Outcomes associated with nonoperative management	1	Any mention of the natural course of injury with nonoperative management (eg, 42% of patients with nonoperative management return to preinjury activity at roughly 24 weeks, may progress to medial epicondylitis, ulnar nerve tensile injury, valgus extension overload)^40,43
Outcomes associated with operative management	1	Any mention of the outcomes after operative management (eg, 90% of patients shown to have excellent outcomes following UCL reconstruction, return to competition at or above preinjury level, of 85.7% mean return to competition of 12.2 months following UCL reconstruction, expected return to competition timeline of roughly 7 to 12 months)^4,51
Complications of operative management	1	Inclusion of ≥2 potential complications (including, but not limited to, ulnar nerve injury, medial antebrachial cutaneous nerve injury, ulnar/medial epicondylar fracture, elbow stiffness, failure to return to preinjury performance)^6,22
Total QAR-T score ^c	16

^a QAR, quality assessment rating; QAR-T, QAR for treatment content; UCL, ulnar collateral ligament.

^b If any medically inaccurate statement or scientifically unproven data were included in a specific category, all points in that category were forfeited.

^c Grading: 13-16, excellent quality; 10-12, high quality; 7-9, moderate quality; 4-6, fair quality; 0-3, poor quality.

It is important to note that if any inaccurate or unfounded information was identified in a video related to a specific QAR item, a score of 0 was assigned for that item. Non–UCL related content, if present, was not evaluated for medical accuracy. The medical judgment of the authors and referencing of peer-reviewed literature was used to determine the accuracy of statements made in videos. The total number and proportion of videos with ≥1 inaccurate statement was also recorded.

Finally, a comprehensibility parameter was developed to evaluate the appropriateness of each video for patient education, much like “readability” parameters are applied to written materials. The comprehensibility scoring system and definitions for each specific score can be found in Table 3. A score of 3 or higher was defined as acceptable for patient education. An intraclass correlation coefficient between the scores of the 2 reviewing authors was calculated for this parameter.

Table 3

Video Comprehensibility

Score	Description
1	Heavy use of medical jargon (80%-100% of concepts explained solely with the use of medical jargon); inappropriate for patient education
2	Moderate use of medical jargon (>50% of concepts explained solely with the use of medical jargon); questionable appropriateness for patient education
3	Minimal use of medical jargon (10%-25% of concepts explained solely with the use of medical jargon); acceptable for patient education
4	Absent or negligible use of unexplained medical jargon (nearly 0% of concepts explained solely with the use of medical jargon); highly appropriate for patient education

Statistical Analysis

Categorical variables were reported as counts and continuous variables were reported as means with standard deviations and ranges. The Shapiro-Wilk test was performed to examine the distribution of individual variables for normality. The 1-way analysis of variance (ANOVA) test was used to compare means of continuous variables between groups. Bonferroni-adjusted pairwise t tests were performed for any significant differences found. The Fisher exact test was used to compare means of categorical variables between groups. The Pearson correlation coefficient was used to assess associations between 2 continuous variables. All analyses were performed with R Version 4.1.0 (R Foundation for Statistical Computation). Alpha level was set at < .05.

Results

A total of 100 videos were included in this analysis. Date of video upload ranged from October 28, 2011, to July 28, 2021. The mean duration of all videos was 6 minutes 31 seconds (range, 44 seconds to 43 minutes 31 seconds), the mean number of views was 28,218 (range, 582 to 487,053), and the mean number of likes was 195.66 (range, 0 to 3100). There was no statistically significant correlation between video duration and number of views (r = 0.07; P = .47). A summary of the top 10 most-viewed videos can be found in Appendix Table A1.

The mean QAR-D of all 100 videos was 4.83 ± 3.41 (fair quality), the median score was 4, and the range was 0 to 15 (Figure 2). The most commonly omitted item in QAR-D was “differential diagnosis” (92 videos failed to include). The mean QAR-T of all 100 videos was 2.76 ± 3.26 (poor quality), the median score was 2, and the range was 0 to 16 (Figure 2). Only 3 videos discussed “indications for UCL repair,” and only 4 videos described needing to “avoid valgus stress postoperatively.” A total of 12 videos included ≥1 inaccuracy. The mean comprehensibility score was 2.66 ± 1.12, with 39 videos falling below the acceptable comprehensibility threshold (ie, score <3). The intraclass correlation coefficient for comprehensibility scores was r = 0.82 (P < .01).

Figure 2.

Distribution of QAR-D and QAR-T scores from the 100 most-viewed videos related to ulnar collateral ligament injury and management. QAR, quality assessment rating; QAR-D, QAR for diagnostic content; QAR-T, QAR for treatment content.

Analysis by QAR-D

Regarding QAR-D categories, 3 videos were of excellent quality, 8 were graded as high quality, 14 were moderate quality, 38 were fair quality, and 37 videos were poor quality. Videos with the highest QAR-D scores are highlighted in Appendix Table A2. After performing ANOVA tests for each of the basic video characteristics (eg, duration, views, likes), the only statistically significant difference between the 5 QAR-D groups was video duration (F = 5.45; P = .022) (Table 4). The QAR-D videos graded as excellent and high quality were the longest (mean duration 14 minutes and 6 seconds and 11 minutes and 9 seconds, respectively) and significantly differed in length from the lesser quality videos (P < .01). Duration was also the only video characteristic found to correlate positively with numerical QAR-D scores (r = 0.26; P < .01). Moderate-quality QAR-D videos garnered the highest mean views (62,246) and mean likes per 1000 views (10.22), although there were no statistically significant differences between the groups in either of these categories.

Table 4

Video Evaluation by QAR-D Category ^a

		QAR-D Category^c
	Total Videos(N = 100) ^b	Excellent Quality(n = 3)	High Quality(n = 8)	Moderate Quality(n = 14)	Fair Quality(n = 38)	Poor Quality(n = 37)	P
Video Characteristic
Duration, minutes:seconds	6:31 ± 6:18	14:06	11:09	7:33	4:23	6:41	.022
Days online	1810 ± 906	2121	1672	1736	1949	1700	.707
Views	28,218 ± 65,929	16,592	36,090	62,246	24,556	18,344	.214
Views per day	32.48 ± 146.37	11.33	175.54	30.82	11.55	25.39	.128
Likes	195.66 ± 462.82	128.00	432.13	331.14	127.47	168.78	.213
Likes per day	0.53 ± 3.00	0.09	3.07	0.29	0.07	0.58	.318
Likes per 1000 views	8.29 ± 8.62	4.91	9.78	10.22	6.41	9.44	.808
Dislikes	6.30 ± 14.26	3.00	6.38	12.86	5.61	4.78	.427
Likes:dislikes ratio	26.25 ± 27.96	21.33	36.17	31.42	19.64	29.33	.805
Video Source							.113
Physician-led educational	35	2 (67.7)	5 (62.5)	5 (35.7)	19 (50.0)	4 (10.8)
Other medical practitioner–led educational	20	1 (33.3)	1 (12.5)	5 (35.7)	6 (15.8)	7 (18.9)
Nonmedical practitioner–led educational	13	0 (0)	1 (12.5)	3 (21.4)	2 (5.3)	7 (18.9)
Surgical technique	16	0 (0)	0 (0)	1 (7.1)	6 (15.8)	9 (24.3)
Patient testimonial	9	0 (0)	1 (12.5)	0 (0)	2 (5.3)	6 (16.2)
News	7	0 (0)	0 (0)	0 (0)	3 (7.9)	4 (10.8)
Comprehensibility Score							.645
4	28	1 (33.3)	4 (50)	4 (28.6)	7 (18.4)	12 (32.4)
3	33	0 (0)	1 (12.5)	6 (42.9)	16 (42.1)	10 (27.0)
2	16	1 (33.3)	1 (12.5)	3 (21.4)	6 (15.8)	5 (13.5)
1	23	1 (33.3)	2 (25)	1 (7.1)	9 (23.6)	10 (27.0)
Inaccuracy							>.999
No medical inaccuracies	88	3 (100)	7 (87.5)	13 (92.9)	33 (86.8)	32 (86.5)
≥1 medical inaccuracy	12	0 (0)	1 (12.5)	1 (7.1)	5 (13.2)	5 (13.5)

^aBoldface P value indicates statistically significant difference between quality groupings (P < .05). QAR, quality assessment rating; QAR-D, QAR for diagnostic content.

^b Data are presented as mean ± SD or No. of overall videos.

^c Data are presented as mean or No. of videos in that category (% of category total).

Each of the 5 QAR-D categories were also stratified by source (Table 4). All 3 excellent-quality QAR-D videos were either physician-led educational or other medical practitioner–led educational videos. The vast majority (24/29; 82.8%) of nonmedical practitioner–led educational, patient testimonial, and news videos fell within the QAR-D categories of fair quality or poor quality. The number of videos achieving each comprehensibility score (1 through 4) from each of the 5 QAR-D categories was also identified, but no statistically significant difference was found between the groups (Table 4).

Analysis by QAR-T

Regarding QAR-T categories, 3 videos were graded as excellent quality, 2 were high quality, 6 were moderate quality, 17 were fair quality, and 72 videos were poor quality. Videos with the highest QAR-T scores are highlighted in Appendix Table A3. Similar to the analysis of diagnostic content, the only statistically significant difference between the QAR-T groups from ANOVA testing of each video characteristic was video duration (F = 36.26; P < .01) (Table 5). Excellent-quality and high-quality QAR-T videos were significantly longer (mean duration 20 minutes 9 seconds and 23 minutes 21 seconds, respectively). There was also a positive correlation between numeric QAR-T scores and duration (r = 0.51; P < .01). Poor-quality QAR-T videos garnered the highest mean views (30,480), but there was no statistically significant difference between groups. Comparison of mean likes of combined poor- and fair-quality QAR-T videos (mean = 209.67) versus combined moderate-, high-, and excellent-quality QAR-T videos (mean = 82.27) revealed that the lower quality videos had significantly more likes (P = .039).

Table 5

Video Evaluation by QAR-T Category ^a

		QAR-T Category ^c
	Total Videos(N = 100) ^b	Excellent Quality(n = 3)	High Quality(n = 2)	Moderate Quality(n = 6)	Fair Quality(n = 17)	Poor Quality(n = 72)	P
Video Characteristic
Duration, minutes:seconds	6:31 ± 6:18	20:09	23:21	7:08	8:09	5:02	<.001
Days online	1810 ± 906	2276	1876	2073	1732	1786	.352
Views	28,218 ± 65,929	23,483	15,349	13,355	26,235	30,480	.572
Views per day	32.48 ± 146.37	13.28	8.08	4.68	88.00	23.17	.864
Likes	195.66 ± 462.82	144.33	107.50	42.83	266.47	196.26	.679
Likes per day	0.53 ± 3.00	0.10	0.06	0.02	1.51	0.37	.929
Likes per 1000 views	8.29 ± 8.62	4.75	6.32	7.59	9.70	8.21	.628
Dislikes	6.30 ± 14.26	4.00	2.50	2.50	5.29	7.06	.392
Likes:dislikes ratio	26.25 ± 27.96	22.33	50.25	26.50	24.39	26.16	.792
Video Source							.505
Physician-led educational	35	3 (100)	1 (50)	3 (50)	10 (58.8)	18 (25.0)
Other medical practitioner–led educational	20	0 (0)	1 (50)	1 (16.7)	2 (11.8)	16 (22.2)
Nonmedical practitioner–led educational	13	0 (0)	0 (0)	1 (16.7)	1 (5.9)	11 (15.3)
Surgical technique	16	0 (0)	0 (0)	1 (16.7)	3 (17.6)	12 (16.7)
Patient testimonial	9	0 (0)	0 (0)	0 (0)	1 (5.9)	8 (11.1)
News	7	0 (0)	0 (0)	0 (0)	0 (0)	7 (9.7)
Comprehensibility Score							.758
4	28	1 (33.3)	0 (0)	1 (16.7)	5 (29.4)	21 (29.2)
3	33	1 (33.3)	2 (100)	1 (16.7)	5 (29.4)	24 (33.3)
2	16	0 (0)	0 (0)	1 (16.7)	4 (23.5)	11 (15.3)
1	23	1 (33.3)	0 (0)	3 (50)	3 (17.6)	16 (22.2)
Inaccuracy							>.999
No medical inaccuracies	88	3 (100)	2 (100)	6 (100)	15 (88.2)	62 (86.1)
≥1 medical inaccuracy	12	0 (0)	0 (0)	0 (0)	2 (11.8)	10 (13.9)

^a Boldface P value indicates statistically significant difference between quality groupings (P < .05). QAR, quality assessment rating; QAR-T, QAR for treatment content.

^b Data are presented as mean ± SD or No. of overall videos.

^c Data are presented as mean or No. of videos in that category (% of category total).

Each of the 5 QAR-T categories was also stratified by source (Table 5). All 5 high-to-excellent quality QAR-T videos were either physician-led educational or other medical practitioner–led educational videos. All but 1 (28/29; 96.6%) of the nonmedical practitioner–led educational, patient testimonial, and news videos fell within the QAR-T category of “fair” or “poor” quality. The QAR-T groups did not differ with regard to the relative proportions of each comprehensibility score (Table 5).

Analysis by Video Source

The most common video source was physician-led educational (n = 35) followed by other medical practitioner–led educational (n = 20), surgical technique (n = 16), nonmedical practitioner–led educational (n = 13), patient testimonial (n = 9), and news (n = 7) (Figure 3). ANOVA tests identified statistically significant differences between video source groups for both QAR-D and QAR-T scores (Table 6). Physician-led educational videos had both the highest mean QAR-D (6.37) and the highest mean QAR-T (4.34), and these means were significantly greater than the combined QAR-D and QAR-T means of the other 5 source categories (P < .01 and P < .01, respectively). While not statistically significant, physician-led educational videos had the lowest percentage of videos containing inaccuracies (2/35; 5.71%). Patient testimonial videos had the highest mean views (70,335) despite having the lowest mean QAR-D (2.33) and second lowest mean QAR-T (1.22), but none of these findings reached statistical significance. Technique videos demonstrated the lowest mean comprehensibility score (1.13) on ANOVA and confirmatory t testing (P < .01).

Figure 3.

Video distribution by source of most-viewed videos related to ulnar collateral ligament injury.

Table 6

Video Evaluation by Source ^a

Variable	Physician-Led Educational(n = 35)	Other Medical Practitioner–Led Educational(n = 20)	Nonmedical Practitioner–Led Educational(n = 13)	Surgical Technique(n = 16)	Patient Testimonial(n = 9)	News(n = 7)	P
Duration, minutes: seconds	6:05	7:42	4:15	8:00	7:25	4:45	.934
QAR-D, mean	6.37	5.10	4.23	4.00	2.33	2.57	<.001
QAR-T, mean	4.34	2.10	1.85	2.56	1.22	0.86	<.001
Comprehensibility, mean	2.91	2.55	2.92	1.13	3.89	3.14	<.001
Inaccuracy, count (%)	2 (5.71)	3 (15)	3 (23.1)	1 (6.3)	1 (11.1)	2 (28.6)	.457
Views, mean	36,560	16,521	29,468	8,248	70,335	9,105	.771
Likes, mean	205.03	132.50	173.23	31.63	743.00	42.14	.540

^a Boldface P values indicate statistically significant difference between sources (P < .05). QAR, quality assessment rating; QAR-D, QAR for diagnostic content; QAR-T, QAR for treatment content.

Discussion

The purpose of this study was to evaluate the top 100 most-viewed YouTube videos related to UCL injury using 4 key parameters: QAR-D, QAR-T, presence of inaccurate information, and comprehensibility. Key statistically significant findings included the association between video quality and duration (P < .01 for both QAR-D and QAR-T) as well as the increased mean video quality scores (QAR-D, 6.37; QAR-T, 4.34) among physician-led educational videos (P < .01 for both). Also striking was the overall predominance of low-quality videos; 75 videos were of fair or poor quality for diagnostic content, 89 videos were of fair or poor quality for treatment content, and 12 videos included ≥1 inaccuracy. Similar findings have been reported by other studies evaluating online video content for numerous orthopaedic conditions.^{2,3,8,9,13,17,29,49,52} In terms of comprehensibility, almost 40% (n = 39) of all videos were deemed inappropriate for patient education.

The only video attribute found to correlate significantly with video quality (both QAR-D and QAR-T scores) in the present study was video duration (r = 0.26, P < .01; and r = 0.51, P < .01, respectively). Videos scoring excellent in QAR-D averaged 14 minutes 6 seconds and videos scoring excellent in QAR-T averaged 20 minutes 9 seconds, while the overall mean duration was 6 minutes 31 seconds. This finding was somewhat expected, as the QAR-D and QAR-T scoring rubrics awarded points based on whether or not a video addressed critical aspects of the diagnosis and management of UCL injuries. As such, videos of longer duration would be expected to contain more content and therefore earn higher scores. Interestingly, video quality did not correlate with likes or views, perhaps because viewers of online content prefer brevity. In fact, previous research has revealed that viewership of online content exponentially decreases after the 6-minute mark.¹⁸ Moving forward, developers of online patient education videos should aim to strike a balance between thoroughness and brevity, so as to present the essential information regarding a certain medical topic while maintaining viewer interest. In addition, future video quality analyses may consider incorporating brevity into the overall quality rating rubric.

Regarding the accuracy of YouTube content related to UCL injuries, the most commonly encountered inaccurate statement was that “throwing curveballs increases the risk of a UCL injury,” which was mentioned in 25% of videos containing an inaccuracy (3 out of 12). This statement is not supported by the current orthopaedic literature. Furthermore, Keller et al²⁶ have in fact demonstrated that a cohort of 83 Major League Baseball pitchers with history of UCLR pitched a higher rate of fastballs compared with matched controls (46.7% vs 39.4%; P = .035), with no difference in rate of curveballs (8.5% vs 8.2%; P = .88). Misconceptions regarding UCL injuries are numerous and widespread, particularly those about treatment and expected postoperative outcomes. A survey of 516 sportswriters, for instance, found that 45.5% either did not know or believed that no injury was necessary for players to benefit from UCLR. Such misinformation can greatly influence patients’ decision making and further supports the need for peer-reviewed patient education materials.¹²

Interestingly, only 3 videos discussed indications for UCL repair (as opposed to UCL reconstruction) despite an abundance of recent research demonstrating considerable success with UCL repair when performed for the proper indications.^15,35,47,48 This finding could be representative of the lag between the publication of novel research in peer-reviewed journals and the incorporation of these findings into freely available online video content. As orthopaedic surgery continues to evolve as a field, there will be a continued need to produce updated online patient education materials so that patients can fully consider all available options.

Several readability studies conducted in the field of orthopaedics have demonstrated that patient education materials often require reading comprehension beyond the recommended sixth-grade level.^{14,31,41,42,53} While several tools exist to assess comprehensibility of written patient education materials (eg, the Flesch Reading Ease Score, the Gunning fog index), most are automated text-evaluation systems that are incompatible with video content.²⁰ As such, a novel 4-point video comprehensibility scale was devised for this study and demonstrated good interrater reliability (r = 0.82; P < .01). This scale also addressed a major limitation of most readability studies, which often only evaluate the length of words and number of words per sentence, without considering the presence of medical jargon.^14,31,53 The mean comprehensibility of videos in the present analysis (2.66) was below the threshold score of 3 (ie, acceptable for patient education), suggesting that both the quality and the comprehensibility of UCL-related content on YouTube is inadequate (Table 3). Problematically, the video source with the highest mean quality scores (physician-led educational: QAR-D, 6.37 [P < .01]; QAR-T, 4.34 [P < .01]) had a mean comprehensibility score below the acceptable threshold (mean comprehensibility, 2.91) (Table 6).

As shared decision making gains wider acceptance and adoption, patient education will become more important than ever. Patients will increasingly turn to the internet as a source of medical information given the tremendous accessibility of online content. As such, it is imperative that patients have unrestricted access to high-quality, accurate, and comprehensible medical content. To this end, the medical community must strive to improve the quality of medical content on the internet and can accomplish this task through a variety of initiatives. Most obviously, medical experts must continue to create evidence-based content that reflects patient preferences in terms of format, content, and duration. Academic promotion committees could consider placing some degree of value to the development of such patient-targeted medical content, similar to what is currently done for the publication of peer-reviewed research. In addition, professional societies could serve as arbiters of online medical content, providing “kitemarks” or seals of approval for content that meets certain quality standards.³³

Limitations

This study is not without its limitations. First, this study provides only a snapshot in time; as new YouTube videos are created, the overall quality of content may change. Second, the scoring rubric utilized to assess video quality emphasized thoroughness and accuracy. It is possible that certain videos contained excellent content but received a low-quality rating due to a narrow focus or a single inaccuracy. For example, a video describing the “modified milking maneuver” was excellent in depicting this physical examination maneuver but scored poorly overall as it did not cover any additional topics related to UCL injury. It is possible that certain individuals utilize YouTube to obtain specific information regarding a certain aspect of a medical condition rather than a comprehensive overview.

Conclusion

Based on an analysis of the top 100 most-viewed videos, the overall quality of YouTube content related to the diagnosis and treatment of UCL injuries was found to be low (mean QAR-D, 4.83 out of 16; mean QAR-T, 2.76 out of 16). In addition, no correlation was observed between video quality and views/likes, suggesting that patients are not preferentially utilizing the limited high-quality content that does exist on the YouTube platform. Similarly, while physician-led educational videos tended to be of higher quality, patient testimonial videos had the most views on average. In addition, almost half of all videos were deemed inappropriate for patient education in terms of comprehensibility, as defined by our novel comprehensibility parameter. These findings represent an evolving problem as patients increasingly turn to the internet as a source of medical information. There is a clear need for physicians and other health care professionals to produce evidence-based and high-quality patient education videos that are appropriate in terms of comprehensibility level and that cater to patient preferences with regard to content (eg, patient testimonials) and duration (preference for brevity).

Footnotes

Final revision submitted September 18, 2022; accepted September 26, 2022.

One or more of the authors has declared the following potential conflict of interest or source of funding: C.A.P. has received education payments from Arthrex. C.S.A. has received consulting fees, nonconsulting fees, and royalties from Arthrex. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.

Ethical approval was not sought for the present study.

Appendix

References

Ahmad

Grantham

Greiwe

. Public perceptions of Tommy John surgery. Phys Sportsmed. 2012;40(2):64–72.

Ahuja

Aggarwal

Sareen

Mohindru

Kandwal

. Comprehensiveness and reliability of YouTube as an information portal for lumbar spinal fusion: a systematic review of video content. Int J Spine Surg. 2021;15(1):179–185.

Akpolat

Kurdal

. Is quality of YouTube content on Bankart lesion and its surgical treatment adequate? J Orthop Surg Res. 2020;15(1):78.

Anderson

MJJ

Crockatt

Mueller

, et al. Return-to-competition criteria after ulnar collateral ligament reconstruction: a systematic review and meta-analysis. Am J Sports Med. 2022;50(4):1157–1165.

Bell

Post

Trigsted

Hetzel

McGuine

Brooks

. Prevalence of sport specialization in high school athletics: a 1-year observational study. Am J Sports Med. 2016;44(6):1469–1474.

Cain

Jr Andrews

Dugas

, et al. Outcome of ulnar collateral ligament reconstruction of the elbow in 1281 athletes: results in 743 athletes with minimum 2-year follow-up. Am J Sports Med. 2010;38(12):2426–2434.

Campbell

McGhee

Freedman

Tjoumakaris

. Diagnostic imaging of ulnar collateral ligament injury: a systematic review. Am J Sports Med. 2020;48(11):2819–2827.

Cassidy

Fitzgerald

Cassidy

, et al. YouTube provides poor information regarding anterior cruciate ligament injury and reconstruction. Knee Surg Sports Traumatol Arthrosc. 2018;26(3):840–845.

Celik

Polat

Ozcan

Camur

Kilinc

Uzun

. Assessment of the quality and reliability of the information on rotator cuff repair on YouTube. Orthop Traumatol Surg Res. 2020;106(1):31–34.

10.

Ciccotti

. Ulnar collateral ligament evaluation and diagnostics. Clin Sports Med. 2020;39(3):503–522.

11.

Conte

Fleisig

Dines

, et al. Prevalence of ulnar collateral ligament surgery in professional baseball players. Am J Sports Med. 2015;43(7):1764–1769.

12.

Conte

Hodgins

ElAttrache

Patterson-Flynn

Ahmad

. Media perceptions of Tommy John surgery. Phys Sportsmed. 2015;43(4):375–380.

13.

Crutchfield

Frank

Anderson

Trofa

Lynch

. A systematic assessment of YouTube content on femoroacetabular impingement: an updated review. Orthop J Sports Med. 2021;9(6):23259671211016340.

14.

Dalton

Kelly

Molony

. Availability of accessible and high-quality information on the Internet for patients regarding the diagnosis and management of rotator cuff tears. J Shoulder Elbow Surg. 2015;24(5):e135–e140.

15.

Dugas

Looze

Capogna

, et al. Ulnar collateral ligament repair with collagen-dipped fibertape augmentation in overhead-throwing athletes. Am J Sports Med. 2019;47(5):1096–1102.

16.

Erickson

Nwachukwu

Rosas

, et al. Trends in medial ulnar collateral ligament reconstruction in the United States: a retrospective review of a large private-payer database from 2007 to 2011. Am J Sports Med. 2015;43(7):1770–1774.

17.

Etzel

Bokshan

Forster

Owens

. A quality assessment of YouTube content on shoulder instability. Phys Sportsmed. 2022;50(4):289–294.

18.

Fishman

. How long should your next video be? Published 2020. Accessed September 1, 2021. https://wistia.com/learn/marketing/optimal-video-length#2-there-is-significant-drop-offbetween-2-and-3-minutes

19.

Fleisig

Andrews

Cutter

, et al. Risk of serious injury for young baseball pitchers: a 10-year prospective study. Am J Sports Med. 2011;39(2):253–257.

20.

Flesch

. A new readability yardstick. J Appl Psychol. 1948;32(3):221–233.

21.

Fox

Ward

O’Rourke

. The “expert patient”: empowerment or medical dominance? The case of weight loss, pharmaceutical drugs and the Internet. Soc Sci Med. 2005;60(6):1299–1309.

22.

Griffith

Ahmad

Gorroochurn

, et al. Comparison of outcomes based on graft type and tunnel configuration for primary ulnar collateral ligament reconstruction in professional baseball pitchers. Am J Sports Med. 2019;47(5):1103–1110.

23.

Hodgins

Vitale

Arons

Ahmad

. Epidemiology of medial ulnar collateral ligament reconstruction: a 10-year study in New York State. Am J Sports Med. 2016;44(3):729–734.

24.

Jayanthi

LaBella

Fischer

Pasulka

Dugas

. Sports-specialized intensive training and the risk of injury in young athletes: a clinical case-control study. Am J Sports Med. 2015;43(4):794–801.

25.

Karbach

Elfar

. Elbow instability: anatomy, biomechanics, diagnostic maneuvers, and testing. J Hand Surg. 2017;42(2):118–126.

26.

Keller

Marshall

Guest

Okoroha

Jung

Moutzouros

. Major League Baseball pitch velocity and pitch type associated with risk of ulnar collateral ligament injury. J Shoulder Elbow Surg. 2016;25(4):671–675.

27.

Leland

Conte

Flynn

, et al. Prevalence of medial ulnar collateral ligament surgery in 6135 current professional baseball players: a 2018 update. Orthop J Sports Med. 2019;7(9):2325967119871442.

28.

Lightsey

Trofa

Sonnenfeld

Swindell

Makhni

Ahmad

. Rehabilitation variability after elbow ulnar collateral ligament reconstruction. Orthop J Sports Med. 2019;7(3):2325967119833363.

29.

MacLeod

Hoppe

Simunovic

Bhandari

Philippon

Ayeni

. YouTube as an information source for femoroacetabular impingement: a systematic review of video content. Arthroscopy. 2015;31(1):136–142.

30.

Madathil

Rivera-Rodriguez

Greenstein

Gramopadhye

. Healthcare information on YouTube: a systematic review. Health Informatics J. 2015;21(3):173–194.

31.

Mehta

Swindell

Westermann

Rosneck

Lynch

. Assessing the readability of online information about hip arthroscopy. Arthroscopy. 2018;34(7):2142–2149.

32.

Morrey

Tanaka

. Valgus stability of the elbow: a definition of primary and secondary constraints. Clin Orthop Relat Res. 1991;265:187–195.

33.

Murray

Pollack

, et al. The impact of health information on the Internet on health care and the physician-patient relationship: national US survey among 1050 US physicians. J Med Internet Res. 2003;5(3):e17.

34.

Nakanishi

Masatomi

Ochi

, et al. MR arthrography of elbow: evaluation of the ulnar collateral ligament of elbow. Skeletal Radiol. 1996;25(7):629–634.

35.

O’Connell

Hoof

Heffernan

O’Brien

Savoie

. Medial ulnar collateral ligament repair with internal brace augmentation: results in 40 consecutive patients. Orthop J Sports Med. 2021;9(7):23259671211014230.

36.

O’Driscoll

Lawton

Smith

. The “moving valgus stress test” for medial collateral ligament tears of the elbow. Am J Sports Med. 2005;33(2):231–239.

37.

Pew-Research-Center. The Internet and Health. Published 2013. Accessed September 1, 2021. https://www.pewresearch.org/internet/2013/02/12/the-internet-and-health/

38.

Popkin

Bayomy

Ahmad

. Early sport specialization. J Am Acad Orthop Surg. 2019;27(22):e995–e1000.

39.

Reiman

Walker

Peters

Kilborn

Thigpen

Garrigues

. Risk factors for ulnar collateral ligament injury in professional and amateur baseball players: a systematic review with meta-analysis. J Shoulder Elbow Surg. 2019;28(1):186–195.

40.

Rettig

Sherrill

Snead

Mendler

Mieling

. Nonoperative treatment of ulnar collateral ligament injuries in throwing athletes. Am J Sports Med. 2001;29(1):15–17.

41.

Roberts

Zhang

Dyer

. The readability of AAOS patient education materials: evaluating the progress since 2008. J Bone Joint Surg Am. 2016;98(17):e70.

42.

Sabharwal

Badarudeen

Unes Kunju

. Readability of online patient education materials from the AAOS web site. Clin Orthop Relat Res. 2008;466(5):1245–1250.

43.

Safran

Ahmad

ElAttrache

. Ulnar collateral ligament of the elbow. Arthroscopy. 2005;21(11):1381–1395.

44.

Singh

. YouTube for information on rheumatoid arthritis–-a wakeup call? J Rheumatol. 2012;39(5):899–903.

45.

Swindell

Trofa

Alexander

Sonnenfeld

Saltzman

Ahmad

. Nonsurgical management of ulnar collateral ligament injuries. J Am Acad Orthop Surg Glob Res Rev. 2021;5(4):e20.00257.

46.

Tan

Goonawardene

. Internet health information seeking and the patient-physician relationship: a systematic review. J Med Internet Res. 2017;19(1):e9.

47.

Torres

Limpisvasti

. Ulnar collateral ligament repair of the elbow–-biomechanics, indications, and outcomes. Curr Rev Musculoskelet Med. 2021;14(2):168–173.

48.

Trofa

Lombardi

Noticewala

Ahmad

. Ulnar collateral ligament repair with suture augmentation. Arthrosc Tech. 2018;7(1):e53–e56.

49.

Uzun

Cingoz

Duran

Varol

Celik

. The videos on YouTube related to hallux valgus surgery have insufficient information. Foot Ankle Surg. 2022;28(4):414–417.

50.

Veltri

O’Brien

Field

Deutsch

Altchek

Potter

. The milking manuever–-a new test to evaluate the MCL of the elbow in the throwing athlete. J Shoulder Elbow Surg. 1995;4(S):10.

51.

Vitale

Ahmad

. The outcome of elbow ulnar collateral ligament reconstruction in overhead athletes: a systematic review. Am J Sports Med. 2008;36(6):1193–1205.

52.

Wong

Desai

Bautista

Kwon

Kolodychuk

Chimento

. YouTube is a poor source of patient information for knee arthroplasty and knee osteoarthritis. Arthroplast Today. 2019;5(1):78–82.

53.

Ganta

Hussein

Frank

Jawa

. Readability of arthroscopy-related patient education materials from the American Academy of Orthopaedic Surgeons and Arthroscopy Association of North America Web sites. Arthroscopy. 2013;29(6):1108–1112.

54.

Carr

JB II

Thomas

, et al. Trends in patient, physician, and public perception of ulnar collateral ligament reconstruction using social media analytics. Orthop J Sports Med. 2021;9(3):2325967121990052.

Systematic Assessment of the Quality and Comprehensibility of YouTube Content on Ulnar Collateral Ligament Injury and Management

Abstract

Background:

Purpose:

Study Design:

Methods:

Results:

Conclusion:

Keywords

Methods

Search Methodology

Video Characteristics

Quality Assessment

Statistical Analysis

Results

Analysis by QAR-D

Analysis by QAR-T

Analysis by Video Source

Discussion

Limitations

Conclusion

Footnotes

Appendix

References