Abstract
Background
One-third of American adults encompassed by current colorectal cancer screening guidelines fail to obtain recommended screening evaluations. Educational videos are a valuable medium through which to educate and encourage recommended health behaviors in patients
Methods
A cross-sectional study reviewing the quality of patient education videos addressing colorectal cancer screening. Video quality was assessed in 3 domains: accountability, content, and production
Results
Forty-four videos met inclusion criteria. Out of 33 possible points, videos scored a median of 15.0 (interquartile range 12.9-16.6). Videos scored 1.0 (interquartile range .8-1.0) out of 4.0 for accountability, 6.0 (interquartile range 4.4-8.0) out of 20 for content, and 8.0 (interquartile range 7.4-8.0) out of 9.0 for production. Colonoscopy was the most frequently discussed method of screening (38, 86%). While 13 (34%) videos discussed the risk of colorectal cancer in the general population and 15 (32%) discussed the risk in those with a family history, few videos addressed those with other risk factors. Most (31, 70%) videos discussed the medical consequences of not receiving screening, but only 1 (2%) video discussed the social consequences. Similarly, medical benefits were discussed in 34 (77%) videos while other benefits were not discussed by any video. Only one-fifth of the videos address three or more barriers to screening
Conclusions
Videos on colorectal cancer screening have excellent production quality but need improvement in the domains of accountability and content. The videos included in this analysis did not adequately address the concerns of viewers nor the benefits of colorectal cancer screening.
Introduction
American adults commonly seek health information online with 58% endorsing doing so in the past year alone. 1 YouTube™ (Google LLC, Mountain View, CA) is a popular social media platform, amassing over 14 billion views in 1 week. 2 Over a third of US adults have watched health-related videos on YouTube™. 3 This platform may be especially beneficial for those with lower health literacy. Colorectal cancer is the second leading cause of death. 4 Colon cancer screening with sigmoidoscopy or colonoscopy has been shown to decrease colorectal cancer mortality by 28% and 68%, respectively. 5 YouTube™ has a large audience and wide reach with can promote colorectal cancer screening. A good video can assist informed decision making by educating the public on the benefits of screening, addressing common concerns regarding screening, and explaining specific screening methods and associated schedules. Furthermore, when patients are involved in shared decision-making and given a choice of screening method they are more likely to adhere to screening recommendations, which is the key factor in effectiveness of screening.6,7
Various professional societies and government organizations provide written patient education material on colorectal cancer screening, but prior analysis has found the information offered by these websites to be limited in addressing patient’s questions. 8 No studies, however, have analyzed the quality of YouTube™ videos on colorectal cancer screening. Previous studies on patient-oriented YouTube™ videos for abdominal aortic aneurysm, anal fissure, umbilical hernia, female urinary incontinence, and urinary tract infection found that YouTube™ videos had higher quality compared to TikTok™ (ByteDance, Beijing, China), videos although video quality in general were poor and in need of improvement.9-13 The DISCERN instrument, a tool designed to evaluate the quality of written information about treatment choices, was the most common used quality assessment tool among these studies. 14 Other assessment tool used are Global Quality scale, Patient Education Materials Assessment Tool, JAMA benchmark, and author developed criteria.9-12 No studies studied the production quality of the videos. We used an instrument adapted from the combination of Ingledew’s website quality assessment tool, the Health Belief Model, and recommendations from Colorado State University specific to video evaluation.15,16 Website quality assessment tool by Ingledew is a validated, comprehensive assessment tool that incorporated HON, DISCERN, and JAMA benchmark as well as criteria deemed important by patient identified through literature review. 15 The Health Belief Model is a framework for individual’s health-related behaviors that focus on 4 dimensions including perceived susceptibility, perceived severity, perceived benefits, and perceived barriers. In addition, we added evaluation of video production based on the recommendations from Colorado State University. 17 This study aims to evaluate the accountability, content, and production quality of YouTube™ videos on colorectal cancer screening.
Material and Methods
Study Design
This was a cross-sectional study of publicly available YouTube™ videos covering colorectal cancer screening oriented towards patients. This study did not require Health Insurance Portability and Accountability Act information; thus, it did not require Institutional Review Board approval or informed consent.
Search Strategy and Video Selection
A cache-cleared browser was used to query the YouTube™ platform using the search terms “colon cancer screening,” “colorectal cancer screening,” and “bowel cancer screening” in October 2020. The top 50 videos ranked by relevance were collected as well as the top 50 videos for each of the 3 search terms ranked by most viewed for a total of 300 videos. Previous studies on written health content demonstrated that the top 25 websites resulting from an Internet search accounted for 87% of websites with the most site traffic; thus, the videos included in this analysis serve as a sampling of those most accessed by patients seeking information on colorectal cancer screening via video. 18 Videos were eligible for inclusion if the predominant focus was educating the public about colorectal cancer screening. Videos were excluded if they were: (1) uploaded more than 5 years ago, (2) oriented towards health providers, (3) focused on specific national colorectal cancer screening programs in Australia or Europe, (4) advertisements, videos served to offer pure instructional information or focused solely on patient experiences, (5) not related to colorectal cancer screening. Advertisement was defined as a video which specifically directed the patient to obtain screening with a specific gastroenterology group or hospital system. Disagreement between reviewers were resolved by discussion with the senior author until an unanimous decision is reached.
Assessment and Outcomes
Scoring Criteria for Colorectal Cancer Screening Media.
Rater Training
Video evaluators (QY and JJ) were trained using videos with examples of good and poor performance in video production. Accountability and content were either present or not and did not require rater judgment. The videos used for training were not eligible for study inclusion. Evaluators scored videos independently first, any discrepancies between the raters were discussed and consensus was reached. To establish consistency between raters, the raters reviewed sets of 10 excluded videos for practice until Kappa correlation >.8. The Kappa correlation coefficients between 2 raters for the included videos was .78. Video assessment was performed between October 2020 and February 2021.
Data Analysis
Categorical variables were reported as a number and percentage. Continuous variables were reported as a median and range or interquartile range (IQR). Statistical analysis was performed in RStudio version 1.2.5001 (RStudio, Inc., Boston, MA).
Results
General Video Information
Forty-four videos met inclusion criteria for evaluation. Reasons for exclusion can be seen in the flow diagram (Figure 1). The majority of videos (25, 57%) were created by hospital or healthcare organizations, while the remainder were by news media (n = 6, 14%), government organizations (n = 4, 9%), professional societies (n = 3, 7%), personal account (n = 3, 7%), nonprofit organizations (n = 2, 5%), and medical device company (n = 1, 2%). Videos were a median of 2.0 (range .5-25.8, IQR 1.2-3.7) minutes long with a median of 4026 (range 70-296,171, IQR 650-19048) views. Videos received a median of 8.5 (range .0-3100.0, IQR 2.0-43.5) likes and .5 (range .0-68.0, IQR .0-2.3) dislikes. Videos scored a median of 15.0 (range 8.0-23.5, IQR 12.9-16.6) out of 33 possible points for overall quality score. Flow diagram for video selection. Out of 300 videos screened, 44 met inclusion criteria.
Content
Barriers to Colorectal Cancer Screening Addressed by YouTube Videos.
Total n = 45.
Accountability
Videos scored a median of 1.0 (range 0-3, IQR .8-1.0) out of 4.0 for accountability. Only 5 (11%) videos included copyright information, 2 (5%) videos had video credits, and 1 (2%) video provided disclosures. Credentials of the main instructor were identifiable in 31 (70%) videos. Of those that disclosed credentials 18 (39%) were MDs, 1 (2%) was DO, 11 (25%) were non-specific “Dr,” and 1 (2%) was a patient.
Production
Videos scored a median of 8.0 (range 6-9, IQR 7.4-8.0) out of 9.0 for production quality. The presence of closed captions was the most missed production criterion, as it was omitted by 35 (80%) videos. Videos performed well on other production qualities including: stable images (40, 91%), appropriate framing (43, 98%), clear contrast (44, 100%), clear resolutions (43, 98%), contain audio commentary (42, 95%), no distracting background noise (41, 93%), appropriate loudness (41, 93%), and lastly appropriate transitions (41, 93%).
Discussion and Conclusion
Colorectal cancer is one of the leading causes of cancer mortality but, through screening efforts, it can be identified early and even be prevented. Colorectal cancer screening, however, is not at goal and general public knowledge related to colorectal cancer is still low. 19 YouTube™ is a popular social media platform with high traffic volume and broad reach. It has previously been utilized as a tool for public health campaigns. 20 Our study found that, while videos educating the public on colorectal cancer screening are of high production quality, accountability and content scores varied considerably between videos. These videos lacked important information that could potentially improve colorectal cancer screening adherence.
YouTube™ videos on colorectal cancer screening performed poorly in regard to accountability, the criteria of which allows viewers to judge the credibility of the content they are consuming. This finding aligns with the lack of accountability found in prior study of written content for patients.18,21 The accountability criterion provided by the greatest number of videos was the credentials of the main educator in the video (31, 70%). Providing credentials of the main educator can lend credibility to the information presented in the video, even so, only just over half of the videos included this information. Disclosure statements are required at scientific meetings but were neglected by most videos geared towards the public. Most videos also omitted copyright and credit information.
The YouTube™ videos included in this study did not address important issues regarding colorectal cancer screening from the patient’s perspective. Less than half of the videos discussed risk of colorectal cancer in the general population. Very few videos discussed risk factors associated with colorectal cancer and no videos gave specific incidence and prevalence information in these groups. Similar findings were seen in the evaluation of written colorectal screening material. 8 While a small percentage (18%) of people would not screen for colorectal cancer regardless of their risk level and about half (55%) of people endorse an intention to screen independent of their risk level, the individual risk of colorectal cancer does affect the decision to screen or not to screen in the remaining participants. 22 Seventy-seven percent expressed the intent to screen if their risk of colorectal cancer was 5% compared to 60% if the risk was 1%. Thus, the risk of colorectal cancer should be discussed, especially in those with identified risk factors such as smoking and diabetes.23,24 Additionally, only about a third of the videos made the distinction between average risk populations and increased risk populations, including people with a personal history of polyps or colorectal cancer, inflammatory bowel disease, genetic syndromes, or a family history of colorectal cancer. 25 These patients are subject to different screening schedules and would benefit from discussion with their primary care physician.
Colonoscopy was the most discussed screening method in these videos followed by stool DNA test. These are also the two most frequently used screening methods although stool DNA test is gaining popularity in the recent years. 26 A study by Deibel et al has shown that the effectiveness of screening is most dependent on adherence rather than the screening method utilized. 7 A randomized controlled trial showed that participants who were given a choice between FOBT and colonoscopy had better adherence compared to those assigned to either screening method. 6 Patient characteristics may affect their preferred screening method. Patients aged 50-54 or of the male sex were more likely to undergo colonoscopy over stool DNA test as compared to those 65 years or older or of the female sex. 26 This re-enforces the value of patient choice and the necessity to educate patients on the differences between these modalities for them to make informed decisions. However, stool studies were discussed only in a third of the videos and only 31% of videos discussed more than one modality for screening.
A major issue with screening methods other than colonoscopy is that if there is abnormal result then colonoscopy would still be required. Prior research has demonstrated that only about 40%–60% of patients with a positive FIT follow up with a colonoscopy.27,28 This caveat was discussed in only 39% of the videos. Other reasons where colonoscopy would be necessary were rarely discussed but may be important information to patients trying to avoid colonoscopy due to fear. Additionally, 12% of the population achieves inadequate screening because of poor adherence to the screening schedule for flexible sigmoidoscopy or FIT/FOBT. 29 Thus, it is important for patients utilizing screening methods other than colonoscopy to be aware of these details.
Most videos performed well discussing the medical consequences and benefit of colorectal cancer screening such as early discovery, prevention of cancer by removing polyps, and better survival in lower stage disease. Only one video, however, discussed the social consequence of not being to be there for their family members. This is similar to the information included in written content provided by cancer societies and government organizations where changes in the quality of life and family life were rarely discussed. 8 Social benefits of screening, such as setting a good example for family members and following doctor’s advice, have been associated with increased likelihood of undergoing sigmoidoscopy or colonoscopy in the African American population. 30 Additionally, a sense of obligation to others, such as genetic screening for family members or cancer screening to save the government money, has been found to motivate patients to get screened. 31 Thus, benefits outside of the pure physical and medical aspects of screening should be explored more in these videos.
For cancer screening, health behavior models have shown that as perceived barriers begin to outweigh the benefits, health behaviors become less likely. 32 Commonly cited barriers to colorectal cancer screening include fear due to pain, difficulty tolerating bowel prep, worry about expense or embarrassment, doctors not recommending the test, and not having any symptoms.30,33,34 Watching videos addressing colorectal cancer screening has been proven to improve knowledge of colorectal cancer and decrease barriers to screening. 35 However, these barriers to screening were not well addressed in the videos included in this analysis. Future colorectal cancer screening videos should incorporate more open dialogue regarding common fear and concerns regarding screening.
YouTube™ videos on colorectal cancer screening uniformly performed well on production quality. The only criterion not met by most videos was closed captions within the video itself. This deficit can be made up by the automatic caption function of YouTube™. Remaining production criteria were met by most videos. This is in contrast with videos geared towards residents on techniques of surgical procedures where a much higher variation of production quality was seen. 36
The internet has become a common source for patients to look up health information because of its convenience, anonymity, and wide scope. 37 Online health information has been shown to influence health decisions. 37 Audiovisual content has the benefit of being easily understandable as written health education content frequently exceed the recommended level of difficulty. 18 Despite the large amount of content available on the internet, YouTube™ videos are not comprehensive and occasionally even contain inaccurate information. One video in our study recommended against routine screening which can have negative impact on viewers’ preventive health behavior. While this study was limited to only videos from YouTube™, other studies have demonstrated even lower quality of medical content on TikTok™ for other disease.10,11
Given the variable quality of online health information on popular social media platforms like YouTube™ or TikTok™, this places a burden on providers to be aware of the content available to patients. While it would not be possible for providers nor health care organizations to police online medical information, it does fall on individual providers to initiate discussions regarding health information obtained online in order to correct any misconceptions or inaccuracies. Individualized discussion initiated by the primary care providers is important to ensure informed decision regarding colorectal screening based on accurate and complete information. Provider should have high quality websites or videos available to provide to interested patients. In addition, we call on medical organizations such as professional societies or health organizations to create high quality medical education material as well as collaborative efforts from all physicians to disseminate them on popular social media. Collaboration with professional video production is also necessary to improve video production quality. Since COVID epidemic, social media has been more diligent in identifying good vs bad health content. For example, YouTube™ provide information regarding health source context and created avenues to report inaccurate health information. Continued efforts from social media platform combined with efforts from provider can made significant strides towards high quality health information on social media.
This study does have some limitations. The internet is highly dynamic with 48,486 hours of content uploaded to YouTube™ weekly. This is a cross-sectional study capturing only a small percentage of videos at a specific time point. However, with the massive amount of content available on YouTube™, it would not be feasible to evaluate all videos available. We used multiple search terms that were most relevant to our topic to broaden our video sample, although patients may use other search terms than the ones included in our study. The videos were ranked by most relevant and most popular to increase the sampling of videos. Top positioned videos were chosen to mimic how viewers select videos. Thus, the videos we evaluated likely reflected those watched by patients and can provide some sights into the quality of audiovisual content on this topic which has not been evaluated in the past. Additionally, we limited out search to YouTube™ and did not evaluate videos from other platforms such as TikTok™ (Tiktok Inc., Culver City, CA) or Instagram™ (Meta Platforms, Menlo Park, CA) which are more often personal testimonies than instructional videos, but this can be a future direction of research. Lastly, it is possible that the goal of the producer was not to be comprehensive. Criteria used in this video was based on Health Belief Model which has been supported by both prospective and retrospective research. 16
In conclusion, while YouTube™ videos on colorectal cancer screening have excellent production quality, these videos performed poorly on accountability and content. Videos do not adequately provide information important from patients’ perspective including perceived susceptibility to colorectal cancer, consequences of disease, benefits of screening, and barriers to screening. Future colorectal cancer screening videos should incorporate more social consequences of disease, benefits of screening, and include more dialogue regarding common barriers to screening.
Footnotes
Author Contributions
Study concept and design: QY, WBP, and MGD
Data acquisition: QY and JEJ
Data analysis and interpretation: QY, JEJ, KJJ, HEDC, AL, WBP, and MGD
Drafting the article: QY and KJJ
Critical revision and intellectual content: JEJ, HEDC, AL, WBP, and MGD
Final approval of the manuscript: QY, JEJ, KJJ, HEDC, AL, WBP, and MGD
Agree to be accountable for all aspects of work to ensure that questions regarding accuracy and integrity investigated and resolved: QY, JEJ, KJJ, HEDC, AL, WBP, and MGD
Author’s Note
Accepted for presentation at American Society of Colon and Rectal Surgeons (Annual Scientific Meeting, San Diego, CA, April 24-28, 2021)
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Code availability
Code is available upon request.
