Abstract
Introduction:
Breast cancer is the most common cancer in women, affecting approximately 1 in 8 women worldwide. Additionally, it is either the most or second-most lethal cancer depending on ethnicity. Many women and concerned family members turn to the internet for information regarding the signs, symptoms, diagnosis and treatment of this life-altering condition. We sought to objectively evaluate the quality of the information available on YouTube concerning breast cancer.
Methods:
We searched YouTube for videos relating to breast cancer. We included videos that were less than 20 minutes in length, were in the top 200 for most views and were available in English, either subtitled or narrated. Videos were independently and blindly scored using a standardized scoring form. Possible scores ranged from negative infinity to 25.5 with one point being deducted for each misleading statement or claim.
Results:
After exclusions, 133 videos in our sample were evaluated. Out of a possible 25.5 points, the true mean was 3.90 with a standard deviation of 1.38. There were only 30 videos (22.5%) that scored more than 7 points. 12 (9.0%) videos scored 0 points or fewer. 12 (9.0%) of the videos in our sample contained at least one misleading statement with 9 of those contained multiple misleading statements with some videos having up to 10 such statements.
Conclusion:
While YouTube is an important source of information regarding breast cancer, few videos contain a significant amount of quality information and many of the videos contain false or misleading statements. In an area where patients are often scared and searching for answers, there is a significant need for more high-quality videos to educate patients and dispel myths and pseudoscience.
Introduction
In 2015, there were 242,476 new cases of female breast cancer and 41,523 deaths due to breast cancer reported [1]. Breast cancer is the most commonly diagnosed cancer among women at 124.8 per 100,000 women and the second most common cause of cancer death at 20.3 per 100,000 women [1]. Early detection and treatment can have an astounding effect on breast cancer prognosis, greatly decreasing overall morbidity and mortality [2]. For these reasons, there is a push from the healthcare community to motivate women to recognize the symptoms of breast cancer and participate in breast cancer screening [3]. In-office visits can be overwhelming for many patients and trying to retain all the information they have been given regarding breast cancer signs, symptoms, screening, diagnosis, risk factors and treatment is unrealistic. Additionally, they will likely have further questions after leaving the office. As such, many patients turn to the internet for answers, explanations and information regarding their health [4].
Six in ten Americans turned to the Internet in the past year seeking health information for either themselves or someone they know [5]. Of these, only 35% would seek out a professional opinion about their condition or confirm the information that they found online. A large portion of the population are using the Internet as their only source for health information [5]. YouTube is a social media platform founded in 2005 that allows users to share videos and has over 1.9 billion users each month [6]. Videos containing breast cancer information can have in excess of 7 million views, demonstrating that a massive audience accesses YouTube for breast cancer information [7]. In the right hands, Youtube and the wider internet can be an excellent tool to educate patients en masse, but there are significant questions as to the quality of the information that is currently being disseminated.
In recent years, several studies have sought to answer the question of whether YouTube is a reliable source for health information. Scientists have investigated the YouTube information quality pertaining to prostate, skin, anal, oral, and colorectal cancers in an attempt to determine what patients are learning online [8–12]. These studies are finding that YouTube is lacking key information that can better inform patients of their condition. Other studies have assessed the possibility of using YouTube as a platform to educate patients about the ties between smoking and breast cancer [12], and as a resource for those affected by familial cancers [13,14]. There are no studies that have critically evaluated the usefulness of YouTube as a resource for all aspects of breast cancer information and misinformation including risk factors, symptoms, screening, diagnosis, and treatment.
Methods
Search strategy
We used a sensitivity-focused search on 1/29/19 of YouTube, for the term “breast cancer”. The search was designed using quotation marks as Boolean operators to provide a specific search for videos that pertain to only “breast cancer” and not simply “breast” OR “cancer”. Videos were sorted based on the number of highest number of views. This keyword search strategy was chosen for its high sensitivity (to include the majority of videos pertaining to breast cancer) and relative specificity (to limit the videos that did not pertain specifically to breast cancer). This search strategy was designed to include the videos on the subject of breast cancer that were most likely seen by patients searching the internet for information.
Screening videos for inclusion
Two investigators screened the top 200 videos from the search results based on number of views. Videos that were not directly related to the clinical aspects of breast cancer, such as music videos and some breast cancer awareness campaign videos, were excluded. Based on YouTube’s definition of a “short video”, videos longer than 20 minutes were excluded. Finally, duplicate videos were excluded as well as those that were unavailable in English, either narrated or subtitled (Fig. 1).
Prisma diagram.
In order to evaluate the breast cancer content of Youtube videos, we developed a scoring rubric for investigators to grade each video. The rubric consisted of items specific to breast cancer, allowing the investigators to assign points to the videos based on the information provided. Clinical practice guidelines and clinical experience were both used in the drafting process. The rubric was developed in collaboration with 4 resident physicians and an attending board certified in obstetrics and gynecology. Following the initial design of the rubric, we assessed it for face validity. Three physicians, who were board certified in Family Medicine, Obstetrics and Gynecology, and InternaI Medicine carefully reviewed all items and provided feedback for the development of the final rubric. Once completed, investigators assigned positive values for evidence-based information and negative values for misinformation. Videos received points for discussing breast cancer symptoms, screening, classifications, risk factors, diagnosis, treatments, and use of graphics/models. Videos were deducted points for each item of misinformation such as presenting “cures for cancer” that were not evidence-based. Maximum total points was 25.5 and minimum total points was negative infinity. The scoring tool is outlined in Table 1.
Scoring system of YouTube “breast cancer” videos
Scoring system of YouTube “breast cancer” videos
Five investigators independently and blindly evaluated each video for the items in the rubric using a pilot-tested Google Form. In order to limit extreme inter-evaluator scores, the maximum and minimum scores for each video were excluded and the remaining three scores were used to calculate a “trimmed mean”. We identified video authors, view count, and comment count for each of the videos to identify possible predictors of either high scoring, or low scoring videos.
We compared the total score of videos produced by healthcare professionals and uncredentialed videos using a t-test. Calculations were done using STATA.
Results
Of the top-200 most-viewed YouTube videos about breast cancer, 133 were included in our sample. 26 videos were irrelevant to the topic, 20 videos were unavailable with English narrating or subtitles, 18 were longer than 20 minutes in length, 2 were duplicates and 1 was removed from YouTube (Fig. 1). The number of views in our sample ranged from 39,010 to 7,583,240 (mean 547,572, median 274,496). The number of likes in our sample ranged from 0 to 226,000 (mean 3874, median 1100). Mean video length was 544 seconds (standard deviation 877 seconds).
Trimmed mean scores of the videos in our sample ranged from −6.33 to 20.08 with a mean of 3.95 and standard deviation of 4.15. The true mean was 3.90 with a standard deviation of 4.154 and a range of −6.6 to 19.85. Approximately 12/133 (9.0%) of the videos in our sample contained one or more misleading statements with 9 of those containing 2 or more misleading statements with a maximum of 10.
The 133 videos in our sample were produced by medical professionals (38/133, 28.6%), health promotional websites (22/133, 16.5%), personal testimonials (22/133, 16.5%), news organizations (14/133, 10.5%), medical colleges and societies (8/133, 6.0%) and other sources (29/133, 21.8%). Videos produced by medical professionals had the highest average score of 4.85, followed by uncategorized videos (4.20), videos produced by medical colleges and societies (3.93), news organizations (3.88), personal testimonials (3.27) and lastly videos produced by health promotional websites (2.78) (Table 2). None of the groups were significantly different than another.
Characteristics of videos by source
Characteristics of videos by source
In our sample, only 30/133 videos (22.56%) averaged greater than 7 points by the trimmed mean, which would be less than
Mean score vs number of views.
Our evaluation of the most-viewed Youtube videos regarding breast cancer revealed a dearth of quality information on the subject. Less than 25% of the videos in our sample received a score of 7 or higher, representing less than
Our results, while disconcerting, were hardly unexpected. A comprehensive review of Youtube videos related to prostate cancer showed that up to 77% of videos contained bias or misinformation, affecting up to 6 million viewers [12]. Other reviews of Youtube videos regarding breast reconstruction, colorectal cancer, transrectal biopsy, laparoscopic pyloromyotomy, glioblastoma, benign prostatic hyperplasia, atrial fibrillation ablation and facial plastic surgery procedures found a paucity of comprehensive educational material and high incidence of misinformation [8,15–21]. Often this misinformation is related to the curability of a disease or associated sequelae [22]. Indeed there was one video in our sample which claimed that a concoction of fruit juices would kill cancer within 42 hours [23]. More sinister and dangerous health information is also available on Youtube. A review of videos regarding skin cancer identified videos advocating for the use of “black salve”, a topical treatment that is toxic to both healthy and cancerous cells [24]. Further review of the salve identified that use led to scarring, deformity and hindered professional, standard-of-care efforts to treat the cancer [25].
While our study showed no significant difference between the means of videos produced by medical professionals vs other sources, several other reviews have identified significant differences in the quality of content produced by or including medical professionals [8,12,26,27]. One explanation for the lack of significance was the breadth of who is considered a medical professional. There were multiple videos in our sample with a negative mean score from medical professionals who have little or no expertise in the field of breast cancer, including therapists and doctors of naturopathic medicine. In the internet age there are no shortage of uncredentialed individuals with social media access who propagate information that may be biased, harmful or sew mistrust or discord between patients and their physicians. It is in the best interest of medical professionals to make evidence-based information accessible and desirable for patients to access.
Physicians explaining a breast cancer diagnosis to a patient have a lot of information that needs to be conveyed from prognosis to staging to risks and benefits of treatment options. Patients are bound to have extensive questions during and after the visit and they may have misconceptions and misunderstandings about their care. Youtube is a powerful educational tool if utilized correctly. Well-made videos should be able to dispel myths, explain misconceptions and soothe fear and anxiety while providing patient realistic expectations on their disease course and what they can expect from their physician regarding diagnosis and treatment. It would not be unreasonable for a physician to direct patients to a video or series of videos that can educate and reinforce what was talked about in the office. In order for patients to make the best decisions about their health, they need evidence-based, well-explained information. Videos can help to fill that need and allow patients collaborate with their physician to make the best decisions regarding their health.
Our study contains several strengths and limitations. Strengths of our study include that all of the videos in our sample were independently and blindly reviewed by 4 resident physicians specializing in obstetrics and gynecology. Our scoring system, while of our own construct, was based on evidence-based guidance and was developed in conjunction with attending and resident physicians with expertise in the field [28]. Adding credence to our evaluation and scoring system is the consistency in scoring as evidenced by the true and trimmed means in Table 2. One factor we were unable to account for in our scoring was the presence of one video series where each video addressed a different aspect of breast cancer diagnosis, screening, symptoms and treatment. By themselves, each video scored poorly, but taken as a whole they would have come close to a perfect score. We believe that the crux of our paper and the need for excellent educational content in this field are unaffected by this isolated anomaly. Our study was cross-sectional in nature, and our results should be interpreted accordingly. Our results are not intended to generalize beyond the videos composing our sample. It is possible that other videos outside of our sample may contain either more or less evidence-based information regarding breast cancer.
Conclusion
While YouTube is an important source of information regarding breast cancer, few contain a significant amount of quality information and many of the videos contain false or misleading statements. In an area where patients are often scared and searching for answers, there is a significant need for more high-quality videos to educate patients and dispel myths and pseudoscience.
Conflict of interest
All authors certify that they are free from any conflicts of interest.