Sage Journals: Discover world-class research

Abstract

Objective:

To evaluate the usefulness of YouTube videos as an educative tool for type 2 diabetes self-management.

Design:

Search terms were “diabetes diet” and “diabetes treatment.” Videos were jointly assessed by two reviewers. A third investigator evaluated a random sample to check for agreement.

Main measures:

Usefulness defined as making reference to AAD7 Self-Care Behaviors^™ and presence of misleading information.

Results:

Of the 393 videos included, 42.2 percent (n = 166) classified as “alternative medicine.” 40.2 percent (n = 158) contained useful information. 25.7 percent (n = 101) videos contained misleading information. Videos displaying “alternative medicine” professionals (60 out of 200) were less useful (17% vs 57%; p < 0.001) and more misleading (40% vs 2%; p < 0.001).

Conclusion:

The probability of finding videos that relate to AADE7^™ self-care behaviors is below 50 percent. The odds of finding misleading information are high. Therefore, it is inadvisable for professionals to recommend the use of YouTube as an educating tool if specific videos are not suggested.

Keywords

health-education type 2 diabetes YouTube

Introduction

In recent years, Internet tools have evolved into the Web 2.0 (social media) environment which allows users to easily create and share content and interact through virtual networks rather than only accessing one-way information. Currently, social media play a large part in the way people communicate.¹ Health information is one of the most searched topics on the Web,² and it can be presumed that social media use is playing a growing role in how health information is delivered and accessed by final users. YouTube (http://www.YouTube.com) is an online public communication web site launched on December 2005, on which users can upload and share videos, as well as view videos and make comments on them. Its ease of access and an intuitive interface makes this web site a popular platform across the globe for sharing information, including that related to health and disease.^3,4 YouTube shows potential as a health education tool since video assisted learning has demonstrated being valuable for the knowledge acquisition of medical students⁵ and patients.⁶ However, free and unregulated information available on the Internet carries the potential hazard of misinformation. Individuals accessing health-related information on the Internet are often oblivious to issues such as who uploaded the material, when the material was last updated, and how accurate the information is.³ Given that diabetes is a chronic disease where patient education is highly advocated,⁷ and bearing in mind the values and hazards of social media, we conducted a descriptive study to characterize how useful YouTube videos are for type 2 diabetes self-education.

Methods

Selection of search terms

In order to get knowledge on the searching habits of actual users we used Google Trends (https://www.google.com/trends/), a public web facility of Google Inc. that allows comparing the relative frequency of use of different search terms on Google and/or YouTube. A query using the term “diabetes” in combination with terms used to define the contents of diabetes education was performed on 10 October 2015. The combinations “diabetes diet” and “diabetes treatment” accounted for the vast majority of searches in YouTube and therefore, were selected as search terms for the study.

Video selection process

The video selection process was based on the snowball technique.⁸ Specifically, we performed a search in YouTube, using a novel Google account and YouTube default settings, with both search terms separately. Videos shown in the first pages of both searches, as well as those suggested when viewing all selected videos, were retrieved and evaluated for inclusion/exclusion criteria. This process was continued until a prespecified number of included videos was reached.

We hypothesized that 50 percent or more of the total number of videos would contain useful information. To test this hypothesis with a precision of 5 percent and a confidence level of 95 percent, a total of 385 videos were needed, and therefore we decided to select 200 videos in each search.

A single inclusion criterion was broadly defined as “videos in English related to type 2 diabetes.” Exclusion criteria were videos not related to diabetes, with no voice, specified as ads, specifically dedicated to type 1 or gestational diabetes, and/or with duration longer than 10 min. This time frame was selected because the time of video engaging significantly decreases for longer videos.⁹

Video characterization

Videos were classified according to the publishing channel, and the characters displayed in the video (Table 1).

Table 1.

Classification of videos according to the publishing channel and the characters displayed.

YouTube channels	• Health professional • Health-related organization (including health institutions, universities, and professional associations) • Lay person • Non-pharmaceutical company • Pharmaceutical company • Nonprofit organization • YouTube channel not otherwise specified dedicated to health contents • YouTube channel not otherwise specified and not dedicated to health contents
Characters in the video	• Health professional • Patient or relative • Educational video without characters • Commercial • Other

In addition, both channels and characters were classified as pertaining to “conventional” or “complementary or alternative” medicine. “Complementary or alternative medicine” (from now on “alternative medicine”) was defined as health and wellness therapies that have typically not been a part of conventional Western medicine, including mind-body medicine, manipulative and body-based practices, homeopathy, and herbs, and dietary and ayurvedic supplements.

We registered the date of publication and the number of views, and calculated the view rate as the number of views per day since publication up to 22 October 2015. Videos were also classified as of amateur or professional production according to published criteria.¹⁰

Video usefulness evaluation

Videos’ usefulness was evaluated using a stepped approach. First, videos were evaluated for reference to the contents and skills specified in the AAD7 Self-Care Behaviors^™ (AADE7^™) of the American Association of Diabetes Educators.¹¹ AADE7 is a well-accepted framework for patient-centered diabetes self-management education that defines self-care behaviors essential for effective diabetes self-management, namely, “healthy eating,” “being active,” “monitoring,” “taking medications,” “problem-solving,” “healthy coping,” and “reducing risks.” Videos that referred to any of the contents and skills specified as being associated with any of AADE7 self-care behaviors¹¹ were considered useful. Second, as videos can also be useful without complying with AADE7 contents, a criterion of usefulness was applied based on the subjective impression of the evaluators.

In these two first steps, no consideration on the veracity of the information was made, in order to avoid considering as “non-useful” all those videos related to alternative ways of approaching type 2 diabetes management. Third, videos were considered misleading if:

Affirmed or insinuated the curability of the disease with any specific approach;

Advocated for specific diets or treatments with terms implying that other approaches were invalid;

Provided pathophysiological explanations or treatment effects without a minimum experimental base.

Misleading videos were considered non-useful, regardless of any additional information contained.

All evaluations were performed jointly by two investigators (G.G.P. and N.R.V.) who achieved a final decision by consensus. A random sample of 50 videos was evaluated by a third investigator (M.T.G.) to check for agreement. G.G.P. is a registered endocrinologist and N.R.V. a registered pediatrician with previous experience in evaluating content’s quality of web pages and Twitter. M.T.G. is resident in geriatrics and received specific instruction for the entrusted task.

Statistical analysis

Qualitative variables are expressed as absolute and relative frequencies with the 95 percent confidence interval (CI). Quantitative variables that do not show a normal distribution are expressed as median plus minimal and maximal values and interquartile range (IQR). Discrete variables were compared with the Chi-square test. Quantitative variables were compared with the Mann–Whitney test if n ⩽ 200 and the student’s T-test if n > 200. To test the null hypothesis of a proportion of useful videos equal to 50 percent, we used the one-tailed Z-test for one sample. Differences were considered significant if p < 0.05. All analyses were performed with the freely online available software VassarStats (http://vassarstats.net/), Social Science Statistics (http://www.socscistatistics.com/Default.aspx) and Epitools epidemiological calculators (http://epitools.ausvet.com.au/content.php?page=home).

Results

To achieve the prespecified number of videos, a total of 2084 videos were screened. Figure 1 shows the flow diagram of the selection process. As shown, after the initial selection, seven additional videos were excluded since they only contained the announcement of curative treatments, not otherwise specified, available through non-operative websites (scam videos). Therefore, the final number of evaluated videos was 393 from 261 different channels. Of them, 199 were obtained from the “diabetes diet” search and 194 from the “diabetes treatment” search.

Figure 1.

Flow diagram of the videos’ selection process.

General characteristics

Mean video duration was 05’:13’’ ± 02’:44’’ (range: 0’:27’’–10’:00’’). The oldest video was of 10 August 2006 and the most recent of 30 September 2015. The median rate of views per day was 12.4 (range: 0.05–4254; IQR 47.8). In all, 59 videos had more than 10,000 and 10 more than 500,000 total views. Amateur produced videos accounted for 47.8 percent (n = 188) of the total sample. A total of 166 videos (42.2%; CI: 37.4%–47.2%), were classified as pertaining to “alternative medicine.”

Health-related channels (professionals, institutions, or YouTube channels) accounted for 45 percent (CI: 38.2%–47.9%; n = 169) of videos. Of these, 77 (45.6%; CI: 38.2%–53.1%) corresponded to YouTube channels devoted to health issues but not linked to health professionals or institutions. From those belonging to health professionals or institutions, 32 (34.7%; CI: 25.8%–45.0%) were classified in the category “alternative medicine.” Channels belonging to pharmaceutical companies accounted for 4.3 percent (n = 17) of the evaluated videos.

In 180 videos (45.8%; CI: 40.9%–50.7%), the main character was a health professional while in 63 (16.0%; CI: 12.7%–20.0%) it was a patient. Twenty of these videos showed both health professionals and patients. “Alternative medicine” channels or characters were present in 135 videos (42%; CI: 37.2%–46.9%). From the 223 videos showing a health professional or patient as the main character, 83 (37.2%; CI: 31.1%–43.7%) were classified in the category “alternative medicine.”

Usefulness assessment according to the presence of AADE7 contents/skills

AADE7 contents/skills were present in 192 (48.8%) videos, of which 34 contained misleading information either when describing the content/skill or as additional information in other parts of the video. Therefore, a total of 158 videos (40.2%; CI: 36.1%–44.3%) contained useful information for type 2 diabetes self-education according to the AADE7 specification. The one-tailed Z value against a proportion equal to 50 percent was 3.9 (p < 0.001), and therefore the null hypothesis was rejected. Most useful videos referred to only one content/skill, with only 40 (25.3%) making reference to two or more content/skills. Most commented content/skills were food choices and planning meals (62 out of 116 comments about “healthy eating”) and drugs action and side effects (46 out of 73 comments about “taking medications”).

Characteristics and differences between useful and non-useful videos according to the AADE7 specification are depicted in Table 2. As it can be seen, useful videos were less frequently of amateur production and from alternative sources and displayed more frequently health-related characters.

Table 2.

Differences between videos classified as useful according to the presence of AADE7 contents/skills.

	Useful videos n = 158	Non-useful videos n = 235	p
View rate (views per day)	8.41 (0.16–2098; IQR 34.38)	17.05 (0.05–4254; IQR 58.17)	0.14
Amateur production	61 (38.6%)	121 (51.9%)	0.012
Health related channels	71 (44.9%)	98 (41.7%)	0.52
Health related characters	113 (68.1%)	135 (54.7%)	0.006
“Alternative medicine” channel or character	34 (21.5%)	131 (55.7%)	<0.001

Subjective usefulness assessment

Independently of the presence of AADE7 contents/skills, evaluators considered that 226 (57.5%; CI: 53.4%–61.6%; Z = 2.77; p < 0.01), and hence a proportion greater than 50 percent, of videos contained information useful for self-education. Most videos considered useful and without references to AADE7 contents/skills referred to the causes and symptoms of type 2 diabetes. Differences between useful and non-useful videos were parallel to those shown in Table 2.

Analysis of videos with misleading information

A total of 101 videos (25.7%; CI: 21.6%–30.2%) contained misleading information. In 78 videos (19.8%), the misleading information referred to the curability of the disease or of its complications.

The median view rate of misleading videos was 17.22 (range: 0.17–1190; IQR 67.4) versus 9.85 (range: 0.05–4254; IQR 46.7) in the case of non-misleading videos (p = 0.094). The proportion of an amateur format was 57.4 percent (CI: 47.2–67.2%; n = 58) for misleading videos versus 42.5 percent (CI: 36.7–48.4%; n = 124) for non-misleading videos (p = 0.009). Chi-square test for multiple categories showed that the distribution of misleading videos was significantly different (p < 0.001) according to the type of publishing channel. Specifically, 41.6 percent (n = 42) of misleading videos were published by non-pharmaceutical companies (Figure 2). The distribution of misleading videos was also significantly different (p < 0.001), according to the characters displayed in the video. Specifically, the difference between health professionals (19.7%; CI: 14.5%–26.1%) and patients (46.1%; CI: 35.3%–57.1%) was significant (p < 0.001).

Figure 2.

Percentage of YouTube videos with misleading information according to the publishing channel (n = 393; Chi-square test for multiple categories p < 0.001).

Analysis of videos by health professionals

We conducted a subanalysis of videos published by or displaying health professionals, depending on whether they were classified as pertaining to “conventional medicine“ or “alternative medicine.” A total of 200 videos were classified as “health professional videos,” of which 140 (70%; CI: 63.3%–75.9%) were considered as pertaining to “conventional medicine.” The median view rate of “conventional medicine” videos was 7.3 (range: 0.6–4252; IQR 34.7) versus 12.3 (range: 0.05–349; IQR 59.9) of “alternative medicine” videos (p < 0.01). Figure 3 shows the proportion of AADE7 contents, subjective usefulness, and misleading information of “health professional videos” according to their classification as “conventional” or “alternative.” As shown, there are marked differences between both types of videos.

Figure 3.

Usefulness and misleading information in videos published or displaying health professionals (n = 200; all comparisons p < 0. 001).

External agreement assessment

Cohen’s Kappa ranged from 0.896 (0.755–1.0; very good) for misleading information, to 0.637 (0.422–0.851; good) for presence of AADE7 contents/skills.

Discussion

In this study the usefulness of YouTube videos for type 2 diabetes self-education, using the AADE7 standards, is below 50 percent, and the proportion of misleading information is around 25 percent. The proportion of useful videos is higher if videos from “conventional medicine” health professionals are only considered.

The easiness for creating contents of available Web 2.0 resources has definitively changed the way people communicate and also how patients and carers access health information. YouTube as a video-sharing platform has unique advantages as an education tool, and it is increasingly being used for disseminating health knowledge to professionals¹² and patients.¹³ In this context and due to the unregulated and free nature of YouTube available videos, it is of critical importance to evaluate the quality and usefulness of their contents. Numerous studies have addressed this issue in different areas of medical knowledge. The results are mixed, the most common conclusion being that health information available is often not useful, misleading,^13–18 or even dangerous¹⁹ with few studies indicating that the information is mostly useful.^20–24 Given that diabetes is a chronic disease where patient education is highly advocated,⁷ it is surprising that only one small study has evaluated the quality of the diabetes information available on YouTube videos, specifically about diabetes foot care.²⁵ Here, we present the first study that evaluates the quality of YouTube diabetes videos as a complementary tool for comprehensive diabetes education.

Diabetes education is a complex task nowadays, based in theories of learning in which the interaction and shared decisions between healthcare providers and patients are crucial.²⁶ Therefore, it is obvious that it cannot rely solely on published materials without the participation of professionals. However, the permanent availability of online materials can help in the educational process and, therefore, it is important to know the possibilities of finding online useful information. The main objective of this study was to evaluate this question. In order to do so, we followed the published recommendations about the design of studies on consumer health YouTube videos regarding sample size, selection of terms, and screening of search results.⁸ The evaluation of quality and usefulness of YouTube videos is not standardized, and is generally based on the accuracy, credibility, and scientific correction of the contents according to the evaluation of experts.²⁷ We chose a different approach trying to evaluate their usefulness as an educational tool bearing in mind that there are different approaches that may be useful for treating diabetes type 2, especially when it comes to dietary recommendations.²⁸ Therefore, the videos of “complementary and alternative” medicine were not a priori inaccurate unless they contained misleading information as specified in the section “Methods.” To evaluate the educative utility of videos we relied primarily on the skills specified in an evidence-based framework for diabetes education such as AADE7. Secondarily, we used the subjective evaluation of researchers. Misleading information was intentionally specified in a generic way without making a detailed assessment of the accuracy of the information.

Using this approach, our results indicate that the chances of finding useful videos, according to the presence of AADE7 contents/skills, are less than 50 percent. These chances are higher if the user discards videos of amateur production, without health-related characters or those using an alternative medicine approach. If causes and symptoms of type 2 diabetes are considered a part of the education program, the chances of finding useful information is above 50 percent. These results are in line with other studies in which useful information is found in proximately 50 percent of retrieved videos^20,21 and similar to the only study related to diabetes,²⁵ but not as high as in some recent ones.^22,24 As in most studies, videos depicting health-related characters were more useful than others.¹³

Approximately a quarter of the videos contained misleading information, a figure in the high-end of published studies¹³ that clearly limits YouTube’s potential as an educational tool for type 2 diabetes. It is worth noting that all videos from pharmaceutical companies were considered “not misleading,” however, the number of videos in this category was very low and this finding might be irrelevant. Surprisingly or not, the videos from pharmaceutical companies were dedicated to drugs only in one case, while the rest were devoted to healthy eating and psychological aspects of the disease.

When analyzing videos published or displaying health professionals, the differences between “conventional” and “alternative” medicine were remarkable in favor of videos of “conventional medicine” with respect to the usefulness and presence of misleading information. Since the accuracy of their content was not studied, but their usefulness as an educational tool, and videos were considered misleading only if they showed clearly false information, it could be hypothesized that a more conventional approach that evaluated the accuracy of the content would have resulted in a higher percentage of misleading or inaccurate information in videos from “alternative medicine” health professionals. It is also remarkable that “alternative medicine” videos had a significant higher view rate.

The study has some limitations. First the number of videos available in YouTube is unattainable and we cannot exclude that another method of selection would have rendered different results. However, the strategy used for retrieving videos and the method for selecting search terms guarantee that the selected videos are near enough to what a lay user would have found.⁸ Second, the exclusion criteria are arbitrary and can be subject to criticism. Specifically, since we intended to evaluate videos as a new way of delivering health information, we dismissed videos with no voice as they were considered equivalent to written information. Third, the method for evaluating video content is arguable. We choose not to evaluate the accuracy from the point of view of conventional medicine admitting that other approaches may be partly useful. Instead, we decided to evaluate their educative usefulness using the AADE7 standards that specify what contents and skills a diabetes education program should contain. Fourth, the number of evaluators was limited and, as in other studies,²⁹ they evaluated the videos jointly. However the external validation by a third investigator gave an acceptable degree of agreement.

In conclusion, our analysis of YouTube videos as a tool for diabetes self-management education indicates that the probability of finding videos that relate to AADE7 self-care behaviors is less than 50 percent. Considering this fact along with the high-enough odds of finding misleading information makes it inadvisable for health professionals to recommend the use of YouTube as an educating tool if suggestion of specific videos are not made.

Footnotes

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr Mauricio has received honoraria for consulting and/or speaking for AstraZeneca, Boehringer Ingelheim, Ferrer, GlaxoSmithKline, Eli Lilly, Janssen, Medtronic, Menarini, Merck Sharp & Dohme, Novartis, Novo Nordisk, Praxis Pharmaceutical, and Sanofi. He is also a principal investigator and research group leader of CIBER of Diabetes and Associated Metabolic Diseases (CIBERDEM), which is an initiative from Instituto de Salud Carlos III (Spain).

All other authors declare that they do not have a conflict of interest.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Gabriel Gimenez-Perez

References

Perrin

Social media usage 2005–2015. Pew Research Center, http://www.pewinternet.org/2015/10/08/social-networking-usage-2005-2015/ (accessed 4 October 2017).

Fox

The social life of health information. Pew Research Center, http://www.pewinternet.org/2011/05/12/the-social-life-of-health-information-2011/ (accessed 4 October 2017).

Pant

Deshmukh

Murugiah

, et al. Assessing the credibility of the "YouTube approach" to health information on acute myocardial infarction. Clin Cardiol 2012; 35(5): 281–285.

Harrison

Sampson

Reszel

, et al. Too many crying babies: a systematic review of pain management practices during immunizations on YouTube. BMC Pediatr 2014; 14: 134.

Jang

Kim

KJ.

Use of online clinical videos for clinical skills training for medical students: benefits and challenges. BMC Med Educ 2014; 14: 56.

El-Jawahri

Paasche-Orlow

Matlock

, et al. Randomized, controlled trial of an advance care planning video decision support tool for patients with advanced heart failure. Circulation 2016; 134(1): 52–60.

Funnell

Brown

Childs

, et al. National standards for diabetes self-management education. Diabetes Care 2012; 35(Suppl 1):S101–S108.

Sampson

Cumber

, et al. A systematic review of methods for studying consumer health YouTube videos, with implications for systematic reviews. Peerj 2013; 1: e147.

Guo

Kim

Runin

How video production affects student engagement: an empirical study of MOOC vídeos. In: Proceedings of the first ACM conference on learning @ scale conference, New York, NY, USA. 4–5 March 2014, pp. 41–50.https://dl.acm.org/citation.cfm?doid=2556325.2566239 (accessed 02 July 2018).

10.

Esser

Gordon

KE.

YouTube: a gauge of public perception and awareness surrounding epilepsy. Epilepsy Behav 2010; 17(4): 541–545.

11.

American Association of Diabetes Educators AADE7 self-care behaviors position statement. https://www.diabeteseducator.org/docs/default-source/legacy-docs/_resources/pdf/publications/aade7_position_statement_final.pdf?sfvrsn=4 (accessed 21 September 2017).

12.

Khandelwal

Devine

Otremba

Quality of widely available video instructional materials for point-of-care ultrasound-guided procedure training in internal medicine. J Ultrasound Med 2017; 36(7): 1445–1452.

13.

Madathil

Rivera-Rodriguez

Greenstein

, et al. Healthcare information on YouTube: a systematic review. Health Informatics J 2015; 21(3): 173–194.

14.

Gonzalez-Estrada

Cuervo-Pardo

Ghosh

, et al. Popular on YouTube: a critical appraisal of the educational quality of information regarding asthma. Allergy Asthma Proc 2015; 36(6): e121–e126.

15.

Stothers

Lazare

, et al. Evaluation of educational content of YouTube videos relating to neurogenic bladder and intermittent catheterization. Can Urol Assoc J 2015; 9(9-10): 320–354.

16.

Bae

Baxter

YouTube videos in the English language as a patient education resource for cataract surgery. Int Ophthalmol 2018; 38(5): 1941–1945.

17.

Kwok

Singla

Phang

, et al. YouTube as a source of patient information for varicose vein treatment options. J Vasc Surg Venous Lymphat Disord 2017; 5(2): 238–243.

18.

Nour

Tsatalou

, et al. Schizophrenia on YouTube. Psychiatr Serv 2017; 68(1): 70–74.

19.

Haymes

Harries

. “How to stop a nosebleed”: an assessment of the quality of epistaxis treatment advice on YouTube. J Laryngol Otol 2016; 130(8): 749–754.

20.

Garg

Venkatraman

Pandey

, et al. YouTube as a source of information on dialysis: a content analysis. Nephrology (Carlton) 2015; 20(5): 315–320.

21.

Delli

Livas

Vissink

, et al. Is YouTube useful as a source of information for Sjögren’s syndrome? Oral Dis 2016; 22: 196–201.

22.

Erdem

Sisik

The reliability of bariatric surgery videos in YouTube platform. Obes Surg 2018; 28(3): 712–716.

23.

Paul

Boraas

Duvet

, et al. YouTube and the single-rod contraceptive implant: a content analysis. J Fam Plann Reprod Health Care 2017; 43(3): 195–200.

24.

Wong

Doong

Trang

, et al. YouTube videos on botulinum toxin A for wrinkles: a useful resource for patient education. Dermatol Surg 2017; 43(12): 1466–1473.

25.

Abedin

Ahmed

AlMamun

, et al. YouTube as a source of useful information on diabetes foot care. Diabetes Res Clin Pract 2015; 110(1): e1–e4.

26.

Powers

Bardsley

Cypress

, et al. Diabetes self-management education and support in type 2 diabetes: a joint position statement of the American Diabetes Association, the American Association of Diabetes Educators, and the Academy of Nutrition and Dietetics. Clin Diabetes 2016; 34(2): 70–80.

27.

Gabarron

Fernandez-Luque

Armayones

, et al. Identifying measures used for assessing quality of YouTube videos with patient health information: a review of current literature. Interact J Med Res 2013; 2(1): e6.

28.

Khazrai

Defeudis

Pozzilli

Effect of diet on type 2 diabetes mellitus: a review. Diabetes Metab Res Rev 2014; 30(Suppl 1): 24–33.

29.

Knosel

Jung

Informational value and bias of videos related to orthodontics screened on a video-sharing web site. Angle Orthod 2011; 81(3): 532–539.

Are YouTube videos useful for patient self-education in type 2 diabetes?