Abstract
Objectives
This study aimed to assess the readability of online information about semaglutide while also assessing understandability and quality.
Methods
Ozempic, Wegovy, and ‘semaglutide’ were individually searched. The non-sponsored results on the first five pages for each search were screened. The text from the included links were evaluated by two researchers for readability using SMOG and Flesch Reading Ease (FRE), for understandability using Patient Education Materials Assessment Tool (PEMAT) and for quality using DISCERN. A statistician ran reports for medians, interquartile ranges, and frequency statistics.
Results
61 links met evaluation criteria. Median scores for SMOG and FRE were 13th grade level and College. Fewer than 10% were at or below the recommended reading grade level. The median score of PEMAT was 62%. The median overall score of DISCERN was 4 out of 5.
Conclusions
Most education available online about semaglutide medications is not written at the recommended reading level. Patient education on semaglutide needs to be rewritten to be at the recommended 8th grade reading level.
1. Introduction
The internet is a vast resource and a part of everyday life that one can use to search for medical and health information. In fact, the CDC found that 58.5% of adults used the internet to search for health information in 2022. 1 When it comes to searching for health information material online, there are many different factors that need to be considered, such as readability, organization, understandability, and use of multimedia. 2 Furthermore, understanding medical information might be difficult without the healthcare professional to interpret the information.
With the average American adult having a reading grade level of the 7th to 8th grade, it is especially important that online medical information be written at the 8th grade or below. 3 The National Institutes of Health, American Medical Association (AMA), and US Department of Health and Human Services even recommend that health information be written at the 6th grade reading level. 4 However, most patient education is written at the 10th grade level, which a significant leap from the recommended grade. 5 Patient education is meant to help patients understand their disease and how to manage it 4 ; therefore, patient education needs to be written at a reading grade level that matches that of the average American to help improve their understanding of disease management.
Studies have also shown that people with lower health literacy may be more likely to have difficulty evaluating and discerning between low- and high-quality health information.6,7 Another study found that patients with more limited health literacy not only utilized online health information as a resource, but had more trust in the information they were provided from virtual sources. 8 This is a concern as patients may be avoiding conversations with their provider and relying on low-quality health information to answer vital questions and guide them in making health-related decisions. Additionally, in the current era of misinformation being shared online, it is vitally important for physicians to be aware of this and collaborate with patients to help support them in accessing and utilizing credible online health information. 9 This type of misinformation can stem from many sources including non-experts, commercial interests, social media, blogs, and other non-peer reviewed websites. Unfortunately, the credible online sources are difficult to distinguish between non-credible ones, and this may influence patients over professional advice which may undermine the trust in the patient-physician relationship and the physician’s ongoing counsel. 10
Numerous studies have evaluated the readability of online patient education materials using various resources. Rathod et al. evaluated the readability, suitability, and quality of online patient education on depression self-management resources and determined that the readability of the education exceeded the recommended reading grade level. 11 In the surgical realm, a similar study compared available online resources regarding robotic cardiac and thoracic surgeries. Those authors found that websites generally overestimate patient understanding and are written beyond a 12th grade reading level. 12 Another study examined the understandability, actionability, and readability of diabetes education available online. 13 The results showed that the materials all scored at the 8th level or higher with poor understandability. 10 A study on endocrine care found that most materials available online were at the 11th grade or higher and were fairly difficult to read. 14 These studies highlight the need for online patient education written at a grade level that the average American adult can understand.
Media attention and public interest in semaglutide injections like Ozempic have led to increased numbers of people searching for information about this drug. 15 More people are taking semaglutide injections with 2.6 million prescriptions being fulfilled in retail pharmacies by December 2023. 16 With high grade levels and low understandability of online patient education, researchers sought to determine the readability of online information about semaglutide medications, including Ozempic and Wegovy.
2. Methods
This quantitative study was completed with data collected in July 2024 and assessed from August 2024 to February 2025 using validated tools for readability, understandability, and quality. Ethics approval was not required for this study. There are no human participants in this study and informed consent is not required.
2.1. Search strategy & web page selection
Google was selected as the main search engine due to its popularity. 17 The two most popular brand names (‘Ozempic’ and ‘Wegovy’) and ‘semaglutide’ were individually searched on Google on July 25, 2024. 15 Each of the three search terms were searched on a separate public-use computer that wipes user data upon restart and in incognito mode. The searches were run in the United States, and the computer networks utilize Network Address Translation (NAT). The non-sponsored results on the first five pages for each search were manually imported into Microsoft Excel and screenshots of each result page were saved. Duplicate links were removed. In efforts to reduce bias, two researchers screened each individual link for the exclusion criteria below and discrepancies were discussed to find agreement. Web links to information not written in English were excluded due to the researchers’ abilities and the tools selected to assess readability. Beyond language limitations, inclusion centered on whether the information was meant for patient education based on the definition that “patient education is the process of enabling individuals to make informed decisions about their personal health-related behaviour.” 18 Given that patients prescribed these medications would likely go to the manufacturer’s website for more information, drug manufacturer webpages were included as long as the main purpose of the page was to educate and it was free of advertisement, patient testimonials, incentives, etc. within the text. Content designed for healthcare professionals, such as prescription guides or research articles, was also excluded since it was not designed for patient consumption. Non-text materials like videos or pictographs were excluded from extraction and evaluation due to the primary objective of assessing readability. However, if there was accompanying text information in addition to the non-text materials that met the inclusion criteria, only the text was extracted. Using the print-friendly option or “save as” on the webpage, it was converted to a PDF. The PDF was converted to Microsoft Word with the document being cleaned up so that only text data was extracted for assessment. Data was collected July 2024 and assessed from August 2024 to February 2025. The list of included URLs is available from Appendix B.
2.2. Assessment & tools
Two researchers independently assessed the text from each included link using four different tools. Readability of the content was measured using Simple Measure of Gobbledygook (SMOG) and Flesch Reading Ease (FRE).19,20 SMOG provides an estimation of the years of education needed to understand a piece of writing on the first reading by assigning a score based on the number of polysyllabic words within the piece of writing. FRE gives text a score from 1-100 with 100 being the easiest to read and a corresponding grade level. To assess understandability, the Patient Education Materials Assessment Tool (PEMAT-P) for print materials was selected. 21 It also assesses actionability. PEMAT-P scores range up to 100, and the higher the score, the more understandable or actionable the material is. Since readability doesn’t equate to accuracy, the quality of the materials was assessed using the DISCERN questionnaire, which asks users to evaluate written consumer health information through 16 questions on a 5-point scale. 22 It has been adapted to assign a single overall quality score. 23 SMOG, FRE, PEMAT-P, and DISCERN have been used in numerous studies to assess online health information.13,24–29
2.3. Statistical analysis
Interrater reliability was tested by using Intraclass Correlation Coefficients (ICC) with 95% confidence intervals (95% CI). Due to non-normal distributions, medians and interquartile ranges (IQR) were used to report the respective metric scores for each rater. Frequency and percentage statistics were employed to analyze the grade-level findings for the patient materials, as well as for the prevalence of patient materials written at higher grade levels that are commonly accepted. Statistical significance was assumed at an alpha value of 0.05 and all analyses were performed using SPSS Version 29. 30
3. Results
3.1. Criteria
There were a total of 143 links from the three searches. After duplicates were removed, 125 unique websites were screened for inclusion. Reasons for exclusion included advertisement (n=40), patient testimonial (n=1), targeted audience being healthcare providers (n=12), non-text material only (n=4), and research article (n=7). Sixty-one met the inclusion criteria and had their text extracted. Statistical tables can be found in Supplemental Material Appendix A.
3.2. Inter-rater reliability
The following coefficients were generated from the reviewers and ranged from poor to excellent inter-rater reliability: SMOG score (ICC = 0.70, 95% CI 0.54 – 0.81, p < 0.001), SMOG Grade Level (ICC = 0.67, 95% CI 0.50 – 0.79, p < 0.001), Flesch Reading Level (ICC 1.00, 95% CI 1.00 – 1.00, p < 0.001), FRE Grade Level (ICC = 1.00, 95% CI 1.00 – 1.00, p < 0.001), PEMAT Understandability (ICC = 0.71, 95% CI 0.56 – 0.82, p < 0.001), PEMAT Actionability (ICC = 0.89, 95% CI 0.82 – 0.93, p < 0.001), and DISCERN (ICC = 0.37, 95% CI 0.14 – 0.57, p = 0.001). See Table 1 for the ICC values and their respective 95% CIs.
3.3. Readability
Readability scores for SMOG fell between 27 to 237 with SMOG Grade Level ranging from 8th to 17th. The median value for SMOG for Reviewer 1 was 96 (IQR 73 – 123), which equates to the 13th grade reading level. For Reviewer 2, the median was 122 (IQR 90 – 146), which equates to the 14th grade reading level. Reviewer 1 found only two websites to be written at the 8th grade level whereas Reviewer 2 found none. Figure 1 shows the grade levels assigned to the documents by each reviewer. SMOG grade level scores.
Readability scores for FRE fell between 0.0 to 71.6 and both reviewers found the median of FRE to be 48.3 (IQR 39.5 – 53.9), which equates to the College level. Researchers found that over 90% of the documents scored above the recommended reading level.
For SMOG, only 3% of the websites were written at the recommended 8th grade reading level or below while FRE found one website to be written at the 7th grade reading level and four written at the 8th to 9th grade reading level. See Figure 2 for distribution of grade levels in the perfect agreement FRE evaluation. FRE grade level percentages.
3.4. Understandability and actionability
Reviewer scores for PEMAT-P understandability fell between 24% to 94%. The median score was 62% for Reviewer 1 (IQR 54 – 69) and 62% for Reviewer 2 (IQR 58 – 69). The median score falls below the criteria for understandability, which is at 70%.
21
For Reviewer 1, 14 (23%), scored above the standard 70% to be considered understandable whereas Reviewer 2 found that 13 (21%), scored above the standard 70%. Figure 3 shows the IQRs for Reviewer 1 and Reviewer 2. IQRs for PEMAT-P understandability scores.
For PEMAT-P actionability, the scores for both reviewers ranged between 0% to 86%. The median was 40% for Reviewer 1 (IQR 0% – 60%) and 40% for Reviewer 2 (IQR 0% – 60%). Reviewer 1 found seven websites to be at 80% or above, the acceptable score for actionability, while Reviewer 2 found only six websites. Sixteen of the websites for Reviewer 1 scored at 0% whereas 18 websites scored at 0% for Reviewer 2. Additionally, 28% of the websites did not have an actionable item that a user could follow.
3.5. Quality
Reviewers used overall score for quality instead of evaluating each aspect of DISCERN. The median for DISCERN for Reviewer 1 was 4 (IQR 4 – 4) while the median for Reviewer 2 was 3 (IQR 2 - 4). Reviewer 1 tended to give higher scores for quality when compared to Reviewer 2. Figure 4 details the overall scores given by each reviewer. Overall DISCERN score.
4. Discussion
By re-creating the same information-seeking experience that the average person would have while searching for information regarding semaglutide, researchers were able to evaluate semaglutide patient information available online. Researchers found that fewer than 10% of the online patient education materials were written at the recommended reading grade level. Those websites that were written at the 8th grade level or lower were from government or hospital websites, including Mayo Clinic in USA, Cleveland Clinic in USA, and NPS MedicineWise in Australia, and one independently owned website, Electronic Medicines Compendium in the UK.31–34 Additionally, these websites all scored well on understandability (62%-94%) and quality (4-5), with the exception being Mayo Clinic, which scored 50% on understandability and 3 for quality. With very few websites on semaglutide written at the 8th grade level or lower, more needs to be done to improve the readability. This can include using plain language, adding visuals, or using a sample patient group to test the education for readability. 35 Of those websites that scored well, they all incorporated a question-and-answer format. This format makes it more engaging through helping the patient focus on one piece of information at a time, breaks the document into sections, and makes it an easier reference for patients to find the information.36,37 Our results are similar to Lipari’s findings of online diabetes education having a high reading grade level with SMOG. 13
The wide range of PEMAT-P scores shows a need for improvement in understandability as well. Most of the education scored below the >70% standard to consider education “understandable.” Lipari also found that online patient education on diabetes scored poorly on PEMAT, with only 23% scoring above the 70% standard. 13 For patient education to be understandable, it needs to make its purpose clear, use common words and the active voice, break the material into short sections, and use images to help make the message clear. 21 The websites evaluated in this study need to improve their understandability to make it understandable for the average American adult.
While actionability was not a primary or secondary objective in this study, researchers found the actionability of the majority of websites to be well below the 80% standard. Actionability provides the patient with at least one action they can take based on the information provided. 21 On average, nearly 30% of the websites did not include at least one action the user can take. Materials were also missing a tangible tool that could help a user take action. Furthermore, 75% of the materials did not break down actions into manageable, explicit steps. Understandability of patient education allows patients to understand the information being given to them while actionability can empower patients to take part in their own health through specific actions they can take, thus making it a very important component of effective health information.
When it came to DISCERN, researchers found the quality to range from average to good. Researchers chose to use one final overall score for DISCERN, which has been supported in literature. 23 While DISCERN does not evaluate readability, it does help assess the overall quality of the written content. Interrater reliability for DISCERN was poor; therefore, it is difficult to assess the quality of the articles. Based on the researchers’ scores, most websites fell between a 3 and 4, indicating there are potentially important, but not serious shortcomings to minimal shortcomings. 38
To date, there are a limited number of published studies that assess the readability, understandability, or quality of semaglutide. While Yeung et al. examined YouTube videos and not text, their study of semaglutide full-length videos found an average of 2.3 out of 5 on the Modified DISCERN scale. 39 As mentioned earlier, the majority of websites in this study scored between 3 and 4, which suggests that websites may have higher quality information than YouTube videos. However, another study assessed the quality of Japanese medical institutions’ websites on glucagon-like peptide-1 receptor agonists like semaglutide and weight loss. 40 Using the full DISCERN instrument which assigns an overall score with a maximum of 80, they found a mean total of 32.6, which falls in the ‘poor’ quality range. Future research could explore the quality of information from both text and audiovisual materials on semaglutide.
This study shows a need for semaglutide information to be written at a lower reading grade level with understandability and actionability in mind. With 2.6 million prescriptions filled and a growing interest in semaglutide, publicly available information should be written in plain language.15,16 For non-healthcare professionals with patient education training who are looking for reliable sources on semaglutide to recommend to patients, this study found that websites like Mayo Clinic, Cleveland Clinic, and international government-partnered websites scored highly. Additionally, both non-healthcare and healthcare professionals can educate patients and family members on how to search for health information online and evaluate it.
For healthcare professionals, this study reinforces the need to engage with patients to make sure they understand medications like semaglutide at the time of prescribing, by using TeachBack or other health literacy principles. In addition, engaging patients in conversations regarding their usage of online health information and ensuring that information is accurate and understandable for the patient would help healthcare professionals collaborate with patients engaging in their own care. This would also open a line of communication to encourage patients to reach out to them with any questions about their medication. For patients who articulate any incorrect benefits or concerns from a misinformed perspective, healthcare professionals should try to review those specific sources of misinformation with their patients to help clarify and correct any misinformed opinions. Understanding where their individual patients are obtaining their information will also help better inform their care and counseling of the practice at large. This may also instigate development of patient-friendly, accurate information geared specifically to each clinic’s patients’ needs. Furthermore, healthcare professionals and those in the medical field, such as librarians, should be aware that education readily available is not always written as easy-to-read, even if it is from a reliable website or author. This study highlights an opportunity for healthcare providers and organizations to create their own patient education on semaglutide that is understandable, actionable, and written at the 8th grade or lower reading grade level. In summary, we recommend using TeachBack or some other method for medication education at the time of prescribing, discuss online health information with patients and utilize this conversation to alleviate misinformation, offer resources for reliable online health information, and consider utilizing good health literacy principles to develop medication information geared toward the providers specific patient population.
4.1. Strengths/Limitations
One strength of this research was having the websites evaluated by two researchers with differing backgrounds: a librarian who is a health literacy expert and an experienced family medicine clinician. This helped prevent a bias or potential gap in knowledge on both the information and clinical sides. A second strength was the inter-rater reliability for readability. Reviewers agreed most strongly on the readability assessments, which increases the validity of the findings as the main objective of this research.
There were some limitations to this research. First, we excluded non-text materials from the evaluation which likely affected the overall understandability scores since visuals were excluded from this study. We attempted to mitigate the effect by using the PEMAT-P, which is for print materials. Future studies could be done that include non-text and audio-visual materials and utilize the PEMAT-AV tool. Second, we included manufacturers’ websites as long as they were free of advertisement, patient testimonials, incentives, etc., but the assessment of what was an advertisement, patient testimonial or incentive is subjective. This potential subjectivity was addressed by having two reviewers screen each website for inclusion and exclusion criteria, including being free of advertisement, etc. Third, there was poor agreement on inter-rater reliability on quality; therefore, the overall conclusions on quality might not be as valid since the reviewers did not reach a moderate level of agreement.
This study was also conducted at the same time that artificial intelligence (AI) use has increased in the general population and healthcare at large. While there is varying quality, accuracy and readability in the most-commonly used generative AI platforms when evaluated for healthcare literacy, we acknowledge that the adaptation of this technology will likely lead to additional patient-centered resources that were not included in this study, including the creation of individualized, reading-level appropriate patient information by patients and physicians alike. 41
5. Conclusion
The majority of patient education available online about semaglutide medications is not written at the appropriate reading level for the average consumer. Authors of the online patient education on semaglutide should consider rewriting the education to be at the recommended 8th grade reading level using plain language principles. The Health Literacy Solutions Center from the Institute for Healthcare Advancement provides helpful tips and examples for creating effective patient education materials. 42 Based on the scores, this may require a great amount of time and effort for some of the online education. These findings are likely not unique to semaglutide medications, but the growing popularity of these medications suggests an increased need for patient education that is more understandable and readable.
Supplemental material
Supplemental material - Readability and understandability of online patient education on semaglutide
Supplemental material for Readability and understandability of online patient education on semaglutide by Alexandria Q. Wilson, MSIS, Shauntá M. Chamberlin, PharmD, FCCP, William S. Dabbs, MD, FAAFP, R. Eric Heidel, PhD, Kelsey L. Grabeel, MSIS in Health Informatics Journal
Supplemental material
Supplemental material - Readability and understandability of online patient education on semaglutide
Supplemental material for Readability and understandability of online patient education on semaglutide by Alexandria Q. Wilson, MSIS, Shauntá M. Chamberlin, PharmD, FCCP, William S. Dabbs, MD, FAAFP, R. Eric Heidel, PhD, Kelsey L. Grabeel, MSIS in Health Informatics Journal
Footnotes
ORCID iDs
Author contributions
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of conflicting interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
