Abstract
People commonly and increasingly rely on the internet to search for health information, including those related to endometriosis-associated dyspareunia. Yet the content of such websites may be of variable accuracy and quality. This review aims to evaluate the quality, readability, and suitability of web-based resources on endometriosis-associated dyspareunia for patients. We searched 3 databases – Google, Bing, and Yahoo – to identify websites related to endometriosis-associated dyspareunia. Two independent reviewers screened the search results against inclusion and exclusion criteria. Another set of two reviewers evaluated the selected websites using validated measurement instruments. Out of 450 websites, 21 met the inclusion criteria and were evaluated. More than half of the websites had information on content updates, reported on authorship, or disclosed sponsorship information. The mean quality and suitability scores were 47.5 (SD = 13.3) and 65.2 (SD = 13.6) respectively, thus suggesting generally adequate quality and suitability levels. However, the mean readability scores exceeded the recommended level for health-related websites. The poor readability of the websites might limit accessibility for a significant proportion of patients with low educational levels. The findings of this review have implications for designing high-quality, readable and up-to-date web interventions for people who rely on web platforms as an alternative or complementary source of health information on dyspareunia.
Introduction
Endometriosis is a chronic inflammatory condition in which endometrial-like tissue grows outside the uterus, typically in the pelvic cavity. 1 The condition has been estimated to affect approximately 10% of women globally 2 and an unknown number of gender-diverse people. Symptoms include chronic pelvic pain, dysmenorrhea, abnormal menstrual bleeding, dysphasia, dysuria, and impaired fertility. One of the cardinal symptoms of endometriosis is dyspareunia, which has been estimated to affect more than 50% of people with this disease.2,3 Dyspareunia (i.e., pain experienced during and/or after penetrative vaginal intercourse) has been found to have detrimental effects on quality of life including reduced sexual function, low self-esteem, and poor interpersonal relationships.3–7 Despite the negative effects of dyspareunia on quality of life, the symptom is often normalized by healthcare providers and in some cases neglected. 8 Patients have also reported ignoring pain symptoms because they are unaware of the link between dyspareunia and endometriosis. 7 For those who may be aware of their dyspareunia symptoms and inclined to seek medical care, the sensitive and potentially stigmatizing nature of pain, intimacy, and genitalia, may prevent patients from disclosing their symptoms to providers and inhibit patient-provider discussions on the topic.
Due to the sensitive nature of dyspareunia and limited patient-provider discussions on the topic, it is highly likely that people have sought related information online. Indeed, the internet has become a major source of health information, with more than 80% of Internet users accessing the world wide web to find health-related information,9,10 and an estimated 6.75 million daily health searches occurring in Google alone. 11 Moreover, people turn to the internet to an even greater extent when searching for information about health conditions perceived as embarrassing and stigmatizing including dyspareunia. 12 For instance, more than 400,000 endometriosis-related Google searches are performed monthly in the United States alone. 13 With the internet being a major source of health-related information, people can explore sensitive and personal questions regarding their health in the privacy and comfort of their homes.14,15
Despite the high use of web-based information, the information provided on these platforms are often complex, and written at an advanced literacy level, with most websites unregulated and of varying quality.14,16 The complexity of internet-based information often compounds people’s ability to read, comprehend, or utilize the information in their daily lives.16,17 Exposure to complex and unregulated web-based information may negatively affect peoples’ understanding, application, and decision-making ability, especially among populations with low literacy or those who experience marginalization.18,19 The use of unregulated web-based information may be prominent in people with sensitive and stigmatizing conditions like dyspareunia because people with such disorders are more likely to resort to web-based resources as complementary or alternative sources of information. 20
The purpose of this review is to evaluate the readability, suitability, and quality of web-based information on endometriosis-associated dyspareunia for end-user patients. It is expected that the findings from this study will inform the design of future tailored web-based interventions for patients on sensitive and potentially stigmatizing health topics who may have low literacy levels. This is important because having quality, suitable and readable web-based resources tailored to patients’ needs and expertise is essential to non-expert patients who rely so much on web-based resources but may not be able to evaluate and distinguish between credible and inaccurate information.
Materials and methods
We conducted a 2-step process in this evaluation. First, we conducted a search for the websites and two independent reviewers screened the search results against inclusion and exclusion criteria. Secondly, another set of two reviewers evaluated the selected websites to assess the quality, suitability, and readability.
Website searches and screening
We conducted a systematic review of English-language web-based resources on endometriosis-associated dyspareunia across the globe. Web searches were conducted across the three most popular search engines including Google, Bing, and Yahoo. 9 Approximately 77% of website users across the globe will begin their search from these 3 sources with an estimated monthly visit of 1.8 billion and 500 million on Google and Bing respectively.9,21 The following search terms were used for each of the 3 search engines: “Endometriosis + painful sex/sexual pain,” “Endometriosis + dyspareunia,” and “Endometriosis + sex”. To prevent biased results from past search history and search location, we erased all search histories, cookies, and cached data from the web browser before beginning each search. It has been established that more than 97% of people might not review results beyond the first 3 pages of a search. 17 Therefore, search results in the first 5 pages of each search engine were extracted. This process was repeated for all the search terms across the 3 databases. The result for each search term across all search engines were compiled in excel and duplicate websites were removed. Two independent reviewers screened the remaining search results against the inclusion and exclusion criteria. Websites were included if they contained >300 words about endometriosis-associated dyspareunia, were written in English, and were publicly available. Websites were excluded if they provided information on endometriosis-associated dyspareunia exclusively to healthcare professionals or researchers, had dead/broken links, or were merely links to documents, blogs, social media, or online newspapers/magazines. Because this review was focused on dyspareunia, websites that contained information on endometriosis without corresponding information on dyspareunia were also excluded. Websites that met the inclusion criteria were entered into an Excel spreadsheet for further data extraction and evaluation. Discrepancies between the two reviewers on the website selection were resolved by discussion with a third reviewer. The search and screening of the websites was conducted between May 2022 – October 2022. A repeated search was conducted in October 2023 but found no additional websites.
Websites evaluation
Following data extraction, another set of two independent reviewers evaluated the websites’ quality and suitability using validated measurement instruments.22–25 The purpose of choosing a different set of reviewers was to reduce biases from the screening process. These two reviewers were given a day’s training on how to use each measurement instrument. The evaluation was done independently by each reviewer and was recorded on an Excel sheet. The country where the website was hosted and maintained was also extracted. Discrepancies between the two evaluators were resolved by an ongoing discussion with a third person until a consensus is reached. The details of each measurement instrument are outlined below. The evaluation of the website content was conducted between November 2022 – March 2023. A re-evaluation was also conducted in October 2023 but found no major changes to the content of the websites.
Quality
Quality was assessed using DISCERN, a 16 item questionnaire that evaluates the quality of health information on a 5-point Likert scale from 0 (very poor quality) to 5 (very high quality). 23 DISCERN has three main sections. Questions 1-8 assess the reliability of the information, 9–15 assess the quality of information on treatment choices, while question 16 evaluates overall quality about treatment choices. A cumulative score of the 16 items was calculated and the results were used to classify the quality of each website as either high quality (53–80), moderate quality (27–52), or low quality (<27). To understand the performance of the websites on the individual components of DISCERN, we calculated the scores for each sub-scale of DISCERN by adding the average scores for each sub-scale (i.e., reliability, quality, and overall quality). The quality of the websites were also assessed using the Journal of American Medical Association (JAMA) benchmark criteria. 26 JAMA is a Yes/No checklist that assesses the quality of websites across four quality attributes including (1) authorship and affiliations, (2) attribution/references 3) disclosure and (4) currency.
Suitability
Suitability was assessed using the Suitability and Comprehensibility Assessment of Materials (SAM) tool.23,24 SAM is a 22-item scale that assesses the suitability of materials across six components including content, literacy demand, graphics, layout and typography, learning stimulation and motivation, and cultural appropriateness. Each component assesses web-based information as either “Superior,” (scored 2), “Adequate,” (scored 1) or not suitable (scored 0). The total scores are then divided by the maximum possible score. Websites with scores between 70%-100% were interpreted as superior, 40% - 69% were interpreted as adequate, and 0%-39% were interpreted as not suitable. Components that did not apply to each website were excluded from the overall maximum possible score.
Readability
Readability was assessed by the third author using the online tool, the Flesch-Kincaid Reading Ease test (https://goodcalculators.com/flesch-kincaid-calculator/). 27 Readability describes the minimum level of education that is needed to read and understand a written text. The Flesch-Kincaid Reading Ease test gives a score of 0–100, with test scores of 0 being the most difficult to read and 100 being the easiest to read. Score ranges were used to categorize the approximate reading grade level needed to read the websites. Because the Flesch-Kincaid Reading Ease test does not provide an understanding of readability from the patients’ perspective, we engaged our two reviewers to provide an additional perspective on readability by commenting on how easy they find the website content to read.
Data analysis
The data were summarized using descriptive statistics. The mean, standard deviation, and ranges for all 21 websites were calculated across the 3 criteria of quality, suitability, and readability. We also calculated the scores for sub-dimensions of DISCERN. For the DISCERN and SAM assessments, an intraclass correlation coefficient (ICC) was calculated to determine the level of agreement between the two evaluators before the final composite scores were computed. The ICC for this study was based on Koo et al 28 guidelines for selecting and reporting intraclass correlation coefficients for reliability research. All analyses were conducted using SPSS Version 28.
Results
The search strategy yielded a total of 450 websites across the 3 search engines. We removed 341 duplicate websites for the remaining 109 web pages, which 2 reviewers assessed based on the inclusion and exclusion criteria outlined above. Following the screening process, 21 websites were included in the final assessment. Figure 1 shows the flowchart for the website selection process. 13 (61.9%) of the websites were hosted and maintained in the United States, 5 (23.8%) were global websites, 2 (9.5%) were from Australia, and 1 (4.7%) was in Canada. Flowchart for the website selection process.
Quality (DISCERN & JAMA)
Summary of websites’ quality, suitability, and readability.
Summary of interrater reliability for DISCERN and SAM assessment.
ICC = Intraclass correlation coefficient, SAM = Suitability assessment of materials.
Suitability
The SAM assessment shows that seven (33.3%) of the websites were found to be highly suitable, 12 (57.1%) were found to be adequately suitable, and 2 (9.5%) were found not to be suitable. Overall, the websites were found to be adequately suitable (Mean SAM = 65.2, SD = 13.6, Range 34.4 – 83.7). The websites’ suitability indicators (i.e, the SAM scores across the 21 websites) are also depicted in Table 1. There was an excellent level of agreement between the two evaluators on the SAM scores for 9 websites (ICC >0.9), a good level of agreement for 1 website (ICC = 0.7 – 0.9), a moderate level of agreement for 10 websites (ICC = 0.5 – 0.7) and a poor level of agreement for 1 website (ICC <0.5). The intraclass correlation coefficients for the SAM scores on each website are shown in Table 2. Graphics and cultural appropriateness components of SAM was not applicable to at least seven of the websites.
Readability
The readability of the websites by US reading age.
Discussion
To the best of our knowledge, this study is the first study to examine the quality, suitability, and readability of websites specifically focused on endometriosis-associated dyspareunia. It is also important to note that while this review was focused on dyspareunia, most of the websites broadly focussed on endometriosis, with few pages dedicated to dyspareunia. This likely stems from the symptoms of dyspareunia occurring, not in isolation, but as one symptom among the numerous other symptoms of endometriosis, such as painful menses, urination, or defecation.3–5 Furthermore, the comparatively limited focus on dyspareunia also suggests that even in the context of endometriosis, the symptom of dyspareunia is not commonly acknowledged, described, explained, or prioritized. This is perhaps explained, in part, by the normalization of dyspareunia generally and the limited awareness and discomfort of healthcare providers involved in generating disease-related information. 7
Even though we searched for endometriosis-associated dyspareunia websites across the global, majority of the websites that meet our inclusion criteria were largely developed and maintained in high income countries. These findings reflected the neglected nature of dyspareunia as a symptom of endometrioses, particularly across developing countries.29,30 The fact that these websites are developed in high income countries may not take into account the cultural nuances of patients in other parts of the world who access these websites.
Overall, there was a wide variability in the readability, suitability, and quality of the websites. All the websites exceeded the recommended grade level for the readability of online health information. Even though the readability of the websites on dyspareunia were consistent with websites on other disease conditions,16,17,31 they might nevertheless affect access to information by patients with low levels of education (i.e., below 9th grade). Considering the importance of health literacy and the generally neglected nature of endometriosis-associated dyspareunia, providing more readable and easy to understand information may optimise patient decision making and improves engagement with online resources, particularly for patients with limited English proficiencies. 32 The quality of the websites was found to be generally moderate (mean DISCERN scores = 47). Even though overall quality was good, a relatively lower score on the reliability sub-scale of DISCERN shows that the websites may be less reliable for patients even though they might have scored high on quality sub-scale. Furthermore, 3 of the websites were rated as poor quality with outdated information. Such poor quality and outdated websites may likely mislead patients to opt for inappropriate treatment or cause financial loss as a result of purchases of ineffective/harmful products. 33
Similar to the quality assessment, the suitability score was also considered adequate (i.e., an average score of 65.2%). Generally, graphics are seen as an effective way of patient education. 34 However, 7 the websites were missing essential elements relating to graphics and cultural appropriateness components of SAM. Given the limited understanding of dyspareunia and the important role of graphics in patient education, 34 graphics and illustrations are needed as a means of conveying complex medical problems. In the context of endometriosis and dyspareunia, we believe that graphics can help break up the complex relationship between endometriosis, pelvic pain, and dyspareunia in an easy-to-understand format, particularly for patients with low educational levels. In addition to our recommendation for graphics, cultural appropriateness should also be thought of as a way of reaching out to patients from ethnic minorities who experience systemic discrimination and inequities in accessing and using healthcare, such as those who are Indigenous and/or racialized, and with limited English language reading comprehension. Cultural appropriateness is important because websites are ubiquitous and could be accessed by people in low income countries even though they are hosted and maintained in high-income countries.35,36
Implications for healthcare
Even though this review was conducted for patients, the findings have implications for everyone involved in endometriosis and dyspareunia care including patients, healthcare professionals, as well as web-based interventions developers. In fact, the websites rated as poor quality or outdated may not only deny patients of the latest evidence and best practices on dyspareunia but also lead to worse health outcomes. Given the increasing reliance on and referral of patients to web-based resources,16,37,38 these findings have important implications for providing healthcare in an increasingly digitalised world. As providers of care, health professionals should advise patients of the inherent inaccuracies, misinformation, or outdated information associated with online platforms. 16 A careful consideration is also required by healthcare professionals and patients to identify available web-based platforms on dyspareunia that might have been exposed to patients. The ability to identify quality web-based platforms might facilitate appropriate decision-making and self-management of endometriosis-associated dyspareunia for patients who rely on web platforms as their main sources of information. While it may be challenging to achieve 100% accuracy on health-related online platforms, efforts must be made to co-create and deploy web-based platforms by engaging with patients in the development process. In cases when co-creation may be challenging, patients should be empowered to identify accurate and credible web-based resources. Furthermore, the accuracy and quality of online health information should not be the only concern for patients and healthcare professionals because a website can be considered as accurate but out of date. Research on endometriosis-associated dyspareunia and its clinical guidelines is constantly evolving and websites but be regularly updated to reflect the latest evidence in the field.39,40
Strengths and limitations
The study had some strengths. Engaging two different set of reviewers for screening and evaluating the websites helped to reduce the biases of the study – improving our findings and applicability of results. For the two website evaluators, there was a higher level of agreement between on both DISCERN and SAM instruments – improving the reliability of our findings. By disabling location services on the computers used for the searches, we were able to eliminate geographical biases thereby ensuring that our search results reflect all websites across the globe.
Despite these strengths, the review had some limitations. Even though the level of agreement between the two evaluators were high, we cannot account for the inherent subjectivity of the DISCERN and SAM instrument, which we believed, might have influenced an assessment of the websites. The SAM and DISCERN instruments were also developed over 20 years ago at the time of this review and their relatively older nature could limit their applicability to modern websites. Even though recent psychometric analysis has proved the relevance of the tools,41–44 they still do not allow for a detailed and comprehensive analysis on the various sub-scales. Further studies should modify and expand on the tools to allowed for detailed and nuanced analysis of the sub-scales as they apply to modern websites. Furthermore, only English-language websites were selected for this review and the findings may not be generalizable to websites written in other languages or for patients in non-English speaking countries. Further research is needed to evaluate websites in other languages and perhaps compare their quality to those in English. While the limitations in this study are similar to those observed across other web-based health information evaluation studies,16,31,45 they nevertheless be taken into consideration when interpreting or applying the findings, particularly for patients in non-English speaking countries or in low and middle income countries.
Conclusion
Overall, the suitability and quality of the information on the websites were generally adequate and the readability level exceeded the recommended levels for health-related websites. The poor readability levels can limit patients’ ability to use health-related websites or lead to serious health consequences if outdated recommendations are applied. Future research should explore insights into how web-based resources should be developed to meet the needs of patients, particularly those with low literacy skills. As modern clinical encounters for patients with stigmatized conditions/disorders gradually move from face-to-face to web-based information-seeking and web-based interactions, 37 regular revisions are warranted to improve the overall quality and readability of health-related websites for patients. This assessment provides valuable information on existing endometriosis-associated dyspareunia websites to guide healthcare providers in directing their patients to credible, quality, and suitable web-based resources.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Canadian Institutes of Health Research.
