Abstract
Introduction
Little is known about the quality of Arabic health information available online, particularly regarding information on head and neck cancer. This study evaluated the quality of web-based Arabic information on head and neck cancer and its treatment.
Methods
Eleven Arabic keywords were used to search for the content using Google (Google.com). The quality of written material was assessed using the Patient Education Materials Assessment Tool (PEMAT)-P and four JAMA Quality Benchmarks (authorship, attributions, disclosure, and currency), whereas the quality of audiovisual (AV) material was assessed using the PEMAT-A/V tool.
Results
A total of 315 websites were analyzed, of which 17 and 298 contained AV and written materials, respectively. Among these, 198 sites were sourced from scientific outlets, such as Medscape, whereas 117 originated from layperson platforms, such as news articles and blogs. The mean PEMAT scores for understandability and actionability were 61% (± 16.03) and 17% (± 24.89), respectively. The average overall PEMAT score was as low as 49% (± 15.55). Additionally, the included websites did not meet the JAMA benchmarks, with an average overall rating of 1.11 (± 0.94) on a total score of 4. Of the websites, none achieved all benchmarks, and 23 achieved three (7%).
Conclusion
Notably poor-quality Arabic information on head and neck cancers online was observed, with questionable actionability, understandability, and veracity of sources for the presented information. Clinicians and healthcare policymakers should consider creating accessible, evidence-based, up-to-date web content on head and neck cancers tailored to patients’ needs and wishes.
Keywords
Introduction
Head and neck cancer (HNC) is a term used for cancer that affects the oral cavity, larynx, and pharynx; individuals with HNC have a 5-year mortality rate of up to 50%. 1 The number of patients diagnosed with HNC continues to increase worldwide, particularly due to the high incidence of oropharyngeal cancers.1,2 In the Gulf Cooperation Council countries (Saudi Arabia, Bahrain, Oman, Kuwait, Qatar, and the United Arab Emirates), the incidence rate of HNC is expected to increase by 200–300% by 2024. 2 The three primary curative treatment modalities for HNCs are surgery, radiotherapy, and chemotherapy, which may be associated with adverse effects that can negatively affect a patient’s physical and psychosocial well-being.3,4
Similar to individuals with other chronic noncommunicable diseases, individuals with HNC require accessible and accurate web-based information regarding the prevention, early detection, risk factors (e.g., tobacco use, alcohol drinking, and HPV-related sexual behaviors), and other information to help them make informed decisions regarding their health care plans.1,5,6 Healthcare providers (HCPs) are often the primary source of health information; however, a short consultation duration and patient anxiety upon receiving a diagnosis may result in insufficient information.
5
As such, patients use online information to find what they need, and questions such as “
In addition to written educational leaflets provided by healthcare services, patient-tailored, easy-to-understand educational materials can play an important role in informing patients about their health conditions. 5 Such educational material may include web-based podcasts, videos, and articles aimed at patients and the public. Patients with a comprehensive understanding of their chronic illness are more likely to engage in health care plans and better interpret their symptoms and procedure-associated complications.3,5 The internet allows for information access at any point, which is particularly relevant for the Saudi Arabian population, where 99% of the population has access to the internet. 10
Nevertheless, finding accurate, high-quality information may not always be possible without the help of HCPs. 11 This is likely due to a shortage of freely accessible, evidence-based, and comprehensible resources, as well as language barriers. 11 Furthermore, the quality and readability of English-language health information regarding HNCs remain below the recommended levels.12,13 Similar assessments of Arabic web-based health content on oral health topics, such as periodontal disease, have found it to be of low quality. 14 Notably, there is a lack of a comprehensive assessment of the quality and readability of Arabic information on HNCs. 15 Such assessments can inform routine HCP–patient consultations and ascertain useful health information sources.5,15
Objectives
This study aimed to assess the quality and readability of web-based Arabic information on HNCs and their treatment.
Methods
Search Strategy
The eleven keywords in Arabic to search for HNC-related web-based content included the following: “head and neck cancer,” “nasopharyngeal cancer,” “laryngeal cancer,” “pharyngeal cancer,” “paranasal sinus cancer,” “treatment of nasopharyngeal cancer,” “treatment of pharyngeal cancer,” “treatment of laryngeal cancer,” “treatment of head and neck cancer,” “treatment of paranasal sinus cancer,” and “radiation therapy” (see Appendix for Arabic translation). The main Google domain (Google.com) was used as the search engine to identify written and audiovisual (AV) materials [
Inclusion & Exclusion Criteria
All Arabic Language HNC-related web-based content written for patients or the public from any source were included (e.g. governmental sources, non-profit organizations, public and private healthcare providers). Exclusion criteria included content promoted by search engines, content on websites with paywalls/restricted access, content written for HCPs (e.g. journal articles), content with broken links, duplication to other searches, advertisements (sponsored results), and those with exclusive on non-Arabic content were manually excluded by a panel of study investigators (AMA, RII, HFA and SAA).
Quality Assessment
The quality of written material was assessed using the Patient Education Materials Assessment Tool for printable materials (PEMAT-P) and JAMA quality benchmarks,16,17 whereas AV material was assessed using the PEMAT-A/V. The PEMAT was used to evaluate the understandability and actionability of patient education materials. The understandability component comprises 13 items that assess how well people with varying levels of health literacy can process and explain key messages, accounting for factors such as word choice, layout, and organization. PEMAT incorporates a criterion related to active voice, which refers to a sentence or clause in which the subject actively performs or causes an action expressed by the verb. 18 As this criterion is not often relevant or employed in the Arabic language, 19 it was not assessed in this study.
The actionability component includes four items that evaluate whether the material provides clear instructions that patients can follow to manage their health. 16 The JAMA quality benchmarks assess the quality of content based on “authorship” (availability of author credentials and affiliations), “attribution” (source/s of the presented information), “disclosure” (whether sponsorship information is available), and “currency” (whether the content is up to date). 17 For quality assurance and to standardize the PEMAT-P and JAMA assessments, a panel of study investigators (AMA, RII, HFA, and SAA) performed a thorough group-based discussion to address the scoring criteria and ensure their accurate implementation. Later, two of them, the medical interns (RII and HFA), performed all analyses independently and met to address any inconsistencies and duplicate data. A few of these remained non-agreeable, thus, the consensus was made at the end by medical professionals with patient education experience (AMA and SAA).
Data Collection and Statistical Analysis
Data were collected in an MS Excel spreadsheet and analyzed using IBM SPSS Statistics for Windows, version 29 (IBM Corp., Armonk, NY, USA). Ethical approval was not required for this study as no patient data or patient accrual was required. Descriptive statistics were used to summarize the data, presenting means and standard deviations where appropriate. A score-based analysis was conducted to evaluate the differences among groups. Statistical significance was determined at a p-value of <0.05.
Results
Out of 330 websites, 15 did not meet the inclusion criteria (Figure 1). Thus, a total of 315 websites were included in the analysis. Of these websites, 17 had AV material, and 298 had written material; 198 had been from sources allied to HCPs (e.g., Medscape), and 117 were from layperson sources (e.g., news articles and blogs). The number of websites obtained from each Arabic keyword is presented in Figure 1. Number of websites obtained from all the search terms (n = 315)
Summary of the Results for all Keywords and Their Characteristics
Abbreviations: AV, audiovisual, HCP, healthcare provider.
*Total number of JAMA benchmarks achieved for each keyword (highest =4, lowest = 1).
JAMA Benchmarks
Jama Benchmark and PEMAT Scores of the 315 Websites
Max, maximum; Min, minimum; SD, standard deviation.
PEMAT
The mean PEMAT understandability and actionability scores were 61% (± 16.03) and 17% (± 24.89), respectively (Table 2). The mean PEMAT total score was 49% (± 15.55). Notably, the items “
When comparing the PEMAT scores across the 11 keywords, both “treatment of head and neck cancer” and “head and neck cancer” had the lowest understandability scores. However, there were no significant differences in the PEMAT actionability scores across the 11 keywords (p > 0.05). Additionally, there was no significant difference in PEMAT scores between written and AV materials (p > 0.05). The PEMAT understandability scores for scientific sources were significantly higher than those for layperson sources (p = 0.014). However, there was no significant difference between the PEMAT actionability scores for scientific and layperson sources (p > 0.05). Regarding the dimensions of items in PEMAT understandability, “use of visual aids” had the lowest scores among all other dimensions. “Content,” “use of numbers,” and “layout and design” had the highest scores among other dimensions. Regarding PEMAT actionability, 37% of the websites clearly identified a single action readers could take. However, only 13% broke down the actions into manageable steps. The item “ Mean PEMAT-P scores for all written materials (n = 298)
Concerning AV materials, understandability assessments indicated that the “organization” item had the lowest scores among all dimensions. Contrastingly, items related to “content,” “design,” and “word choice and style” had the highest scores. Furthermore, the actionability items were notably low with scores of 18% or less (Figure 3). Notably, all items from the PEMAT actionability scored less than 40% for both written and AV materials. Furthermore, the item “ Mean PEMAT-AV scores for all audiovisual materials (n = 17)
Discussion
This study presents a comprehensive analysis of web-based Arabic information on HNCs, using multiple keywords and assessment methods. We found that the content was limited and of notably low quality; furthermore, scores assessing patients’ ability to comprehend and act on what they read remained below the recommended levels. Our findings have twice the non-JAMA criterion benchmarks as previous assessments of Arabic patient educational materials. In a study on oral cancer, 15% of the websites did not meet the JAMA benchmarks. 20 Our results reflect that 32% of the websites did not meet any of the criteria. Additionally, the JAMA benchmark score of 1.11 observed in our study was lower than the reported score of 1.6 in a study on Arabic breast cancer content. 21 These results indicated a consistent trend in low-quality Arabic web-based information on various chronic health topics. Therefore, significant improvements in the content and quality of health information available in the Arabic language on the internet are urgently needed.
Notably, none of the analyzed websites fulfilled all the JAMA benchmarks. Similarly, in another Arabic study that assessed online information on oral cancer, only 2% of the websites fulfilled all the JAMA benchmarks. 20 In comparison, studies evaluating English information on HNC reported a significantly higher fulfilment rate of 45%. 11 Additionally, in another study examining English-language information on HNC, 17% of the sources partially or fully met the JAMA benchmarks. 22 These findings indicate the disparity between our study and English-based studies in meeting the JAMA benchmarks, highlighting the need to improve the quality and comprehensiveness of Arabic web-based health information on HNCs.
Furthermore, in our study, the least commonly fulfilled benchmark among the four JAMA benchmarks was “disclosure,” followed by “attribution” and “authorship”. Contrastingly, a study evaluating English web-based HNC information identified “authorship” as the least commonly fulfilled benchmark, followed by “attribution.” These findings highlight specific areas of weakness in both Arabic and English web-based health information. 11 The absence of “authorship” and “attribution” raises concerns regarding the credibility and accountability of Arabic information concerning HNCs. Failure to include references and authors on most Arabic cancer-related websites significantly affected the quality scores of these sites. This omission could hinder patients’ ability to make well-informed, independent decisions. 20
Concerning the PEMAT assessments, website scores for understandability and actionability were significantly low at 61% and 17%, respectively. These scores were notably lower than those reported for information regarding type 2 diabetes provided in hospitals (78% and 66%, respectively). These findings highlight the disparity in the effectiveness of web-based information and printed materials in conveying understandable, actionable content to Arabic-speaking individuals. 23
One of the strengths of our study lies in the inclusion of 315 websites, in contrast to the limited sample sizes typically seen in similar studies, which assessed only 19, 20, or 100 websites.6,20,24 Analyzing such an extensive number of websites enriched the depth and breadth of our study. Furthermore, the use of two quality assessment methods (JAMA benchmarks and PEMAT) improved the robustness and reproducibility of our approach. Notably, our study marks the first evaluation of Arabic language content for HNC.
The websites analyzed in this study had a high degree of similarity. Many websites cite a single source, resulting in highly repetitive content and concerns about the reliability and accuracy of the information presented. Pop-up ads can also hinder the users’ ability to find relevant information, making it difficult to access the desired content. A noteworthy observation is the frequent use of automated translation tools, particularly for websites with non-English content. 24 Despite their ability to make information accessible to a broader audience, these tools often render content incomprehensible because of linguistic errors. Consequently, individuals who rely on these translations to understand critical information for HNC treatment face significant challenges.
Our study contributes significantly to advancing patient education in the context of HNC by highlighting inadequacies in existing online resources. Many of the websites included in our analysis were from private clinics and hospitals. While these sources can provide valuable insights, their content may be biased toward promoting their services. Websites often include brief overviews that read like advertisements, emphasize the clinic’s capabilities, and encourage visitors to book appointments. This commercial bias could affect the objectivity and comprehensiveness of the available information. Thus, it is critical to recognize that our selective data collection method is highly subjective. Moreover, many studies, including ours, have been limited to evaluating websites using only one language. 6 It is also not clear whether specific content creators (e.g., governmental organizations) would present favorable understandability and actionability scores – this has not been analyzed to date.
Although the present study included a relatively large dataset of 315 websites, only 17 contained AV materials, resulting in limited statistical power for comparative analyses. Readability metrics (e.g., an Arabic adaptation of the Flesch–Kincaid index) were not employed as they do not apply to Arabic text due to the structural and grammatical differences compared to English (e.g. using diacritical marks, less use of medical terms and acronyms, and multiple meanings of the same word). 25 Some novel Arabic-specific tools were developed to overcome these limitations, but concerns remain about the language’s morphological variations and their ease of use for laypeople. 26 These differences were also notable in relation to PEMAT’s active voice criteria, which were not addressed at present, given the relatively low relevance of passive/active voice and readability compared to English and French. 19
Furthermore, the search strategy was restricted to Google; other platforms and search engines, such as Bing and AV-focused platforms (e.g., YouTube), were not evaluated, potentially limiting the generalizability of the findings. Some other limitations were also related to the assessment tools. For instance, the JAMA benchmark emphasizes scientific rigor by defining standards for authorship, attribution, disclosure, and currency. Therefore, it is necessary to guarantee the accuracy of medical information. However, this approach may ignore websites that include helpful details regarding patient experiences. In contrast, the PEMAT focuses on evaluating the representational and actionable qualities of patient education materials. While this standard is critical for making information accessible and actionable to patients, it may neglect the scientific reliability of websites. The DISCERN instrument is widely used in comparable studies and facilitates the assessment of treatment-related information quality and comparison with existing literature. 16 Nevertheless, the more recent PEMAT was more commonly used in recent infodemiological studies to assess whether a patient can comprehend and act on what they read, compared to its quality and robustness relative to the former.15,27 Also, its inter-rater reliability is notably good 28 – this was, however, not tested at present.
Future assessments need to address webpage traffic, visibility, and user engagement metrics (e.g., view counts, visitation statistics, or popularity rankings), as these factors were not considered during the study design phase. Such metrics are commonly incorporated in infodemiology studies to contextualize information quality and public health impact. Consequently, the absence of these data limits the ability to determine the real-world influence and reach of lower-quality Arabic-language-based websites. To enhance the quality and reliability of online information on HNC, we recommend developing standardized guidelines for website authors and administrators to ensure accurate, up-to-date, evidence-based content. Another recommendation is to implement interventions to reduce the occurrence of repetitive and plagiarized content and excessive use of pop-up advertisements. Furthermore, it is important to devise strategies to improve the precision and readability of translated texts, especially in languages with grammar and sentence structures that differ somewhat from the original text. Collectively, language experts and medical practitioners may be able to provide adequately translated information or create more dependable translation systems.
Conclusion
Our study showed that while web-based Arabic information on HNC serves as a valuable resource for patients, there is a concerning trend of poor quality. This trend may lead to misleading or biased information that is often not actionable. Patients seeking online Arabic language information about HNC may encounter challenges in finding practical guidance. Given current limitations, clinicians and healthcare policymakers should create accessible, evidence-based, up-to-date, actionable, and patient-centered web content on HNCs.
Footnotes
Ethical Considerations
No ethical approval was needed for the present infodemiological assessments as no patient data or patient accrual was required.
Author Contributions
Reem Bin Idris: Investigation, Writing – original draft preparation, Formal Analysis, and Visualization. Hasah F Alaluan: Investigation and Writing – original draft preparation, and Visualization. Saif A. Aljabab: Conceptualization, Supervision, and Writing – review & editing. Abdullah M Alsogheir: Conceptualization, Supervision, Writing – review & editing, and Methodology.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to thank the College of Dentistry Research Centre and the Deanship of Scientific Research at King Saud University, Saudi Arabia, for funding this research project.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The data supporting this study’s findings are not publicly available due to detailed analysis of numerous search terms and assessment methods but are available from the corresponding author upon request [AA,
