To evaluate the quality, understandability, actionability and readability of English-language Internet-based resources about autism spectrum disorder.
Design:
Cross-sectional observational study evaluating the quality, accountability, understandability, actionability and readability of English-language Internet-based resources related to autism spectrum disorder.
Setting:
The study was conducted using publicly available webpages accessed through Google and Yahoo search engines, identified based on search terms relevant to autism spectrum disorder, as determined by Google Trends and SEMRUSH data. Data collection occurred in March 2023.
Method:
Google Trends was used to identify the terms most likely searched by consumers when looking for information related to autism. The HON, URAC, JAMA benchmark criteria and the DISCERN instrument were used to evaluate webpage quality and accountability. We used the PEMAT-P to evaluate webpage understandability and actionability. The Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning Fog Index and the Simple Measure of Gobbledygook were used to evaluate webpage readability.
Results:
In total, 103 webpages were analysed. Nine webpages displayed the HON seal. Six webpages met all core JAMA standards, 23 met three core standards, 35 met one core standard and six met none of the core standards. DISCERN scores ranged from 16 to 53 (Mdn = 33.00; IQR = 41.00–27.00). PEMAT-P understandability and actionability scores were 8.52 ± 2.10 and 1.30 ± 1.04, respectively. Readability scores, on average, exceeded the recommended grade reading levels (FRE = 27.00 ± 12.19, F-KGL = 15.20 ± 2.32, FOG = 12.70 ± 2.09 and SMOG = 15.70 ± 1.78).
Conclusion:
Most webpages did not meet the criteria for quality, understandability, actionability and readability. The developers of websites aimed at autistic persons and their care partners should consider the findings of this study when developing patient education materials. Doing so will facilitate improved communication with healthcare consumers, resulting in improved outcomes.
AdemiluyiGReesCESheardCE (2003) Evaluating the reliability and validity of three tools to assess the quality of health information on the internet. Patient Education and Counseling50(2): 151–155.
AltmanD (1991) Statistics in medical journals: Developments into the 1980s. Statistics in Medicine10(12): 1897–1913.
5.
AtchersonS, et al (2014) A computer-based readability analysis of consumer materials on the American Speech-Language-Hearing Association website. Contemporary Issues in Communication Sciences and Disorders41: 12–23.
6.
AziosJH, et al (2017) Quality and readability of English-language internet information for aphasia. International Journal of Speech-Language Pathology21(1): 1–9.
7.
BadarudeenSSabharwalS (2010). Assessing readability of patient education materials: Current role in orthopaedics. Clinical Orthopaedics and Related Research468(10): 2572–2580.
8.
BailinAGrafsteinA (2001) The linguistic assumptions underlying readability formulae. Language & Communication21(3): 285–301.
9.
Batt-RawdenSFlickingerTWeinerJ, et al (2014) The role of social media in clinical excellence. The Clinical Teacher11(4): 264–269.
10.
BeerAJEggerstedtMUrbanMJ, et al (2022) Quality-based evaluation of patient-facing online education Materials regarding facial filler procedures. Facial Plastic Surgery38(3): 300–306.
11.
CasanovaELWidmanCJ (2021) A sociological treatment exploring the medical model in relation to the neurodiversity movement with reference to policy and practice. Evidence & Policy17(2): 363–381.
12.
CharnockDShepperdS (2004) Learning to DISCERN online: Applying an appraisal tool to health websites in a workshop setting. Health Education Research19(4): 440–446.
13.
CharnockDShepperdSNeedhamG, et al (1999) DISCERN: An instrument for judging the quality of written consumer health information on treatment choices. Journal of Epidemiology and Community Health53(2): 105–111.
14.
ChowdhuryJ (2002) Content analysis of online autism specific sites. Journal of Developmental Disabilities9(2): 157–165.
15.
CocoL, et al (2017) Readability level of Spanish-language patient-reported outcome measures in audiology and otolaryngology. American Journal of Audiology26(3): 309–317.
16.
CroenL, et al (2015) The health status of adults on the autism spectrum. Autism19(7): 814–823.
17.
CronbachJ (1951) Coefficient alpha and the internal structures of tests. Psychometrika16(3): 297–334.
18.
DanielWWCrossCL (2013) Biostatistics: A Foundation for Analysis in the Health Sciences. Hoboken, NJ: Wiley.
19.
DeanB (2023) We analysed 5 million Google search results. Here’s what we learned about organic click through rate. Backlinko. Available at: http://www.backlinko.com/google-ctr-stats
20.
Finney RuttenLJBlakeKDGreenberg-WorisekAJ, et al (2019) Online health information seeking among US adults: Measuring progress toward a healthy people 2020 objective. Public Health Reports134(6): 617–625.
21.
FleissJLCohenJ (1973) The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement33: 613–619.
FleschR (1948) A new readability yardstick. The Journal of Applied Psychology32(3): 221–233.
24.
GhidellaC, et al (2005) Aphasia websites: An examination of their quality and communicative accessibility. Aphasiology19: 1134–1146.
25.
GrantNRodgerSHoffmannT (2015) Evaluation of autism-related health information on the Web. Journal of Applied Research in Intellectual Disabilities28(4): 276–282.
26.
GrayS, et al (2019) Readability of individuals with Disabilities Education Act Part B procedural safeguards: An update. Language of Speech and Hearing Services in Schools50(8): 373–384.
27.
GreenwellTWalshB (2021) Evidence-based practice in speech-language pathology: Where are we now?American Journal of Speech-Language Pathology30(1): 186–198.
28.
GunningR (1952) The Technique of Clear Writing. New York: McGraw-Hill.
29.
HallCM, et al (2016) Online dissemination of resources and services for parents of children with autism spectrum disorders (ASDs): A systematic review of evidence. Review Journal of Autism and Developmental Disorders3: 273–285.
30.
HasanLAbuelrubE (2011) Assessing the quality of web sites. Applied Computing and Informatics9(1): 11–29.
31.
IversonSA, et al (2008) Impact of Internet use on health-related behaviours and the patient-physician relationship: A survey-based study and review. Journal of Osteopathic Medicine108(12): 699–711.
JiaX, et al (2021) Online health information seeking behaviour: A systematic review. Healthcare9(12): 1740.
34.
KaickerJ, et al (2013) Assessing the quality and reliability of health information on ERCP using the DISCERN instrument. Health Care Current Reviews1(1): 1–4.
35.
KimH, et al (2017) Seeking medical information using mobile apps and the internet: Are family caregivers different from the general public?Journal of Medical Systems41(3): 38.
36.
KoganMD, et al (2018) The prevalence of parent-reported autism spectrum disorder among US children. Pediatrics142(6): e20174161.
37.
KoganMDStricklandBBBlumbergSJ, et al (2008) A national profile of the health care experiences and family impact of autism spectrum disorder among children in the United States, 2005–2006. Pediatrics122(6): e1149–58.
38.
KolliaB, et al (2019) Testing the readability of online content on autism spectrum disorders. Advances in Neurodevelopmental Disorders3(1): 85–90.
39.
KristAH, et al (2017) Engaging patients in decision-making and behaviour change to promote prevention. Information Services & Use37(2): 105–122.
40.
KutnerM, et al (2006) The health literacy of America’s adults: Results from the 2003 National Assessment of Adult Literacy. Available at: https://nces.ed.gov/pubs2006/2006483.pdf
41.
LandisJRKochGG (1977) The measurement of observer agreement for categorical data. Biometrics33(1): 159–174.
42.
LindlyOJCabralJMohammedR, et al (2022) ‘I don’t do much without researching things myself’: A mixed methods study exploring the role of parent health literacy in autism services use for young children. Journal of Autism and Developmental Disorders52(8): 3598–3611.
43.
LordC, et al (2018) Autism spectrum disorder. Lancet392(10146): 508–520.
44.
MackintoshVH, et al (2005) Sources of information and support used by parents of children with autism spectrum disorders. Journal of Developmental Disabilities12(1): 41–51.
45.
MaennerMJ, et al (2023) Prevalence and characteristics of autism spectrum disorder among children aged 8 years – Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2020. MMWR Surveillance Summaries72(2): 1–14.
46.
ManchaiahVDockensALFlaggeA, et al (2019) Quality and readability of English language internet information for tinnitus. Journal of the American Academy of Audiology30(1): 31–40.
47.
McLaughlinG (1969) SMOG grading: A new readability formula. Journal of Reading12(8): 639–646.
48.
Morahan-MartinJAndersonCD (2004) Information and misinformation online: Recommendations for facilitating accurate mental health information retrieval and evaluation. Cyberpsychology & Behaviour3(5): 731–746.
NicholsonNC, et al (2016) Readability, user-friendliness, and key content analysis of newborn hearing screening brochures. Journal of Early Hearing Detection and Intervention1(1): 66–77.
51.
NicolaidisC, et al (2014) Primary care for adults on the autism spectrum. Medical Clinics of North America98(5): 1169–1191.
RedishJCSelzerJ (1985) The place of readability formulas in technical communication. Technical Communication32(4): 46–52.
54.
ReichowBShefcykABruderMB (2013) Quality comparison of websites related to developmental disabilities. Research in Developmental Disabilities34(10): 3077–3083.
55.
RosenNELordCVolkmarFR (2021) The diagnosis of autism: From Kanner to DSM-III to DSM-5 and beyond. Journal of Autism and Developmental Disorders51(12): 4253–4270.
SeelyKD, et al. (2022) The “teach back” method improves surgical informed consent and shared decision-making: A proof of concept study. Patient Safety in Surgery16(1): 33–39.
58.
ShneydermanMDavisRSnowG, et al (2022) Zenker’s diverticulum: Readability and quality of online written education materials. Dysphagia37(6): 1461–1467.
59.
ShoemakerSJWolfMSBrachC (2014) Development of the patient education materials assessment tool (PEMAT): A new measure of understandability and actionability for print and audiovisual patient information. Patient Education and Counseling96(3): 395–403.
60.
SilbergWM (1997) Assessing, controlling, and assuring the quality of medical information on the internet. JAMA277(15): 1244.
61.
SkalitzkyMKGulbrandsenTRLorentzenW, et al (2021) Health Literacy in clubfoot: A quantitative assessment of the readability, understandability and actionability of online patient education material. The Iowa Orthopaedic Journal41(1): 61–67.
62.
SlavychBK, et al (2024) Using ChatGPT to improve health communication and plain language writing for students in Communication Sciences and Disorders. Perspectives of the ASHA Special Interest Groups9(3): 599–612.
63.
SlavychBK, et al (2013) Quality of life in persons with voice disorders: A review of the patient-reported outcome measures. International Journal of Therapy and Rehabilitation20(6): 308–315.
64.
SlavychBK, et al (2021) A systematic review of voice-related patient-reported outcome measures for use with adults. Journal of Voice38(2): P544.E1–P544.E14.
StefuJSlavychBKZraickRI (2023) Patient-reported outcome measures in voice: An updated readability analysis. Journal of Voice37(3): 465e27–465.
67.
SteinerSMSlavychBKZraickRI (2023). Assessment of online patient education material about dysphagia. Dysphagia38(3): 990–1000.
68.
SviderPFAgarwalNChoudhryOJ, et al (2013) Readability assessment of online patient education materials from academic otolaryngology–head and neck surgery departments. American Journal of Otolaryngology34(1): 31–35.
69.
TanSS-LGoonawardeneN (2017) Internet health information seeking and the patient-physician relationship: A systematic review. Journal of Medical Internet Research19(1): e9.
70.
ThomasKCEllisARMcLaurinC, et al (2007) Access to care for autism-related services. Journal of Autism and Developmental Disorders37(10): 1902–1912.
71.
ThyerBA (2019) Pseudoscientific therapies for autism spectrum disorder. In: MatsonJL (ed.) Handbook of Intellectual Disabilities: Integrating Theory, Research, and Practice. Cham: Springer, pp. 501–515.
72.
TonsakerT, et al (2014) Health information on the internet: Gold mine or minefield?Canadian Family Physician60(5): 407–408.
US Department of Health and Human Services (2023) Healthy people 2030: Building a healthier future for all. Available at: https://health.gov/healthypeople
75.
US National Library of Medicine (n.d.) Evaluating Internet health information: A tutorial from the National Library of Medicine. Available at: https://medlineplus.gov/webeval/webeval.html
76.
VandenbrouckeJP, et al (2007) Strengthening the reporting of observational studies in Epidemiology (STROBE). Epidemiology18(6): 805–835.
YeapZ, et al (2023) Online health literacy resources for people with intellectual disability: Protocol for a grey literature scoping review. BMJ Open14(11): e088509.
80.
YoungbloodNE (2014) Revisiting Alabama state website accessibility. Government Information Quarterly31(3): 476–487.
81.
ZamanianMHeydariP (2012) Readability of texts: State of the art. Theory and Practice in Language Studies2(1): 43–53.
82.
ZhouS, et al (2017) How consistent are the best-known readability equations in estimating the readability of design standards?IEEE Transactions on Professional Communication60(1): 97–111.
83.
ZraickRIAtchersonSR (2012) Readability of patient-reported outcome questionnaires for use for persons with dysphonia. Journal of Voice26(5): 635–641.
84.
ZraickRIAtchersonSRHamBK (2012) Readability of patient-reported outcome questionnaires for use with persons with swallowing disorders. Dysphagia27(3): 346–352.
85.
ZraickRIAziosMHandleyMM, et al (2021) Quality and readability of internet information about stuttering. Journal of Fluency Disorders67: 105824–105812.
