Readability of Retina Patient Education Materials Generated With a Large Language Model

Abstract

Purpose: To determine whether large language models (LLMs) can be harnessed to improve the readability of educational material for retina patients. Methods: Forty-one documents (fact sheets presented in portable document format), each representing a vitreoretinal condition, from the American Society of Retina Specialists (ASRS) Retina Health Fact Sheets website, were downloaded in November of 2024. The multimodal LLM Generative Pre-trained Transformer 4 (GPT-4) was accessed through ChatGPT to generate patient education material on the same 41 vitreoretinal conditions. The model was then prompted to adjust the texts to a sixth-grade reading level. The text outputs for each of the 41 conditions were then analyzed through a readability calculator, and the Average Reading Level Consensus Calc (ARLCalc) score, a normalized average of 8 validated readability formulas that reflect a consensus readability grade level of the text, was recorded. Results: The ARLCalc scores for the ASRS Fact Sheet, GPT-4 Response, GPT-4 Enhanced, and ASRS Enhanced responses were 12.85 (± 0.89), 12.37 (± 0.97), 8.66 (± 0.87), and 9.37 (± 1.09), respectively. A statistically significant difference was found between the 4 groups (P < .001). Conclusions: LLMs may be used as a tool to improve the readability of patient-facing text. Patient education material created by specialty-trained authorship committees remains the gold standard for providing accurate medical information.