Enhancing readability and understandability of vascular surgery discharge summaries using artificial intelligence

Abstract

Objectives

This study aimed to assess whether Large Language Models (LLMs), like ChatGPT-4, could simplify discharge summaries for vascular surgery patients while maintaining accuracy and completeness, ultimately improving patient comprehension and engagement in their postoperative care.

Methods

In this cross-sectional multicentric study, discharge summaries from 90 vascular surgery patients across three centers were collected. These were divided into three groups based on patient pathology: aortic pathology, peripheral artery disease, and carotid artery disease. Summaries were processed by LLMs to create patient-friendly versions with a target reading level suitable for a 6th-grade education. The readability of the original and AI-generated summaries was evaluated using the Flesch-Kincaid Grade Level and Ease Score. Understandability and actionability were assessed with the Patient Education Materials Assessment Tool for Print (PEMAT-P), which evaluates the clarity, organization, and actionable nature of the text. Accuracy and completeness were rated using a 6-point Likert scale and a 3-point Likert scale, respectively. Statistical analyses, including paired-samples t-tests, ANOVA, and post-hoc tests, were performed to assess the differences between the original and AI-modified summaries.

Results

AI-generated summaries demonstrated significant improvements in readability, with a 39.6% reduction in Flesch-Kincaid Grade Level and a 106.37% increase in Ease Score. The mean understandability score based on PEMAT-P was 77.71, while the actionability score was 52.12. Accuracy was rated highly (mean score of 5.21), and completeness had a mean score of 2.61. However, 10.8% of summaries had omissions, and 7.5% had hallucinations, with corrections made in some cases.

Conclusions

LLMs like ChatGPT-4 can significantly improve the readability and accessibility of discharge summaries for vascular surgery patients, enhancing their understanding and engagement in postoperative care. While the summaries were accurate and complete, the occurrence of errors suggests the need for further refinement to minimize omissions and hallucinations. These findings indicate that AI can be a valuable tool in improving communication between healthcare providers and patients. Future research should focus on reducing errors and enhancing actionability.