Generative AI and large language model evaluation of kidney donation websites: Benchmarking health literacy,sentiment,and digital engagement to optimize donor recruitment

Abstract

Background

Kidney transplantation offers substantial clinical and economic benefits for patients with end-stage kidney disease (ESKD), yet organ shortages persist. Enhancing public awareness and health literacy regarding kidney donation is essential for effective donor recruitment. While online patient education materials are primary drivers of public perception, their readability, digital engagement, and accessibility remain underexplored barriers.

Methods

We analyzed the most prominent kidney donation websites identified through U.S.-based Google searches, including 11 primary organizations and 16 affiliated subdomains, for a total of 27 websites. Readability was benchmarked using Flesch-Kincaid, SMOG, and Gunning Fog indices. To assess qualitative metrics, we deployed a generative AI framework utilizing a Large Language Model (Claude) to conduct automated sentiment analysis and tone evaluation, validated by human review. We systematically mapped digital engagement features, including multimedia, interactive tools, and multilingual support, to determine content comprehensiveness.

Results

Websites consistently provided accurate information with a generally positive or neutral tone. Average readability exceeded recommended levels, with a combined mean grade of 12.3; 34% of websites were written at a college-level reading standard. Consensus across content was high. Multimedia elements were widely used, but engagement features were limited; only 30% of sites included extensive testimonials, and interactive tools were absent. AI-based analysis enabled standardized and reproducible evaluation, highlighting opportunities to improve accessibility, tone, and inclusivity.

Conclusion

Current U.S. kidney donation digital resources present a barrier to health equity due to excessive reading complexity and static engagement models. AI provides a scalable, reproducible framework to audit and optimize patient education materials. Future initiatives must leverage AI-guided content optimization to bridge the literacy gap, potentially increasing donor registration and access to transplantation.

1. Introduction

In 2024, kidney transplantation reached a historic milestone in the USA, with a total of 27,759 kidney transplants performed, the highest number recorded to date, continuing a trend observed since 2021. ¹ Despite this progress, more than 90,000 individuals remain on the kidney transplant waitlist. ² Kidney transplantation offers clear advantages over hemodialysis (HD), including reduced mortality, improved quality of life, and greater cost-effectiveness for patients with end-stage kidney disease (ESKD).^3,4 Although recent policy initiatives have sought to increase organ supply and improve equitable access to transplantation, a critical shortage of donor organs persists. ⁵

Surveys conducted by the U.S. Department of Health and Human Services indicate that while over 90% of Americans support organ donation, only approximately 50% are registered donors. Digital platforms, including search engines, health websites, and social media, now represent the primary sources of health information for patients and the public and play a major role in disseminating information about organ donation. Indeed, 40.9% of respondents report learning about organ donation through social media. Age-related differences are notable: younger adults (18 to 34 years) are substantially more likely than those aged 50 to 64 years to obtain information via social media (58.7% vs. 29.9%) or search engines (45.1% vs. 18.2%). ⁶ These findings highlight that efforts to expand kidney donation increasingly unfold within a digitally mediated information environment that strongly shapes public understanding, perceptions, and decision-making.

Living and deceased kidney donation differ substantially. Living donation is primarily altruistic and ideally free of external pressure, whereas deceased donation often relies on prior patient documentation or family decisions, which are frequently unplanned.^7,8 These differences highlight the need for clear, tailored online information for both living and deceased donor pathways, which could improve awareness and support for all donors. Over the past decade, the internet, including search engines and social media, has become the dominant source of medical information and misinformation for patients. Surveys reported that up to 98% of individuals rely on search engines to obtain health-related content.^9,10

As such, the way information is presented online, including its accuracy, accessibility, readability, and tone, plays a critical role in shaping public understanding of health-related topics, including organ donation. These factors could influence informed decision-making and the willingness to participate in living or deceased donor programs. Kidney donation and organ transplantation are complex and nuanced topics that require a foundational understanding of medical principles. Public perceptions are shaped by varying cultural and religious beliefs and remain vulnerable to misinformation and disinformation. The ramifications of social and traditional media attention are substantial, reflecting the susceptibility of public perceptions to rapidly circulating narratives. Following a widely publicized September 2024 House Energy and Commerce Oversight Subcommittee hearing that included testimony regarding alleged organ procurement irregularities, the Kentucky Donor Registry experienced a significant rise in removals; the number of removals in the month following the hearing was threefold higher than in the preceding month. ¹¹ Studies have not only identified a substantial burden of mis- and disinformation on social media but have also shown that false claims travel faster and farther across social platforms.^12,13

The American Medical Association (AMA) recommends that patient education materials be easy to follow, clearly illustrated, and written at or below a sixth-grade reading level. ¹⁴ However, across medical fields, most resources exceed this guideline, limiting accessibility for patients with low health literacy^15–17; less than 3% of materials meet the sixth-grade standard. ¹⁸ This concern is heightened by declining literacy in the U.S., with 28% of adults aged 16–65 demonstrating Level 1 or below proficiency, meaning they can understand simple texts but have difficulty with complex or detailed information. ¹⁹

Recent advances in artificial intelligence (AI), particularly large language models (LLM), offer new opportunities to systematically evaluate large volumes of unstructured, patient-facing digital content. ²⁰ These models enable scalable assessment of readability, sentiment, tone, and engagement-related features across online health information, providing insights that are difficult to obtain through manual review alone.^20,21 In this context, AI-based analysis can serve as a tool to characterize the quality and framing of information that patients and potential donors encounter in real-world digital environments.

This study aims to assess the sentiment, readability, and user engagement of widely accessed online kidney donation resources using AI. By identifying patterns in public-facing information, we seek to highlight opportunities to enhance awareness, address misconceptions, and support donor recruitment and informed consent.

2. Materials and methods

2.1. Selection and comprehensive analysis of top-ranked websites

In the initial phase of our methodology, we selected the top 10 suggested websites on Google when searched for the term “Kidney Donation”. Device location tracking settings were disabled and set to United States, but not to a specific city, county or state. We did not differentiate between living or deceased donations based on our search. An initial Google search was conducted to identify the top 10 most frequently accessed websites related to kidney donation. To account for potential temporal variability in search engine rankings and website content, the search was repeated three months later. This follow-up search allowed us to assess the stability of search results over time and to ensure that the final set of analyzed websites reflected current, widely accessed public resources at the time of detailed evaluation. We performed an in-depth analysis, examining not only the content but also the layout, design elements, and navigational ease, as these factors can significantly influence user engagement and comprehension. We intentionally did not apply strict inclusion or exclusion criteria for the websites to better reflect a real-world user experience and its corresponding outcomes.

For each top-ranked website, we included the main kidney donation page and up to 4 directly linked subpages (single click away) that provided core content (e.g. eligibility, risks, living donation, FAQs, testimonials). Main pages and associated subdomains were treated as individual analytic units for page-level assessments. Content-based features, including testimonials, multimedia elements, multilingual content, and deceased donor information, were recorded as present if identified on any page within a website or its associated subdomains. Quantitative metrics such as tone, sentiment, and readability were calculated separately for each page and subdomain and then summarized at the parent-domain level, typically by averaging, to generate website-level estimates. Reporting occasionally focused on main landing pages for clarity, but all subdomains were included in the underlying analyses. This approach allowed comprehensive capture of website content while maintaining a consistent and interpretable analytic framework.

In addition to overall content, we assessed specific website features for their potential relevance to patient engagement and decision-making. Testimonials were included as they provide personal perspectives that may influence readers’ understanding of donation. Interactive elements (e.g., videos, comment sections) were considered for their potential to enhance user engagement and accessibility. Links to decision aids were evaluated because they may support informed decision-making. These features were selected to capture both informational content and aspects of the user experience that could affect comprehension and engagement.

2.2. Utilization of claude AI for advanced sentiment analysis

Sentiment analysis in this study was carried out using Claude Sonnet 4.0, an LLM developed by Anthropic, designed for natural language processing tasks such as text classification and emotional inference. The model leveraged semantic comprehension, tone identification, and emotional mapping to assign a sentiment label (positive, neutral, or negative) while accounting for broader linguistic context. Using a multi-layer attention framework, the model assessed the relative importance of words and phrases to detect nuanced emotional content.

Claude Sonnet 4.0 (Anthropic) was used to perform sentiment classification and tone identification for each website using standardized prompts. For each output, the model provided sentiment labels and supporting textual examples drawn directly from the source material to justify its classifications. All AI-generated outputs were reviewed by the research team to ensure plausibility and concordance with the original content. No formal quantitative accuracy grading was performed, as the sentiment analysis was intended to support qualitative interpretation rather than provide definitive measurement.

The top 10 websites were identified through Google search ranking results. Both the website URL and the full text transcript were provided to Claude to ensure accurate interpretation of content. Standardized criteria were applied across all sources for assessment of sentiment, engagement, and consensus, ensuring comparability and reproducibility. This AI-assisted workflow added analytical depth, reduced subjectivity, and enhanced the quality of the qualitative evaluation.

2.2.1. Grading criteria for ‘consensus level’ and accuracy

The ‘Consensus Level’ was assessed based on the uniformity of information presented across the included websites and its alignment with established, widely accepted information on kidney donation intended for patient education. This involved analyzing the degree to which the websites agreed on key points about kidney donation and information was considered accurate if it reflected current, commonly accepted understanding of kidney donation for a lay audience. The grading was as follows:

- High Consensus: If 8–10 websites shared identical or very similar views or information on a specific aspect of kidney donation.

- Moderate Consensus: If 5–7 websites exhibited similar viewpoints or information, with some minor variations.

- Low Consensus: If fewer than 5 websites agreed, indicating significant variation in the presentation or interpretation of information.

- Accuracy Assessment: Independently of consensus level, each website was also assessed for factual accuracy and the presence of potentially misleading information.

2.2.2. Grading criteria for ‘engagement'

‘Engagement’ was evaluated based on the presence and extent of interactive elements such as comments, shares, likes, or other forms of user interaction on the websites. The grading scale was:

- High Engagement: Websites with active user comments, frequent shares, and other interactive elements indicating significant public interaction.

- Moderate Engagement: Websites with some user comments and shares, but less frequent or extensive than those classified as high engagement.

- Low Engagement: Websites with minimal to no user comments, shares, or other forms of interaction, indicating a primarily informational rather than interactive focus

2.2.3. Grading criteria for ‘testimonials'

‘Testimonials’ refers to website content that features real-life experiences shared by donors or recipients, offering personal perspectives on the donation and transplantation process.

- High Testimonials Presence: Website includes multiple detailed personal account from donor or recipients. Opportunities for users to engage with or submit their own stories may also be present.

- Moderate Testimonials Presence: Website features some personal experiences but are limited in number and depth. Little to no user interaction beyond viewing the testimonials

- Low/No Testimonials Presence: Website contains no testimonials or only vague references to individual experiences without personal narrative.

2.2.4. Grading criteria for ‘sentiment'

The ‘Sentiment’ of the content was gauged based on the tone and emotional quality of the information provided. This was analyzed using sentiment analysis tools. Manual review was utilized to ensure appropriate analysis. Sentiment was assessed using the URL link and transcript of the website. The grading was:

- Positive Sentiment: Content that predominantly conveyed optimistic or hopeful tones about kidney donation.

- Neutral Sentiment: Content that was primarily informative, factual, and objective, without explicit positive or negative biases.

- Negative Sentiment: Content that conveyed cautious, skeptical, or pessimistic tones regarding kidney donation.

Claude 4.0 was also provided with a list of 40 words to describe sentiment to choose from. In our prompt, we instructed the analysis tool to identify and report the three most dominant tones in the text based on its assessment. These grading criteria provided a structured and systematic approach to evaluating the content of the websites. By applying these scales, we aimed to quantitatively and qualitatively analyze the data provided on the websites. (List of words: Optimistic, Pessimistic, Anxious/Concerned, Reassuring, Excited/Enthusiastic, Authoritative, Cautious, Confident, Uncertain/Tentative, Objective/Clinical, Empathetic, Supportive, Empowering, Judgmental, Inclusive, Educational/Informative, Urgent, Formal, Conversational, Alarmist, Skeptical, Defensive, Apologetic, Promotional/Marketing, Balanced/Neutral, Dramatic, Condescending, Respectful, Inspiring, Threatening, Vulnerable, Practical, Technical, Compassionate, Critical, Motivational, Warning, Celebratory, Frustrated, Patient-Centered).

2.3. In-depth evaluation of readability and Content complexity

This portion of the study focused on systematically analyzing how readable and complex the content was across the selected websites. To assess this, we applied the Flesch-Kincaid Grade Level formula, the Gunning Fog Index and the SMOG (Simple Measure of Gobbledygook) Index, all of which are well-established tools for estimating the educational level needed to understand written material. Although all three indices report readability in U.S. grade levels, the SMOG and Gunning Fog indices typically provide stricter estimates than the Flesch-Kincaid Grade Level, frequently assigning higher grade requirements for comparable material. ²² We calculated each score individually for every website and calculated an average between the three scores. We utilized our AI tool to calculate each score and the average. We verified with independent calculations using transcripts of the website and online calculators to ensure accurate reading grade assessment by the AI tool. In addition to the reading level, we examined the use of medical terminology and whether the websites included supportive explanations or clarifying resources. When subpages were present, an average readability score across all pages was calculated per domain.

The goal was to evaluate whether the information was accessible to users with a wide range of educational backgrounds and health literacy, particularly those reading at a high school or early college level. Readability and text complexity are essential factors in effective public health messaging.

2.4. In-depth analysis of the website structure and content

Besides the above-mentioned AI-driven analysis of the websites, we also performed a thorough manual evaluation of each website and subdomains to identify key features. These key features included if the author of the text or website is easily identifiable and if the most recent date of update is available.

In addition, we examined each main and subdomain for multilingual content availability. We also separately identified if there is Spanish content available on the website.

We assessed multimedia utilization by examining whether content included media beyond text alone. Multimedia was defined as any combination of video, audio, interactive elements (such as clickable buttons or forms), or emerging technologies (virtual reality, augmented reality). Content was classified as utilizing multimedia if it incorporated any of these media types alongside or instead of text.

Website ownership characteristics were documented, including the geographic location of domain owners (within or outside the United States), organizational type (non-profit versus for-profit), and healthcare system affiliation. Additionally, we evaluated ease of communication by examining the accessibility of contact information and the presence of direct links to contact the operating organization.

3. Results

This analysis reviewed the top 10 websites providing educational content on kidney donation and identified several notable trends. In total, 11 primary websites from 10 different organizations were included, along with 16 subdomains, resulting in 27 websites analyzed. Despite variations in presentation, the sites consistently addressed fundamental topics such as donor eligibility, donation methods (living and deceased), surgical approaches, potential complications, and long-term outcomes. Beyond content, the analysis considered factors influencing user experience including tone, visual engagement, emotional framing, frequency of content updates, and reading complexity. The websites analyzed represent a wide spectrum of trusted sources, including national health agencies, academic institutions, and leading hospitals across both the United States and the United Kingdom.

3.1. Google search

A Google search was conducted in February of 2025 and in July 2025, with greater focus on the search in July of 2025. 11 Websites were included in the analysis as one organization had two websites in the most popular search in July. Out of the 11 websites and 10 organizations, only one website was by an organization from outside of the USA. Only one website had no subdomains, brochures or additional information available.

Compared to the February 2025 search, two websites in the list of the 10 most recommended were replaced, and two others moved down in ranking.

3.2. Key features of websites

Most websites featured an intuitive layout with straightforward navigation and easy access to subdomains and supplementary informational materials where available. The structure and flow of the content were generally clear and user-friendly. Of the websites analyzed, nine were operated by non-profit organizations, while only one was owned by a for-profit entity (10%). Additionally, three websites were affiliated with major healthcare systems. Out of the 10 websites, only 50% had multilingual content available, with Spanish being the most common. Only three of the 10 (30%) main websites identified their authors, and a “last updated” date was available on the same websites. The average word count was 1625 including the main websites and all subdomains.

Multimedia use was prevalent across the websites, with only one site relying solely on text on both its main page and subdomains. All other websites incorporated multimedia elements on either their main page, subdomains, or both.

60% of the websites provided information on deceased kidney donation, though depth of content varied substantially across sites (Table 1).

Table 1.

Characteristics of the top 10 U.S.-based kidney donation websites, including testimonials, peer engagement features, multimedia and multilingual content, tone and sentiment, readability, and deceased donor information.

Organization	Country	Testimonials and engagement	Multimedia	Multilingual content	Tone	Sentiment	Readability grade level	Deceased donor information
National Kidney Foundation	USA	Testimonials and peer connection present	Present	Yes	50% Neutral	Empowering	10-12th	Limited
					40% Positive	Reassuring
					10% Negative	educational
National Kidney Registry	USA	Testimonials present, no peer connection	Present	Yes	15% Neutral	Promotional	College	None
					75% Positive	Inspiring
					10% Negative	Empowering
American Kidney Fund	USA	Testimonials present, no peer connection	Present on subdomain	Yes	45% Neutral	educational	10-12th	Present
					45% Positive	Balanced
					10% Negative	supportive
Johns Hopkins Medicine	USA	Testimonials present, no peer connection	Present	Yes	25% neutral	Inspiring	College	Present on subdomain
					65% positive	Supportive
					10% negative	Authoritative
University of Wisconsin	USA	Testimonials present, limited peer connection	Present	Yes	30% neutral	Authoritative	College	Limited, on subdomain
					65% positive	Empowering
					5% negative	Supportive
United Network for Organ Sharing	USA	Testimonials present, no peer connection	Present	No	70% neutral	Educational	College	Present
					20% positive	Cautious
					10% negative	Balanced
Alliance for paired Kidney Donation	USA	Testimonials present, no peer connection	None	No	15% neutral	Inspiring	College	None
					80% positive	Authoritative
					5% negative	Supportive
National Health Service	UK	Testimonials present, no peer connection	Present	No	50% neutral	Educational	10-12th	Present
					45% positive	Reassuring
					5% negative	Balanced
University of Michigan	USA	Testimonials and peer connection present	Present	Yes	30% neutral	Encouraging	10-12th	None
					60% positive	Supportive
					5% negative	Educational
American Medical Association	USA	No testimonials or peer connection	Present	No	80% neutral	Inspiring	10-12th	None
					15% positive	Reassuring
					5% negative	Empowering

Abbreviations: UK = United Kingdom; USA = United States of America; UW = University of Wisconsin. Sentiment and tone were classified (neutral, positive, negative) using Claude 4.0. Readability reflects the mean Flesch–Kincaid, SMOG, and Gunning Fog scores. Testimonials were defined as first-person donor or recipient narratives, peer engagement as interactive or connection-enabling features, and deceased donor information as present, limited, or absent.

3.3. Consensus and accuracy

A high level of consensus was observed, defined as eight or more websites presenting similar information on kidney donation. Overall, the accuracy of information across the included websites was high, and no significant misinformation was identified. However, variability was noted in the level of detail and depth provided, likely reflecting the varying priorities and target audiences of the website creators. Two websites deviated from the general pattern observed across the sample; one presented the information in a less structured format with limited donor-specific detail, while the other lacked sufficient depth overall. However, no misinformation was identified.

3.4. Testimonials and engagement

Only one website (10%) lacked testimonials on both its main site and subdomains. The extent of testimonial availability varied across websites. Three websites (30%) were classified as having a high level of testimonial content, characterized by robust donor networks and opportunities for prospective donors to connect with previous donors. These sites also offered peer or mentorship programs for potential donors.

Across all analyzed websites, engagement features were limited. No site offered interactive tools for user comments or real-time communication, and none met criteria for High engagement, per predefined thresholds. Social media links were present on all sites (100%), whereas direct “share” functions (e.g., email or platform-specific buttons) appeared on half of the sites (50%). Accordingly, all websites were classified as low engagement. This definition does not capture functional engagement features such as decision aids or interactive tools.

3.5. Readability of websites

Readability of the websites was successfully assessed using Claude AI. There was minimal variation between scores generated from analyzing the full website URLs versus the corresponding text transcripts. Specifically, the difference in readability scores was less than 1% across the Flesch-Kincaid Grade Level, the Gunning Fog Index, and the SMOG formula, indicating consistency in the assessment method.

The average readability scores across all websites and subdomains were as follows: Flesch-Kincaid Grade Level of 10.75, SMOG Index of 11.9, and Gunning Fog Index of 13.17. The combined average readability score was 12.3, reflecting a high school to college-level reading difficulty. Only one subdomain demonstrated a reading level below 8th grade. The most observed reading level was between 10th and 12th grade. Notably, 34% of the websites were written at a freshman college reading level or higher. When comparing main pages to subdomains, the average reading grade level was comparable, with main pages averaging 12.97 and subdomains 12.19 (Figure 1).

Figure 1.

Distribution of grade-based readability levels across kidney donation websites. Readability was assessed across each website’s main landing page and associated subdomains. Scores were calculated as the average of three validated readability metrics (Flesch–Kincaid Grade Level, SMOG, and Gunning Fog) and categorized into four grade-level groups: 6th–8th, 8th–10th, 10th–12th, and college level. Values represent the proportion of websites falling within each category.

The websites from the three health systems demonstrated a range of readability levels: one had a below-average score (10.1), one was approximately average (12.6), and one exceeded the overall average (14.1). This variation highlights the inconsistency in readability even among institutional sources.

The only non-U.S.-based website included in the analysis had an average readability score of 10.7. A difference was observed between websites that offered multilingual content and those that provided information only in English. Websites with multilingual information had a higher average readability score of 13.07, compared to 11.74 for English-only websites, suggesting that multilingual sites may present more complex language or content structure.

3.6. Sentiment analysis

In the sentiment analysis, there was again no significant difference between results obtained using Claude AI with full-text transcripts versus direct website URLs. The classification of overall tone into positive, neutral, or negative categories was consistent across both input methods, demonstrating high reliability of sentiment detection.

The most frequently identified sentiment was positive, accounting for 48.5% of the analyzed text, followed by neutral at 44.3%. Only 6.3% of the content was categorized as having a negative tone (Figure 2).

Figure 2.

Average sentiment distribution across kidney donation websites based on AI-driven textual analysis. For each website, the percentage of content categorized as neutral, positive, or negative tone was calculated across the main page and subdomains using Claude 4.0. These percentages were then averaged across all websites to produce the values shown.

The most positively framed website contained over 75% positive language, whereas the least positive website, with 15% of its language classified as positive, was mostly neutral (80%) and therefore did not convey an overall negative tone. Notably, no website exhibited more than 10% negative language use. In addition to the primary sentiment categories (positive, neutral, and negative), other commonly identified tones included educational, supportive, inspiring, and authoritative. Among these, the educational tone was the most frequently observed, appearing in content from more than 14 websites and subdomains (Figure 3).

Figure 3.

Frequency of dominant sentiment words across kidney donation websites. For each site, including all subdomains, AI identified the top three dominant sentiments from a predefined list of 40 sentiment descriptors representing a broad spectrum of tones. This figure shows the cumulative frequency of each sentiment word selected across all websites.

A promotional tone was identified in content from only two distinct website owners, one affiliated with a non-profit organization and the other with a health system, suggesting that overtly marketing-oriented language was relatively uncommon.

Claude AI also provided examples from the text for each of the sentiments identified.

4. Discussion

Deciding to donate a kidney is a major commitment, yet studies indicate that donors typically maintain or even experience improvements in quality of life.^23,24 While most donors report they would choose to donate again, some may still experience feelings of regret. ²⁵ This underscores the importance of providing potential donors with information that is comprehensive and tailored to their individual circumstances, including language and cultural background, to support truly informed decision-making.

Our analysis of top-ranked patient-facing websites highlights several key findings. First, the average readability grade of patient-facing websites was 12.3, which exceeds the recommended 6th–8th grade level. High readability may limit accessibility for individuals with lower literacy or non-native language speakers, potentially contributing to disparities in understanding and engagement with kidney donation information. By presenting information at a level that is too complex, websites risk reinforcing existing inequities in understanding donation options and participating in the transplant process. Stratifying by domain owner or subdomains revealed no significant differences in readability, and roughly one-third of sites were written at a college reading level. Addressing this issue through simplified language, plain-language summaries, or multilingual content could help reduce these barriers, improve comprehension, and support equitable access to donor education across diverse populations.

Second, engagement features were generally limited. In this study, engagement was assessed using a standardized rubric focused on publicly observable social features, such as comments, shares, and likes, to enable consistent comparison across websites. However, this operational definition does not capture all meaningful forms of user interaction. Several websites included high-value interactive elements, including decision aids, quizzes, personalized eligibility tools, downloadable educational materials, and links to donor mentorship programs. These elements were documented qualitatively but were not incorporated into the standardized engagement score. As a result, our engagement metric may underestimate certain types of user interaction that support informed decision-making, highlighting an important limitation and an area for refinement in future analyses.

Third, sentiment analysis revealed the overall tone of the websites was predominantly neutral or positive, with only a minority (6%) classified as negative. The most common sentiments identified by Claude AI were “educational”, followed by “supportive” and “inspiring”, suggesting that websites aim to inform while providing reassurance to potential donors. Using a predefined sentiment lexicon and standardized prompts improved the reproducibility of classification, while human review of AI outputs ensured plausibility and alignment with source content. Sentiment analysis may help website creators refine content, ensuring it is meeting their desired tone and readability level. Overly positive or negative information may underrepresent the challenges of kidney donation or present an unrealistic portrayal of the experience.

Coverage of living versus deceased kidney donation varied considerably. While most websites included information on deceased donation, detailed descriptions were rare. Emphasis on living kidney donation is understandable, as living donor transplants account for less than 30% of kidney transplants, yet generally achieve superior graft and patient survival compared with deceased donor transplants.^26–28 Nonetheless, deceased donation remains the predominant source of kidneys, and initiatives that make simplify registration have demonstrated substantial impact, such as in California where 95% of registered deceased donors enrolled via the DMV. ²⁹ Incorporating more comprehensive information on deceased donation, even within sections focused on living donation, could further encourage participation and improve overall organ donation rates.

Our findings build on prior work such as Garcia et al., who applied AI to evaluate patient-facing resources for pancreas transplantation, by demonstrating how AI can be used not only to assess readability and sentiment but also to identify gaps in engagement and content comprehensiveness. ³⁰ The application of AI in our study allowed for systematic, reproducible assessment of readability, sentiment, and engagement, highlighting specific gaps that can be addressed to improve patient-facing resources. Practical applications include:

• Simplifying language and clarifying medical terminology to lower reading grade levels.

• Incorporating interactive tools (scenario simulators, more decision aids, virtual assistants, discussion forums) to increase user engagement.

• Using AI-assisted sentiment review to balance tone, ensuring information is realistic yet tailored towards desired tone.

• Expanding content coverage for deceased donation to ensure both donation pathways are adequately represented.

Despite these findings, several limitations must be acknowledged. Pre-trained AI models may contain biases and may not fully capture nuances of language, culture, or patient experience. Search engine algorithms, device location, and settings influence which websites are accessed, and individual perception and interpretation of content are inherently variable. Nonetheless, standardized AI-supported evaluation offers a scalable framework for identifying gaps and informing improvements in public health education materials.

5. Conclusion

Patient-facing kidney donation websites are generally accurate and positively framed, yet gaps persist in readability, engagement, and content depth, particularly regarding deceased donation. AI-assisted analysis offers a standardized, scalable, and reproducible framework for evaluating online health information, enabling identification of patterns that might otherwise be overlooked.

The use of AI to tailor content to diverse audiences, optimize readability, refine tone, and produce multilingual versions has the potential to reduce barriers to access and make educational materials more inclusive. By addressing gaps in content, readability, and interactivity, patient-directed websites can play a crucial role in increasing awareness, fostering informed decision-making, and contributing to efforts to expand the kidney donor pool. Future research should assess how these improvements influence donor behavior across diverse populations and further validate AI’s role in optimizing patient education materials.

Supplemental material

Supplemental material - Generative AI and large language model evaluation of kidney donation websites: Benchmarking health literacy, sentiment, and digital engagement to optimize donor recruitment

Supplemental material for Generative AI and large language model evaluation of kidney donation websites: Benchmarking health literacy, sentiment, and digital engagement to optimize donor recruitment by Benjamin Bizer, Oscar Garcia-Valencia, Jose Arriola-Montenegro, Charat Thongprayoon, Jing Miao, Iasmina M. Craici, Wisit Cheungpasitporn in Digital Health.