Improving tuberculosis-related knowledge in tuberculosis patients: Protocol for the development and validation of an evidence-based Q&A robot powered by large language models

Abstract

Background

Inadequate health knowledge of tuberculosis patients is one of the causes of poor adherence among tuberculosis patients in China's tuberculosis control. In this study, we will develop and validate the effectiveness of a large language model (LLM) to improve the health knowledge of tuberculosis patients.

Methods

We will design a LLM application tailored to tuberculosis scenarios and evaluate its effectiveness in tuberculosis patient health education through a single-center, factorial-design randomized controlled trial. The study will feature a factorial design with two factors: LLMs-based health education model and a peer-intervention health education model, each with two levels (yes/no). A total of 148 tuberculosis (TB) patients in the intensive treatment phase will be randomly allocated to four groups through simple randomization. The primary outcome will be the patients’ level of personal health knowledge about tuberculosis, measured through questionnaires administered at discharge and three months later.

Conclusion

We are the first study in China to apply LLMs to tuberculosis health education. Tailored specifically for TB, our model uses certified guidelines and expert consensus to minimize inaccuracies. Large language models provide access to personalized, private health information, and reducing stigma. Instead of creating a new platform, we use the popular WeChat platform to deliver education via videos, text, and images, enhancing accessibility and engagement. This innovative approach aims to improve patient adherence and contribute to better TB management and disease control outcomes.

Introduction

Tuberculosis (TB) is a chronic infectious disease caused by Bacterium tuberculosis and remains a significant global public health challenge. ¹ In 2023, the global TB burden has surged dramatically, overtaking COVID-19 and reclaiming its position as the world's deadliest infectious disease. ² Without treatment, a person with active TB can infect an average of 10 to 15 individuals annually. ³ And the high morbidity and mortality associated with TB are more likely to occur in vulnerable populations in low- and middle-income countries. ⁴ China faces a substantial TB burden, ranking third among the 30 high-burden countries in 2023, accounting for 6.8% of global cases, following Indonesia (10%) and India (26%). ² Effective treatment and management are key to controlling the spread of TB and reducing morbidity and mortality to achieve the goals of the End TB Epidemic Strategy ⁵

In TB treatment, patient adherence is critical to treatment success.^6–9 In the 1990s, the World Health Organization (WHO) advocated for directly observed therapy to enhance patient adherence, but in practice, a substantial number of patients in China still self-manage their treatment. ¹⁰ According to the China National Tuberculosis Survey, 25.3% of TB patients missed more than one dose of their anti-TB medication. ¹¹ Research has demonstrated that educating TB patients can significantly improve treatment completion rates.^12,13 Health education aims to ensure that patients have sufficient knowledge and understanding to make informed decisions in their healthcare. ¹⁴ Through structured health education, patients can learn about the causes of TB, its transmission, treatment methods, and the serious consequences of nonadherence to treatment. Additionally, health education helps patients identify and manage potential side effects, enhancing their ability to cope with the disease and maintain a high level of adherence throughout the treatment process.^15,16 However, current health education for TB patients remains insufficient.^17,18 In a qualitative study, more than half of the nonadherent patients interviewed were found to lack essential knowledge about TB, despite all participants acknowledging that their doctors had provided them with relevant information. ¹⁹

Several existing approaches to TB patient health education have distinct advantages and disadvantages. Face-to-face and group education provide valuable opportunities for direct interaction, allowing immediate clarification of patients’ concerns and misconceptions, but are limited by time and resource constraints. ²⁰ Written materials and e-health education can reach a wider audience, yet their effectiveness is hindered by patients’ varying levels of comprehension and the quality of the information provided. ¹³ Telephone follow-ups offer personalized support but require substantial human resources to ensure long-term educational and monitoring effects. ²¹ One widely used approach to TB health education is peer intervention, which involves five key peer roles: peer counselor, peer educator, peer supporter, peer facilitator, and peer mentor. ²² The primary goal of peer interventions is to help TB patients better understand their disease and adhere to treatment. In recent years, a variety of digital technologies have been introduced into TB care, though most have focused on medication reminders and monitoring. Researchers have suggested that two-way communication-based chatbots in TB treatment may lead to improved self-management in patients with TB.^23,24 In this context, large language models (LLMs) have gained increasing attention in the field of patient health education. ²⁵ By drawing from vast amounts of medical knowledge and context, LLMs can accurately recognize and interpret patient symptoms and disease risks, enhancing patients’ understanding of their conditions. ²⁶ Furthermore, patients can access LLMs through digital platforms such as mobile apps, allowing for on-demand, scalable, and time-independent access to reliable health information and guidance. Despite this potential, few studies have explored the development and implementation of LLM-based tools specifically designed for TB patients.^23,27 Moreover, there remains limited evidence regarding the effectiveness of LLMs in enhancing knowledge among TB patients and the most effective combination of health education strategies.^25,28,29

Therefore, to address these gaps, our study includes two primary components: first, the development of an LLM application specifically designed for TB scenarios; and second, an evaluation of the optimal combination of health education methods and their effectiveness in improving TB patients’ knowledge through a factorial-design randomized controlled trial. Ultimately, our study aims to offer technical and policy support for the Global Strategy to End Tuberculosis.

Methods

Study setting and period

We will conduct this study from January 2025 to May 2026 at Shenzhen Third People's Hospital. Shenzhen, as China's first special economic zone and an emerging city with a large migrant population, faces significant challenges in TB screening and management. The Shenzhen Third People's Hospital is a designated specialist facility for TB and manages the diagnosis and treatment of approximately one-third of the city's registered TB patients.

Study design

The study will be conducted in two phases. We summarize the procedure of the study in Figure 1. The first phase involved developing a LLM application for TB scenarios. The second phase of the study aims to evaluate the effectiveness of an evidence-based Q&A robot powered by LLMs in TB patient health education, using a factorial design randomized controlled trial.

Figure 1.

Flowchart of the study.

Developing a knowledge base of TB guidelines and the Koch Q&A robot

The first step involves constructing a comprehensive knowledge base of TB guidelines. We will systematically review and select relevant clinical guidelines and expert consensus on TB from reliable sources, such as academic databases (e.g., PubMed, CNKI), professional organizations (e.g., WHO, Chinese Antituberculosis Association), and health departments. Given the varying timeliness of guidelines, we will include clinical guidelines and expert consensus from the past five years and group standards from the past 10 years. An appropriate search strategy will be formulated to establish inclusion and exclusion criteria for screening the guidelines, such as relevance to TB management, evidence-based quality (e.g., graded using AGREE II appraisal tool), and endorsement by recognized authorities. Selected guidelines will be validated by a panel of TB experts (including clinicians and researchers from Shenzhen Third People's Hospital and collaborating universities) to ensure accuracy, currency, and applicability before vectorization. This validation process includes cross-referencing with peer-reviewed literature and consensus-building discussions to exclude outdated or conflicting information. Subsequently, this phase involves two major subcomponents: (1) Knowledge base vectorization and (2) Evidence-Based Q&A processing. First, guideline documents are processed via embedding models such as Qwen or LLaMA2 using the LangChain framework (Steps ② and ③). These embeddings are subsequently used to construct a vector index using the LlamaIndex tool (Step ④), which is stored locally to serve as the underlying knowledge base. In the second stage, users input queries in natural language. The system utilizes LlamaIndex to perform semantic vector search and identify relevant content from the indexed library. The retrieved information, along with the original query, is then formatted into a prompt via LangChain and passed to an LLM (e.g., Qwen or LLaMA2) to generate a coherent, natural language response, which is then delivered back to the user. And, in honor of Robert Koch, the “Father of Infectious Diseases” who discovered Mycobacterium tuberculosis, we named the evidence-based Q&A robot “Koch.” The development of Koch will be led by a team of professors from the University.

Additionally, Koch will be integrated with the WeChat platform to facilitate seamless interaction with users through the WeChat interface. This integration will be achieved via WeChat Mini Programs development, enabling users to access the model directly within the app, or through open-source frameworks such as those available at https://github.com/zhayujie/chatgpt-on-wechat, which allow for secure, bot-based information delivery in a conversational format. The Koch system will store a comprehensive database and cache data, utilizing WeChat's existing social media infrastructure to connect efficiently with users. To ensure user data privacy and security, especially given WeChat's integration, all interactions will comply with China's Personal Information Protection Law and GDPR-inspired standards. User data (e.g., queries, responses, and metadata) will be anonymized, encrypted using AES-256 standards, and stored on secure, local servers with access restricted to authorized study personnel. No personal identifiers will be retained beyond what is necessary for trial purposes, and data will be processed in a privacy-by-design manner, with regular audits by an independent ethics committee. Informed consent will explicitly cover data handling, and users can opt out or request data deletion at any time.

Application development for Koch health education for TB patients

After the initial development of the LLM application platform, health education materials can be added in parallel. Healthcare professionals then manually evaluate the platform's output, checking for message consistency and assessing whether it meets the expected quality standards. The evaluation process for determining accuracy involves a team of three independent evaluators (two TB clinical experts and one AI specialist) who review a sample of 100 generated responses against predefined criteria based on certified TB guidelines and expert consensus. Accuracy is calculated as the percentage of responses that fully align with evidence-based sources, with a threshold of 90% required for deployment. Discrepancies among evaluators are resolved through a consensus meeting, where majority agreement is sought; if needed, a fourth senior expert arbitrates to ensure objectivity. Quality checks will be conducted biweekly during the three months of deployment, involving random sampling of 20 user interactions per check to monitor ongoing performance and make iterative improvements. If the model consistently generates answers with 90% accuracy and the content aligns with expectations, it can be put into clinical use. The Agent is a Koch-powered robot equipped with specialized functions to deliver health information to patients. It seamlessly integrates automation technology with a human touch to offer personalized management services. With interactive capabilities, patients can inquire about health issues anytime via the platform, and the Agent will instantly provide scientific and reliable answers through Koch. See Figure 2 for details.

Figure 2.

Development of a health education platform for tuberculosis patients with large language models.

The second phase consisted of a randomized controlled trial using a single-center, single-blind, parallel, factorial design to validate the effectiveness of a health education approach with the application of an LLM in improving the knowledge of TB patients. Participants were randomly assigned to groups in a 1:1 ratio to evaluate the optimal effectiveness of the health education approach.

Second phase: Randomized controlled trial with a factorial design

The study includes two interventions: LLMs and a peer intervention health education approach. Specifically, the study design included two factors—LLMs’ health education and peer intervention health education—each with two levels (yes or no), categorizing the study population into four groups (Table 1):

Group A (Standard care) will receive education based on the routine health education model for TB patients. The standard process for health education and management of TB patients involves several stages. After diagnosis, healthcare providers at designated medical institutions provide educational sessions to registered patients with pathologically confirmed TB. During hospitalization, patients may receive additional health education and exercise guidance from healthcare staff within their departments. In the later stages, patients primarily self-manage their condition, with regular follow-ups conducted by community healthcare managers. However, in practice, telephone or home visits are limited.

Group B (LLM health education) will receive regular health education supplemented with access to the LLM platform, allowing patients to seek immediate, evidence-based responses to their queries at any time. Patient responses are drawn from our preestablished database and are labeled with the source of the information to enhance patient confidence.

Group C (Peer intervention health education) will receive peer intervention health education in addition to regular health education. Patients in this group will be organized into peer groups of 7–10 individuals. There will be two roles of peer educators: First, TB specialists will serve as peer counselors, delivering TB-related knowledge, practical guidance, and tools to support patients in achieving their health goals. They will provide health education materials or videos monthly and conduct peer counseling sessions lasting approximately 20–25 min. During these sessions, patients will receive tailored verbal and written guidance based on their individual needs, aimed at addressing both general and personal barriers to treatment adherence and self-management. Second, recovered TB patients will serve as peer facilitators, sharing their personal experiences with patients after the counseling sessions to foster encouragement and mutual understanding. All peer counselors and facilitators will undergo standardized training in TB knowledge, communication skills, and motivational interviewing before the start of the intervention.

Group D (LLM + peer intervention health education) will receive routine health education combined with both LLM support and peer intervention.

Table 1.

Subgroups of randomized controlled trials with factorial design.

Group	Factors
	LLMs health education (+yes, -no)	Peer intervention health education (+yes, -no)
A	-	-
B	+	-
C	-	+
D	+	+

Participants and sampling

Shenzhen, as China's first Special Economic Zone and a rapidly developing city, has a large migrant population, placing a significant burden on TB screening and management. The Third People's Hospital of Shenzhen is the only National Clinical Medical Research Center for Infectious Diseases in the city. As the designated specialist hospital for TB, it is also responsible for managing one-third of all TB cases in Shenzhen. In this study, a randomized controlled trial with a factorial design will be conducted in the Third People's Hospital of Shenzhen to verify the effectiveness of different health education methods in improving the health knowledge of TB patients. Our study participants consisted of discharged TB patients who met the following criteria: (1) aged 18–50 years; (2) discharged patients in the intensive treatment phase (the first two months of the standard six-month TB treatment); (3) had no communication deficits (visual, auditory, or psychiatric) and were independently proficient in using a smartphone; and (4) had no other household members participating in the study.

The primary outcome measure of the study will be participants’ knowledge scores. Based on previous research, a 4.8-point increase in TB-related knowledge score was deemed clinically significant. ³⁰ Thus, we set a significance level of α = 0.05 (two sided) and a power of 0.8. Using the MOST-R package and a 2 × 2 factorial design, the required sample size was calculated as 128 participants, with 32 individuals per group. Accounting for a 15% attrition rate, the final sample size was 148, with 37 participants in each group. Each group will receive a three-month follow-up survey.

Randomization and blinding

Study participants will be randomly assigned to groups using a simple randomization method. Computer software will generate random numbers for each group, and patients will be assigned based on the corresponding number in the order of their discharge from the hospital. The grouping process is conducted by a research assistant before participant recruitment to ensure proper randomization. The study will employ a single-blind design, with data analysts remaining unaware of the participants’ group assignments. While an independent researcher explains the study's purpose to participants at discharge registration and obtains informed consent, and assign them to the appropriate group.

Data collection and variables

Data on study outcomes will be gathered from multiple sources, including the hospital information system and patient self-reports. All data will be stored and managed using the Research Electronic Data Capture (REDCap) system. Widely used in clinical research, REDCap offers features such as tracking data changes and full data de-identification. ³¹ It also has bank-level security, ensuring that only investigators and project members associated with the study have access.

The primary outcome variable in this study is the level of TB-related knowledge among TB patients (Table 2). The same set of questionnaires will be administered to all four patient groups at baseline during the intervention, and again three months postintervention. The questionnaire will be developed based on those used in previous studies and TB guidelines, and TB experts will review and refine the questionnaire to ensure its scientific validity and practical relevance.^30,32,33 It is expected to include 20–40 items covering topics such as TB infection knowledge, prevention and screening, treatment adherence, daily life practices for preventing transmission, and nutrition and exercise. The level of TB-related knowledge will be determined using a cumulative score: one point for each correct answer and zero points for incorrect or “not sure” responses.

Table 2.

Data variables.

Outcome variables	Variables	Description	Data collection tool
Primary outcome	Patient Health Knowledge Level	Extent of patient knowledge about TB prevention and treatment	Patient Questionnaire
Secondary outcome	User Satisfaction	Patient experience and feedback on usage	Patient Self-Report and Questionnaire
Other variables	Demographic information	Patient age, gender, education level, and household economic status	Data Collection Form, Hospital Information System

The secondary outcome is user satisfaction, will also be measured through a questionnaire using a Likert scale. Each dimension will have several items rated from 1 to 5, where 1 represents “very dissatisfied” and 5 represents “very satisfied.”

Data analysis

Statistical analysis will be conducted using IBM SPSS 26 software. First, we will use paired t-tests or the Wilcoxon test to evaluate changes in health knowledge levels before and after the intervention across the four groups. Second, we will perform descriptive analyses to examine differences in health education knowledge levels based on patient age, gender, and education level. Lastly, we will employ a generalized linear regression model to analyze the main effects of the intervention components on the quality of consultation and the interaction effects when combined. For the primary outcome indicators (continuous variables), which are normally distributed, a constant link function will be used. Results will be presented as mean differences with 95% confidence intervals. Covariates such as baseline health knowledge level, age, gender, annual income, place of residence, and education level will be controlled for in order to adjust for their potential impact and validate the intervention's effectiveness in TB patient health education.

For sensitivity analysis, missing outcome data will be addressed using multiple imputations to ensure the robustness of the primary analysis. Subgroup analyses of the primary outcomes will be conducted, including age, gender (male or female), place of residence (rural or urban), education level (undergraduate or higher, technical degree or high school, other), and per capita annual household income (<20,000; 20,000–100,000; >100,000).

Finally, we will assess user satisfaction with the application of the LLMs through a patient survey. This will gather user experiences and feedback to further refine and improve the models.

Discussion

We are the first study in China to apply LLMs to health education for TB patients, aiming to improve patient adherence. This study offers several key advantages. Firstly, our development approach and data training set are specifically tailored for TB. Rather than directly adopting existing LLMs, the model will be adapted to address the unique needs of the TB field. The data sources include widely certified guidelines and expert consensus, which will be further rated and refined to serve as Koch's training data. Such steps are designed to minimize “hallucinations” and prevent undesirable outcomes. To enhance the generalizability of our findings from this single-center study conducted at Shenzhen Third People's Hospital, we emphasize that the LLM-based Q&A robot (“Koch”) will be grounded in universally applicable, evidence-based guidelines from sources such as the WHO and Chinese national standards, which are not region-specific. This design will facilitate adaptation to other high-burden regions, such as those in Southeast Asia or Africa, by incorporating local language models, culturally relevant examples, and region-specific TB epidemiology data during deployment. For instance, the WeChat-integrated platform could be scaled via similar popular messaging apps in other countries (e.g., WhatsApp in India or Line in Southeast Asia), with minimal modifications to the underlying knowledge base. Future multicenter studies could validate these adaptations, potentially informing policy frameworks like the WHO's End TB Strategy by demonstrating cost-effective, technology-driven health education models that transcend single-site limitations.

Secondly, using an LLM application for health education has inherent benefits over traditional methods. One of the main advantages is that it will not be restricted by time or location, allowing patients to access study content whenever needed and receive personalized responses to their questions. ³⁴ Additionally, it will offer a discreet and private way for patients to engage with educational material, reducing feelings of discomfort or stigma associated with TB. ³⁵

Finally, only innovative technologies that enhance patient communication can effectively contribute to improved disease control outcomes. ³⁶ In our study, rather than creating a new application, we will leverage the existing and widely used WeChat platform. Health education will be delivered through various formats, including videos, text, and images, making it accessible and engaging for patients.

Several other limitations and challenges need to be noted. First, we cannot blind the health providers or patients, which is very likely to introduce biases. However, every effort will be made to ensure that healthcare providers deliver consistent care to all patient populations. Second, the current application of the LLM requires patients to have some proficiency in smartphone use. Also, patients with hearing or visual impairments will be excluded at this stage, which may limit the generalizability of the findings. Additionally, while we have implemented logging, review, and user-flagging mechanisms to detect and prevent misinformation (as detailed in the Methods section), these rely on active user participation and ongoing oversight, which could be resource-intensive.