Impact of Syndrome Differentiation on Treatment Effects and side Effects of Chinese Herbal Medicine in Randomized Controlled Trials: A Meta-Epidemiological Study

Abstract

Objective

In Traditional Chinese medicine (TCM), syndrome differentiation is considered to be an essential diagnostic process for tailoring Chinese herbal medicine (CHM) treatments to individual patients. We conducted a meta-epidemiological study to evaluate the potential impact of syndrome differentiation on improving effect estimates in randomized controlled trials (RCTs).

Study Design and Setting

Seven databases were searched to identify systematic reviews (SRs) published between 2021 and 2022 that pooled RCTs evaluating CHM treatment effects or side effects. Meta-epidemiological analyses were conducted using a two-step approach. Subgroup analyses were performed based on the disease category, outcome nature, and funding.

Results

We included 137 SRs comprising 2,064 RCTs. The results demonstrated small and statistically nonsignificant differences in both treatment effects (ROR: 0.93, 95% confidence interval (CI): 0.86 to 1.00, P = .04 for binary outcomes; dSMD: −1.44, 95% CI: −3.95 to 1.08, P = .26 for continuous outcomes) and side effects (ROR: 1.13, 95% CI: 0.66 to 1.92, P = .66) between RCTs that incorporated syndrome differentiation and those did not. Subgroup analysis showed RCTs of gastrointestinal disease that incorporated syndrome differentiation exhibited a larger magnitude of side effects than that did not. For binary outcomes, subgroup analyses of RCTs of circulatory diseases or those reporting subjective outcomes showed that syndrome differentiation led to a statistically significantly smaller treatment effect.

Conclusion

Individualizing CHM treatments through syndrome differentiation provide no significant evidence of improvement in TCM practice outcomes in RCTs. Furthermore, subgroup analyses suggest that syndrome differentiation may be associated with a smaller treatment effect among RCTs of CHM for circulatory diseases and a possible trend toward increased side effects in patients with gastrointestinal diseases.

Keywords

drugs Chinese herbal randomized controlled trial diagnosis differential epidemiologic studies meta-Analysis as topic treatment outcome

Introduction

Traditional Chinese medicine (TCM) has a long-standing history and is widely practiced and used in China¹ and in overseas Chinese communities.² Coinciding with the inclusion of a chapter on conditions recognized by TCM in the International Classification of Diseases (ICD-11) by the World Health Organization,³ syndromes and disorders derived from the paradigm of TCM have started to receive attention from the international medical community. In TCM, “syndrome,” also known as “pattern” or “Zheng”, refers to the pathologic changes at a specific stage of a disease and concerns their cause, nature, and location.⁴ Syndrome differentiation and treatment, also known as “bianzhenglunzhi”, is a classic diagnostic procedure and clinical decision-making approach guided by TCM theories. It involves four examinations: inspection, listening and smelling, inquiry, and palpation.^4,5

By analyzing the diagnostic information gathered through a range of syndrome differentiation methods, such as the “eight principal syndrome differentiation” approach, personalized treatment strategies can be developed to address the underlying cause of the condition. Patients who would all be diagnosed with the same disease in conventional medicine may be diagnosed with different TCM syndromes. Accordingly, patients are prescribed specific Chinese herbal medicines (CHMs) according to their TCM diagnosis. For example, two patients with functional dyspepsia may be diagnosed with different TCM syndromes, ie, liver qi invasion of the stomach or spleen–stomach qi deficiency,⁶ despite these two patients having the same diagnosis according to conventional medical standards. Commonly prescribed CHMs to treat the pathogenesis of these two TCM diagnoses are Xiao Yao pills and Xiang Sha Liu Jun Zi granules, respectively.⁷

CHM is one of the main TCM treatment modalities.⁸ Due to its widespread use in China and overseas, assessing the effectiveness of CHM is an important research topic. Well-designed randomized controlled trials (RCTs) are widely recognized in evidence-based medicine as the most reliable study design for evaluating treatment effectiveness.^9,10 In the past two decades, RCTs have emerged as the preferred method for evaluating the therapeutic effects of CHM. In the CONSORT (Consolidated Standards of Reporting Trials) Statement Extensions for Chinese Herbal Medicine Formulas 2017,¹¹ trialists are expected to report details on how syndrome differentiation is incorporated into the RCT design. For some TCM practitioners, syndrome differentiation is considered as a key step for identifying appropriate personalized CHM treatment strategies, based on centuries of practical experience and observations.¹² RCTs that do not incorporate syndrome differentiation are not considered as an accurate representation of TCM practice, limiting the validity and generalizability of the trials’ results.^5,13 It is believed that the absence of syndrome differentiation can lead to inappropriate patient and treatment selection,¹⁴ subsequently leading to an underestimation of the treatment effect.¹⁵

However, a systematic review of 2955 TCM trials from 1999 to 2017 revealed that only 25 trials (6.3%) evaluating Chinese patent medicines had incorporated the syndrome differentiation process into their design.¹⁶ In many RCTs on CHM, a specific CHM formula was chosen to treat a disease diagnosed using conventional medical definitions only. For instance, in a placebo-controlled trial of the efficacy of Xiao Yao pills for treating functional dyspepsia, patients might be eligible regardless of their TCM syndrome. Such a design is considered to be of low model validity,¹³ as the TCM indication of Xiao Yao pills, ie, a diagnosis of liver qi invading the stomach, is not included as an eligibility criterion for patient enrolment. Nevertheless, this has been the mainstream RCT design for CHM in the past two decades. Indeed, the prescription of TCM without syndrome differentiation is not uncommon. For example, the Chinese National Guideline for COVID-19 recommends the use of various CHM formulae based on syndrome differentiation, but many guidelines published by specific provinces and healthcare organizations suggest the use of a fixed CHM formula.¹⁷

To guide future trial design, it is necessary to determine whether the incorporation of syndrome differentiation influences the effect sizes of outcomes in CHM RCTs. We conducted a meta-epidemiological study to compare the magnitudes of the effects (both treatment and side effects) in CHM RCTs that (i) incorporated syndrome differentiation for tailoring CHM versus (ii) those that prescribed a fixed CHM based on a conventional disease diagnosis.

Methods

We report this study according to the guidelines for reporting meta-epidemiological research methodology.¹⁸ The protocol of this study was registered in PROSPERO (ID: CRD4202340761).¹⁹

Inclusion Criteria

In the Cochrane Handbook for Systematic Reviews of Interventions, a systematic review (SR) is defined as a structured review with a comprehensively planned process for identifying, evaluating, and synthesizing all evidence that meets predetermined eligibility criteria to answer a specific research question.²⁰ In the present study, eligible SRs were defined as those including at least one eligible meta-analysis (MA) that pools RCTs evaluating the treatment effects or side effects of CHM. An eligible MA was required to include both RCTs incorporating syndrome differentiation and RCTs that did not (ie, those that prescribed a fixed CHM based on a conventional disease diagnosis). MAs involving only RCTs that did or did not incorporate syndrome differentiation were excluded, as they would not have contributed to the analyses. We required the MAs to include at least 10 eligible RCTs to ensure sufficient statistical power for analysis.²⁰

If there were multiple eligible MAs, we selected the MA including the largest number of RCTs.²¹ MAs with overlapping trials were identified, and all duplicate RCTs were removed, starting from the MA with the largest number of RCTs and moving sequentially to the MA with the smallest number of RCTs. When this procedure resulted in an MA with fewer than 10 unique RCTs, that MA was excluded. This harmonization process was used to generate a dataset with no overlap between MAs.²² No eligibility criteria relating to participant characteristics or language restrictions were applied to the RCTs embedded within the MAs, but they were required to possess the following characteristics with regards to interventions, comparisons, and outcomes:

Interventions and comparisons: The interventions were required to have been orally administrated CHMs of any type, with at least one ingredient listed in the Chinese Pharmacopoeia (2020 edition). Other modalities of TCM practice, such as acupuncture and therapeutic massage (Tuina), were not included. Eligible comparisons were i) CHM versus conventional treatments; ii) CHM plus conventional treatments versus conventional treatments; and iii) CHM versus placebo. Within each MA, the comparisons of the included RCTs were required to be consistent with each other. This implies that each eligible MA can include only a single comparison, as outlined earlier, throughout all RCTs within that MA. When eligible RCTs were identified as having multiple arms, we selected a comparison consistent with other RCTs in the same MA.

Outcomes: The outcomes of interest for each RCT were the treatment effects and side effects of the CHM reported in the extracted MA, regardless of whether they were binary or continuous outcomes.

Exclusion Criteria

Overviews of SRs, network MAs, protocols, conference abstracts, animal studies, SRs not focusing on treatment effects or side effects of CHM, and SRs not published in Chinese were excluded. SRs that included only non-RCTs or did not include MAs were also excluded. When multiple versions of SRs were identified, the most up-to-date version was selected.

Literature Search

We searched for eligible SRs published between January 2021 and September 2022 in seven electronic databases: MEDLINE (Ovid), EMBASE (Ovid), the Cochrane Database of Systematic Reviews, China National Knowledge Infrastructure, WanFang, SinoMed, and Airiti Library. We applied specialized filters for SRs in MEDLINE²³ and EMBASE.²⁴ No restrictions on publication status were imposed. The detailed search strategy is shown in Appendix 1.

Literature Screening and Data Extraction

After the literature search, all citations were imported into EndNote version 20 for de-duplication. The titles and abstracts of the retrieved citations were then screened against the eligibility criteria. The full texts of the potentially eligible SRs were downloaded for further evaluation. Before inclusion, the full texts of the embedded RCTs retrieved from potentially eligible MAs were extracted and assessed for eligibility. If the full text of an RCT was unavailable, we excluded the RCT.

Prior to the formal launch of the study, training and calibration among the reviewers were conducted until good inter-rater agreement was reached. Literature screening, data extraction, and risk-of-bias assessment were carried out by one reviewer. A second reviewer independently duplicated the literature screening, data extraction, and risk-of-bias assessment in a random sample of one fifth of the trials. Any discrepancy was resolved by both reviewers checking against the final publication. A third reviewer was consulted to address unresolved discrepancies. The bibliographical characteristics of the included MAs and embedded RCTs were extracted using a prespecified data extraction form. The details are shown in Appendix 2. Binary and continuous outcome data regarding the treatment effects and side effects were also extracted and were classified as objective or subjective.

Assessment of the use of Syndrome Differentiation

We considered an RCT as having incorporated syndrome differentiation when the treatment strategies for patients were tailored based on TCM diagnostic theories.^4,5 The detailed operational definitions are as follows:

An RCT incorporated syndrome differentiation if it used any of the following designs:

Patients recruited to the RCT were diagnosed with the same TCM syndrome type at baseline, and they received the same CHM prescription according to their common TCM syndrome type.

Patients recruited to the RCT were diagnosed with different TCM syndrome types at baseline, and they received different CHM prescriptions from a fixed selection according to their specific TCM syndrome type.

Patients were assessed by a TCM practitioner at baseline and received an individualized TCM syndrome type diagnosis. Tailored treatments were then prescribed according to the syndrome type, with possible adjustment during the course of treatment.

An RCT did not incorporate syndrome differentiation if it used either of the following designs:

Patients recruited to the RCT were not diagnosed with any TCM syndrome type at any time point, and they received the same CHM prescription based on their conventional medicine diagnosis.

Patients recruited to the RCT were diagnosed with different TCM syndrome types at baseline, but they received the same CHM prescription based on their conventional medicine diagnosis.

Risk-of-Bias Assessment

We assessed the risk of bias of the included RCTs using the Cochrane Collaboration's risk-of-bias tool,²⁵ which evaluates the performance of each RCT in the following domains: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of the outcome assessment, incomplete outcome data, and selective reporting. Each domain was assessed as having a low, unclear, or high risk of bias.

Data Analysis

The results of descriptive analyses were presented as frequencies (percentages) for categorical outcomes and medians (ranges) for continuous outcomes. Chi-square tests were used to evaluate the differences between different types of RCTs. We analyzed the association between syndrome differentiation and effect sizes using the two-step approach described by Sterne and colleagues.²⁶ All analyses were stratified by whether the outcome was binary or continuous.

The first step involved the calculation of effect sizes within each MA. We calculated the odds ratios (ORs) for binary outcomes and standardized mean differences (SMDs) for continuous outcomes. The direction of each outcome indicator was recoded so that an OR > 1 or SMD > 0 indicated more favorable treatment effects or more severe side effects of CHM, as compared to the respective control interventions. For each MA, we estimated the differences in effect sizes between RCTs that incorporated syndrome differentiation and those that did not by using random effects meta-regression analyses.²⁷ The coefficients from each regression express the estimated differences in effect sizes between the two types of RCTs. The differences in effect sizes are expressed as ratios of odds ratios (RORs) for binary outcomes and differences in standardized mean differences (dSMD) for continuous outcomes. An ROR > 1 or a dSMD > 0 indicates that the RCTs incorporating syndrome differentiation yielded larger treatment effects or more severe side effects of CHM than those that did not.

In the second step, among the MAs, each ROR or dSMD was combined by applying both fixed and random-effect MA models. We used the I² statistic and the between-MA variance τ² to assess heterogeneity across differences in RORs or dSMDs.^28,29 An I² value <25% was considered to indicate a low level of heterogeneity, 25%-50% to indicate a moderate level of heterogeneity, and >50% to indicate a high level of heterogeneity.²⁹ A two-tailed P-value of <.05 was considered to indicate significance, except for heterogeneity testing, for which a threshold of <0.10 was used.³⁰

Subgroup and Sensitivity Analysis

Clinical conditions, the nature of outcomes (objective vs subjective), and funding support (with funding vs no funding information reported) were prespecified factors for consideration in subgroup analyses. We carried out interaction tests using a random-effects meta-regression model to assess whether the differences in effect sizes varied by these subgroups.

To control for potential confounding, we adjusted the meta-regression models for important trial characteristics: sample size, funding support of RCTs, and the six risk-of-bias domains. In these analyses, we re-evaluated the combined differences in effect sizes. on 4.2.2. Descriptive analyses were conducted using IBM SPSS Statistics version 24.0, and meta-epidemiological analyses were run in R versi

Results

Literature Screening and Selection

A total of 6,140 records were retrieved through the database searches. After excluding 2,158 duplicates, a further 2,851 records were removed through screening of the titles and abstracts. Then, a further 838 SRs were excluded after full-text assessment, and an additional 156 SRs were excluded after assessing the eligibility of the embedded RCTs. Finally, 137 SRs that met the eligibility criteria were included. Details of the literature screening and selection process are presented in Figure 1. A full list of included SRs is provided in Appendix 3.

Figure 1.

Flowchart of literature search and selection (from Jan 2021 to Sep 15, 2022).

Characteristics of the Included SRs and RCTs

The 137 SRs encompassed a total of 2,064 embedded RCTs, involving 187,503 participants (Tables 1 and 2). The included SRs covered a wide range of clinical conditions, including diseases of the circulatory system (35, 25.5%), diseases of the genitourinary system (27, 19.7%), diseases of the digestive system (23, 16.8%), endocrine, nutritional or metabolic diseases (10, 7.3%), and other conditions (42, 30.7%) (detailed in Table 1). A total of 84 (61.3%) SRs were supported by non-industry funding. For the risk-of-bias assessment, a large proportion of the SRs (105, 76.6%) used the Cochrane risk-of-bias tool, 16 (11.7%) used the Jadad scale, 15 (10.9%) used both the Cochrane risk-of-bias tool and the Jadad scale, and only one (0.7%) applied the Cochrane risk-of-bias tool 2.0. The median number of eligible RCTs per MA was 14, ranging from 10 to 36. All MAs reported treatment effect outcomes. The majority of these treatment effect outcomes were binary (110, 80.3%) and subjective (109, 79.6%) in nature. For side effects, only 11 (8%) MAs pooled data on adverse events related to CHM, and all of these outcomes were binary and subjective. Detailed characteristics of the included SRs are shown in Table 1.

Table 1.

General Characteristics of the 137 Systematic Reviews.

Characteristics	Results*
Characteristics of the systematic reviews
Year of publication
2021	94 (68.6)
2022	43 (31.4)
Clinical conditions classified by ICD-11
Diseases of the circulatory system	35 (25.5)
Diseases of the genitourinary system	27 (19.7)
Diseases of the digestive system	23 (16.8)
Endocrine, nutritional or metabolic diseases	10 (7.3)
Other conditions^#	42 (30.7)
Funding support
Non-industry support	84 (61.3)
Both industry and non-industry support	1 (0.7)
Not reported	52 (38.0)
Risk-of-bias assessment tool applied
Cochrane risk-of-bias tool	105 (76.6)
Both Cochrane risk-of-bias tool and Jadad scale	16 (11.7)
Jadad scale	15 (10.9)
Cochrane risk-of-bias tool 2.0	1 (0.7)
Characteristics of meta-analyses conducted in the systematic reviews
Median number of eligible RCTs included per meta-analysis, range	14, 10-36
No. of meta-analyses reporting treatment effects	137 (100.0)
Treatment effect data
Binary outcome	110 (80.3)
Continuous outcome	27 (19.7)
Treatment effect nature
Subjective outcome	109 (79.6)
Objective outcome	28 (20.4)
No. of meta-analyses reporting side effects	11 (8.0)
Side effect outcome data
Binary outcome	11 (100.0)
Continuous outcome	0 (0.0)
Side effect outcome nature
Subjective outcome	11 (100.0)
Objective outcome	0 (0.0)

*Values are frequency numbers (percentages) unless stated otherwise; ^#Other clinical conditions cover 11 categories, including mental, behavioral or neurodevelopmental disorders (8, 5.8%), sleep-wake disorders (6, 4.4%), diseases of the musculoskeletal system or connective tissue (5, 3.6%), diseases of the nervous system (4, 2.9%), diseases of the immune system (4, 2.9%), diseases of the respiratory system (4, 2.9%), certain infectious or parasitic diseases (3, 2.2%), neoplasms (3, 2.2%), symptoms, signs or clinical findings, not elsewhere classified (2, 1.5%), injury, poisoning or certain other consequences of external causes (2, 1.5%), and diseases of the visual system (1, 0.7%). RCTs: randomized controlled trials; ICD-11: International Classification of Diseases 11th Revision.

Table 2.

General Characteristics of the 2,064 Randomized Controlled Trials *.

Characteristics	Overall(N = 2,064)	RCTs Not Incorporating Syndrome Differentiation (n = 1,015)	RCTs Incorporating Syndrome Differentiation (n = 1,049)	P^#
Median year of publication, range	2016, 1997-2021	2016, 1997-2021	2017, 1997-2021	<.001
Median follow-up duration for the outcome of interest^a, range (unit: weeks)	8, 0.14-114	8, 0.14-92	8, 0.29-114	.341
Total number of participants included in primary studies	185,333	95,982	89,351	-
Median sample size, range	80, 28-560	82, 28-560	78, 30-480	<.001
Publication language
Chinese	2,060 (99.8)	1,013 (99.8)	1,047 (99.8)	1.000
English	4 (0.2)	2 (0.2)	2 (0.2)	1.000
Types of comparisons
CHM ＋conventional treatments versus conventional treatments	1,377 (66.7)	685 (67.5)	692 (66.0)	.464
CHM versus conventional treatments	687 (33.3)	330 (32.5)	357 (34.0)	.464
Types of CHM
Prescribed Chinese materia medica	1,625 (78.7)	774 (76.3)	851 (81.3)	<.001
Chinese patent medicine	231 (11.2)	159 (15.7)	72 (6.9)
Prescribed CHM Granules	125 (6.1)	41 (4.0)	84 (8.0)
CHM extracts	3 (0.1)	3 (0.3)	0 (0.0)
Not reported	77 (3.7)	37 (3.6)	40 (3.8)
Types of RCTs
Two-arm	1,993 (96.6)	981 (96.7)	1,012 (96.5)	.825
Multi-arm	71 (3.4)	34 (3.3)	37 (3.5)	.825
Number of centers
Single-center	1,921 (93.1)	937 (92.3)	984 (93.8)	.010
Multi-center	49 (2.4)	19 (1.9)	30 (2.9)
Not reported	94 (4.6)	59 (5.8)	35 (3.3)
Funding support
Non-industry support	380 (18.4)	139 (13.7)	241 (23.0)	<.001
Industry support	1 (0.1)	1 (0.1)	0 (0.0)
Not reported	1,683 (81.5)	875 (86.2)	808 (77.0)

*Values are numbers (percentages) unless stated otherwise; RCTs: randomized controlled trials; CHM, Chinese herbal medicine; ^aMedian follow-up duration for the outcome of interest refers to the median duration between baseline and follow-up when the outcome is being measured, as reported in the trial. ^#P values associated with cluster adjusted χ2 or t tests.

The characteristics of the included RCTs are shown in Table 2. Among the 2,064 RCTs, 1,049 (50.8%) incorporated syndrome differentiation. The median year of publication for RCTs was 2016, ranging from 1997 to 2021. Across all RCTs, the median sample size was 80, ranging from 28 to 560 participants. RCTs incorporating syndrome differentiation had a smaller sample size than those that did not (P < .001). All but four RCTs (2,060, 99.8%) were published in Chinese. The comparisons evaluated in the RCTs included i) CHM versus conventional treatments (687 RCTs, 33.3%) and ii) CHM plus conventional treatments versus conventional treatments (1,377 RCTs, 66.7%). The most commonly prescribed CHM treatment type was Chinese materia medica (1,625 RCTs, 78.7%), followed by Chinese patent medicine (231 RCTs, 11.2%). Chinese materia medica were more commonly prescribed in RCTs incorporating syndrome differentiation than in those that did not (81.3% vs 76.3%, P < .001). The majority of the RCTs (1,993, 96.6%) had a two-armed design, and 93.1% of the RCTs were single-center studies. Finally, 81.5% of the RCTs did not disclose any funding support.

Risk of Bias of the Included RCTs

Regarding the risk of bias, the included RCTs performed well in the domain of incomplete outcome data, with 92.9% having a low risk of bias. However, the performances with respect to allocation concealment and the blinding of participants and personnel were unsatisfactory, with only 1.6% and 2.9% of the RCTs having a low risk of bias in these domains, respectively. In terms of random sequence generation and blinding of the outcome assessment, 50.8% and 77.8% of the included RCTs, respectively, were assessed as having an unclear risk of bias. Notably, 96.4% of the RCTs had an unclear risk of bias in selective reporting due to the unavailability of open-access registration information or protocols.

Comparing the RCTs incorporating syndrome differentiation with those that did not, those incorporating syndrome differentiation had a larger proportion assessed as having a low risk of bias in random sequence generation (52.0% vs 36.0%, P < .001) and blinding of outcome assessment (25.2% vs 18.8%, P < .001). However, RCTs incorporating syndrome differentiation showed a poorer performance in the domain of incomplete outcome data (90.7% vs 95.3%, P < .001). Detailed results can be found in Table 3.

Table 3.

Risk-of-Bias Among 2064 Randomized Controlled Trials*.

Risk-of-Bias Domains and Risk Levels	Overall(N = 2,064)	RCTs Not Incorporating Syndrome Differentiation (N = 1,015)	RCTs Incorporating Syndrome Differentiation (N = 1,049)	P ^#
Random sequence generation
Low	910 (44.1)	365 (36.0)	545 (52.0)	<.001
Unclear	1,049 (50.8)	597 (58.8)	452 (43.1)
High	105 (5.1)	53 (5.2)	52 (5.0)
Allocation concealment
Low	34 (1.6)	12 (1.2)	22 (2.1)	.257
Unclear	1,925 (93.3)	950 (93.6)	975 (92.9)
High	105 (5.1)	53 (5.2)	52 (5.0)
Blinding of participants and personnel
Low	59 (2.9)	30 (3.0)	29 (2.8)	.664
Unclear	1,996 (96.7)	982 (96.7)	1,014 (96.7)
High	9 (0.4)	3 (0.3)	6 (0.6)
Blinding of outcome assessment
Low	455 (22.0)	191 (18.8)	264 (25.2)	<.001
Unclear	1,605 (77.8)	823 (81.1)	782 (74.5)
High	4 (0.2)	1 (0.1)	3 (0.3)
Incomplete outcome data
Low	1,918 (92.9)	967 (95.3)	951 (90.7)	<.001
Unclear	9 (0.4)	4 (0.4)	5 (0.5)
High	137 (6.6)	44 (4.3)	93 (8.9)
Selective outcome reporting
Low	0(0)	0(0)	0(0)	.658
Unclear	1,989 (96.4)	980 (96.6)	1,009 (96.2)
High	75 (3.6)	35 (3.4)	40 (3.8)

* Values are numbers (percentages) unless stated otherwise; Low: low risk-of-bias; unclear: unclear risk-of-bias; high: high risk-of-bias. ^#P values associated with cluster adjusted χ².

Differences in Treatment Effects and side Effects Between RCTs Incorporating Syndrome Differentiation and Those That did not

Figures 2 and 3 present the differences in binary and continuous treatment effects between RCTs incorporating syndrome differentiation and those that did not. The binary treatment effects of CHM were slightly smaller in RCTs incorporating syndrome differentiation (ROR: 0.93, 95% CI: 0.86 to 1.00, P = .04). However, although this P value was less than .05, the 95% CI of the ROR included the null value of 1, suggesting that the difference in binary treatment effects of CHM between RCTs incorporating syndrome differentiation and those that did not was not statistically significant. Similar findings were observed for the MAs of continuous treatment effects (dSMD: −1.44, 95% CI: −3.95 to 1.08, P = .26). For both types of treatment effects, heterogeneity among the MAs was low (I² = 0%, between-MA variance τ² = 0.00).

Figure 2.

Difference in binary treatment effects of Chinese herbal medicine between RCTs incorporating syndrome differentiation and those that did not.

Figure 3.

Difference in continuous treatment effects of Chinese herbal medicine between RCTs incorporating syndrome differentiation and those that did not.

Figure 4 shows the difference in the magnitude of side effects of CHM between RCTs incorporating syndrome differentiation and those that did not. There was no statistically significant interaction between RCTs incorporating syndrome differentiation and those that did not in terms of the magnitude of side effects (ROR: 1.13, 95% CI: 0.66 to 1.92, P = .66). Moderate heterogeneity was found among the MAs (I² = 48%, between-MA variance τ² = 0.28).

Figure 4.

Difference in the side effects of Chinese herbal medicine between RCTs incorporating syndrome differentiation and those that did not.

Subgroup and Sensitivity Analyses

Figure 5 presents the results of subgroup analyses based on different clinical conditions, the nature of outcomes, and funding support of SRs. In the subgroup of studies of circulatory system diseases, RCTs incorporating syndrome differentiation showed a smaller binary treatment effect of CHM than those that did not (ROR: 0.82, 95% CI: 0.69 to 0.97, P = .02). Among MAs of subjective outcomes, the binary treatment effects of CHM in RCTs incorporating syndrome differentiation were also slightly smaller than in those that did not (ROR: 0.91, 95% CI: 0.85 to 0.99, P = .02). For side effects, in the subgroup of studies of gastrointestinal diseases, RCTs incorporating syndrome differentiation reported a higher risk of side effects than those that did not (ROR: 2.45, 95% CI: 1.15 to 5.24, P = .02). For the remaining subgroups, there were no significant differences in treatment effects or side effects between the two types of RCT.

Figure 5.

Subgroup analyses based on different clinical conditions, nature of outcomes, and funding support of systematic reviews.

Figure 6 shows the results of sensitivity analyses, where adjustments were made for RCT sample size, RCT funding, and each risk-of-bias domain. For continuous treatment effects and side effects, adjusting for RCT sample size, RCT funding, and each risk-of-bias domain did not result in significant changes in the findings. For binary treatment effects, after adjusting for blinding of the participants and personnel (ROR: 0.92, 95% CI: 0.85 to 0.99, P = .03) or blinding of the outcome assessment (ROR: 0.92, 95% CI: 0.86 to 0.99, P = .03), a slightly smaller effect size was found in RCTs incorporating syndrome differentiation than in those that did not.

Figure 6.

Sensitivity analyses adjusting for RCT sample size, RCT funding, and each domain of risk-of-bias.

Discussion

Summary of Findings

We carried out a meta-epidemiological study of 137 SRs that included 2,064 RCTs to investigate the impact of syndrome differentiation on the treatment effects and side effects of CHM in RCTs. Overall, the magnitudes of differences in the treatment and side effects were small and nonsignificant, with low to moderate heterogeneity across the MAs. In sensitivity analyses, these findings remained unchanged after adjusting for sample size, funding support, and the RCT risk-of-bias domains of random sequence generation, allocation concealment, incomplete outcome data, and selective outcome reporting.

For binary outcomes, subgroup analyses of RCTs of circulatory diseases or those reporting subjective outcomes showed that the incorporation of syndrome differentiation led to a statistically significantly smaller treatment effect. This trend was also apparent, as expected, in the sensitivity analyses adjusting for blinding of the participants and personnel, as well as blinding of the outcome assessment. In the subgroup analysis of RCTs of gastrointestinal diseases, a larger magnitude of side effects was observed when syndrome differentiation was applied.

Possible Explanations of the Results

While our results suggest that the prescription of CHM in accordance with a conventional diagnosis tends to generate similar outcomes to those obtained when incorporating syndrome differentiation in RCTs, the rationale for choosing a specific CHM formula for RCT evaluation is expected to be influenced by syndrome differentiation. A WHO report³¹ mentioned that as COVID-19 patients often present with a common set of symptoms, they are often diagnosed with similar TCM syndromes. This commonality in TCM syndrome diagnosis allows the prescription of only a few specific CHM formula designs for the treatments of COVID-19. A similar observation can be noted regarding the relatively high prevalence of a few TCM syndrome diagnoses among patients with functional dyspepsia. In a cross-sectional diagnostic study, liver qi invading the stomach was found to be a common TCM syndrome diagnosis among patients with functional dyspepsia in routine practice.⁶ Interestingly, according to two network MAs,^7,32 liver qi invading the stomach is the target TCM syndrome of the best-performing CHM formulae for treating functional dyspepsia: Xiao Yao Pills, Modified Zhi Zhu Decoction, and Xiao Pi Kuan Wei Decoction. This coincidence may explain why these CHM formulae, which have the functions of liver qi soothing and spleen nourishment, outperformed all other CHM formulae. The RCTs included in these two network MAs did not incorporate syndrome differentiation. If the majority of the participants in these trials had experienced liver qi invading the stomach, as suggested by the results of diagnostic studies, then the use of Xiao Yao Pills, Modified Zhi Zhu Decoction, and Xiao Pi Kuan Wei Decoction would have been the most suitable options according to TCM theory.

In clinical contexts such as those above, individualized syndrome differentiation may be less important for conditions that mostly correspond to a particular TCM syndrome diagnosis, because the prescription of specific CHM formulae targeting that TCM syndrome would have a high chance of fitting many patients. This may explain our current results: RCTs enroll patients with a clear conventional diagnosis who often have a high pre-test probability of having a particular TCM syndrome diagnosis. Based on TCM experts’ clinical experience, the CHM formula chosen for these RCTs is typically one that is known to match this highly prevalent TCM syndrome. In this case, an explicit syndrome differentiation process in the RCT design may be deemed less relevant, as the CHM treatment design already implicitly considers the relevant TCM syndrome. As all RCTs either explicitly or implicitly consider TCM syndromes in their CHM treatment design, the lack of differences between the results of the two types of RCT studied here is not unreasonable.

Future Research Opportunities

As our meta-epidemiological study included a wide range of RCTs from different areas of medicine, it can be hypothesized that the above explanation may apply to many conventional conditions. However, we acknowledge that the role of TCM syndrome differentiation may vary across disease types, and that disease-specific factors could influence its methodological impact. To complement our broad analysis, future research should consider focusing on individual diseases to assess whether the observed patterns hold at a more granular level. In line with this, we have conducted a follow-up network meta-analysis on gastroesophageal reflux disease, a condition selected through expert Delphi consensus, to compare the effectiveness of CHM interventions with and without syndrome differentiation. The findings of this study, currently under review, may provide further insight into the disease-specific implications of syndrome differentiation in CHM trials.

Indeed, our current findings already suggest that the impact of syndrome differentiation may differ by disease area. For example, the incorporation of syndrome differentiation led to a smaller treatment effect among RCTs of CHM for circulatory diseases than those that did not incorporate syndrome differentiation. This observation may be attributable to the poor reliability of the TCM diagnostic procedure,³³ leading to a mismatch between the TCM syndrome and individualized CHM treatments. Aside from lower effectiveness, this mismatch may also cause a higher occurrence of side effects, as we observed among RCTs of gastrointestinal diseases. To address these challenges, the recent emergence of data-driven approaches for identifying TCM syndromes in a replicable and quantifiable manner is a promising solution to the deep-rooted problem of poor diagnostic reliability.³⁴ Its application may help researchers to better characterize the prevalent TCM syndrome diagnoses of patients with various diseases as defined in conventional medicine. Based on the most common TCM syndromes corresponding to each disease, CHM formulae that are shown by network MAs to have promising effects and that fit the corresponding TCM syndrome could be chosen for further RCT evaluation, or even routine practice.

To further improve the methodological rigor of TCM clinical research, future studies could explore more standardized and objective approaches to syndrome differentiation. For instance, patient-reported outcome measures have been developed for specific TCM syndromes, such as the TCM Spleen Deficiency Syndrome (TCM-SDS) scale and the TCM Kidney Deficiency Pattern (TCM-KDP) scale.^35,36 These tools offer a promising direction for enhancing diagnostic consistency and may facilitate the integration of syndrome differentiation into clinical trials in a more replicable manner.

Nevertheless, it is worth considering whether syndrome differentiation is always necessary in CHM trials. While it is widely accepted in TCM clinical practice that the absence of syndrome differentiation could lead to inappropriate patient and treatment selection, overemphasizing this concept in clinical research may hinder communication and acceptance among researchers from other medical traditions. Our findings suggest that the necessity of syndrome differentiation in CHM clinical research, especially in studies aiming for broader international acceptance, may be context-dependent. Exploring both syndrome-based and non-syndrome-based approaches in CHM trials could help bridge the gap between TCM and Western medicine paradigms, and expand the applicability of CHM interventions in global health research.

Strengths and Limitations

This meta-epidemiological study has several strengths. First, the inclusion of a large number of recently published SRs and MAs allowed us to cover a wide range of clinical conditions, which provides a representative sample for answering the research question. Second, this study's rigor is strengthened by the use of stringent inclusion criteria during the study selection process, ensuring a clear comparison between trials with and without the use of syndrome differentiation. Finally, the use of a standardized data extraction form and audit procedure minimized errors in the data extraction process.

There are also some limitations. First, although sensitivity analyses considering several potential confounders, ie, the sample size, funding support, and risk of bias of RCTs, were conducted, other unobserved confounding factors may still have existed and were not adjusted for. Second, the comprehensiveness of data extraction from existing publications may have been influenced by those publications’ reporting quality, and this issue reflects a need to promote better adherence to reporting guidelines in the research community. Third, minor adjustments of CHM ingredients based on patient-reported symptoms were not considered to qualify as the use of syndrome differentiation in this study, as we believe that such adjustments do not constitute a full use of TCM diagnostic theory. Future research may explore the potential impact of such adjustments on the overall outcomes and side effects. Lastly, the literature search for this study was conducted up to 2022. Given the rapid development of clinical research, more recent trials may have been published. However, due to the methodological nature and large scope of this meta-epidemiological analysis, which involved extensive screening, data extraction, and synthesis across multiple disease areas, updating the entire dataset was not feasible within the current project timeline. This is a common limitation in large-scale methodological research. Future research could build upon our findings using updated datasets to further validate and expand the current evidence base.

Conclusions

Overall, no significant difference was found in the magnitude of treatment effects or side effects between RCTs with and without the use of syndrome differentiation for guiding CHM treatment prescription. Individualizing CHM treatments based on syndrome differentiation does not improve the outcomes of TCM practice in RCTs, and its use may lead to reduced effectiveness or stronger side effects in patients with certain diseases. Research on the most prevalent TCM syndromes diagnosed among patients with a specific disease according to conventional medicine may allow the discovery of specific CHM treatments for particular conditions. Evaluating these specific treatments could be a way forward for developing CHM in an evidence-based manner.

Supplemental Material

sj-docx-1-chp-10.1177_2515690X261422020 - Supplemental material for Impact of Syndrome Differentiation on Treatment Effects and side Effects of Chinese Herbal Medicine in Randomized Controlled Trials: A Meta-Epidemiological Study

Supplemental material, sj-docx-1-chp-10.1177_2515690X261422020 for Impact of Syndrome Differentiation on Treatment Effects and side Effects of Chinese Herbal Medicine in Randomized Controlled Trials: A Meta-Epidemiological Study by Claire C.W. Zhong, Betty Huan Wang, Mary Y. Jiang, Irene X.Y. Wu, Leonard Ho, Fai Fai Ho, Shan Shan Xu, Ming Hong Kwong, Joson H.S. Zhou, Jason K.C. Lam and Vincent C.H. Chung in Journal of Evidence-Based Integrative Medicine

Footnotes

Abbreviations

Acknowledgments

Professional English language editing support was provided by AsiaEdit (asiaedit.com).

ORCID iDs

Claire C.W. Zhong

Betty Huan Wang

Authors’ Contributions

CZ: Data curation, formal analysis, methodology, software, visualization, writing - original draft; BW: Data curation, formal analysis, methodology, writing - review & editing; MY: Data curation, conceptualization, writing - review & editing; IW: Conceptualization, methodology, writing - review & editing; LH, FH, SX, MK, JZ, JL: Data curation, writing - review & editing; VC: Conceptualization, validation, project administration, supervision, funding acquisition, writing - review & editing. The authors read and approved the final manuscript. VC is responsible for the overall content as the guarantor.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Chinese Medicine Development Fund, (grant number 21B2_018A).

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Availability of Data and Materials

The data that support the findings of this study are available on request from the corresponding author.

Supplemental Material

Supplemental material for this article is available online.

References

Griffiths

Chung

Tang

. Integrating traditional Chinese medicine: Experiences from China, Australas. Medical J (Online). 2010;3(7):385.

Chung

Lau

Wong

Yeoh

Griffiths

. Views on traditional Chinese medicine among Chinese population: A systematic review of qualitative and quantitative studies. Health Expect. 2014;17(5):622–636.

International Classification of Diseases. Eleventh Revision (ICD-11). World Health Organization (WHO); 2023. https://icd.who.int/browse11/l-m/en .

World Health Organization (WHO). 2007. WHO international standard terminologies on traditional medicine in the western pacific region.

Jiang

Zhang

, et al. Syndrome differentiation in modern research of traditional Chinese medicine. J Ethnopharmacol. 2012;140(3):634-642.

Zhang

, et al. Quantification of prevalence, clinical characteristics, co-existence, and geographic variations of traditional Chinese medicine diagnostic patterns via latent tree analysis-based differentiation rules among functional dyspepsia patients. Chin Med. 2022a;17(1):101.

Zhong

CCW

Wong

CHL

, et al. Herbal medicine for functional dyspepsia: Network meta-analysis of placebo-controlled randomised trials. J Ethnopharmacol. 2022b;283(2022):114665.

Pei

. Innovating Chinese herbal medicine: From traditional health practice to scientific drug discovery. Front Pharmacol. 2017;8:381.

Higgins

Green

. 2008. Cochrane handbook for systematic reviews of interventions.

10.

Kabisch

Ruckes

Seibert-Grafe

Blettner

. Randomized controlled trials: Part 17 of a series on evaluation of scientific publications. Dtsch Arztebl Int. 2011;108(39):663-668.

11.

Cheng

Shang

, et al. CONSORT Extension for Chinese herbal medicine formulas 2017: Recommendations, explanation, and elaboration (traditional Chinese version). Ann Intern Med. 2017;167(2):W7-W20.

12.

Dong

. The relationship between traditional Chinese medicine and modern medicine. Evid-Based Complement Altern Med. 2013;2013(1):153148.

13.

Chung

. Incorporating traditional Chinese medicine syndrome differentiation in randomized trials: Methodological issues. Eur J Integr Med. 2016;8(6):898-904.

14.

Zha

Chang

. Pattern differentiation in traditional Chinese medicine can help define specific indications for biomedical therapy in the treatment of rheumatoid arthritis. J Altern Complement Med. 2009;15(9):1021-1025.

15.

Jiang

Yang

Zhang

, et al. Clinical studies with traditional Chinese medicine in the past decade and future research and development. Planta Med. 2010;76(17):2048-2064.

16.

Zhang

Tian

Zhao

, et al. The use of pattern differentiation in WHO-registered traditional Chinese medicine trials – A systematic review. Eur J Integr Med. 2019;30(2019):100945.

17.

Chung

. Chinese Medicine diagnosis and treatment for COVID-2019: Is China ready for implementing a national guideline? Adv Integr Med. 2020;7(2):51-54.

18.

Murad

Wang

. Guidelines for reporting meta-epidemiological methodology research. Evid Based Med. 2017;22(4):139-142.

19.

Zhong

Wang

Jiang

, et al. Influence of syndrome differentiation on treatment effect and safety effect estimates in randomized controlled trials of Chinese herbal medicine: A meta-epidemiological study. PROSPERO: International prospective register of systematic reviews. 2023.

20.

Higgins

JPT

Thomas

Chandler

, et al. (eds). 2022. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 Cochrane.

21.

Haring

Ghannad

Bertizzolo

Page

. No evidence found for an association between trial characteristics and treatment effects in randomized trials of testosterone therapy in men: A meta-epidemiological study. J Clin Epidemiol. 2020;122(2020):12-19.

22.

Savović

Harris

Wood

, et al. Development of a combined database for meta-epidemiological research. Res Synth Methods. 2010;1(3-4):212-225.

23.

Montori

Wilczynski

Morgan

Haynes

. Optimal search strategies for retrieving systematic reviews from Medline: Analytical survey. Br Med J. 2005;330(7482):68.

24.

Wilczynski

Haynes

Team

. EMBASE Search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews. J Clin Epidemiol. 2007;60(1):29-33.

25.

Higgins

Altman

Gøtzsche

, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. Br Med J. 2011;343:d5928.

26.

Sterne

Jüni

Schulz

Altman

Bartlett

Egger

. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med. 2002;21(11):1513-1524.

27.

Berkey

Hoaglin

Mosteller

Colditz

. A random-effects regression model for meta-analysis. Stat Med. 1995;14(4):395-411.

28.

DerSimonian

Laird

. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188.

29.

Higgins

Thompson

Deeks

Altman

. Measuring inconsistency in meta-analyses. Br Med J. 2003;327(7414):557-560.

30.

Ioannidis

. Interpretation of tests of heterogeneity and bias in meta-analysis. J Eval Clin Pract. 2008;14(5):951-957.

31.

World Health Organization (WHO). WHO Expert Meeting on Evaluation of Traditional Chinese Medicine in the Treatment of COVID-19. World Health Organization; 2022.

32.

Zhong

CCW

Wong

CHL

, et al. Chinese Herbal medicine for functional dyspepsia: A network meta-analysis of prokinetic-controlled randomised trials. Chin Med. 2021;16(1):140.

33.

Jacobson

Conboy

Tsering

, et al. Experimental studies of inter-rater agreement in traditional Chinese medicine: A systematic review. J Altern Complement Med. 2019;25(11):1085-1096.

34.

Zhang

, et al. Latent tree analysis for the identification and differentiation of evidence-based traditional Chinese medicine diagnostic patterns: A primer for clinicians. Phytomedicine. 2022c;106(2022):154392.

35.

Sun

Mao

Yan

Yang

. Patient reported traditional Chinese medicine spleen deficiency syndrome (TCM-SDS) scale for colorectal cancer: Development and validation in China. Integr Cancer Ther. 2021;20:15347354211020105. 10.1177/15347354211020105

36.

Yan

Yang

Ning

Yan

Sun

. Development and validation of the patient-reported traditional Chinese medicine kidney deficiency pattern scale for patients with colorectal cancer. J Trad Chin Med Sci. 2025;12(3):358-363. 10.1016/j.jtcms.2025.04.003

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB