Sage Journals: Discover world-class research

Abstract

Performance evaluation is important for improving medical quality and services. But, there is a lack of research for medical quality in traditional Chinese medicine (TCM) hospitals. This study examines the medical quality and various indicators of tertiary public traditional Chinese medicine hospitals in Gansu Province, to establish a foundation for improving the medical and management standards of these hospitals. This study collected performance assessment data from 10 tertiary TCM hospitals in Gansu Province from 2019 to 2022. Thirteen indicators with TCM characteristics were selected and categorized into 3 aspects: control of medical costs, internal operational dimensions, and comprehensive management. The level of medical quality in different hospitals and in different years were determined using the TOPSIS method for ranking and the RSR method for grading. Firstly, in terms of TCM characteristic indicators, hospital H had the highest control of medical costs and comprehensive management among different hospitals, with 45.87% and 24.20% respectively. The highest values for control of medical costs and comprehensive management were observed in 2020, with 40.65% and 18.69% respectively among different years. When evaluating the medical quality of different hospitals using the TOPSIS method, it was found that hospital H had the highest ranking from 2020 to 2022, with C_i values of 0.725, 0.778, and 0.667 respectively. Additionally, the RSR method indicated that hospital H had a high level of grading from 2020 to 2022, with P_i values of 0.687, 0.690, and 0.723 respectively. These findings suggest that the medical quality of hospital H is at a high and stable level of development. Based on the TOPSIS method to evaluate the performance appraisal results and ranking of different hospitals from 2019 to 2022. The results showed that the highest ranking was hospital B(C_i = 0.913) in 2019. The highest ranking was hospital C(C_i = 0.809)in 2020. The highest ranking was hospital D(C_i = 0.689) in 2021. The highest ranking was hospital J(C_i = 0.865) in 2022. The RSR method indicated that high grading level were hospitals B(P_i = 0.899),F(P_i = 0.795) in 2019. The highest grading level was hospital C(P_i = 0.809) in 2020. The highest grading level were hospitals A(P_i = 0.868), D(P_i = 0.813), E(P_i = 0.689), G(P_i = 0.873), J(P_i = 0.813), K(P_i = 0.842) in 2022. Based on the above results indicate that there is a large variation in the medical quality profile of different hospitals from 2019 to 2022. By comparing the results of TOPSIS and RSR method from 2019 to 2022, we found that the hospitals with identical ranking were D and J, and the hospitals with ≤2 difference in ranking was A,B,C,E in 2019, the hospitals with >2 ranking was A, F in 2020, the hospitals with >2 ranking were C, G in 2021, and the hospitals with identical ranking results were B,D,E,G,J in 2022. Comparing the ranking results of TOPSIS and RSR methods, showed that the hospitals with identical rankings were B, F from 2019 to 2022. The difference in ranking results ≤2 were A, C, D, E, G, H, J, K, indicating that high consistency between TOPSIS and RSR methods and credible results. The findings reveal significant fluctuations in medical quality across different years, while the overall level of medical quality remains relatively stable among the various hospitals. It is recommended that TCM hospitals focus on improving management efficiency, optimizing hospital operations, enhancing the utilization of medical resources, and fostering the efficient development of hospitals.

Keywords

Traditional Chinese Medical Hospitals TOPSIS method RSR method medical quality performance appraisal

What do we already know about this topic?

Previous studies have shown that evaluating medical quality is an important part of new medical reform and is necessary to solve the problem of limited and unbalanced medical resources.

How does your research contribute to the field?

This study aimed to evaluate the degree of excellence of medical quality and other indicators in tertiary public TCM hospitals in Gansu Province, evaluating the relative efficiency of hospital operations, and determine the influencing factors of hospital efficiency by the TOPSIS and the RSR method. Provide a basis for comprehensively improving the medical level and management level of tertiary public TCM hospitals in Gansu Province.

What are your research’s implications toward theory, practice, or policy?

The tertiary public TCM hospitals in Gansu Province have different developmental shortcomings in terms of medical quality, and need to take appropriate measures to continuously improve the quality of their medical services. The government should strive to narrow the gap in the efficiency of medical resources utilization in different regions. Hospital managers should improve their ability to promote the effective utilization of medical resources. This will result in improved medical technology ability and management ability.

Introduction

Medical quality played a crucial role in assessing the performance of tertiary hospitals, and enhancing the management levels was imperative for the continuous improvement of medical quality and services for the people.¹ Performance appraisal not only aligned with development goals, but also continuously improved the performance allocation scheme, leading to enhanced efficiency and motivation.² Therefore, performance appraisal has a significant value in improving medical quality and services.

Combined Chinese medicine and clinical practice could promote the ability of research for formulating strategic goals and fulfilling the public welfare duties.^3,4 The evaluation of Chinese medicine clinical served as a basis for formulating national health policies and promoting medical fairness in China.^5,6 The United States integrated Chinese medicine with clinical practice to provide restorative treatment for patients with long-term COVID infections.⁷ However, there is a lack of research on the performance appraisal of Traditional Chinese Medicine (TCM) hospitals in China.

Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) methodology determined the best option by performance scores and ranked.^8,9 TCM hospital performance appraisal is usually scored by a percentage system, exists 1-sided and cannot reflect the fairness of the assessment.¹⁰ Human and technological factors were the most important factors for medical tourism adoption in Malaysia by Fuzzy TOPSIS.¹¹ Base on the shortcomings of the traditional TOPSIS method, this paper introduces the principle of RSR, aiming at achieving the sorting of the relative closeness sought above, so that it can be more objective, comprehensive and real analysis and assessment. Rank-sum ratio (RSR) solved the problem of grading the evaluation object by combining the concepts and methods of parametric analysis.¹² RSR method are a relatively new way to evaluate TCM hospitals performance appraisals, which are effective methods frequently utilized in multi-objective decision analysis,¹³ the policy and measures implement in the healthcare reform promoted child healthcare in China by RSR.¹⁴ Currently, TOPSIS and RSR method were a relatively new way to evaluate TCM hospital performance appraisals, which were effective methods frequently utilized in multi-objective decision analysis.¹⁵ Health education oriented strategic model was the best model for managing schistosomiasis transmission by TOPSIS and RSR method.¹⁶ The effectiveness of community-based public health services was evaluated using the TOPSIS and RSR methods in Inner Mongolia Autonomous Region.¹⁷ The TOPSIS method and the RSR method were used to analyze the health resource allocation in the Chengdu-Chongqing from 2017 to 2021.¹⁸ To comprehensively evaluate have jointly promoted the development of child health care in China in the past 20 years by TOPSIS and RSR method, providing a scientific theoretical basis for future policy-making to promote child health care provide theoretical support for future decision-making on the policies of child health care in China.¹⁴ However, fewer studies have been conducted to evaluate the quality of Chinese hospitals using these 2 methods.

In this study, 13 performance assessment indicators in TCM hospitals from Gansu Province were chosen based on the Internal Performance Assessment and Evaluation System of tertiary Hospitals. Then, based on the Pilot Project: Establishing Performance Indicator Systems for Public Chinese Medicine Hospitals that Combine Western and Traditional Medicines developed by China, we categorized these indicators into 3 groups: Control of Medical Costs, Internal Operational Dimensions, and Comprehensive Management, a comparison of data from these 3 aspects reveals trends in medical quality changes and provides a dynamic view of hospital management. The strengths and weaknesses of medical quality were evaluated by TOPSIS and RSR method for identifying the areas for improvement, addressing shortcomings, and enhancing the medical and management levels. The results will provide data support for Chinese hospitals to improve medical quality.

Materials and Methods

Data Sources

We collected performance appraisal data from 10 tertiary TCM hospitals in Gansu Province from 2019 to 2022, with the names of the hospitals denoted by A, B, C, D, E, F, G, H, J, K. The data came from 6 hospitals (A, B, C, D, E, J) in 2019 and from 10 hospitals in 2020, 2021 and 2022.

Indicators Determination

The evaluation indicators adopted herein are based on the Pilot Project: Establishing Performance Indicator Systems for Public Chinese Medicine Hospitals that Combine Western and Traditional Medicines developed by China. To evaluate the performance appraisal of 10 tertiary TCM hospitals in Gansu Province from 2019 to 202l, we selected 13 indicators (X1-X13) including Control of medical costs (X1-X4), Internal Operational Dimensions (X5-X9) and Comprehensive management (X10-X13). X1 stand for the proportion of outpatient Chinese medicine prescriptions. X2 stand for the proportion of outpatient prescriptions for bulk and sachet Chinese divine tablets. X3 stand for the utilization rate of outpatient Chinese medicine tablets. X4 stand for the utilization rate of discharged patients’ Chinese medicine tablets. X5 stand for the proportion of outpatients using non-pharmacological therapies of traditional Chinese medicine (X6) stand for the proportion of discharged patients using non-pharmacological therapies of traditional Chinese medicine. X7 stand for the proportion of discharged patients who were mainly treated by traditional Chinese medicine. X8 stand for the proportion of medical service income (excluding income from medicines, consumables, examinations, and tests) to medical income. X9 stand for the proportion of income from traditional Chinese medicines to medicine income. X10 stand for the proportion of income from TCM to medicine income. X11 stand for the proportion of income from traditional Chinese medicine preparations to medicine income in medical institutions, X12 stand for the proportion of income from outpatient TCM medical service items to outpatient medical income. X13 stand for the proportion of income from inpatient TCM to hospitalization income.

Methods

TOPSIS

The TOPSIS method is a technique for order preference according to similarity to the ideal solution. Based on the normalized raw data matrix, this method forms a space for both positive and negative ideal solutions of priority solutions. The distance between a point and the positive and ideal solutions is obtained. This distance helps identify the relative closeness between the solution to be evaluated and the positive ideal solution and provides a basis for evaluating the advantages and disadvantages of this solution.^19,20 In order to ensure that the evaluation indicators have a positive trend, we transformed the 13 indicators of 10 hospitals with the same trend, normalized the indicators to find out the optimal and least optimal matrix vectors, calculated the distance of the indicators of each Chinese hospital from the positive and negative ideals (D+ and D−), and ranked the C_i value indicating the close to the optimal solution.

Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS).²¹ Basic steps:

(1) The same trending method was used for all the indicators.

(2) The original data matrix with the same trend was normalized.

Z_{i j} = \frac{x_{i j}}{\sqrt{\sum_{i = 1}^{n} x_{i j}^{2}}}

(1)

In the equation, x_ij represents the value of the i-th evaluation object of the j-th indicator. The normalized matrix Z can be presented as

Z = [\begin{matrix} Z_{11} & Z_{12} & \dots & Z_{1 m} \\ Z_{21} & Z_{22} & \dots & Z_{2 m} \\ \dots & \dots & \dots & \dots \\ Z_{n 1} & Z_{n 2} & \dots & Z_{n m} \end{matrix}]

(2)

(3) According to the normalized matrix Z, the positive ideal (optimal vector) and negative ideal solutions (worst-case vector) were calculated as follows:

Positive ideal solution: $Z^{+} = (z_{i 1}^{+}, z_{i 2}^{+}, \dots, z_{i m}^{+}) .$

Negative ideal solution: $Z^{-} = (z_{i 1}^{-}, z_{i 2}^{-}, \dots, z_{i m}^{-}) .$

(4) The weighted Euclidean distance between D+ i (positive ideal solution) and D− i (negative ideal solution) for each indicator values of every laboratory were calculated as follows:

D_{i}^{+} = \sqrt{\sum_{j = 1}^{m} {[ω_{j} (Z_{i j} - Z_{i j}^{+})]}^{2}}

(3)

D_{i}^{-} = \sqrt{\sum_{j \equiv 1}^{m} {[ω_{j} (Z_{i j} \cdot Z_{i j})]}^{2}}

(4)

(5) The relative closeness between the positive ideal and negative ideal solutions as well as the indicator values were calculated as follows:

C_{i} = \frac{D_{i}^{-}}{D_{i}^{+} + D_{i}^{-}}

(5)

The value of C_i was set between 0 and 1. C_i value close to 1 indicates a higher likelihood of positive ideal solution (optimal level), whereas a C_i value close to 0 indicates a higher likelihood of negative ideal solution (worst level).

RSR

The RSR method is a comprehensive evaluation and analysis method proposed by the Chinese statistician, Professor Fengtiao Tian in 1988. RSR method is performed by ranking the indicators to obtain the RSR value of the dimensionless statistic, which grades the evaluated objects depending on the RSR value. In this study, all indicators were high superiority indicators. Combining the principle of optimal grading and the table of reasonable grading numbers,¹⁷ the grading level is divided into upper, middle and lower levels, which are expressed as 3, 2, and 1.

Rank Sum Ratio (RSR) Basic steps:

(1) Data matrix

In order to calculate the computed P_i values, we ranked the raw data as a matrix of m×n, (where m denotes the hospitals and n is the 13 evaluation indicators).

(2) Rank

Then applies the parametric statistical analysis to determine a set of quantitative methods. and ranked them from smallest to largest for the high superiority indicators and from largest to smallest for the low superiority indicators.

(3) Calculation of RSR

In a matrix with n rows and m columns, the RSR was calculated as follows:

R S R_{i} = \frac{1}{m \cdot n} \sum_{j = 1}^{m} R_{i j}

(6)

(4) Determination of RSR distribution

The distribution of RSR refers to the specific downward cumulative frequency of RSR values expressed in the probability unit (P_i), the calculation of the P_i value is carried out and sorted into grades, and the larger the value, the higher the level.

Combination of TOPSIS and RSR methods

The TOPSIS method can be applied to the relative closeness value C_i, which ranges from 0 to 1, and the RSR value distributed in the same rank and ratio method can be analyzed using the RSR method. Here are the basic steps involved:

(1) Determination of RSR distribution for C_i. The RSR distribution of C_i refers to the speciﬁc downward cumulative frequency of C_i expressed in the Probit. Speciﬁc steps include compiling the frequency distribution table of the C_i values; listing the frequency f of different groups and calculating the cumulative frequency $Σ f$ of each group; determining the rank R, average rank R, and R/n values for each group of C_i values.

(2) Calculation of the regression equation.

Using Y as the independent variable and C_i value as the dependent variable, the regression equation becomes C_i = a+b×Probit

(3) Hypothesis testing of the regression equation.

Determine whether the C_i value is normally distributed and whether the regression equation is relevant.

(4) Rank by level.

The evaluation objects are ranked by level based on the reasonable RSR rank method and the corresponding estimated values obtained using the regression equation.

Statistical Analysis

The data was collected and organized using Excel. The 13 evaluation indicators were statistically described in the form of component ratios. The data was then synthesized and ranked using the TOPSIS and RSR methods of SPSS 22.0 statistical analysis software.

Results

Description of Indicators With Chinese Medicine Characteristics

In different hospitals, we found that the highest control of medical costs was H hospital (45.87%), and the lowest was D hospital (22.10%). The highest for the Internal Operational Dimensions indicator was G hospital (91.16%), and the lowest was D hospital (11.23%). The highest for the Comprehensive management Indicator was H hospital (24.20%), and the lowest was J hospital (1.33%; Table 1).

Table 1.

Describing Indicators of Chinese Medicine in Different Hospitals.

Hospitals	Control of medical costs		Internal operational dimensions		Comprehensive management
Hospitals	N	n (%)	N	n (%)	N	n (%)
A	10 350 894	4 578 127 (44.23)	892 211	423 484.00 (47.76)	705 824	116 860.0 (16.56)
B	6 545 184	2 568 816 (39.25)	2 303 349	358 506.12 (15.56)	339 193	22 453.62 (6.62)
C	6 434 201	2 727 509 (42.39)	2 025 606	324 658.93 (16.03)	191 558	23 154.78 (12.09)
D	3 519 457	777 827 (22.10)	1 982 496	222 565.15 (11.23)	193 224	23 085.07 (11.95)
E	5 442 235	2 062 639 (37.90)	2 212 038	304 696.26 (13.77)	173 347	15 947.94 (9.20)
F	1 652 044	473 397 (28.66)	570 287	91 321.90 (16.01)	52 299	8528.69 (16.31)
G	1 307 644	453 655 (34.69)	553 010	504 103.41 (91.16)	57 581	12 101.92 (21.02)
H	1 220 367	559 754 (45.87)	641 117	74 507.36 (11.62)	40 012	9764.08 (24.20)
J	4 454 245	1 415 096 (31.77)	1 964 240	323 209.65 (16.45)	218 710	2901.36 (1.33)
K	1 196 964	489 730 (40.91)	577 147	53 528.31 (9.27)	33 044	6502.12 (19.68)

Note. N stands for target value, n stands for actual value.

In different years, we found that the highest Control of medical costs was 2020 (40.65%), and the lowest was 2022 (36.30%). The highest for the Internal Operational Dimensions indicator was 2019 (87.50%), and the lowest was 2022 (9.22%). The highest for the Comprehensive management indicator was 2020 (18.59%), and the lows was 2019 (10.87%; Table 2).

Table 2.

Describing Indicators of Chinese Medicine From 2019 to 2022.

Years	Control of medical costs		Internal operational dimensions		Comprehensive management
Years	N	n (%)	N	n (%)	N	n (%)
2019	8 204 708	3 233 419 (39.41)	3 150 670.53	2 756 782.95 (87.50)	472 351.23	51 321.83 (10.87)
2020	10 261 521	4 171 006 (40.65)	3 799 203.41	905 362.28 (23.83)	423 449.64	78 721.90 (18.59)
2021	11 510 302	4 326 491 (37.59)	3 958 057.55	857 927.46 (21.68)	533 331.91	81 123.33 (15.21)
2022	12 146 704	4 409 730 (36.30)	5 278 304.79	486 646.57 (9.22)	482 306.53	75 182.34 (15.59)

Note. N stands for target value, n stands for actual value.

Evaluation of the Medical Quality in Different Hospitals

Ranking for medical quality by the TOPSIS method

Based on the concept of sorting after TOPSIS normalization process, we analyzed the ranking of 13 indicators with TCM characteristics in different hospitals from 2019 to 2022. We found that the highest ranked hospital in 2019 was C(C_i = 0.630), and the lowest ranked hospital was D(C_i = 0.265). The highest ranked hospital in from 2020 to 2022 was H(C_i = 0.725,0.778,0.667). The lowest ranked was F(C_i = 0.334) in 2020 and the lowest ranked was D(C_i = 0.287,0.216) in from 2021 to 2022. The indicated that higher ranking was H, and lower ranking was D from 2021 to 2022 (Table 3, Figure 1A).

Table 3.

Ranking of C_i Values of Different Hospitals by TOPSIS Method.

Hospitals	2019				2020				2021				2022
Hospitals	D+	D−	C_i	Rank	D+	D−	C_i	Rank	D+	D−	C_i	Rank	D+	D−	C_i	Rank
A	47.701	61.292	0.562	4	74.913	105.537	0.585	4	48.564	79.146	0.620	2	54.757	68.066	0.554	3
B	42.380	61.884	0.594	2	83.152	91.229	0.523	7	62.371	57.519	0.480	5	75.932	40.542	0.348	8
C	41.366	70.374	0.630	1	80.803	100.248	0.554	5	63.509	68.109	0.517	3	69.037	58.296	0.458	5
D	76.902	27.717	0.265	6	104.223	70.903	0.405	8	88.556	35.574	0.287	10	91.273	25.166	0.216	10
E	42.114	55.885	0.570	3	81.160	97.638	0.546	6	63.975	58.309	0.477	6	68.319	55.394	0.448	6
F	-	-	-	-	114.842	57.713	0.334	10	68.994	64.985	0.485	4	81.320	47.262	0.368	7
G	-	-	-	-	66.383	98.603	0.598	3	68.179	54.427	0.444	7	55.053	66.223	0.546	4
H	-	-	-	-	44.783	118.109	0.725	1	27.784	97.573	0.778	1	45.126	90.377	0.667	1
J	74.055	41.355	0.358	5	113.562	74.884	0.397	9	94.242	41.004	0.303	9	90.819	35.821	0.283	9
K	-	-	-	-	50.020	116.864	0.676	2	72.025	55.349	0.435	8	58.453	72.887	0.555	2

Note. D+: positive ideal solution distance; D−: negative ideal solution distance.

Grading for medical quality by the RSR method

Based on the relevant principles of RSR grading method, we analyzed the 4-year performance data analysis of 10 tertiary public TCM hospitals in Gansu Province. The results showed that, the highest grading level was B(P_i = 0.739) in 2019, the highest grading level was A(Pi = 0.772) and H(P_i = 0.687), the lowest was J(Pi = 0.364) in 2020. The top of grading level were A(Pi = 0.777) and H(P_i = 0.690) in 2021. The top of grading levels were A(P_i = 0.832) and H(P_i = 0.723) in 2022. The findings indicated that the gradings of hospitals A and H showed stable trend from 2019 to 2022 (Table 4, Figure 1B).

Table 4.

Grading of P_i Values of Different Hospitals by RSR Method.

Hospitals	2019				2020				2021				2022
Hospitals	Value	Rank	P_i	Grade	Value	Rank	P_i	Grade	Value	Rank	P_i	Grade	Value	Rank	P_i	Grade
A	0.679	3	0.599	2	0.70	1	0.772	3	0.785	1	0.777	3	0.800	1	0.832	3
B	0.692	1	0.739	3	0.50	8	0.459	2	0.546	6	0.526	2	0.440	8	0.435	2
C	0.679	2	0.657	2	0.577	5	0.557	2	0.485	7	0.493	2	0.480	7	0.478	2
D	0.436	6	0.448	2	0.354	9	0.419	2	0.431	9	0.417	2	0.290	10	0.314	1
E	0.538	4	0.552	2	0.569	6	0.525	2	0.569	4	0.575	2	0.550	6	0.519	2
F	-	-	-	-	0.546	7	0.493	2	0.569	4	0.575	2	0.570	5	0.559	2
G	-	-	-	-	0.615	4	0.591	2	0.577	3	0.634	2	0.630	4	0.602	2
H	-	-	-	-	0.7	2	0.687	3	0.723	2	0.690	3	0.710	2	0.723	3
J	0.474	5	0.506	2	0.3	10	0.364	1	0.362	10	0.316	1	0.360	9	0.384	2
K	-	-	-	-	0.638	3	0.631	2	0.454	8	0.458	2	0.670	3	0.653	2

Figure 1.

Comparison of differences in TOPSIS and RSR Methods Across Hospitals and Years. (A) Ranking of different hospitals by TOPSIS method, (B) Grading of different hospitals by RSR method, (C) Ranking of hospitals by using TOPSIS method from 2019 to 2022, (D) Grading of hospitals by using RSR method from 2019 to 2022.

By comparing the results of TOPSIS and RSR method from 2019 to 2022, we found that the hospitals with identical ranking was D and J, and the hospitals with ≤2 difference in ranking was A,B,C,E in 2019, the hospitals with >2 ranking was A,F in 2020, the hospitals with >2 ranking were C,G in 2021, and the hospitals with identical ranking results were B,D,E,G,J in 2022 (Table 5).

Table 5.

Comparing the Results of TOPSIS and RSR Method From 2019 to 2022.

Sorting results	2019	2020	2021	2022
Identical	2	2	2	5
Sorting gap ≤ 2	4	6	6	5
Sorting gap>2	0	2	2	0
Total	6	10	10	10

Evaluation of the Medical Quality From 2019 to 2022

Ranking for medical quality by the TOPSIS method

Based on the TOPSIS method to evaluate the performance appraisal results and ranking of different hospitals from 2019 to 2022. The results showed that the highest ranking was hospital B(C_i = 0.913) and the lowest was hospital E(C_i = 0.312) in 2019. The highest ranking was hospital C(C_i = 0.809) and the lowest was hospital J(C_i= 0.168) in 2020. The highest ranking was hospital D(C_i = 0.689) and the lowest was hospital K(C_i = 0.338) in 2021. The highest ranking was hospital J(C_i = 0.865) and the lowest was hospital B(C_i = 0.155) in 2022 (Table 6, Figure 1C).

Table 6.

Comprehensive Evaluation Results and Ranking of Hospitals by Using TOPSIS Method From 2019 to 2022.

Years	A		B		C		D		E		F		G		H		J		K
Years	C_i	Rank KKkKk	C_i	RankNKnk	C_i	Rank	C_i	Rank	C_i	Rank	C_i	Rank	C_i	Rank	C_i	Rank	C_i	Rank	C_i	Rank
2019	0.548	3	0.913	1	0.591	2	0.473	3	0.312	4	0.716	1	0.777	1	0.654	2	0.635	2	0.477	3
2020	0.737	1	0.627	2	0.809	1	0.442	4	0.547	1	0.644	2	0.548	2	0.667	1	0.168	4	0.536	1
2021	0.657	2	0.540	3	0.365	3	0.689	1	0.414	3	0.161	4	0.360	4	0.339	4	0.507	3	0.338	4
2022	0.314	4	0.155	4	0.346	4	0.627	2	0.438	2	0.285	3	0.537	3	0.354	3	0.865	1	0.504	2

Grading for medical quality by the RSR method

The performance appraisal indicators of different hospitals were evaluated from 2019 to 2022 grading levels. The indicated that high grading level were hospitals B(0.899), F(0.795) in 2019. The highest grading level was hospital C(0.809) in 2020. The highest grading level were hospitals A(P_i = 0.868), D(P_i = 0.813),E(P_i = 0.689),G(P_i = 0.873),J(P_i = 0.813),K(P_i = 0.842) in 2022 (Table 7, Figure 1D).

Table 7.

Comprehensive Evaluation Results and Grading of A,B,C,D by Using RSR Method From 2019 to 2022.

Years	A		B		C		D		E		F		G		H		J		K
Years	P_i	Grade	P_i	Grade	P_i	Grade	P_i	Grade	P_i	Grade	P_i	Grade	P_i	Grade	P_i	Grade	P_i	Grade	P_i	Grade
2019	0.402	2	0.899	3	0.546	2	0.562	2	0.566	2	0.795	3	0.688	2	0.578	2	0.562	2	0.553	2
2020	0.544	2	0.694	2	0.711	3	0.452	2	0.604	2	0.668	2	0.542	2	0.496	2	0.452	2	0.680	2
2021	0.686	2	0.534	2	0.647	2	0.673	2	0.641	2	0.469	2	0.397	2	0.661	2	0.673	2	0.425	2
2022	0.868	3	0.373	2	0.596	2	0.813	3	0.689	3	0.568	2	0.873	3	0.766	3	0.813	3	0.842	3

Comparing the consistency of results between TOPSIS and RSR methods

Comparison of TOPSIS and RSR ranking in different hospitals from 2019 to 2022 showed that the hospitals with identical rankings were B, F. The difference in ranking results ≤2 were A, C, D, E, G, H, J, K (Table 8). The findings indicated that the rankings of hospitals B and F showed decreasing trend in the ranking, while the ranking of A and E showed an overall increasing trend, and the D, H, and J showed decreasing trends in the rankings from 2019 to 2022.

Table 8.

Comparison of the Results of TOPSIS and RSR Method of Ranking in Different Hospitals From 2019 to 2022.

Sorting results	Number	Items
Identical	2	B, F
Sorting gap ≤ 2	8	A, C, D, E, G, H, J, K
Sorting gap>2	0
Total	10

Discussion

The evaluation indicators are influenced by factors such as the number of visits between hospitals and the number of patients, which can be easily affected by the social environment and resident population.²² According to the TOPSIS method, Hospital F is ranked the lowest due to its low number of patients. This can be attributed to the small resident population in the city where Hospital F is located, which mainly consists of young adults who have their own affiliated medical institutions at their workplaces. The evaluation was also influenced by the services and positioning of the hospitals themselves.²³ The RSR grading level reveals that hospitals A and H are ranked at the top, while hospital J was ranked at the bottom. One possible reason was that Hospital J was a combined Chinese and Western medicine hospital, and patients who seek Western medical services at the hospital may give lower evaluation scores to the Chinese medicine aspects. The development of the Chinese herbal medicine industry, supported by government and policies, also has an impact on the evaluation indicators.²⁴ Hospital H has the highest combined indicators related to integrated management and cost control, which may be attributed to the development of the Chinese herbal medicine cultivation industry in the city where Hospital H is located. On the other hand, hospital D has the lowest level of internal operation and cost control indicators. This could be due to the slow economic development of the city and state where hospital D is located, resulting in a lack of medical resources and a lag in talent cultivation.²⁵

The combined use of the RSR and TOSIS methods can objectively categorize hospital assessments, resulting in additional hospital construction and reform, as well as an improvement in overall hospital quality. The study’s analysis results show that the performance appraisal ranking results from the TOPSIS and RSR methods are highly consistent. Only 2 of the 10 Chinese hospitals in 2020 and 2021 had a difference in performance appraisal rankings >2. In 2019, there were 2 hospitals with identical rankings in 6 Chinese hospitals, and 4 hospitals with a ranking >2. In 2022, only 2 of the 10 Chinese hospitals that used the TOSIS and RSR methods had a ranking >2. Similarly, of 10 Chinese hospitals using these methods, only 2 had a ranking ≤2. In addition, the TOPSIS method and RSR method sorting results of 10 Chinese medicine hospitals’ performance appraisal for 4 consecutive years showed that there were 2 hospitals with identical sorting results and 8 hospitals with a sorting result gap of ≤2. The findings of this study confirm that the combination of the 2 evaluation methods is more practical, and the results obtained are more objective, making it useful as a reference for hospital managers and providing a scientific theoretical basis for further improving the quality of hospital work.

In recent years, the growing of demand for medical treatment and influence of Chinese medicine led to a continuous workload increasing in TCM hospitals under the context of modernized healthcare development.^26,27 The healthcare strength of the hospital’s location significantly influences the hospital’s performance appraisal.²⁸ Hospitals H demonstrate a stronger integrated medical strength, leading to higher performance assessment results. This attributed to the rapid development of the Chinese herbal medicine planting industry in the city where H Institute is situated. The city has a long history, receives governmental attention and policy support, and considers the Chinese herbal medicine industry as a crucial initiative to adjust the structure of agricultural production. Consequently, it has become one of the fastest-growing and advantageous industries in the local national economy, with immense potential for further development.²⁵ The indicators of hospital D are lower compared to all other hospitals. This could be attributed to the slower economic development of the city and state where hospital D is located. Additionally, there may be issues such as a lack of medical resources, insufficient talent pool, and inadequate talent development, all of which can contribute to unsatisfactory performance appraisal results for the hospitals.^29,30 The TOPSIS method and the RSR method were used to evaluate the health resource allocation in the Chengdu-Chongqing from 2017 to 2021, the health resource allocation in Chongqing was more polarized and the health resource allocation in Sichuan Province was more balanced, but the advantaged regions were not prominent enough.¹⁸ These findings align with the results of our study. Therefore, it is important for our tertiary TCM hospitals in Gansu Province to maintain their operational efficiency while addressing the shortcomings in medical quality development. This will help promote overall improvement in the medical quality of each hospital.

The study findings reveal significant fluctuations in performance assessment results,³¹ which align with our own research. Notably, Hospitals B and F experienced a decline in rankings from 2019 to 2022, while Hospitals A and E demonstrated overall improvements in their rankings over the years. These changes were attributed to multiple factors. Firstly, the widespread implementation of home quarantine measures during the 3-year epidemic led to a substantial decrease in hospital visits.³² Additionally, considerable medical resources were allocated toward the detection, isolation, and treatment of patients infected with the coronavirus.³³ Moreover, the utilization of Chinese medicine in the prevention, treatment, and rehabilitation of coronavirus pneumonia may have contributed to better performance evaluation results for certain hospitals.³⁴ Therefore, this could have resulted in higher healthcare expenditure indicators and general management indicators in Chinese hospitals. This implies that hospitals within the same region with significant disparities in medical quality, as well as those with notable differences in medical quality rankings from previous years, should consider the indicators of the national tertiary hospital performance assessment.²⁵ They should strive to understand their own shortcomings and actively seek policy support to enhance various management systems to continually improve the quality of medical care.

This study focuses on the tertiary in Gansu Province as the research sample. However, it does not address the variations between different levels and types of medical institutions within each tertiary, nor the differences between departments within the same medical institution. While using the region as a research sample provides some insights into the variations in medical quality among hospitals, it does not thoroughly analyze the factors influencing efficiency. Further research is needed to delve into this aspect in detail. Although we used the entropy weight-TOPSIS method and RSR method to conduct a comprehensive evaluation of the performance appraisal of Gansu Provincial Hospital of Traditional Chinese Medicine in 2019 to 2022, there are still some limitations in this study. First, the evaluation indexes were selected with reference to the evaluation indexes commonly used in similar studies. Second, Chinese medicine hospitals in Gansu Province can conduct a horizontal comparison study with other regions, but this study does not. Third, this study mainly analyzes from the perspective of performance evaluation efficiency and does not consider the issue of health resource allocation.

Conclusions

This article evaluates the medical quality capacity of 10 tertiary public Traditional Chinese Medicine hospitals in Gansu Province from 2019 to 2022 by the TOPSIS and RSR methods. The findings reveal significant fluctuations in medical quality across different years, while the overall level of medical quality remains relatively stable among the various hospitals. The TOPSIS and RSR methods were used to evaluate the medical quality of different hospitals with consistent and scientifically credible results. These include improving the hospitals’ attractiveness to patients, enhancing the existing medical service measures, and diversifying the range of medical services offered to enhance the overall medical quality of Chinese medicine hospitals.

Footnotes

Acknowledgements

The authors thank TCM monitoring and direct reporting network and all hospitals which supplied the datum.

Author Contributions

All authors were involved in the conceptualization and design of the study. Lijiao Ma and Jianlan Xue contributed to the research methodology. Lin Xu and Xianwen Li collected, processed, and normalized the data. Lijiao Ma and Xuhong Chang wrote the original draft of the manuscript. Xuhong Chang, Yuanyuan Li reviewed and edited the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research has been funded by Gansu Administration of Traditional Chinese Medicine Research Program /grant number GZKB-2022-40, Monitoring and Statistical Research Center, National Administration of Traditional Chinese Medicine Policy Research program/grant number YGZXKT2024366.

ORCID iDs

Yuanyuan Li

Xuhong Chang

References

Amraie

Mahfoozpour

Vahdat

Hesam

Performance appraisal models in the county health centers of Iran: a systematic review. J Biol Res Accad Soc Ital Med Biol Sper. 2022;95(2):10536. doi:10.4081/jbr.2022.10536

Longaray

Marube

Ensslin

Dutra

Ensslin

Performance evaluation of public healthcare supply chain management: a critical literature review. Int J Healthc Technol Manag. 2023;20(2):144-170. doi:10.1504/ijhtm.2023.131518

Zhuang

Jia

, et al. Interaction between Chinese medicine and digoxin: clinical and research update. Front Pharmacol. 2023;14:1663-9812. doi:10.3389/fphar.2023.1040778

Theodosiou

Zhang

, et al. Developing TCM clinical practice guidelines: a comparison between traditional Chinese medicine and western medicine. Integr Med Res. 2023;12(2):100952. doi:10.1016/j.imr.2023.100952

Xie

Y-M

Wang

L-X.

Combined application of Chinese and western medicine, precise positioning for clinical efficacy. China J Chin Mater Med.2016;41(24):4479-4482. doi:10.4268/cjcmm20162401

Guo

Lin

, et al. Identification of active compounds of traditional chinese medicine derived from maxing shigan decoction for COVID-19 treatment: a meta-analysis and in silico study. Expert Rev Anti-Infect Ther. 2023;21(8):871-889. doi:10.1080/14787210.2023.2238899

Estores

Ackerman

Integrative medicine in long COVID. Phys Med Rehabil Clin N Am. 2023;34(3):677-688. doi:10.1016/j.pmr.2023.03.006

Mahmood

Abbas

Influence model and doubly extended TOPSIS with TOPSIS based matrix of interpersonal influences. J Intell Fuzzy Syst. 2020;39(5):7537-7546. doi:10.3233/jifs-200833

. Evaluation of medical and health service ability of TCM hospitals in Yangtze River delta region based on entropy weight-TOPSIS method. Adv Appl Math. 2022;11(01):84-92. doi:10.12677/aam.2022.111013

10.

Vărzaru

AA.

An empirical framework for assessing the balanced scorecard impact on sustainable development in healthcare performance measurement. Int J Environ Res Public Health. 2022;19(22):15155. doi:10.3390/ijerph192215155

11.

Liou

JJH

. Comments on “Factors influencing medical tourism adoption in Malaysia: a DEMATEL-Fuzzy TOPSIS approach.” Comput Ind Eng. 2021;157:157107333. doi:10.1016/j.cie.2021.107333

12.

Yin

Chen

Zhou

Huang

Contribution quality evaluation of table tennis match by using TOPSIS-RSR method - an empirical study. BMC Sports Sci Med Rehabil. 2023;15(1):132-132. doi:10.1186/s13102-023-00738-9

13.

Huang

Shen

. Comprehensive evaluation of medical service ability of TCM hospitals in 30 provinces. Autonomous Regions and Municipalities of China in 2017 Based on Entropy Weight TOPSIS Method and RSR Method. ACM; 2019:108-112.

14.

Liu

Wang

A comparative evaluation of child health care in China using multicriteria decision analysis methods. BMC Health Serv Res. 2023;23(1):1217. doi:10.1186/s12913-023-10204-4

15.

Huang

Shen

. Comprehensive Evaluation of Medical Service Capability in TCM Hospitals by Using the Fuzzy Combined Method of TOPSIS and Rank Sum Ratio. IEEE; 2019:244-248.

16.

Feng

Guo

Qian

, et al. Field validation of different intervention modes for the potential transmission risk of schistosomiasis during post-transmission interruption phase. PLoS Negl Trop Dis. 2023;17(11):e0011739. doi:10.1371/journal.pntd.0011739

17.

Dai

Jiang

, et al. Evaluation of community basic public health service effect in a city in Inner Mongolia Autonomous Region—based on entropy weight TOPSIS method and RSR fuzzy set. Arch Public Health. 2023;81(1):149. doi:10.1186/s13690-023-01151-x

18.

Zhou

Regional differences in health resource allocation: a longitudinal study in the Chengdu-Chongqing economic circle, China. BMJ Open. 2024;14(3):e082721. doi:10.1136/bmjopen-2023-082721

19.

Wang

Wan

Song

Coal miners' livelihood vulnerability to economic shock: multi-criteria assessment and policy implications. Energy Policy. 2018;114:301-314. doi:10.1016/j.enpol.2017.12.017

20.

Chen

Ding

Wang

, et al. A Comprehensive Evaluation Method on Operation Quality of Power Backbone Optical Communication Network Based On Improved TOPSIS. 2018 IEEE 3rd International Conference on Cloud Computing and Internet of Things (CCIOT), Dalian, China, October 20-21, 2018, IEEE. pp. 506–511.

21.

Yoon

Systems Selection by Multiple Attribute Decision Making. Kansas State University. 1980.

22.

Angelini

Teglia

Casolari

Astolfi

Boffetta

Decrease of visits and hospital admissions for cancer patients during the COVID-19 pandemic. a systematic review and meta-analysis. J Public Health. 2023;32(6):909-915. doi:10.1007/s10389-023-01857-w

23.

Ranchon

Chanoine

Lambert-Lacroix

, et al. Development of artificial intelligence powered apps and tools for clinical pharmacy services: a systematic review. Int J Med Inform. 2023;172:172104983. doi:10.1016/j.ijmedinf.2022.104983

24.

Zhang

Wang

, et al. Application of tissue culture technology of medicinal plants in sustainable development of Chinese medicinal resources. China J Chin Mater Med. 2023;48(5):1186-1193. doi:10.19540/j.cnki.cjcmm.20221017.104

25.

Ahumada-Canale

Jeet

Bilgrami

, et al. Barriers and facilitators to implementing priority setting and resource allocation tools in hospital decisions: a systematic review. Soc Sci Med. 2023;322:115790. doi:10.1016/j.socscimed.2023.115790

26.

Baek

Analysis on the determinants of medical demand in China: focusing on the different effects of supplier-induced demand. J Manag Econ. 2020;42(1):61-80. doi:10.22828/meri.2020.42.1.004

27.

Wang

Suo

, et al. An investigation into traditional Chinese medicine hospitals in China: development trend and medical service innovation. Int J Health Policy Manag. 2016;6(1):19-25. doi:10.15171/ijhpm.2016.72

28.

Sheng

Yang

Zhang

The unmet medical demand among China's urban residents. Int J Environ Res Public Health. 2021;18(21):11708. doi:10.3390/ijerph182111708

29.

Al-Hussaniy

HA.

Medical scientific research challenges in Iraq. med Pharm J. 2023;2(1):1-3. doi:10.55940/medphar202228

30.

DeClercq

Gharipour

Mohagheghi

, et al. A novel approach to locating community clinics to promote health care accessibility and reduce health disparities in Baltimore, Maryland. Inq J Heal Care Organ Provis Financ. 2023;60:00469580221135953. doi:10.1177/00469580221135953

31.

Wei

A comprehensive evaluation of China's TCM medical service system: an empirical research by integrated factor analysis and TOPSIS. Front Public Health.2020;8:532420. doi:10.3389/fpubh.2020.532420

32.

Ayittey

Chiwero

Kamasah

Dzuvor

Economic impacts of Wuhan 2019-nCoV on China and the world. J Med Virol. 2020;92(5):473-475. doi:10.1002/jmv.25706

33.

Liu

Jia

Cheng

, et al. Measuring productivity and relative efficiency of public tertiary traditional Chinese Medicine Hospitals in Hubei, China. Inq J Heal Care Organ Provis Financ. 2023;60:00469580231190576. doi:10.1177/00469580231190576

34.

Wang

Pang

Analysis on the selection of herbs in TCM COVID-19 treatment protocols between Malaysia and China. Int J Gen Med. 2023;16:3655-3663. doi:10.2147/IJGM.S426168

Characteristics of Medical Quality in Tertiary Traditional Chinese Medicine Hospitals by TOPSIS and RSR Methods

Abstract

Keywords

Introduction

Materials and Methods

Data Sources

Indicators Determination

Methods

TOPSIS

RSR

Combination of TOPSIS and RSR methods

Statistical Analysis

Results

Description of Indicators With Chinese Medicine Characteristics

Evaluation of the Medical Quality in Different Hospitals

Ranking for medical quality by the TOPSIS method

Grading for medical quality by the RSR method

Evaluation of the Medical Quality From 2019 to 2022

Ranking for medical quality by the TOPSIS method

Grading for medical quality by the RSR method

Comparing the consistency of results between TOPSIS and RSR methods

Discussion

Conclusions

Footnotes

Acknowledgements

Author Contributions

Declaration of Conflicting Interests

Funding

ORCID iDs

References