Longitudinal Evaluation on the Operation Index Applied to Public Hospitals in Pudong New District of Shanghai,China

Abstract

The public hospital reform has lasted 5 years in China; however, the operation development status and trends of public hospitals have not been systematically evaluated in Pudong New District. We first applied the technology of longitudinal index to assess the development of public hospitals there. The quantitative data were mainly gathered by taking health statistics database from 2009 to 2014. The results showed that overall operating index presented a down-up trend, with the highest point in 2014 and the lowest point in 2012. Overall operating index, development foundation index, and management condition index were found to be statistically different (P = .010, P = .016, P = .031) in different years, whereas the service operation index and financial risk index were not so (P = .543, P = .228). Moreover, the results demonstrated that no obvious difference was observed in the overall operating index between the general and specialized hospitals (P = .327), which was the same in the 4 first-class indexes. However, there were statistical differences in the overall operating index and development foundation index among these 5 years (P = .018, P = .036), but none in the service operation index, management condition index, and financial risk index (P = .503, P = .062, P = .177). No interaction effects were discovered between year and hospital categories in the current study (P = .673, P = .375, P = .885, P = .152, P = .288).

Keywords

hospitals public operation index evaluation longitudinal study

What do we already know about this topic?

The index methods are widely applied to social economic and social practices, such as the real estate price index (REPI), the consumer price index (CPI), the life meteorological index, and the stock index, rarely in public hospitals’ operation evaluation.

How does your research contribute to the field?

We perform the longitudinal index research on the operation of Pudong public hospitals undergoing the new health care reform to track the longitudinal operation statuses and developmental trends, to evaluate the effect of the reform measures implemented, and to promote the hospital development in terms of exploring potential and improving service efficiency and quality.

What are your research’s implications toward theory, practice, or policy?

The longitudinal index method can be emphasized on the appraisal of the operation and management of public medical institutions and reflects the development and variation tendency of the institutions by converting the absolute numbers to relative numbers and making the same variables in the heterogeneity hospitals comparable.

Background

Opinions of the Central Committee of Communist Party of China and the State Council on Deepening the Health Care System Reform, published in 2009, unveiled the prelude to a new round of medical and health system reform in China.¹ In terms of public hospitals reform, the Ministry of Health has made clear the measures to advance in 2 directions: one is to separate between governance and administration, rowing and steering, clinic and pharmacy, and profit and nonprofit; the other is to further expand the scope of high-quality nursing care, build a long-term mechanism to support the grassroots and rural areas, escalate efforts in basic medical expenses direct settlement, and conduct diagnosis and treatment before payment.²

At present, Pudong New District (Pudong), located in the east of Shanghai, China, has 16 public hospitals and 45 community health service centers belonging to the health sector. Since Chinese 12th five-year plan of health programs was implemented in 2011, Pudong pioneered in deepening the medical and health system reform, exploring the medical resource integration, clinical pathway, medical association, family doctor system, basic drug system, drug income, and medical drug for public medical institutions,^3-8 as indicated by the further exploration of the complex patterns of secondary public hospitals and community health service centers, by the management of clinical pathways in 7 secondary-class or above general hospitals, and by the propelling of the major project construction of the health informatization in Pudong since 2011. With health care policies implemented, it would be of far-reaching influence on the operation and development of the public hospitals in the area. It is imperative to evaluate how the public hospitals adapted to the in-deep development of the medical reform policies in terms of operating status and developing trend.

In the previous assessment of the public medical institutions, the indicators had mainly referred to absolute amount, characterized by horizontal comparisons, and the analysis results had just expressed the cross-sectional situation, which made it difficult to reflect the longitudinal development and variation tendency of the institutions; furthermore, the cross-sectional data in the heterogeneity hospitals were incomparable.^9-11 However, index is a multivariate, synthetic, abstract amount,¹² which is generally of relative numbers measuring the certain variables changing in time or space.¹³ In theory, the index evaluation research assesses the longitudinal tendency of variation and emphasizes the investigation of the increment development when compared with the previous cycle, rather than with the direct comparison of absolute numbers. In practice, therefore, the index becomes an important indicator to reflect the actual changes in macro- and micro-economy, which is a major problem of social economic and social practices and receives considerable public attention.¹⁴ Nowadays, the index methods are widely applied to many fields and gradually penetrate into various industries. Illustrations of this are the real estate price index (REPI), which reflects the boom of the real estate market; the consumer price index (CPI), which indicates the national consumer price; the life meteorological index, which shows the change of atmosphere climate; and the stock index, which describes the price fluctuation in the stock market or bond market.^15-19 It should be noted that an index method is to be applied based on the continuous monitoring data. Therefore, we intended to apply the longitudinal index method to evaluating the public hospitals’ operation. There is a dearth of relevant literature in such databases as CNKI (China National Knowledge Infrastructure), Wanfang, Weipu Information Network, and PubMed; therefore, it was wise to keep track of all kinds of the public hospitals’ operation and development and their measures to improve the service efficiency and quality in Pudong.

Material and Methods

Participant Selection

All 16 public hospitals in the administrative area were enrolled in the current study. The data collection was mainly accomplished based on its health statistics databases from 2009 to 2014, part of the human resource data collected from the information system of its health resources management center. In the processing of missing data and the extreme value, as data missing and extreme value can cause certain bias, we took the following measures to avoid them. For some extreme value, we viewed the content of health statistics annals of Pudong or took the initiative connection to the investigation units to verify the relative data; for the missing data of the previous year, we set the indicator to 0 (namely, the growth rate as 0), which was not included in the operation index calculation.

Methods of Developing Operating Evaluation Indexes

Determining the operating index evaluating indicators

On the basis of our pre-research, we invited 10 experts from the government departments concerned, public hospitals, and universities for consultation, after which we revised the indicators which had been built by referring to the literature.^20-23 By analyzing qualitatively and quantitatively, we made a factor analysis on third-level indicators using the public hospital data, which aims to lower the number of unobserved variables. We selected the common factors according to the standards of characteristic roots value greater than 1, whose cumulative contribution of variance were all greater than 50%.²⁴ The results showed that resource level extracted of 3 common factors; continuing education training indicators, 2 common factors; medical efficiency indicators, 1 common factor; medical quality indicators, 2 common factors; profitability indicators, 2 common factors; cost control, 2 common factors; and financial risk indicators, 2 common factors (Table 1). We further improved the existing index system and confirmed the final operating index evaluation system, which contained 4 first-class indicators (development foundation, service operation, management condition, and financial risk, respectively), 7 second-class indicators (resource level, continuing education training, medical efficiency, medical quality, profitability, cost control, and financial risk), and 36 third-class indicators (including the ratio of doctors and nurses, clinical cure rate, medical cost rate, total assets turnover, etc.) (Table 1).

Table 1.

Factor Analysis of Public Hospitals on Operation Evaluating Indicators in Pudong.

Category	Common factors	Characteristic root	Variance contribution ratio (%)	Cumulative variance contribution rate (%)	KMO test statistics	P value in the Bartlett sphericity test
Resource level	1	2.57	32.18	32.18
	2	1.44	18.00	50.18
	3	1.28	15.98	66.17
	4	0.99	12.35	78.52	0.477	<.001
	5	0.87	10.83	89.35
	6	0.48	5.96	95.31
	7	0.20	2.56	97.87
	8	0.17	2.13	100.00
Continuing education training	1	1.24	41.22	41.22
	2	1.07	35.73	76.95	0.441	.484
	3	0.69	23.05	100.00
Medical efficiency	1	1.62	53.92	53.92
	2	0.96	31.91	85.83	0.473	<.001
	3	0.43	14.17	100.00
Medical quality	1	2.67	44.56	44.56
	2	1.10	18.28	62.83
	3	0.93	15.53	78.36	0.705	<.001
	4	0.82	13.59	91.95
	5	0.32	5.39	97.33
	6	0.16	2.67	100.00
Profitability	1	2.07	41.33	41.33
	2	1.26	25.20	66.53
	3	0.96	19.15	85.68	0.497	<.001
	4	0.71	14.22	99.90
	5	0.01	0.10	100.00
Cost control	1	2.37	39.52	39.52
	2	1.36	22.66	62.19
	3	0.99	16.56	78.75	0.455	<.001
	4	0.71	11.89	90.63
	5	0.44	7.25	97.88
	6	0.13	2.12	100.00
Financial risk	1	2.36	47.20	47.20
	2	1.17	23.48	70.68
	3	0.86	17.21	87.89	0.592	<.05
	4	0.41	8.18	96.07
	5	0.20	3.93	100.00

Note. KMO = Kaiser-Meyer-Olkin. The bold-faced values shows that the characteristic roots value were greater than 1.

Assigning every indicator weight in operating index system

To determine the first and second index weight: According to their suggestions from the key interviewers invited and expert consultations conducted, we sorted data and obtained the first and second index weight. To determine the third index weight: Based on the consultants’ relative weights out of the analysis of the data collected from 2009 to 2014 on the index evaluation to get the initial weight for reference, we calculated the arithmetic mean scores of each index and processed normalization to obtain the final weight of the third indicators. The details were as follows:

Determining the initial weight

According to the variance contribution ratio of the selected common factor and their value, we calculated the weighted average of each index, as indicated by the formula. For the index weighted average positive, we directly normalized weighted average of the indicators as the initial weights of every index, and for the index weighted average negative, we normalized the weighted average of the indicators after subtracting the minimum weighted average value as the initial weights of every index. On the basic of factor analysis, we acquired primary indicator weight in the operating index system of public hospitals (Table 2).

Table 2.

Determination of the Initial Weights of the Operating Index of Public Hospitals in Pudong, Shanghai.

Item		Extracted common factors			Calculation of the initial weight
Item		Common factor 1	Common factor 2	Common factor 3	Weighted mean	Positive conversion value	Initial weight
Resource level index	Variance contribution rate (%)	32.18	18.00	15.98
	Percentage of the master degree or above in doctors (%)	0.75	−0.31	0.30	23.18	44.42	0.1560
	Percentage of senior title in doctors (%)	0.90	0.04	0.05	30.37	51.61	0.1812
	Percentage of college or above degree in nurses (%)	−0.10	0.67	0.61	18.57	39.81	0.1398
	Percentage of intermediate title and above in nurses (%)	0.14	0.78	−0.25	14.56	35.80	0.1257
	Ratio of health technical personnel in working staffs (%)	0.51	0.49	−0.18	22.46	43.70	0.1534
	Fixed assets per capita (10 000 yuan/person)	0.34	−0.22	−0.09	5.57	26.81	0.0942
	Ratio of doctors and nurses	0.32	−0.07	0.77	21.38	42.62	0.1497
	Ratio of bed and nurse	−0.84	−0.01	0.36	−21.24	0.00	0.0000
	Total				114.87	284.79	1.0000
Continuing education training index	Variance contribution rate (%)	41.22	35.73
	On-the-job training per capita (time/person)	−0.08	0.93		29.82	29.82	0.3030
	Percentage of the staff for further education (%)	0.77	0.38		44.97	44.97	0.4570
	Percentage of science and education cost in total expenditure (%)	0.80	−0.26		23.63	23.63	0.2401
	Total				98.42	98.42	1.0000
Medical efficiency index	Variance contribution rate (%)	53.92
	Workload per capita (person-time/person)	0.55			29.44	29.44	0.4105
	Bed utilization rate (%)	0.31			16.66	16.66	0.2323
	Average hospital stay for discharged patients (day)	0.48			25.61	25.61	0.3571
	Total				71.71	71.71	1.0000
Medical quality index	Variance contribution rate (%)	44.56	18.28
	Treatment and improvement ratio (%)	−0.37	0.62		−5.21	9.15	0.0434
	Incidence of operation complication (%)	0.17	0.82		22.62	36.97	0.1756
	The successful rescue rate of critically ill in-patients (%)	−0.34	0.05		−14.35	0.00	0.0000
	The coincidence rate of outpatient and discharge diagnoses (%)	0.87	−0.10		36.78	51.13	0.2429
	The coincidence rate of admission and discharge diagnoses (%)	0.92	0.06		42.21	56.56	0.2687
	The coincidence rate before and after the operation (%)	0.89	0.15		42.37	56.72	0.2694
	Total				124.41	210.53	1.0000
Profitability index	Variance contribution rate (%)	41.33	25.20
Profitability index	Medical income by every 10 000 yuan fixed assets (yuan/10 000 yuan)	0.32	0.71		30.81	30.81	0.2215
	Average business income per worker (10 000 yuan/person)	−0.06	0.78		17.28	17.28	0.1242
	Rate of business income and expense balances (%)	0.98	−0.12		37.63	37.63	0.2705
	Percentage of business income accounted for total expenditure (%)	0.99	−0.12		37.74	37.74	0.2713
	Percentage of drug income accounted for business income (%)	0.17	0.35		15.64	15.64	0.1124
	Total				139.09	139.09	1.0000
Cost control index	Variance contribution rate (%)	39.52	22.66
	Medical cost rate (%)	0.65	−0.58		12.60	43.17	0.1679
	Management cost rate (%)	−0.43	0.70		−1.00	29.58	0.1150
	Health material cost by 10 000 yuan medical income (yuan/10 000 yuan)	0.80	0.31		38.75	69.33	0.2696
	Medical expenses by every emergency case and outpatient (yuan/person-time)	0.04	0.54		13.73	44.31	0.1723
	Cost by every discharged patient (yuan/person)	−0.60	−0.30		−30.58	0.00	0.0000
	Cost by every bed-day in hospital (yuan/bed-day)	0.87	0.26		40.23	70.81	0.2753
	Total				73.74	257.20	1.0000
Financial risk index	Variance contribution rate (%)	47.20	23.48
	Total assets turnover (%)	0.82	−0.26		32.62	77.94	0.2574
	Rate of net asset growth (%)	−0.02	0.89		19.76	65.08	0.2149
	Rate of net asset return (%)	0.51	0.36		32.71	78.03	0.2577
	Current ratio (%)	−0.78	−0.36		−45.32	0.00	0.0000
	Asset-liability ratio (%)	0.90	−0.25		36.42	81.74	0.2700
	Total				76.21	302.79	1.0000

Determining the final weight

The formula for the third terminal index was as follows. Based on the initial weights of every index, we consulted the experts, who scored the importance of evaluating indexes. Then, we came to obtain the operating index evaluation system for public hospitals with the final weight of all levels of indicators (Table 3).

Table 3.

The Operating Index Evaluation System for Public Hospitals in Pudong.

First-class index (W_1r)	Second-class index (W_2s)	Third-class index (W_3t)		Final weight (W_rst)
Development foundation (0.3100)	Resource level (0.6800)	Percentage of the master degree or above in doctors (%)	0.1324	0.0279
		Percentage of senior title in doctors (%)	0.1466	0.0309
		Percentage of college or above degree in nurses (%)	0.1200	0.0253
		Percentage of intermediate title and above in nurses (%)	0.1466	0.0309
		Ratio of health technical personnel in working staffs (%)	0.1281	0.0270
		Fixed assets per capita (10 000 yuan/person)	0.1020	0.0215
		Ratio of doctors and nurses	0.1058	0.0223
		Ratio of bed and nurse	0.1181	0.0249
	Continuing education training (0.3200)	On-the-job training per capita (time/person)	0.3337	0.0331
		Percentage of the staff for further education (%)	0.3276	0.0325
		Percentage of science and education cost in total expenditure (%)	0.3387	0.0336
Service operation (0.3150)	Medical efficiency (0.5100)	Workload per capita (person-time/person)	0.3436	0.0552
Service operation (0.3150)	Medical efficiency (0.5100)	Bed utilization rate (%)	0.3436	0.0552
		Average hospital stay for discharged patients (day)	0.3131	0.0503
	Medical quality (0.4900)	Treatment and improvement ratio (%)	0.1788	0.0276
	Medical quality (0.4900)	Incidence of operation complication (%)	0.1497	0.0231
		The successful rescue rate of critically ill in-patients (%)	0.1659	0.0256
		The coincidence rate of outpatient and discharge diagnoses (%)	0.1736	0.0268
		The coincidence rate of admission and discharge diagnoses (%)	0.1821	0.0281
		The coincidence rate of before and after the operation (%)	0.1497	0.0231
Management condition (0.2550)	Profitability (0.4650)	Medical income by every 10 000 yuan fixed assets (yuan/10 000 yuan)	0.1931	0.0229
		Average business income per worker (10 000 yuan/person)	0.2184	0.0259
		Rate of business income and expenses balances (%)	0.1872	0.0222
		Percentage of business income accounted for total expenditure (%)	0.1965	0.0233
		Percentage of drugs income accounted for business income (%)	0.2058	0.0244
	Cost control (0.5350)	Medical cost rate (%)	0.1708	0.0233
		Management cost rate (%)	0.1657	0.0226
		Health materials cost by 10 000 yuan medical income (yuan/10 000 yuan)	0.1635	0.0223
		Medical expenses by every emergency case and outpatient (yuan/person-time)	0.1840	0.0251
		Cost by every discharged patient (yuan/person)	0.1473	0.0201
		Cost by every bed-day in hospital (yuan/bed-day)	0.1686	0.0230
Financial risk (0.1200)	Financial risk (1.000)	Total assets turnover (%)	0.1800	0.0216
		Rate of net asset growth (%)	0.2150	0.0258
		Rate of net asset return (%)	0.2250	0.0270
		Current ratio (%)	0.1958	0.0235
		Asset-liability ratio (%)	0.1833	0.0220

Calculating the operation index of public hospitals

We started with the conversion of the basic investigated data into parameter values according to the formula of each indicator. When we treated the value of the previous year as the benchmark, we calculated the increasing speed with the link relative method of each index, using the formula ( $[{X^{'}}_{i} - X_{i}] / X_{i}$ ). And then we multiplied the increasing speed with the link relative method by their corresponding weights to sum them up. After that, we multiplied the summation by 100 and plus 100 (defining the point of previous year as 100 point), before we obtained the first-class operation index and the total operating index. If the indicator was negative, the increasing speed with the link relative method would be multiplied by −1; hence, the larger of the index, the better the operating condition. The computation formula of all levels of operating index was as follows:

The first-class operation index of certain public hospital:

I = (\sum_{i = 1}^{n} W_{i} \sum_{j = 1}^{n_{i}} W_{i j} \frac{{X^{'}}_{i j} - X_{i j}}{X_{i j}}) \times 100 + 100,

(Formula 1)

where n is the number of second-class indicators in the first-class index, W_i is the weight of the ith second-class index, n_i is the number of third-class indicators in the second-class index, W_ij is the weight of jth third-class index in the ith second-class index, X_ij is the value of the baseline third-class indicators, and ${X^{'}}_{i j}$ is the value of the third-class indicators that very year.

Total operating index of certain public hospital:

I = (\sum_{i = 1}^{n} W_{i} \sum_{j = 1}^{n_{i}} W_{i j} \sum_{k = 1}^{n_{i j}} W_{i j k} \frac{{X^{'}}_{i j k} - X_{i j k}}{X_{i j k}}) \times 100 + 100,

(Formula 2)

where n is the number of the first-class index, W_i is the weight of the ith first-class index, n_i is the number of the second-class index in the ith first-class index, W_ij is the weight of jth second-class index in the ith first-class index, n_ij is the number of third indicators in the ith first-class index and the jth second-class index, W_ijk is the weight of the kth third-class index in the ith first-class index and the jth second-class index, X_ijk is the value of the baseline third-class indicators, and ${X^{'}}_{i j k}$ is the value of the third-class indicators that very year.

Statistical Analysis

The data were recorded and collated on EpiData 3.1, analyzed by the statistical software package SPSS 17.0. Applied to the analysis were descriptive statistics, 1-way analysis of variance (ANOVA), and ANOVA for repeated measurement data. The threshold of statistical significance was set at P < .05 (2-tailed).

Results

The Trend Description of the Public Hospitals’ Operating Indexes and Comparative Analysis From 2009 to 2014

The overall operating index presented a down-up trend, with the highest point in 2014 (104.41) and the lowest point in 2012 (98.96). Of the first-class index, the development foundation index presented a small fluctuation from 2010 to 2012, followed by a significant improvement in 2013 and 2014, respectively. The service operation index was little changed in these 5 years, whereas the management operation index rose after showing the falling trends, a turning point hitting 2012, and the financial risk index showed a rising trend after the fall in 2013 (Figure 1).

Figure 1.

2010-2014 operating index trends.

By repeated measurement data ANOVA, we found that the overall operating index in different years presented a statistical difference (P = .010). In the first-class index, development foundation index was found to be significantly different during the 5 years (P = .016), whereas the service operation index was not so (P = .543). The management condition index was observed to have a statistical difference (P = .031); however, no significant difference was found in the financial risk index (P = .228).

Analysis of 2009-2014 Overall Operating Index of Different Types

The overall operating index of the general public hospitals showed an increasing trend following a slight 3-year decline, with the highest value in 2014 (103.81). The overall operating index of the specialized public hospitals went down sharply first and then began to rise, with the lowest point in 2012 and the highest in 2014 (Figure 2).

Figure 2.

2010-2014 overall operating index trends of different categories.

By repeated measurement data ANOVA, statistical differences of the overall operating index were found in the general public hospitals and in the specialized ones in different years (P = .018); however, no significant differences were observed between the general public hospitals and specialized ones (P = .327).

Analysis of the Operation Indexes of the General and Specialized Hospitals

Through ANOVA for repeated measurement, the results demonstrated no significant difference in overall operating index between the general and specialized hospitals (P = .327), as in the case of the development foundation index, service operation index, business operation index, management condition index, and financial risk index comparing between groups. Within the groups, statistical differences were observed in the overall operating index and development foundation index (P = .018, P = .036), whereas no significant differences were found during the 5 years in the service operation index, business operation index, management condition index, and financial risk index (P = .503, P = .062, P = .177); furthermore, no interaction effects between time (year) and hospital categories were discovered in the current study (P = .673, P = .375, P = .885, P = .152, P = .288) (Table 4).

Table 4.

Analysis of 2009-2014 Operating Indexes Between Different Types of Public Hospitals.

Index	Category of hospital	Year (mean ± SD)					Between-subjects effects		Within-subjects effects
Index	Category of hospital	2010	2011	2012	2013	2014	F	P	F	P
Overall operating^a	General	100.56 ± 0.61	99.36 ± 4.71	99.70 ± 0.85	99.21 ± 1.36	103.81 ± 4.36	1.039	.327	4.419	.018
	Specialized	100.39 ± 0.92	100.71 ± 0.93	98.47 ± 1.14	101.03 ± 7.56	104.87 ± 4.52			4.419	.018
Development foundation^b	General	100.37 ± 1.24	100.93 ± 0.76	100.18 ± 1.41	101.78 ± 3.06	107.46 ± 2.92	2.807	.118	4.332	.036
	Specialized	100.03 ± 1.59	100.08 ± 1.60	100.17 ± 1.16	108.71 ± 22.10	116.90 ± 13.69	2.807	.118	4.332	.036
Service operation^c	General	100.24 ± 0.28	100.93 ± 0.76	100.21 ± 0.44	99.89 ± 0.60	100.46 ± 0.69	1.194	.294	0.844	.503
	Specialized	109.95 ± 15.03	100.33 ± 0.56	100.30 ± 0.60	100.06 ± 0.42	100.19 ± 0.50	1.194	.294	0.844	.503
Management condition^d	General	101.41 ± 1.12	95.43 ±18.23	98.23 ±3.75	96.75 ±4.16	101.21 ±4.06	0.318	.582	2.842	.062
Management condition^d	Specialized	100.01 ± 0.30	102.10 ± 3.16	93.37 ± 3.68	95.66 ± 8.58	98.73 ± 3.50	0.318	.582	2.842	.062
Financial risk^e	General	100.07 ± 0.19	100.23 ± 0.46	100.26 ± 0.64	96.00 ± 4.18	108.68 ± 28.80	1.571	.232	2.005	.177
Financial risk^e	Specialized	100.01 ± 0.30	100.37 ± 0.40	100.09 ± 0.47	95.15 ± 6.88	99.15 ± 8.50	1.571	.232	2.005	.177

Interaction effect: ^aYear × Hospital Category (F = 0.434; P = .673). ^bYear × Hospital Category (F = 0.970; P = .375). ^cYear × Hospital Category (F = 0.287; P = .885). ^dYear × Hospital Category (F = 1.933; P = .152). ^eYear × Hospital Category (F = 1.260; P = .288).

Discussion

Longitudinal Index Method Emphasizes Evaluation of the Incremental Development to Promote Internal Potential

The health care plan, as part of the new round of the medical and health care reform, has been earnestly implemented in Pudong. Investments have been strengthened, which systematically stimulated some special reform measurements, including compensation mechanism, the construction of regional medical associations, the building of competent talents in the public health service system, the innovation reform of traditional Chinese medicine, and the construction of health information system. As such reforms are furthered and deepened, it is unclear in what way the public hospitals’ operations have changed. Previous assessments had been performed based on the transverse index comparison; therefore, it was difficult to reflect the longitudinal development and changing tendency. It is believed that longitudinal index study can measure the relative ratio changes of some variables along with time or space. In the current study, we performed the longitudinal index research on the operation of Pudong public hospitals undergoing the new health care reform to track the longitudinal operation statuses and developmental trends, to evaluate the effect of the reform measures implemented, and to promote the hospital development in terms of exploring potential and improving service efficiency and quality.

The Construction and Application of Operating Index System Differ From the Performance Evaluation

The word of operation is a term in economics, but its role varies with a particular field. In terms of application to medical and health institutions, basically it refers to the operations management of medical service and product, medical material supply, medical quality control, health budgets, medical equipment maintenance, health human resources, and so on, with the purpose of improving medical service quality and cost-efficiency value as much as possible, so as to meet the demand of the service object.²⁵ Given that, in the current study, the operating index evaluation system of the public hospitals was constructed, including 4 first-class indicators: development foundation, service operation, management condition, and financial risk. While performance is a management concept, Kane argued that it was the result of the work because it had close correlation with the organization’s strategic objectives, customers’ satisfaction, as well as the investment money.²⁶ Hospital performance refers to the performance and efficiency of the hospital, and the performance appraisal is the quantitative and qualitative assessment to operating conditions, operating efficiency of hospitals in a certain period of time.^27-30 In fact, the hospital performance evaluation is mostly based on achieving the organization’s goals; therefore, the index selection of the performance evaluation system should take the function of the hospital, service content, performance status, financial status, and the government demand into consideration, mostly for administration purposes. In view of which, the operation index of evaluation was associated with the traditional hospital performance evaluation but differed from the latter which laid an emphasis on the appraisal of the operation and management of public medical institutions.

The Overall Operation Runs Steadily, But Still Needs to Attach Importance to Management Condition Index

The results showed that the overall operating index presented a downward and then an upward tendency between 2009 and 2014 in Pudong. As a whole, however, the overall operating index growing steadily may be due to advocating patient-centered practice and promoting the mission of medical quality, shifting from extensive to intensive changes as the management mode required by the new medical reform, and embodying the connotation construction. Such measures were mainly aimed at improving the total efficiency and maintaining a sustainable development. The current results showed that except the service operation index, the other 3 indexes of the development foundation, management condition, and financial risk fluctuated significantly, especially the management condition index. The development foundation index showed a trend of rise, especially in 2012-2014, which may have been attributed to the investment in constructing the talent teams as well as to the constant financial input.

The annual average scores of the management condition index were lower than 100.00 points in 2011-2014, which witnessed statistical differences. This suggested that the new medical reform pushed forward the management system reform of the public hospitals in further harmonizing the interests in all aspects. Originating in the 1950s, the public hospitals had been compensated for their operating costs through drug addition, but in 2012 the pilot reform of public hospitals was launched in Pudong to implement the government policy of “pharmaceutical and clinical separation,” thus ultimately demolishing the phenomenon of medical practice maintained by the pharmaceutical income in the public hospital. The change would cause the problems of reducing the drug prices and making the hospital more nonprofit. In reality, the investment was not enough, the compensation mechanism was deficient, and the internal management was weak, which could limit the management condition index.

The financial risk index was found to have decreased significantly in 2013, but without statistical differences from 2011 to 2014. The reasons behind this might go as follows: The internal economic management was extensive, but the external supervision did not reach the designated position, which led to the lax cost control and high financial risk; moreover, the notion still remained that the public hospitals should emphasize on fiscal income rather than on expenditure and on medical business management rather than on financial management. Therein lies the necessity of establishing an internal control mechanism for the public hospitals to standardize their financial behaviors such as strengthening the comprehensive budget control, standardizing the use and purpose of funds, ensuring risk assessments to make investment feasibility analyses and do moderate debt management, and improving the permission system for the cost to strengthen the management of incomes and expenses and improve the economic efficiency.

The General Hospitals Are Superior to the Specialized Despite Their Recent Rapid Advances

As to the developing trend of the operation index, although the overall operating index and development foundation index were slightly higher in the general public hospitals than in the specialized public hospitals, the service operation index, the management condition index, and the financial risk index were observed to be lower. Moreover, no statistical differences were observed in the operating index of the different types of the public hospitals. For a long time, the central government attached great importance to the general public hospitals so that they had better medical conditions and accesses to larger financial investment and to more resource allocations, while most of the specialized hospitals were quite inferior in terms of investment and research. As indicated by the current results, the general public hospitals showed a better developing trend than the specialized ones.

In addition, such a better tendency may have related to the particular development stage of the general public hospitals. The overall operating index of Shanghai Seventh People’s Hospital ranked the top, which was formally recognized as a tertiary hospital in 2013, and during the development stage, the hospital received a substantial support for the hardware facilities, research projects, and informatization construction, thus greatly improving the capacity of service operation and capital operation efficiency.³¹

In recent years, it is undeniable that the new medical reform has created challenges as well as opportunities for the specialized public hospitals, as indicated in 2014 by the higher operation indexes of Pudong New Area Geriatric Hospital and Nanhui Mental Health Center than those of the strong general public hospitals such as Pudong New Area Gongli Hospital, Punan Hospital, and Pudong New Area People’s Hospital. Nowadays, with the aging populations accelerating and life and work stress building up, the numbers of chronic diseases and mental illnesses increase, which imposes growing service workload on the specialized hospitals. Moreover, medical services and payment of several chronic diseases and mental illnesses are covered in the national basic public health services; consequently, the financial risk index was low, whereas service efficiency and operation management ability were relatively high in the specialized hospitals. Therefore, it is imperative that the specialized public hospitals exploit the advantages of operation management and cost control to the full, with an emphasis on the training and nurturing of medical talents, advancing specialties and disciplines, and enhancing the core competition of the specialized public hospitals.

Conclusions

Longitudinal index method, which can be applied to evaluating the incremental development with a view to promoting internal potential of public hospitals, distinguishes with the performance assessment of public health institutions. In general, the overall operating of the public hospitals develops steadily in Pudong; however, they still need to take business operation indexes into account, for different types of the public hospitals develop unbalanced, and the general hospitals run better than the specialized ones. Thus, it is imperative that in Pudong the business development be further strengthened, that growing attention be paid to institution management and financial risk prevention, and that a balanced development be struck between the specialized public hospitals and the general ones as a growing demand is made of specialized services. We believe that the current study could provide experiences and references for the public hospitals in other parts of Shanghai and even for the counterparts across the country. Further studies are needed to expand the evaluating scope of operating index to enlarge the sample size and increase the comparability.

Footnotes

Acknowledgements

The authors would like to thank all interview participants for their support and cooperation. They are also grateful to the Health and Family Planning Commission of Pudong New District and Pudong New District Health Resources Management Center, Shanghai, for their technical support.

Author Contributions

Z.Y.M. is responsible for research design. L.S.S. drafted the manuscript and H.J.L. kept on modifying the draft according to all the reviewers. L.S.S. and H.J.L. are responsible for data collection and processing. F.J.C., L.H., L.J.Q., and J.Y. provided suggestions on research design, paper writing, and modification. All authors read and approved the final draft.

Ethics Approval and Consent to Participate

Ethical approval was granted by the Academic Ethics Committee of Shanghai Pudong Institute for Health Development, and written informed consent was obtained from the participants.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (71403171), Academic Leaders Training Program of Pudong Health and Family Planning Commission of Shanghai (No. PWRd2014-13), Shanghai Municipal Health and Family Planning Commission Research Project (No. 201740016), and the Technology Project of the Pudong Health and Family Planning Commission of Shanghai (No. PW2017A-3).

ORCID iD

Yimin Zhang

References

The Communist Party of China and State Council. The central committee of the communist party of China and the state council’ s discussion about guidelines on deepening the reform of the medical and health care system. New Laws and Regulations. 2009;15:3-18.

Jin

PM.

The impact and thinking of the health care reform on public hospitals. HER. 2011;(7):20-21.

Dan

Zhang

GT.

The practice and discussion on medical resources integration in Waigaoqiao, Pudong new area of Shanghai. CHR. 2011;14(5):325-326.

Chen

The effect evaluation of the clinical pathway management on Hypertensive self-management. CGP. 2011;14(22):2558-2560.

Che

LH.

The Status Analysis and Evaluation Research of the Hospital Scale and Layout Construction in Shanghai. Shanghai, China: Fudan University; 2012.

Jin

Sun

Lou

et al . The practice and reflections of the family doctor in Pudong new area. HER. 2013;(8):36-39.

Wang

Yang

et al . The publicity effect evaluation of the nation system for basic drugs in Pudong new area of Shanghai. Health Education and Health Promotion. 2011;6(3):203-204.

Jin

Sun

Liang

et al . How to cancel the medical cost through drug-selling profits: based on empirical studies on Pudong new area of Shanghai. CHR. 2013;32(6):18-20.

Piper

Iedema

Gray

Verma

Holmes

Manning

Utilizing experience-based co-design to improve the experience of patients accessing emergency departments in New South Wales public hospitals: an evaluation study. Health Serv Manage Res. 2012;25(4):162-172. doi:10.1177/0951484812474247.

10.

Gautam

Hicks

Johnson

Mishra

Measuring the performance of critical access hospitals in Missouri using data envelopment analysis. J Rural Health. 2013;29(2):150-158. doi:10.1111/j.1748-0361.2012.00439.x.

11.

Lotfi

Kalhor

Bastani

et al . Various indicators for the assessment of hospitals’ performance status: differences and similarities. Iran Red Crescent Med J. 2014;16(4):e12950.

12.

Jia

JP.

The Statistics. Beijing: Renmin University of China; 2008.

13.

Liu

SC.

The Modern Economics Dictionary. Nanjing, China: Jiangsu Renmin Press; 2005.

14.

GX.

The frontier of the statistics index theory and application research in China. Stat Decis. 2007;(10):169-170.

15.

LaMontagne

Stoddard

Roelofs

Sembajwe

Sapp

Sorensen

A hazardous substance exposure prevention rating method for intervention needs assessment and effectiveness evaluation: the Small Business Exposure Index. Environ Health. 2009;8:10.

16.

Bos

Sturms

Schrijvers

van Stel

HF.

The Consumer Quality index (CQ-index) in an accident and emergency department: development and first evaluation. BMC Health Serv Res. 2012;12:284.

17.

Gao

Jia

Zhan

LF.

The research of founded the booming index in land market of Hang Zhou. China Real Estate. 2011;(10):63-69.

18.

The discussion of a few of the basic questions on CPI. Stat Res. 2007;24(8):30-35.

19.

Ding

Zhang

QZ.

The construction of service systems of live weather index in Beijing. Global Marhet Information Guide. 2013;(8):29-30.

20.

Zhao

Wang

WP.

Analysis of “accident” of performance appraisal results. Modern Management Science. 2008;(6):82-83.

21.

Wei

et al . Public-interest-oriented indicator system for performance assessment in public hospitals. Chinese J Health Pol. 2014;7(6):16-21.

22.

Xiong

JX.

Public hospital achievementsappraisal index system suggestions. Chinese Health Service Management. 2014;(6):404-406.

23.

Fang

et al . Performance evaluation index system and its application of public hospitals. Chinese Journal of Health Statistics. 2013;30(2):267-269.

24.

Hamilton

WT.

Mathematical Analysis. New York, NY: Harper & Brothers; 1956.

25.

Gao

MH.

The Report of the Public Company’s Financial Governance Index in China. Beijing, China: Economic Science; 2011.

26.

Kane

Bernardin

Villanova

Peyrefitte

Stability of rater leniency: three studies. Acad Manag J. 1995;38(4):1036-1051.

27.

Nuti

Wang

Masoudi

et al . Improvements in the distribution of hospital performance for the care of patients with acute myocardial infarction, heart failure, and pneumonia. Med Care. 2015;53(6):485-491. doi:10.1097/MLR.0000000000000358.

28.

Rezaee

Karimdadi

Do geographical locations affect in hospitals performance? a multi-group data envelopment analysis. J Med Syst. 2015;39(9):85. doi:10.1007/s10916-015-0278-3.

29.

Haddad

Cecatti

Souza

et al . Applying the maternal near miss approach for the evaluation of quality of obstetric care: a worked example from a multicenter surveillance study. Biomed Res Int. 2014: 989815. doi:10.1155/2014/989815.

30.

Berkowitz

Traore

Singer

Atlas

SJ.

Evaluating area-based socioeconomic status indicators for monitoring disparities within health care systems: results from a primary care network. Health Serv Res. 2015;50(2):398-417. doi:10.1111/1475-6773.12229.

31.

Sun

XM.

The Health Development Report in Pudong New District in 2013. Shanghai, China: Science and Technology; 2013.