Performance Evaluation of University-Based Scientific Research Institutions With a Non-Independent Parallel System

Abstract

Performance management in university-based scientific research institutions is essential for driving reform, advancing education quality, and fostering innovation. However, current performance evaluation models often focus solely on research indicators, neglecting the critical interdependence between the education and research systems. This oversight leads to inefficiencies in resource allocation and an underestimation of overall institutional performance, particularly in universities with varying development levels. To address these deficiencies, this study introduces two innovative two-stage data envelopment analysis models: the group-frontier and meta-frontier evaluation models. The findings are validated using data from 144 universities across China. They demonstrate that the proposed models effectively mitigate the underestimation of efficiency in traditional models and accurately reflect the intertwined nature of university subsystems and the disparities in university development. These results offer valuable insights for improving the performance of scientific research institutions and informing the strategic decisions of university administrators and government education departments.

Keywords

university scientific research institutions performance evaluation non-independence difference

Get full access to this article

View all access options for this article.

References

Abramo

D’Angelo

C. A.

Pugini

(2008). The measurement of Italian universities’ research productivity by a non parametric-bibliometric methodology. Scientometrics, 76(2), 225–244. https://doi.org/10.1007/s11192-007-1942-2

Acs

Z. J.

Song

A. K.

Szerb

Audretsch

D. B.

Komlósi

É.

(2021). The evolution of the global digital platform economy: 1971–2021. Small Business Economics, 57(4), 1629–1659. https://doi.org/10.1007/s11187-021-00561-x

Bogt

H. J. t.

Scapens

R. W.

(2012). Performance management in universities: Effects of the transition to more quantitative measurement systems. European Accounting Review, 21(3), 451–497. DOI: 10.1080/09638180.2012.668323.

Cadez

Dimovski

Zaman Groff

(2017). Research, teaching and performance evaluation in academia: The salience of quality. Studies in Higher Education, 42(8), 1455–1473. https://doi.org/10.1080/03075079.2015.1104659

Chambers

R. G.

Chung

Färe

(1996). Benefit and distance functions. Journal of Economic Theory, 70(2), 407–419. https://doi.org/10.1006/jeth.1996.0096

Chen

Patton

Kenney

(2016). University technology transfer in China: A literature review and taxonomy. The Journal of Technology Transfer, 41(5), 891–929. https://doi.org/10.1007/s10961-016-9487-2

Chen

Liu

(2023). Regret-based cross efficiency evaluation method in a general two-stage DEA system. Computers & Industrial Engineering, 175, 108828. https://doi.org/10.1016/j.cie.2022.108828

Chen

W. C.

(2021). On performance evaluation with a dual-role factor. Annals of Operations Research, 304(1-2), 63–84. https://doi.org/10.1007/s10479-021-04102-3

Chen

Cook

W. D.

Zhu

(2009). Additive efficiency decomposition in two-stage DEA. European Journal of Operational Research, 196(3), 1170–1176. https://doi.org/10.1016/j.ejor.2008.05.011

10.

Chen

Yan

Wang

(2021). Operating efficiency in Chinese universities: An extended two-stage network DEA approach. Journal of Management Science and Engineering, 6(4), 482–498. https://doi.org/10.1016/j.jmse.2021.08.005

11.

Chung

Y. H.

Fare

Grosskopf

(1997). Productivity and undesirable outputs: A directional distance function approach. Journal of Environmental Management, 51(3), 229–240. https://doi.org/10.1006/jema.1997.0146

12.

Compagnucci

Spigarelli

(2020). The Third mission of the university: A systematic literature review on potentials and constraints. Technological Forecasting and Social Change, 161, 120284. https://doi.org/10.1016/j.techfore.2020.120284

13.

Dimyati

Hermanu

A. I.

(2023). Evaluating research efficiency in Indonesian higher education institution. Evaluation Review, 47(2), 155–181. https://doi.org/10.1177/0193841X221118181

14.

Ding

Zhang

(2023). Performance evaluation of Chinese research universities: A parallel interactive network DEA approach with shared and fixed sum inputs. Socio-Economic Planning Sciences, 87, 101582. https://doi.org/10.1016/j.seps.2023.101582

15.

Gallardo

(2021). The importance of assessment literacy: Formative and summative assessment instruments and techniques. In Babo

Dey

Ashour

A.S.

(Eds.), Workgroups assessment: Planning, implementing and analysing frameworks. Intelligent systems reference library (Vol. 199, pp. 3–25). Springer. https://doi.org/10.1007/978-981-15-9908-8_1

16.

Gralka

Wohlrabe

Bornmann

(2019). How to measure research efficiency in higher education? Research grants vs. publication output. Journal of Higher Education Policy and Management, 41(3), 322–341. https://doi.org/10.1080/1360080x.2019.1588492

17.

Heil

Ifenthaler

(2023). Online assessment in higher education: A systematic review. Online Learning, 27(1), 187–218. https://doi.org/10.24059/olj.v27i1.3398

18.

Huang

C. W.

Ting

C. T.

Lin

C. H.

Lin

C. T.

(2013). Measuring non-convex metafrontier efficiency in international tourist hotels. Journal of the Operational Research Society, 64(2), 250–259. https://doi.org/10.1057/jors.2012.52

19.

Jiang

J. L.

Lee

S. K.

Rah

M. J.

(2020). Assessing the research efficiency of Chinese higher education institutions by data envelopment analysis. Asia Pacific Education Review, 21(3), 423–440. https://doi.org/10.1007/s12564-020-09634-0

20.

Johnes

(2008). Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Economic Review, 19(4), 679–696. https://doi.org/10.1016/j.chieco.2008.08.004

21.

Kang

Y. K.

Liu

R. M.

(2021). Does the merger of universities promote their scientific research performance? Evidence from China. Research Policy, 50(1), 104098. https://doi.org/10.1016/j.respol.2020.104098

22.

Kao

C. A.

Hwang

S. N.

(2011). Decomposition of technical and scale efficiencies in two-stage production systems. European Journal of Operational Research, 211(3), 515–519. https://doi.org/10.1016/j.ejor.2011.01.010

23.

Chen

Cook

W. D.

Zhang

Zhu

(2018). Two-stage network DEA: Who is the leader? Omega, 74, 15–19. https://doi.org/10.1016/j.omega.2016.12.009

24.

Liang

Yang

Cook

W. D.

Zhu

(2006). DEA models for supply chain efficiency evaluation. Annals of Operations Research, 145(1), 35–49. https://doi.org/10.1007/s10479-006-0026-7

25.

Liu

Zhou

Liu

Shen

(2015). Two-stage DEA models with undesirable input-intermediate-outputs. Omega, 56, 74–87. https://doi.org/10.1016/j.omega.2015.03.009

26.

Liu

X. H.

Zhang

W. P.

(2024). A new DEA model and its application in performance evaluation of scientific research activities in the universities of China’s double first-class initiative. Socio-Economic Planning Sciences, 92, 101839. https://doi.org/10.1016/j.seps.2024.101839

27.

J. F.

(2015). A two-stage DEA model considering shared inputs and free intermediate measures. Expert Systems with Applications, 42(9), 4339–4347. https://doi.org/10.1016/j.eswa.2015.01.040

28.

Meng

Xiong

(2021). Logical efficiency decomposition for general two-stage systems in view of cross efficiency. European Journal of Operational Research, 294(2), 622–632. https://doi.org/10.1016/j.ejor.2021.01.044

29.

Nazarko

Šaparauskas

(2014). Application of DEA method in efficiency evaluation of public higher education institutions. Technological and Economic Development of Economy, 20(1), 25–44. https://doi.org/10.3846/20294913.2014.837116

30.

Nemati

Kazemi Matin

Toloo

(2020). A two-stage DEA model with partial impacts between inputs and outputs: Application in refinery industries. Annals of Operations Research, 295(1), 285–312. https://doi.org/10.1007/s10479-020-03665-x

31.

D. H.

(2010). A metafrontier approach for measuring an environmentally sensitive productivity growth index. Energy Economics, 32(1), 146–157. https://doi.org/10.1016/j.eneco.2009.07.006

32.

Pastor

J. M.

Serrano

(2016). The determinants of the research output of universities: Specialization, quality and inefficiencies. Scientometrics, 109(2), 1255–1281. https://doi.org/10.1007/s11192-016-2102-3

33.

Qin

X. H.

D. B.

(2018). Measuring universities’ R&D performance in China’s provinces: A multistage efficiency and effectiveness perspective. Technology Analysis & Strategic Management, 30(12), 1392–1408. https://doi.org/10.1080/09537325.2018.1473849

34.

Seiford

L. M.

Zhu

(1999). Profitability and marketability of the top 55 US commercial banks. Management Science, 45(9), 1270–1288. https://doi.org/10.1287/mnsc.45.9.1270

35.

Sexton

T. R.

Lewis

H. F.

(2003). Two-stage DEA: An application to major league baseball. Journal of Productivity Analysis, 19(2/3), 227–249. https://doi.org/10.1023/a:1022861618317

36.

Sudakova

N. E.

Savina

T. N.

Masalimova

A. R.

Mikhaylovsky

M. N.

Karandeeva

L. G.

Zhdanov

S. P.

(2022). Online formative assessment in higher education: Bibliometric analysis. Education Sciences, 12(3), 209. https://doi.org/10.3390/educsci12030209

37.

Tiedemann

Francksen

Latacz-Lohmann

(2011). Assessing the performance of German bundesliga football players: A non-parametric metafrontier approach. Central European Journal of Operations Research, 19(4), 571–587. https://doi.org/10.1007/s10100-010-0146-7

38.

Worthington

A. C.

Lee

B. L.

(2008). Efficiency, technology and productivity change in Australian universities, 1998–2003. Economics of Education Review, 27(3), 285–298. https://doi.org/10.1016/j.econedurev.2006.09.012

39.

Zhu

Zhang

(2021). Two-stage DEA models with fairness concern: Modelling and computational aspects. Omega, 105, 102521. https://doi.org/10.1016/j.omega.2021.102521

40.

Yin

Sun

Chu

Liang

(2016a). Evaluating the environmental efficiency of a two-stage system with undesired outputs by a DEA approach: An interest preference perspective. European Journal of Operational Research, 254(3), 1047–1062. https://doi.org/10.1016/j.ejor.2016.04.034

41.

Zhu

Chu

Liang

(2016b). Two-stage network processes with shared resources and resources recovered from undesirable outputs. European Journal of Operational Research, 251(1), 182–197. https://doi.org/10.1016/j.ejor.2015.10.049

42.

Zhang

C. H.

Jiang

N. Y.

T. T.

Chen

Streimikiene

Balezentis

(2022). Spreading knowledge and technology: Research efficiency at universities based on the three-stage MCDM-NRSDEA method with bootstrapping. Technology in Society, 68, 101915. https://doi.org/10.1016/j.techsoc.2022.101915

43.

Zhang

Zhao

L. L.

Zha

(2021). Efficiency evaluation of Chinese regional industrial systems using a dynamic two-stage DEA approach. Socio-Economic Planning Sciences, 77, 101031. https://doi.org/10.1016/j.seps.2021.101031

44.

Zhang

Xia

Wei

(2023). Efficiency evaluation of two-stage parallel-series structures with fixed-sum outputs: An approach based on SMAA and DEA. Expert Systems With Applications, 227, 120264. https://doi.org/10.1016/j.eswa.2023.120264