Data Envelopment Analysis and Higher Education: A Systematic Review of the 2018–2022 Literature and Bibliometric Analysis of the Past 30 Years of Literature

Abstract

The interest in Data Envelopment Analysis (DEA) has grown since its first put forward in 1978. In response to the overwhelming interest, systematic literature reviews, as well as bibliometric studies, have been performed in describing the state-of-the-art and offering quantitative outlines with regard to the high-impact papers on global applications of DEA and the higher education system (DEA-HE). This study examines 75 systematic literature review (SLR) studies published between 2018 and 2022 and 508 bibliometric studies published between 1992and 2022. Four performance-focused areas are identified through SLR analysis: institutional performance, departmental performance, performance of study program, and performance of other higher education (HE) activities. This study highlights issues, methods, and resolutions in selected SLR literature. Bibliometric analysis revealed an increasing trend in DEA-HE since 2003, with the highest number of publications in 2021. Tommaso Agasisti was the most productive author, and Jill Johnes was the most influential. The journal Scientometric had the most publications in the area. This study lays the groundwork for future research. Future reviewers may find the common practises, constraints, and underlying assumptions presented in this study useful for the selection and analysis of relevant studies.

Plain language summary

This study undertakes a comprehensive review of the evolution of Data Envelopment Analysis (DEA) in the field of higher education. The aim is to conduct an in-depth examination, encompassing 75 systematic literature reviews (SLRs) published between 2018 and 2022, along with 508 bibliometric studies spanning from 1992 to 2022. In the SLR analysis, we have identified four main areas of research: institutional performance, departmental performance, study program performance, and performance in other aspects of higher education (HE). This detailed investigation helps us discover important issues, research methods, and solutions frequently found in the SLR literature. Our bibliometric analysis shows a clear increase in DEA-HE research since 2003, peaking in the number of publications in 2021. Tommaso Agasisti is the most prolific author, and Jill Johnes is the most influential in this field. The journal Scientometric has the most publications in this area. In summary, this study provides valuable insights for future research endeavors. Researchers and reviewers can leverage the established practices and duly acknowledge the inherent limitations when assessing relevant studies. This contribution lays the foundation for a more informed exploration within this academic discipline.

Keywords

bibliometrics data envelopment analysis higher education performance PRISMA systematic literature review

Introduction

The field of higher education (HE) has a well-established history of adjusting to changing societal and labor market needs. During the 1980s, there was a notable decrease in public expenditure across several developed nations, which sparked a discussion regarding the efficiency of HE. This conversation has continued to evolve as student enrollments surged, prompting concerns about higher education institutions’ (HEIs) performance (Kupriyanova et al., 2018). The recent emergence of the COVID-19 pandemic has heightened the challenges currently faced by HE. These new challenges have placed HEIs under unprecedented pressure, testing their adaptability and resilience.

For HEIs and decision-makers, increasing efficiency is acknowledged as a highly desired goal. The challenge of meeting the increasing demand for HE has compelled governments to prioritize increased efficiency as a central policy objective. Efficiency demonstrates responsible management by making the most use of available resources and minimizing waste.

A nonparametric Data Envelopment Analysis (DEA) is a common tool used by researchers in evaluating efficiency and productivity. Over four decades, DEA has evolved into a key tool for assessing the performance of entities engaged in diverse activities. It excels in analyzing complex relations that involve multiple inputs and outputs. Initially established in 1978 through the seminal contributions of Charnes, Cooper, and Rhodes, DEA has found significant application across various fields, including transportation, finance, healthcare, agriculture, tourism, construction, and education. In their originating article, DEA is described by Charnes et al. (1978) as a data-driven tool that uses mathematical programing for performance evaluation and benchmarking. The article also introduced the initial DEA model, which is commonly referred to as the CCR model. In another study, Banker et al. (1984) introduced the BCC model, which is a second DEA model. Both models are classical DEA models. There are now several DEA extensions available for evaluating performance.

In academia, various researchers have explored different aspects of the DEA literature. Cooper et al. (2007) explored the development of models and methodologies designed to address issues within DEA formulation. Kao (2014) conducted a review of the network DEA (NDEA), scrutinizing both the models employed and the structure of the network systems under investigation. Mardani et al. (2017) reviewed DEA models related to energy efficiency. Mariz et al. (2018) conducted a comprehensive review paper on the existing literature pertaining to the evolution of dynamic DEA. Cui and Yu (2021) conducted a detailed review of the literature pertaining to the application of DEA models in assessing airline efficiency. Panwar et al. (2022) address the benefits, drawbacks, and uses of several distinct DEA models. Additionally, a paper authored by de Oliveira et al. (2023) undertook a thorough examination of DEA, Multi-criteria Decision Analysis (MCDA), and Cluster Analysis models. The objective was to identify existing and potential applications, as well as emerging trends.

Setting the Stage: Context and Objectives

With the continuous expansion of the HE landscape, there has been a corresponding increase in the amount of literature available in this field. Over the years, numerous studies have extended, proposed, and applied DEA models to assess efficiency in various aspects of HE across different countries. As this body of work continues to grow, it becomes increasingly imperative to gather and synthesize the insights derived from research employing DEA methodology in this field. This study, therefore, embarks on an examination of the application of DEA in evaluating the performance of HEIs, referred to as DEA-HE, through a systematic literature review (SLR) and bibliometric analysis. Throughout this review, we present examples of how various models have been utilized to gage HE efficiency, shedding light on both the identified issues and their corresponding solutions within each selected paper. Notably, research encompassing SLR and bibliometric analysis that delves into the implementation of various DEA models within HE is currently scarce. Hence, we endeavor to fill this gap by conducting a review of this topic. Consequently, the primary aim of this study is to compile existing literature related to the application of DEA in HE, evaluate the current research landscape, and offer insights into potential opportunities for future research within this domain. To address our primary research objective, we have structured five key research questions (RQ):

RQ1: What is the current research on DEA-HE areas under investigation, and what are the different emerging variants of DEA that have arisen in this field in recent years?

RQ2: What key information can be collected from these studies, and how have publication trends evolved over time in the field of DEA applied to HE?

RQ3: Which journals are the most relevant, and how do they influence and impact DEA-HE studies? Furthermore, which articles have the most significant influence, who are the most prolific authors, and what impact have they had on the field of DEA-HE studies?

RQ4: What are the trend topics of DEA-HE studies?

RQ5: What are the bibliographic diagrams and graphs for studies of DEA-HE?

SLR and bibliometric analyses are performed through systematic framework and expert evaluation to understand various applications of DEA in HE. Data from two datasets (N = 74 and 508) sourced from the Thomson Reuters Web of Science (WoS) database were used to address the research questions. The first research question is addressed through SLR on 75 articles published between 2018 and 2022 to validate the recent information in DEA-HE. The remaining research questions (2–5) are addressed through a bibliometric analysis of 508 articles published between 1992 and 2022. This study covers both descriptive and network analysis of bibliometrics to gain an understanding of the present state of the field and guide future research directions. This process helps to identify gaps in the existing literature and highlights areas that can be explored in future research. Moreover, there exists an opportunity to enhance the understanding of DEA as an instrument for appraising the efficiency and productivity of HEIs, as well as its prospective utilization across various domains. The outcomes of this study are anticipated to provide policymakers, administrators, and academics with valuable insights regarding the integration of DEA into the decision-making and performance evaluation disciplines.

The following is how the paper is set up. The study’s objectives and research questions are described in the first part. The methodology and search techniques used to locate DEA-related publications in the field of HE are described in the second section. The third section presents and discusses the results of the bibliometric and SLR analyses. In the final section, conclusions are made, and recommendations for future research are offered.

Methodology

This study follows Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) standards when conducting systematic literature reviews and bibliometric analyses.

As Figure 1 illustrates, we initiated the process by conducting a literature search of the WoS Core Collection database, utilizing the “topic search” option. This feature allowed us to search across titles, abstracts, author keywords, and keywords plus fields. The search was performed on October 3, 2022. The reason why we mainly prefer WoS over other databases is because it looks up information across publishers and is not biased against them (Bartolacci et al., 2020). More importantly, WoS ensures the inclusion of the most significant journals (Bartolacci et al., 2020; Kullenberg & Kasperowski, 2016).

Figure 1.

Workflow for analyzing DEA-HE literature.

Our article selection process adheres to specific criteria, which we have conveniently summarized in Table 1. This screening process narrows down our findings to encompass only peer-reviewed articles in English published between 2018 and 2022, resulting in a total of 398 articles. Articles that did not align with the research topic were flagged for removal after the titles and abstracts had been independently read. If the titles and abstracts did not provide sufficient information for screening, we conducted a full-text review. At the conclusion of this rigorous process, we selected 75 recent studies on DEA in HE for inclusion in our SLR analysis.

Table 1.

Article Selection Criteria.

No.	Criteria	Including
1.	Papers type	Research article (peer-reviewed)
2.	Language	English
3.	Period	2018–2022 (SLR analysis)
		1992–2022 (Bibliometric analysis)
4.	Type of database	Web of Science (WoS) Core Collection
5.	Scope	Only articles with DEA applications in higher education are considered.
6.	Search strings	‘The keywords chosen are associated with the notion of DEA-HE and its synonyms(“data envelopment analysis” OR “DEA”) AND (“efficient” OR “performance”) AND (“highereducation” OR “tertiary education” OR “universit*”)
Literature assessment method
1.	SLR	To address RQ1, a SLR is conducted on 75 relevant literature from the year 2018 to 2022.
2.	Bibliometric analysis	To address RQ2-RQ5, a bibliometric analysis is conducted on 508 bibliographic data from the year1992 to 2022. The analysis is carried out using an “R-tool”“bibliometrix” and “biblioshiny.”

In Joshi’s (2014) paper, he discussed how one can evaluate the quality and quantity of publications using statistical and mathematical measures called bibliometric indicators. Additionally, within his work, he made distinctions among three types of bibliometric indicators: quantitative, performance, and structural. Quantitative indicators are used to gage a researcher’s productivity. Performance indicators, on the other hand, consist of metrics like the Impact Factor that assess the quality of journals or researchers. Nevertheless, structural indicators help find connections between publications, authors, and study areas. This study specifically focuses on quantitative and structural bibliometric indicators. This choice stems from the fact that quantitative indicators can evaluate researcher productivity based on variables such as the number of articles, citations, and the prevalence of highly cited papers. In contrast, structural indicators, such as network analysis, are used to reveal the relationships between publications, authors, and research fields. Consequently, these indicators provide a deeper understanding of the field of academic research.

In this study, the identification of bibliographic data follows a process akin to the search method used for obtaining data for the SLR, with the exception of excluding time filtration. This means that, in arriving at the bibliographic data, the records are identified using the same keywords in the WoS Core Collection database, and articles are filtered to only include English peer-reviewed articles without any limitation on the period of publication. The search process initially resulted in the identification of 805 articles. From these values, 297 articles are removed after thorough screening, and finally, a total of 508 articles are included for bibliometric analysis. The process of data retrieval from the WoS database for SLR and bibliometric analysis is shown in Figure 2.

Figure 2.

Data retrieval process.

Findings

The SLR findings of the current DEA-HE efforts are presented in the first subsection. The bibliometric analysis findings, which include descriptive and network analyses, are covered in the second subsection.

Findings on SLR Analysis

The review resulted in four performance-based themes or areas that are dependent on decision-making-unit (DMU) or entities. The definition of DMU is broad and flexible, and because of that, this study is constructed around the performance of various DMUs being evaluated, institutional performance, departmental performance, performance of study program, and performance of other HE activities. We highlight the issues of each study, the DEA-methods used, and the strategy used to address the issues, as displayed in Tables 2 to 5.

Table 2.

Institutional Performance.

No.	Author/Year	Country	Method	Problem addressed	Proposed solution
1	Canto and Lopez (2018)	Spain	DEA-VRS (O-O)	The efficiency of 42 European universities in terms of academic function, university graduate labour insertion, technological innovation, and global university efficiency.	DEA is employed in assessing efficiency. The results list universities based on the overall level of efficiency in which they are located, and specific aspects of global efficiency should be focused on.
2	de Jorge Morenoet al. (2019)	Spain	Windows DEA, Debreu-Farrell (O-O), Russell non-radial measure (O-O)	The efficiency of Spanish universities in a longitudinal framework between 2008/9–2014/15.	The study performed two windows, DEA and intertemporal, radial (Debreu-Farrell O-O) and non-radial analysis (Russell non-radial O-O) methodologies are compared.
3	de la Torre et al. (2018)	Spain	Bootstrap-DEA, multidimensional scaling analysis, cluster analysis	To provide a different option for institutional evaluation that respects and fosters university diversity.	The study suggests bootstrap DEA and MDS as a novel approach combination that delivers reliable findings. This analysis also gives information on the institutional elements that affect how technical efficiency varies by mission among Spanish universities.
4	Di (2018)	China	Bootstrap-DEA, Tobit regression	The factors that influence conversion efficiency of S&T achievements in colleges and universities.	This study employs the two-step DEA and Tobit regression methodology. The output measurement index used to assess the conversion efficiency of S&T achievements at 24 Chinese universities is the conversion efficiency of S&T achievements in colleges and universities. Additionally, significant variables affecting the conversion efficiency of S&T achievements are examined.
5	Lita (2018)	Romania	DEA-VRS, DEA-CRS, MPI	Technical and allocative efficiencies of Romanian public universities.	The study was carried out using DEA-VRS and DEA-CRS oriented to both the output and the input. The universities are also ranked using the Pareto efficiency model.
6	Qin and Du (2018)	China	NDEA, Gaussian kernel, Tobit regression	The R&D performance of universities relies on two dimensions, which are effectiveness and efficiency.	The R&D effectiveness of universities is evaluated using a multi-index comprehensive evaluation technique, and their R&D efficiency is determined using NDEA. Then, a thorough evaluation of universities’ R&D performance in terms of efficiency and effectiveness is provided, together with visual representations of these outcomes.
7	Quiroga-Martinez et al. (2018)	Argentina	DEA-CCR (O-O), fixed effect estimator	The efficiency of Argentinean public universities and its determinants from the year 2004 to 2013. The effect of institutional variables on the efficiency scores is examined.	The paper employed a two-stage process. The DEA-CCR was first applied annually from 2004 to 2013 along the entire period. Second, stage 1 efficiency scores are employed as a dependent variable and regressed on a number of variables that may have an impact on them. A fixed effect (FE) estimator is applied in controlling the university-specific heterogeneity.
8	C. T. T. Tran and Villano (2018)	Vietnam	Dynamic NDEA, DEA-VRS, Spearman’s rank correlation	To look into dynamic shifts in public college efficiency in a network structure.	The study offers insights into the dynamic change in the efficiency of Vietnamese public colleges from 2011 to 2013 using the dynamic NDEA model.
9	Villano and Tran (2018)	Australia	bootstrap-DEA	The efficiency of private universities in Vietnam for the year 2011/12 to 2013/14.	Employing the subsampling bootstrap directional distance function approach with quasi-fixed inputs, the technical efficiency of private universities in Vietnam is studied, and the corresponding confidence interval of the true efficiency level is computed. This approach takes into account nondiscretionary inputs that would normally be ignored in the production process, for instance, HE, and it is effective for locating the most extreme universities with extremely high inefficiency ratings and very wide confidence intervals.
10	Wijesundara and Prabodanie (2022)	Sri Lanka	DEA (I-O, O-O)	Efficiency and capacity of universities to increase students’ enrollments.	Efficiency assessment via DEA.
11	Wolszczak-Derlacz (2016)	Poland	MPI, generalised least squares (GLS)	To examine and compare the productivity of 500 universities in 10 European countries and the USA from year 2000 to 2010.	There are several phases to this investigation. To compare total factor productivity (TFP) in European and American HEIs, a time series analysis is first conducted. Second, the MPI indices are broken down into technology effects and pure efficiency. Third, the primary causes of productivity shifts are looked into. Finally, statistical analysis using bootstrap techniques is used to evaluate the statistical significance of MPI indices.
12	Yang et al. (2018)	China	NDEA, network Luenberger productivity indicator	The inefficiency of Chinese research universities is based on the two-stage NDEA.	In order to evaluate university inefficiencies, the study first builds a general two-stage network directional distance framework involving carryover variables. To evaluate productivity changes over time and indicator decompositions, the network Luenberger productivity indicator is suggested.
13	Salas-Velasco (2020)	Spain	DEA-CRS (O-O), DEA-VRS (O-O), bootstrapped truncated regression, Spearman’s rank correlation	The efficiency and rank of universities based on teaching and research. Also, the factors affecting the technical efficiency of universities are investigated.	A 2-stage procedure is used in assessing the efficiency of universities with respect to teaching and research. First, DEA-CRS and DEA-VRS are employed in evaluating the efficiency in the first stage. To explore the factors influencing the efficiency of DMUs, DEA efficiency scores are utilized as a dependent variable in a bootstrapped truncated regression. Additionally, Spearman’s rank correlations between efficiency scores and universities’ placement in university rankings were performed in the study (SCImago and Shanghai rankings).
14	Shamohammadi and Oh (2019)	Korea	DEA-CRS, NDEA	The changes in the efficiency of universities based on teaching and research are examined.	A two-stage NDEA—a feedback model with a dynamic-based measurement is utilized to evaluate the efficiency changes of universities. The variations in the efficiency ranking from DEA-CRS are also presented.
15	Gao et al. (2019)	China	Two-stage DEA, Tobit regression	Technology transfer efficiency of Chinese universities and the effects of government funding and reputation on technology transfer efficiency are studied.	The technology transfer process is divided into two: research innovation and value creation. In order to evaluate how well universities transfer technology, a two-stage DEA model is used. Tobit regression was used in the study to analyze how government funding and reputation affected the efficiency of technology transfer.
16	Zhang and Shi (2019)	China	DEA-CCR (O-O), principal component analysis (PCA)	Reduce the size of the variables, get rid of correlation, and compare the efficiency of HEIs (colleges and universities).	To assess HEIs’ efficiency, we employ PCA for performance assessment indexes in scientific and engineering fields, along with DEA-CCR.
17	Duan (2019)	Australia	DEA-VRS (O-O), strategic group analysis, multivariateregression analysis.	Efficiency and source of inefficiency of Australian universities are based on three perspectives: teaching efficiency, overall university operations efficiency, and research efficiency.	The efficiency of using DEA-VRS is examined. To set a baseline for inefficient institutions in each strategic group, a strategic group analysis is carried out. To comprehend the contribution of teaching and research efficiency to total operations efficiency, a multivariate regression analysis is conducted.
18	Szuwarzynski (2019)	Australia	Composite indicators (CIs), benefit of the doubt (BoD)	A model to evaluate the performance of research activities is proposed and to make recommendations for the worst-performing units relying on types and degrees of changes required to raise efficiency.	The CIs and BoD measurement and evaluation techniques are discussed. Due to these approaches’ flaws, the empirical verification produced a range of outcomes. It was discovered that the weighting system used was the root of the disparities. The issue was resolved by adding more constraint conditions to the fundamental BoD algorithm, doing away with the issue of zero weights, and making sure that all factors are taken into account when calculating the efficiency score. The results are employed to create rankings and to do more thorough investigations of the causes (better or worse position in the ranking). It is feasible to identify the advantages and disadvantages of each university using CIs. It is feasible to determine projections for the worst-performing universities using the BoD model.
19	Mousa and Ghulam (2019)	Saudi Arabia	Free disposal hull (FDH), DEA, order-m, α-quantile, second stage regression analysis	Efficiency of HEIs in Saudi Arabia and to identify the main determinants of efficiency scores.	The study highlighted technical efficiency estimates derived from FDH, simple DEA, and order-m and α-quantile (conditional and unconditional) estimators. All HEIs are evaluated using their efficiency scores and hyperbolic graphs (in the case of DEA). Additionally, a second-stage regression is employed to better comprehend how linked elements affect efficiency.
20	Song et al. (2019)	China	MPI	Efficiency changes of universities.	The MPI is used in the study to look into the scientific output of Chinese universities. In China, university productivity is measured and explained using the MPI and its decompositions.
21	Wohlrabe et al. (2019)	Germany	FDH, DEA, order-m,α-quantile, regression analysis	Research efficiency of US universities.	The work uses FDH and partial frontier analysis, which uses the order-m and order-α to nest FDH and DEA. Rankings are created using efficiency scores that have been calculated and modified for institutional context and research focus.
22	Myeki and Temoso (2019)	Australia	DEA-VRS (I-O)	Technical efficiency of South African public universities.	DEA-VRS is employed in assessing the technical efficiency of universities.
23	Firsova and Chernyshova (2019)	Russia	DEA-VRS (O-O)	Develop mathematical models for the evaluation of HEIs effectiveness at a regional level.	The study suggested three distinct models for evaluating the efficiency of HEIs: scientific research, educational activity, innovations, and regional partnerships in the regional economy. DEA-VRS (O-O) is implemented to assess the efficiency of HEIs.
24	Navas et al. (2020)	Columbia	Cluster analysis, Extended Cross-efficiency DEA	A model in evaluating the efficiency of HEIs in the presence of flexible measures.	In the presence of flexible measures, a model to assess the efficiency and cross-efficiency of HEIs is suggested. The study also suggested four models (General, Teaching, Employment, as well as Research) for assessing the effectiveness of HEIs on several fronts. These models are helpful in figuring out how sensitive the model is to the choice and quantity of measures.
25	Wang et al. (2020)	Taiwan	Pearson correlation coefficient, MPI	Efficiency of New Zealand’s eight universities based on MPI.	The study tests the correlation between variables using Pearson to determine whether they are negatively or positively related. A positive correlation indicates consistency with the DEA rule. MPI is used to evaluate the productivity of universities.
26	Jiang et al. (2020)	China	Statistical analysis (average, standard deviation, frequency, ANOVA), DEA-CRS and DEA-VRS	Research efficiency of universities in China and selected universities’ efficiency with respect to region and type of universities.	DEA-CRS, DEA-VRS, and scale efficiency are performed to determine the research efficiency of universities.
27	Ghasemi et al. (2020)	Iran	DEA-CRS (O-O), Hierarchical network DEA (H-NDEA)	The study aims to develop models to measure the efficiency of a university’s campuses with a hierarchical structure.	By taking into account the internal structure, this study develops models to evaluate the efficiency of the whole DMU and its constituent units. The proposed model’s efficiency is contrasted with that of the DEA-CCR model.
28	Mammadov and Aypay (2020)	Turkey	DEA-CRS (I-O) (O-O, DEA-VRS (I-O) (O-O), Tobit regression	Efficiency of research universities and to examine the factors that impact the efficiency of the research universities.	The DEA-CRS and DEA-VRS are utilised to evaluate the efficiency of research universities. The Tobit model is employed in identifying coefficient values and factors affecting efficiency.
29	Moncayo-Martinez et al. (2020)	Mexico	Dual (enveloped) form (O-O), DEA-CRS (I-O)	Performance of HEIs based on teaching, research and knowledge dissemination.	The dual (enveloped) form of DEA-CRS (O-O) is used in evaluating the efficiency of the three models (teaching, research, and knowledge dissemination).
30	Tran et al. (2020)	Vietnam	NDEA	Efficiency of ‘teaching and research’ and ‘revenue’ efficiency of universities, network size economies.	Using the scale elasticity decomposition approach, the research develops a two-stage network structure, which includes “Revenue Efficiency” and “Teaching and Research Efficiency” to assess university performance. The universities are then ranked based on the outcomes of the scale elasticity of a two-stage NDEA.
31	Brzezicki et al. (2020)	Poland	DEA- Epsilon-based measure (EBM) (O-O)	Total teaching activity efficiency of HEIs. The teaching-related revenues and costs in the context of total efficiency indicators are also examined.	EBM (O-O) is used in evaluating the total efficiency of teaching activity.
32	Brzezicki (2020)	Poland	DEA-VRS (O-O), double- bootstrapped truncated regression	The efficiency of HEIs and the impact of external variables are examined.	The DEA-VRS is used to evaluate the efficiency of HEIs. The effect of environmental factors on efficiency is then researched. Both stages employ the Simar and Wilson bootstrapping.
33	Wu et al. (2020)	China	DEA-VRS (I-O), stochastic frontier analysis (SFA)	The efficiency of HEIs by considering the external environmental impact.	The HEIs efficiency is assessed using a three-stage process. The DEA-VRS is used initially. Second, SFA reduces the impact of random errors and environmental variables. Third, in order to analyze efficiency, DEA-VRS is once again performed using modified input or output data based on the orientation.
34	Moreno-Gomez et al. (2019)	Columbia	DEA-VRS (O-O), truncated regression	The efficiency of HEIs based on three models (teaching (T), research (R), and global T&R) and factors that contribute to universities’ performance are examined.	The study applies a two-stage analysis. First, DEA-VRS evaluates efficiency, then DEA scores are related to the additional variables via a parametric model, and a truncated regression model is used to evaluate efficiency.
35	Ranjan and Singh (2021)	India	DEA (categorical)	Efficiency of colleges with categorical variables.	Categorical DEA is applied in measuring the efficiency of colleges.
36	Gebru et al. (2021)	Pakistan	Joint DEA, bootstrapping	Efficiency of HEIs with shared inputs.	The combined DEA model is used in the study to examine the teaching, research, and overall efficiency of HEIs, and bootstrapping is utilized to remove the sensitivity of efficiency scores brought on by the distribution of (in)efficiency in the sample.
37	Wang and Wang (2021)	China	DEA-CRS (I-O), DEA-VRS(I-O)	Efficiency of scientific research.	The DEA-CRS and DEA-VRS were used in the study to examine the general status and trend in scientific research efficiency depending on level, type, and location. Additionally assessed are the distribution and utilization rates for funding for scientific research.
38	Li et al. (2021)	China	Slack-based Measure model (SBM) using super-efficiency DEA	The teaching efficiency of universities and the relation between teaching efficiency and dual prices based on the analyzed index are measured.	SBM model using super-efficiency DEA is used in assessing teaching efficiency.
39	Navickas et al. (2021)	Slovak Republic	DEA-CRS	Efficiency of universities.	The study measures efficiency using DEA-CRS and provides a ranking for universities.
40	Xia et al. (2021)	China	PCA, entropy method, MPI	Scientific research (SR) performance and its development trend.	The study uses PCA as well as the entropy method to form an evaluation index system for universities’ SR and MPI to measure the trend in the SR performance.
41	Ho and Lee (2021)	Taiwan	ANOVA, DEA-CRS (O-O)	The study examined differences in resources and entrepreneurial activity of universities and benchmarked their research commercialization performance.	The study employs ANOVA to determine major differences in resources, size, as well as pattern activity. Additionally, it uses DEA-CRS to evaluate the performance of research commercialization.
42	Xue et al. (2021)	China	DEA, MPI	The static and dynamic scientific research efficiency of HEIs.	A 3-stage DEA and MPI are applied in measuring the static and dynamic efficiencies of HEIs’ scientific research.
43	Thai and Noguchi (2021)	Japan	DEA, double-bootstrapped truncated regression	Technical efficiency with respect to universities.	The study utilizes DEA to investigate the trends in the efficiency of universities and a double-bootstrapped truncated regression model to determine the possible determinants of efficiency.
44	Ghimire et al. (2021)	Canada	Stochastic DEA	Assess the efficiency of three types of universities by developing a new DEA model.	A stochastic DEA model is developed and is used to measure the efficiency and rank three types of universities.
45	Kocak and Orkcu (2021)	Turkey	Two-stage DEA model,DEA-CCR (O-O)	Performance of universities through different two-stage DEA models and the development of an alternative model in the research universities’ assignment.	The performance of universities was analyzed using basic two-stage DEA models, both independent and related. A two-stage system was adopted to more thoroughly examine university performance, with the stages focusing on graduate education and scientific and technological research competency correspondingly.
46	Nkohla et al. (2021)	South Africa	DEA-VRS (O-O), DEA-CRS, ANOVA	Technical efficiency of universities with academic staff and non-academic staff are included as inputs.	The study employed DEA-VRS to evaluate the technical efficiency of universities and also calculated technical efficiency under DEA-CRS to evaluate scale efficiency. Additionally, one-way ANOVAF-statistics were used to determine the significant difference of variables across time.
47	Zhu et al. (2023)	China	Hierarchical clustering method, Tobit regression, DEA	Efficiency of innovation entities, research institutes, universities, and firms.	Efficiency assessment using nonhomogeneous DEA model. Provinces are clustered using the hierarchical clustering method. Meanwhile, Tobit regressions are employed to estimate the environmental factors’ impact on DMUs.
48	Dimyati and Hermanu (2022)	Indonesia	DEA (O-O)	Research efficiency of universities	Efficiency assessment using DEA.
49	Du and Seo (2022)	China	MPI	Efficiency changes in input and output of R&D activity	Changes in the input and output of R&D activity are accessed using MPI.
50	Sun et al. (2021)	China	MPI, enhanced Russell measure, Pearson correlation	To create a model that can evaluate the performance and output of universities in terms of scientific and technological (S&T) in a dynamic manner.	The integrated DEA (enhanced Russell) and MPI, which are used to assess the performance of DMUs, can detect temporal variations caused by improvements using historical data panels. The integrated evaluation methodology is employed to form an effective dynamic evaluation model for university S&T output of university technical and allocative efficiencies.
					Five performance indices are utilized in this research: production technology change, pure technology efficiency change, organizational management performance, scale effect change, and comprehensive efficiency change.
51	Ma et al. (2022)	China	Dynamic Network Slacks-based Measure, MPI	Efficiency of university technology transfer from a sequential process and decomposed perspective.	The stage efficiencies, inter-stage linkage efficiencies, and inter-period carryover efficiencies are evaluated via the Dynamic Network Slacks-based Measurement model. An MPI decomposition is then performed to determine the paths and constraints of efficiency improvement.
52	Pedro et al. (2022)	Portugal	DEA, Tobit regression	Efficiency of HEIs through research and technology (R&T), teaching and learning (T&L), and social responsibility (SR) activities. Moreover, the study assessed external factors influencing T&L, R&T, and SR, as well as the impact of efficiency on sustainable regional economic growth and innovation intensity.	Efficiency assessment through a two-step DEA and evaluation of the influence of the factors affecting HEI efficiency employing Tobit regression.
53	Cossani et al. (2022)	Chile	Dynamic Network Slacks-based Measure	Technical efficiency of universities via teaching and research. Here, the study also addressed the development of a model that incorporates network structures.	The study suggested a model that incorporates network structures, carryover activities, and shared inputs using a dynamic approach. The model was then applied to the university, and the outcomes were assessed against each institution’s ranking and accreditation status. The suggested model possesses the advantages that (a) there exist no subjective criteria and (b) it takes into account inputs, outputs over time, and the internal structure of the university production model.
54	Shi et al. (2022)	China	DEA-VRS, MPI	Categorical assessment in measuring efficiency with respect to scientific research.	The scientific research efficiency with respect to universities was analyzed using DEA-VRS and MPI.
55	Loganathan and Subrahmanya (2023)	India	SBM	The study compared the efficiencies of universities in providing entrepreneurial support to their traditional outcomes and also examined the extent to which universities can pursue entrepreneurial support.	The efficiencies of universities are assessed using DEA-SBM.
56	Zhang et al. (2022)	China	MCDM non-radial super-efficiency DEA, bootstrapping	Research efficiency at universities is addressed.	A model based on 3-stage MCDM non-radial super-efficiency DEA is proposed to measure the research efficiency. Bootstrapping is performed to correct the measurement errors.
57	Zhao et al. (2022)	China	NDEA	Efficiency with respect to universities’ scientific and technological (S&T) activities with shared input and regional differences are examined.	A 2-stage NDEA model with shared input is utilized in measuring the provincial efficiency of universities’ S&T activities.
58	Singh et al. (2022)	India	Super-efficiency DEA	Efficiency of HEIs and their ranking.	The super-efficiency DEA model is applied in measuring efficiency. The ranking of HEIs is assessed based on the efficiency scores.
59	Andersson and Sund (2022)	Sweden	DEA-CRS (O-O), MPI, bootstrapping	Technical efficiency and productivity of universities.	The efficiency of HEIs in the Nordic nations is being compared for the first time in this study. It also addresses methodological flaws in prior research by (a) adjusting the variable input quality of students entering HEIs based on their prerequisites, (b) addressing subject heterogeneity between HEIs by employing the Nordic countries’ national resource allocation systems, and (c) resolving the lack of statistical inference in DEA using bootstrapping.
60	Agasisti and Shibanova (2022)	Italy	BoD, DEA-VRS (O-O), panel-FE regression	It explores how performance and efficiency relate to issues of institutional autonomy (both formal and informal) at universities.	The study first calculates the “autonomy-in-use” (AiU) index via BoD methodology, which compares the actual level of a specific indicator to the ideal benchmark one. Subsequently, the efficiency of universities is assessed using DEA-VRS. The study also employed FE regression to investigate the effect of both formal autonomy and the AiU, utilizing panel data.
61	Lee and Johnes (2022)	Australia	NDEA	Efficiency with respect to universities’ teaching quality by incorporating the Teaching Excellence Framework (TEF) quality metrics and the factors underlying efficiency scores.	Universities use NDEA to assess the efficiency of their teaching by combining qualitative and quantitative data from the TEF. The efficiency effects of high-quality instruction and successful graduate job outcomes are recorded. In the second analysis, NDEA efficiency scores are contrasted with actual TEF results. Lastly, the factors that contributed to the efficiency scores are identified.

Table 3.

Departmental Performance.

No.	Author/Year	Country	Method	Problem addressed	Proposed solution
1	Gnewuch andWohlrabe (2018)	Germany	DEA-VRS (O-O), FDH,super-efficient DEA, order-m,order-α, Spearman’s rank.	Super-efficiency of economic departments around the world.	The study employed five methods for measuring efficiency, all of which are output-oriented and VRS assumption. The methods include DEA-VRS, super-efficient DEA, FDH, order-m, and order-α.
2	Kashim et al. (2018)	Malaysia	H-NDEA, DEA-CCR	A university faculty efficiency model for systems with a hierarchical structure.	The study proposed an efficiency model for systems with a hierarchical structure, which includes new sub-functions based on NDEA.
3	W. Zhu et al. (2018)	China	Fuzzy DEA-CCR	Teaching performance evaluation of a university subordinate collages (faculties).	The teaching performance with respect to the subordinate colleges of a university is assessed using the fuzzy DEA-CCR model.
4	González-Garayet al. (2019)	United Kingdom	DEA-VRS (O-O),	UK HEIs’ performance in chemical engineering studies.	The research implemented DEA-VRS to evaluate the efficiency of HEIs offering chemical engineering. The study discussed how the DEA approach, in combination with sensitivity analysis and a multi-period analysis, can add value to traditional rankings not only by securing high rankings but also by determining the optimal use of resources.
5	Kumar and Thakur(2019)	India	Dynamic DEA (DDEA)	Performance of institutions providing managementeducation in various parts of India and a proposal for a comprehensive efficiency measurement that could be used for all HEIs.	This study developed a methodology to measure efficiency while taking into account the time factor using DDEA and a holistic approach to rank HEIs.
6	Ding et al. (2021)	China	NDEA	Non-homogeneous DMUs’ performance.	An additive DEA model is proposed in assessing the overall efficiency of non-homogeneous DMUs (departments) with 2-stage NDEA.
7	S.-P. Chen and Chang (2021)	Taiwan	DEA-VRS (O-O),cluster analysis	Relative efficiency with respect to academic departments in a university and categorize the departments into groups.	To assess the relative efficiency of academic departments, the study used DEA-VRS. The study separated the departments into three groups—teaching, research, and general—based on the efficiency contributions of output variables, which are decomposed from aggregate efficiency and then divided into subgroups with comparable features.
8	Ersoy (2021)	Turkey	DEA-CCR, TOPSIS method, super-efficiency	Performance of distance education departments of public universities and their ranks.	The research employed the DEA-CCR to determine the efficiency of universities and then appliedthe super-efficiency DEA-CCR and TOPSIS methods to rank the universities.

Table 4.

Performance of Study Program.

No.	Author/Year	Country	Method	Problem addressed	Proposed solution
1	De La Hozet al. (2021)	Columbia	Cluster analysis, DEA,random forestmodel	Classification of university profiles and its academic efficiency. Also, to forecast the categories of academic efficiency, a machine learning model is considered.	To evaluate and project the academic efficiency of engineering programs, a three-phase technique is used. It starts using cluster analysis to create academic profiles and then employs DEA to judge how efficient these profiles are academically. Finally, the findings of the random forest model are presented in the prediction of efficiency profiles.
2	Delahoz-Dominguezet al. (2022)	Columbia	DEA-CCR,DEA-VRS, FDH	Measuring educational quality based on academic efficiency with Engineering programs as DMUs.	There are four phases of analysis: (a) context analysis, (b) database development and adequacy assessment (DEA), (c) exploratory data analysis, as well as (d) outcome analysis. End- and of-high-school standardized exam results and end-of-college exam results are introduced as new variables in the DEA model. Efficiency is assessed using DEA-CCR, DEA-VRS, and FDH models.
3	Olariu andBrad (2022)	Romania	CRS-DEA (O-O), VRS-DEA(O-O), Bootstrap	Efficiency of study programs.	Efficiency assessment using DEA and bootstrap method to correct DEA efficiencies.
4	Sarac andAlptekin (2022)	Turkey	SBM	Efficiency of associate degree programs in terms of open and distance education.	The efficiency of study programs based on open and distance education is assessed and ranked using SBM.

Table 5.

Performance of Other Higher Education Activities.

No.	Author/Year	Country	Method	Problem addressed	Proposed solution
1	Amaraet al. (2020)	Canada	DEA-VRS (O-O), ordinary least square (OLS), Tobit regression, and fuzzy set qualitative comparative analysis (fsQCA)	Canadian scholars’ research efficiency in the management field, as well as the combination of causal conditions that can affect efficiency.	The study evaluates research productivity by combining symmetric (DEA, OLS, and Tobit regression) and asymmetric (fsQCA) methods. With this combination technique, it is possible to examine both technical efficiency and a combination of causal factors that could either improve or decrease efficiency.
2	H. Li et al. (2021),Z.-H. Li et al.(2021)	China	Fuzzy, bootstrap-DEA, classical DEA	Efficiency with respect to national quality online courses (NQOOCs) during the coronavirus epidemic.	Fuzzy logic and bootstrap-DEA are used to evaluate NQOOCs efficiency, and results show that bootstrap-DEA results in greater variation in university rankings compared to classical DEA.

Data exploration revealed that in 2018 and 2019, there were 14 similar publications. In 2020, this number decreased by 2, but in 2022, it increased to 16. The peak year for DEA-HE publications was 2021, with 19 articles indicating a growing interest in the topic in recent years. Notably, China contributed the most, with 30% of the articles, followed by Australia at 7%.

The performance and qualities of an institution’s educational offering are determined by assessments of activities in HE. Such activities include research, teaching, technology, innovation, and more. As shown in Tables 2 to 5, 81% of the works published between 2018 and 2022 are heavily focused on institutional performance, 11% on department or faculty performance, 5% on study program performance, and the remaining 3% on the performance of other higher educational activities. Looking at the important aspects the recent works with regards to the performance of HE had focused on, we noticed that most recent studies focused on evaluating the total performance with regards to HE. Apart from total performance, individual evaluations of research, teaching, and their combinations were superior to performance evaluations of other activities.

Recent works have made substantial use of both classical DEA and DEA extensions. It was discovered that classical DEA was applied in more than half of the literature, and it is mostly used in conjunction with other techniques. Multi-criteria decision-making (MCDM), extensions of DEA like NDEA, super-efficiency DEA, cross-efficiency, and more are observed to be used alongside classical DEA. However, it is mostly used in conjunction with statistical techniques like regression analysis, Analysis of Variance (ANOVA), correlation analysis, cluster analysis, and more. Besides classical DEA, 44% of articles reported the usage of DEA extensions, and 15% reported the application of the Malmquist productivity index (MPI). Generally, the reported figures may not add up to 100 because there may be more than one method utilized in an article.

Findings on Bibliometric Analysis

Table 6 shows the main information that sums up the data. As evidenced by more than twenty average citations per paper, the findings indicate that DEA and performance in HE are popular topics in research. Moreover, as addressed in the first research question on the trend of publication of this topic, Figure 3 confirms the growing interest in understanding this topic in recent years. Although the first publication was in 1992, approximately more than 90% of the papers were published after 2003. The publication of research on DEA and HE performance peaked in 2021, with 52 papers.

Table 6.

Main Information.

Description	WoS database
Time span	1992–2022
Sources	238
Documents	508
Annual growth rate	13.44%
Authors	996
Authors of single-authored documents	60
International co-authorship	23.23%
Co-authors per document	2.64
Author’s keyword	1,194
Average citations per document	20.45
References	12,614

Figure 3.

Annual publication trends.

Most Relevant Journals, Their Influence, and Impact

Table 7 lists the most relevant journals based on their publications on DEA and HE performance. The journal Scientometrics tops the list, representing approximately 4.33% of the papers in this study. The second-most appropriate journal is the Journal of the Operational Research Society, which contributes about 3.74% of the papers. It is interesting to note that the top 10 journals account for 24.21% of the total number of papers examined.

Table 7.

Most Relevant Journals, Their Influence and Impact.

Journal	No. of papers	Total citation	h-Index
Scientometrics	22	465	10
Journal of the Operational Research Society	19	787	12
European Journal of Operational Research	18	966	15
Socio-Economic Planning Sciences	15	206	9
Omega-International Journal of Management Science	14	566	11
Sustainability	14	80	4
Education Economics	11	327	8
Economics of Education Review	10	1,062	9
Applied Economics	8	62	5

Total citations, which represent journal influence, show that the journal Economics and Education Review is the most influential (Total citation (TC) = 1,062), followed by the European Journal of Operational Research (TC = 966). In terms of h-index, which measures the impact of a journal, the European Journal of Operational Research (h-index = 15) is the most influential journal, followed by the Journal of the Operational Research Society (h-index = 12) and Omega-International Journal of Management Science (h-index = 11).

Most Prolific Authors and Authors’ Impact

Figures 4 and 5 enlist prolific authors and authors’ impact. Tommaso Agasisti tops the list with 23 publications between 2006 and 2022, followed by Jill Johnes and Guo-liang Yang with 13 and 12 papers, respectively. From the perspective of author influence, Jill Johnes is the author with the most citations (TC = 1,132), followed by Tommaso Agasisti (TC = 695) and Geraint Johnes (TC = 576).

Figure 4.

Authors with most publications.

Figure 5.

Most influence authors.

Most Influential Articles

Table 8 lists the top ten most influential articles in terms of local (LC) and global citations (GC). The most influential article, with the highest LC and GC, is “The efficiency of Australian universities: A data envelopment analysis” by Abbot and Doucouliagos, which is followed by Jill Johnes’ article, “Data envelopment analysis and its application to the measurement of efficiency in higher education.” Among these top ten papers, it is interesting to note that Jill Johnes’s work appears in half of them.

Table 8.

Top Ten of Most Influential Articles and Their Cited References.

No.	Title	Author(s)	Country	LC	GC
1.	The efficiency of Australian universities: a data envelopment analysis	Abbott and Doucouliagos (2003)	Australia	154	351
2.	Data envelopment analysis and its application to the measurement of efficiency in higher education	Johnes (2006b)	UK	150	311
3.	Determining Teaching and Research Efficiencies	Beasley (1995)	UK	76	255
4.	Measuring the research performance of Chinese higher education institutions using data envelopment analysis	Johnes and Yu (2008)	UK	66	132
5.	Measuring teaching efficiency in higher education: An application of data envelopment analysis to economics graduates from UK Universities 1993	Johnes (2006a, 2006b)	UK	60	112
6.	Research Funding and Performance in UK University Departments of Economics: A Frontier Analysis	Johnes and Johnes (1995)	UK	58	130
7.	Efficiency, technology and productivity change in Australian universities, 1998–2003	Worthington and Lee (2008)	Australia	55	128
8.	Efficiency of European public higher education institutions: a two-stage multicountry approach	Wolszczak-Derlacz and Parteka (2011)	Poland	54	109
9.	Costs and efficiency of higher education institutions in England: A DEA analysis	Thanassoulis et al. (2011)	UK	53	110
10.	Beyond frontiers: comparing the efficiency of higher education decision-making units across more than one country	Agasisti and Johnes (2009)	Italy	48	78

Trend Topics

The trend of the topics, as in Figure 6, shows that in the earlier research years, the main topics were about the performance, efficiency, and productivity of the decision-making units, for instance, universities, as well as departments and processes, such as research. However, recent topics starting from 2019 have shifted to quality, impact, cost, innovation, model, scope, and knowledge-transfer. The interest in knowledge transfer, in particular, has been rising since 2021.

Figure 6.

Trend topics.

Network Analysis of the Literature

Keyword co-occurrence analysis is used to investigate research trends and knowledge structures. In this analysis, the size of the nodes denotes the number of documents, and a link between two nodes is resembled by a line. A strong link is shown by a short line connecting two nodes.

In keyword plus co-occurrence network analysis, three clusters represented by three different colors (green, red, and blue) are observed. According to Rojas-Sánchez et al. (2023), each cluster denotes a keyword and displays the most linked and repeated keywords in the publications. In this bibliometric analysis, the keywords given in every cluster are identified. In a green-colored cluster: data envelopment analysis,’“performance,” and “efficiency”; red-colored clusters: “DEA,”“higher education,”“models,” and “departments”; blue-colored cluster: “productivity,”“universities,”“institutions,” and “scale.” As illustrated in Figure 7, three groups had a stronger relationship: “Performance or efficiency assessment through data envelopment analysis,”“Higher education or departments models using DEA,” and “University or institution productivity.”

Figure 7.

Co-occurrence network based on keywords plus.

Finally, we visualized the evolution over time of this area through a Sankey diagram—as illustrated in Figure 8. This diagram aids in understanding the thematic flow, the direction of thematic flow, as well as conversion relationships in the field of DEA and HE performance (Rojas-Sánchez et al., 2023; Soundararajan et al., 2014).

Figure 8.

Thematic evolution.

Concluding Remarks and Directions for Future Research

This paper reviews the application of DEA methods in the context of HE performance evaluation. To accomplish this, we conducted two primary analyses: an SLR of 75 articles published between 2018 and 2022 and a bibliometric analysis of 508 articles spanning the past 30 years, with a particular emphasis on those published in high-impact journals.

During this study, we organized the articles chosen for the SLR into four categories based on performance. Most recent literature has primarily focused on institutional performance, with the next emphasis being on departmental performance. Consequently, upcoming research should investigate the performance of study programs or other educational activities.

Also, our findings reveal that classical DEA is often complemented by other methods, especially statistical techniques. While classical DEA is a prevalent method in the assessment of HE performance, the adoption of EBM, SBM, and Fuzzy DEA models remains limited. In comparative terms, the findings of this study demonstrate both similarities and distinctions in relation to the research conducted by Cui and Yu (2021), who explored DEA models in the airline industry. Both studies concur regarding the widespread utilization of classical DEA in conjunction with other methods. However, a noteworthy discrepancy arises as Cui and Yu (2021) reported an extensive prevalence of SBM models within the airline industry alongside a limited adoption of EBM. In contrast, within the HE sector, both SBM and EBM applications remain restricted.

Additionally, we observed a rise in the use of the MPI to investigate productivity changes and the application of DEA extensions like NDEA for evaluating HE performance over the past 5 years. Within the NDEA structure implemented in HE, significant opportunities for further exploration have emerged. Beyond the conventional roles of teaching, research, and services provided by the HE sector, other aspects, such as other operational processes or subprocesses related to HE performance, demand exploration. Furthermore, it is essential to assess whether any previously overlooked outputs or shared inputs require thorough examination.

Subsequently, this study embarks on a more comprehensive exploration of the research landscape within this domain. The bibliometric analysis serves as a robust foundation for our investigation, unveiling a significant surge in publications. This surge, commencing in 2003 and peaking in 2021, is particularly noteworthy, with over 90% of these articles published after 2003.

Additionally, this review identifies the most influential journals, authors, publications, and trending topics. That being said, Scientometric tops our list, followed by the Journal of Operational Research. Among authors, Tommaso Agasisti has produced the most written material, while Jill Johnes has had the most significant influence on DEA-HE research. The article titled “The efficiency of Australian universities: A data envelopment analysis” by Abbott and Doucouliagos (2003) is identified as the most influential article.

Trending topics include quality, impact, cost, innovation, model, scope, and knowledge transfer. The study also explores bibliometrics’ network analysis, revealing three groups with strong relationships: “Performance or efficiency assessment through Data Envelopment Analysis,”“Higher education or departmental models using DEA,” and “University or institution productivity.”

In essence, this study touched on various DEA applications in an effort to increase the readers’ understanding of how DEA is used in evaluating HE performances. By providing a comprehensive review and addressing fundamental research issues essential to this field, it provides a solid framework for further investigation. Nevertheless, some limitations provide opportunities for future review papers. This review does not include scholarly works that examine HE efficiency through parametric approaches. Additionally, studies from other fields were not considered, as the focus was strictly on DEA in HE. Another limitation of this review is its focus on English-language literature, which excluded publications in other languages. However, the majority of the assessed papers were from international journals.

This study meticulously selected articles from various publishers within the WoS Core Collection database. However, some content may have fallen outside the scope of this review paper. Therefore, future researchers might consider evaluating works that were not included in the current analysis. Future bibliometric investigations could benefit from creating new techniques and utilizing new keywords to incorporate additional relevant publications for more comprehensive metadata analysis.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Anis Mardiana Ahmad

Data Availability Statement

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

Abbott

Doucouliagos

(2003). The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review, 22(1), 89–97.

Agasisti

Johnes

(2009). Beyond frontiers: Comparing the efficiency of higher education decision-making units across more than one country. Education Economics, 17(1), 59–79.

Agasisti

Shibanova

(2022). Actual autonomy, efficiency and performance of universities: Insights from the Russian case. International Journal of Public Administration, 45(2), 121–134.

Amara

Rhaiem

Halilem

(2020). Assessing the research efficiency of Canadian scholars in the management field: Evidence from the DEA and fsQCA. Journal of Business Research, 115, 296–306.

Andersson

Sund

(2022). Technical efficiency and productivity of higher education institutions in the Nordic countries. International Journal of Public Administration, 45(2), 107–120.

Banker

R. D.

Charnes

Cooper

W. W.

(1984). Some models for estimating technical and scale inefficiencies in data envelopment analysis. Management Science, 30(9), 1078–1092.

Bartolacci

Caputo

Soverchia

(2020). Sustainability and financial performance of small and medium sized enterprises: A bibliometric and systematic literature review. Business Strategy and the Environment, 29(3), 1297–1309.

Beasley

J. E.

(1995). Determining teaching and research efficiencies. Journal of the Operational Research Society, 46(4), 441.

Brzezicki Pietrzak

Cieciora

(2020). The total efficiency of teaching activity of Polish higher education institutions. Foundations of Management, 12(1), 19–30.

10.

Brzezicki

Ł.

(2020). The efficiency of public and private higher education institutions in Poland. Gospodarka Narodowa, 304(4), 33–51.

11.

Canto

M. B.

Lopez

L. B.

(2018). Ranking of global efficiency of the best universities in Europe. Marketing and Management of Innovations, 9(3), 34–47.

12.

Charnes

Cooper

W. W.

Rhodes

(1978). Measuring the efficiency of decision making units. European Journal of Operational Research, 2(6), 429–444.

13.

Chen

S.-P.

Chang

C.-W.

(2021). Measuring the efficiency of university departments: An empirical study using data envelopment analysis and cluster analysis. Scientometrics, 126(6), 5263–5284.

14.

Cooper

W. W.

Seiford

L. M.

Tone

Zhu

(2007). Some models and measures for evaluating performances with DEA: Past accomplishments and future prospects. Journal of Productivity Analysis, 28(3), 151–163.

15.

Cossani

Codoceo

Cáceres

Tabilo

(2022). Technical efficiency in Chile’s higher education system: A comparison of rankings and accreditation. Evaluation and Program Planning, 92, 1–14.

16.

Cui

L. T.

(2021). A review of data envelopment analysis in airline efficiency: State of the art and prospects. Journal of Advanced Transportation, 2021, 1–13.

17.

de Jorge Moreno

González Robles

Martinez

Minero Calvo

Miron

A. G.

(2019). Assessing efficiency in the Spanish public universities through comparative non-radial and radial data envelopment analysis. Tertiary Education and Management, 25(3), 195–210.

18.

Delahoz-Dominguez

Mendoza-Mendoza

Visbal-Cadavid

(2022). Efficiency of academic engineering programs in Colombia: An approach through data envelopment analysis. Journal of Engineering Science and Technology, 17(2), 1105–1118.

19.

De La Hoz

Zuluaga

Mendoza

(2021). Assessing and classification of academic efficiency in engineering teaching programs. Journal on Efficiency and Responsibility in Education and Science, 14(1), 41–52.

20.

de la Torre

E. M.

Casani

Sagarra

(2018). Defining typologies of universities through a DEA-MDS analysis: An institutional characterization for formative evaluation purposes. Research Evaluation, 27(4), 388–403.

21.

de Oliveira

M. S.

Steffen

de Francisco

A. C.

Trojan

(2023). Integrated data envelopment analysis, multi-criteria decision making, and cluster analysis methods: Trends and perspectives. Decision Analytics Journal, 8, 1–54.

22.

(2018). Empirical research on the influencing factors of the conversion efficiency of scientific and technological achievements in colleges and universities based on Tobit model. Educational Sciences Theory & Practice, 18(6), 2794–2807.

23.

Dimyati

Hermanu

A. I.

(2023). Evaluating research efficiency in Indonesian higher education institution. Evaluation Review, 47, 155–181.

24.

Ding

Yang

Wen

Tan

Liang

(2021). Research performance evaluation of Chinese university: A non-homogeneous network DEA approach. Journal of Management Science and Engineering, 6(4), 467–481.

25.

Duan

S. X.

(2019). Measuring university efficiency: An application of data envelopment analysis and strategic group analysis to Australian universities. Benchmarking An International Journal, 26(4), 1161–1173.

26.

Seo

(2022). A comparative study on the efficiency of R&D activities of universities in China by region using DEA–Malmquist. Sustainability, 14, 1–13.

27.

Ersoy

(2021). Performance evaluation in distance education by using data envelopment analysis (DEA) and TOPSIS methods. Arabian Journal for Science and Engineering, 46(2), 1803–1817.

28.

Firsova

A. A.

Chernyshova

G. Y.

(2019). Mathematical models for evaluation of the higher education system functions with DEA approach. Izvestiya Saratovskogo Universiteta Novaya Seriya-Matematika Mekhanika Informatika, 19(3), 351–362.

29.

Gao

Moosa

(2019). The efficiency of technology transfer in Chinese key universities. Journal of Scientific and Industrial Research, 78(9), 582–588.

30.

Gebru

M. G.

Raza

Shaukat Khan

(2021). Efficiency of higher education in the presence of shared inputs using data envelopment analysis. Sains Malaysiana, 50(11), 3421–3438.

31.

Ghasemi

Najafi

Hoseinzadeh Lotfi

Movahedi Sobhani

(2020). Assessing the performance of organizations with the hierarchical structure using data envelopment analysis: An efficiency analysis of Farhangian university. Measurement, 156, 1–19.

32.

Ghimire

Amin

S. H.

Wardley

L. J.

(2021). Developing new data envelopment analysis models to evaluate the efficiency in Ontario universities. Journal of Informetrics, 15(3), 1–17.

33.

Gnewuch

Wohlrabe

(2018). Super-efficiency of education institutions: An application to economics departments. Education Economics, 26(6), 610–623.

34.

González-Garay

Pozo

Galán-Martín Brechtelsbauer

Chachuat

Chadha

Hale

Hellgardt

Kogelbauer

Matar

O. K

McDowell

Shah

Guillén-Gosálbez

(2019). Assessing the performance of UK universities in the field of chemical engineering using data envelopment analysis. Education for Chemical Engineers, 29, 29–41.

35.

J. C.

Lee

(2021). Research commercialisation performance in different types of universities: Case from Taiwan. Scientometrics, 126(10), 8617–8634.

36.

Jiang

Lee

S. K.

Rah

M.-J.

(2020). Assessing the research efficiency of Chinese higher education institutions by data envelopment analysis. Asia Pacific Education Review, 21(3), 423–440.

37.

Johnes

(2006a). Data envelopment analysis and its application to the measurement of efficiency in higher education. Economics of Education Review, 25(3), 273–288.

38.

Johnes

(2006b). Measuring teaching efficiency in higher education: An application of data envelopment analysis to economics graduates from UK Universities 1993. European Journal of Operational Research, 174(1), 443–456.

39.

Johnes

(1995). Research funding and performance in UK university departments of economics: A frontier analysis. Economics of Education Review, 14(3), 301–314.

40.

Johnes

(2008). Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Economic Review, 19(4), 679–696.

41.

Joshi

M. A.

(2014). Bibliometric indicators for evaluating the quality of scientifc publications. Journal of Contemporary Dental Practice, 15(2), 258–262.

42.

Kao

(2014). Network data envelopment analysis: A review. European Journal of Operational Research, 239(1), 1–16.

43.

Kashim

Mat Kasim

Abd Rahman

(2018). Measuring efficiency of a university faculty using a hierarchical network data envelopment analysis model. ITB Journal of Information and Communication Technology, 17(4), 569–585.

44.

Koçak

Örkcü

H. H.

(2021). Measuring the efficiency of Turkish state universities based on a two-stage DEA model. Gazi University Journal of Science, 34(4), 1210–1220.

45.

Kullenberg

Kasperowski

(2016). What is citizen science?–A scientometric meta-analysis. Plos One, 11(1), 1–16.

46.

Kumar

Thakur

R. R.

(2019). Objectivity in performance ranking of higher education institutions using dynamic data envelopment analysis. International Journal of Productivity and Performance Management, 68(4), 774–796.

47.

Kupriyanova

Estermann

Sabic

(2018). Efficiency of universities: Drivers, enablers and limitations. In Curaj

Deca

Pricopie

(Eds.), European higher education area: The impact of past and future policies (pp. 603–618). Springer International Publishing.

48.

Lee

B. L.

Johnes

(2022). Using network DEA to inform policy: The case of the teaching quality of higher education in England. Higher Education Quarterly, 76(2), 399–421.

49.

Pang

Zuo

Tao

(2021). The evaluation and its key influencing factors analysis of “double First-Class” universities’ teaching efficiency in China based on SBM-DEA and its visual decision analysis system development. Discrete Dynamics in Nature and Society, 2021, 2021–2114.

50.

Lita

(2018). Data envelopment analysis techniques – Dea and malmquist indicators, in CRS mode for measuring the efficiency of Romanian public higher education institutions. Economic Computation and Economic Cybernetics Studies and Research/Academy of Economic Studies, 52(3/2018), 249–264.

51.

Z.-H.

Yang

J.-J.

Qin

H.-Q.

Xia

Y.-W.

Zhuang

M. E.

(2021). A study of efficiency evaluation of national quality online courses during the epidemic: Based on fuzzy logic calculation and bootstrap-DEA. Mathematical Problems in Engineering, 2021, 1–7.

52.

Loganathan

Subrahmanya

M. H. B.

(2023). Efficiency of entrepreneurial universities in India: A data envelopment analysis. Journal of the Knowledge Economy, 14, 1120–1144.

53.

Cai

Zhu

(2022). Technology transfer efficiency of universities in China: A three-stage framework based on the dynamic network slacks-based measurement model. Technology in Society, 70, 1–12.

54.

Mammadov

Aypay

(2020). Efficiency analysis of research universities in Turkey. International Journal of Educational Development, 75, 1–11.

55.

Mardani

Zavadskas

E. K.

Streimikiene

Jusoh

Khoshnoudi

(2017). A comprehensive review of data envelopment analysis (DEA) approach in energy efficiency. Renewable and Sustainable Energy Reviews, 70, 1298–1322.

56.

Mariz

F. B. A. R.

Almeida

M. R.

Aloise

(2018). A review of dynamic data envelopment analysis: State of the art and applications. International Transactions in Operational Research, 25(2), 469–505.

57.

Moncayo–Martínez

L. A.

Ramírez–Nafarrate

Hernández–Balderrama

M. G.

(2020). Evaluation of public HEI on teaching, research, and knowledge dissemination by data envelopment analysis. Socio-Economic Planning Sciences, 69, 100718.

58.

Moreno-Gómez

Calleja-Blanco

Moreno-Gómez

(2019). Measuring the efficiency of the Colombian higher education system: A two-stage approach. International Journal of Educational Management, 34(4), 794–804.

59.

Mousa

Ghulam

(2019). Exploring efficiency differentials between Saudi higher education institutions. Managerial and Decision Economics, 40(2), 180–199.

60.

Myeki

L. W.

Temoso

(2019). Efficiency assessment of public universities in South Africa, 2009-2013: Panel data evidence. South African Journal of Higher Education, 33(5), 264–280.

61.

Navas

L. P.

Montes

Abolghasem

Salas

R. J.

Toloo

Zarama

(2020). Colombian higher education institutions evaluation. Socio-Economic Planning Sciences, 71, 1–11.

62.

Navickas

Grenčíková

Krajčo

(2021). DEA model and efficiency of universities - Case study in Slovak Republic. Entrepreneurship and Sustainability Issues, 9(1), 348–362.

63.

Nkohla

T. V.

Munacinga

Marwa

Ncwadi

(2021). A nonparametric assessment of efficiency of South African public universities. South African Journal of Higher Education, 35(2), 158–187.

64.

Olariu

G. V.

Brad

(2022). Preventive risk management of resource allocation in Romanian higher education by assessing relative performance of study programs with DEA method. Sustainability, 14(19), 1–17.

65.

Panwar

Olfati

Pant

Snasel

(2022). A review on the 40 years of existence of data envelopment analysis models: Historic development and current trends. Archives of Computational Methods in Engineering, 29(7), 5397–5426.

66.

Pedro

E. D. M.

Leitão

Alves

(2022). Do socially responsible higher education institutions contribute to sustainable regional growth and innovation? International Journal of Sustainability in Higher Education, 23(8), 232–254.

67.

Qin

(2018). Measuring universities’ R&D performance in China’s provinces: A multistage efficiency and effectiveness perspective. Technology Analysis and Strategic Management, 30(12), 1392–1408.

68.

Quiroga-Martinez

Fernandez-Vazquez

Lucia Alberto

(2018). Efficiency in public higher education on Argentina 2004-2013: Institutional decisions and university-specific effects. Latin American Economic Review, 27(1), 14.

69.

Ranjan

Singh

(2021). Efficiency analysis of higher education institutions: Use of categorical variables. International Journal of Mathematical, Engineering and Management Sciences, 6(6), 1518–1532.

70.

Rojas-Sánchez

M. A.

Palos-Sánchez

P. R.

Folgado-Fernández

J. A.

(2023). Systematic literature review and bibliometric analysis on virtual reality and education. Education and Information Technologies, 28, 155–192.

71.

Salas-Velasco

(2020). The technical efficiency performance of the higher education systems based on data envelopment analysis with an illustration for the Spanish case. Educational Research for Policy and Practice, 19(2), 159–180.

72.

Sarac

Alptekin

(2022). Efficiency in open and distance education: A research at Anadolu University. Turkish Online Journal of Distance Education, 23(2), 153–168.

73.

Shamohammadi

D. H.

(2019). Measuring the efficiency changes of private universities of Korea: A two-stage network data envelopment analysis. Technological Forecasting and Social Change, 148, 1–15.

74.

Shi

Wang

Zhang

(2022). Categorical evaluation of scientific research efficiency in Chinese universities: Basic and applied research. Sustainability, 14(8), 4402.

75.

Singh

A. P.

Yadav

S. P.

Tyagi

(2022). Performance assessment of higher educational institutions in India using data envelopment analysis and re-evaluation of NIRF rankings. International Journal of Systems Assurance Engineering and Management, 13(2), 1024–1035.

76.

Song

Schubert

Liu

Yang

(2019). Measuring scientific productivity in China using Malmquist productivity index. Journal of Data and Information Science, 4(1), 32–59.

77.

Soundararajan

H. K.

(2014). Sankey diagram framework for energy and exergy flows. Applied Energy, 136, 1035–1042.

78.

Sun

Yuan

Chen

(2021). The construction and empirical research on the dynamic evaluation model of university science and technology output. IEEE Access, 9, 57280–57290.

79.

Szuwarzyński

(2019). Benefit of the doubt approach to assessing the research performance of Australian universities. Higher Education Quarterly, 73(2), 235–250.

80.

Thai

K. Q.

Noguchi

(2021). Investigating the technical efficiency of Japanese national universities following corporatization: A two-stage data envelopment analysis approach. International Journal of Educational Management, 35(6), 1297–1311.

81.

Thanassoulis

Kortelainen

Johnes

(2011). Costs and efficiency of higher education institutions in England: A DEA analysis. Journal of the Operational Research Society, 62(7), 1282–1297.

82.

Tran

C. T. T.

Villano

R. A.

(2018). Measuring efficiency of Vietnamese public colleges: An application of the DEA-based dynamic network approach. International Transactions in Operational Research, 25(2), 683–703.

83.

Tran

P. P.

Kuo

K.-C.

W.-M.

Kweh

Q. L.

(2020). Benchmarking in Vietnam universities: Teaching and research and revenue efficiencies. Asia Pacific Education Review, 21(2), 197–209.

84.

Villano

R. A.

Tran

C.-D. T. T.

(2018). Performance of private higher education institutions in Vietnam: Evidence using DEA-based bootstrap directional distance approach with quasi-fixed inputs. Applied Economics, 50(55), 5966–5978.

85.

Wang

C.-N.

Tibo

Nguyen

V. T.

Duong

D. H.

(2020). Effects of the performance-based research fund and other factors on the efficiency of New Zealand universities: A Malmquist productivity approach. Sustainability, 12(15), 5939.

86.

Wang

(2021). Research on the scientific research efficiency of provincial universities based on the DEA model. Mobile Information Systems, 2021, 1–12.

87.

Wijesundara

W. M. T. H.

Prabodanie

R. A. R.

(2022). Relative efficiencies of the state universities in Sri Lanka: Can they enroll more students? International Journal of Educational Management, 36(7), 1284–1296.

88.

Wohlrabe

de Moya Anegon

Bornmann

(2019). How efficiently do elite US universities produce highly cited papers? Publications, 7(1), 4.

89.

Wolszczak-Derlacz

(2016). Assessment of tfp in European and American higher education institutions – Application of Malmquist indices. Technological and Economic Development of Economy, 24(2), 467–488.

90.

Wolszczak-Derlacz

Parteka

(2011). Efficiency of European public higher education institutions: A two-stage multicountry approach. Scientometrics, 89(3), 887–917.

91.

Worthington

A. C.

Lee

B. L.

(2008). Efficiency, technology and productivity change in Australian universities, 1998–2003. Economics of Education Review, 27(3), 285–298.

92.

Zhang

Zhu

Zhou

(2020). An efficiency analysis of higher education institutions in China from a regional perspective considering the external environmental impact. Scientometrics, 122(1), 57–70.

93.

Xia

C.-Y.

Yuan

Z.-H.

W.-Y.

Zhao

J.-H.

(2021). An empirical study on scientific research performance of universities in different regions of China based on PCA and Malmquist index method. Education Research International, 2021(5), 1–11.

94.

Xue

Ali

Rehman

R. U.

Fernández-Sánchez

(2021). Assessing the static and dynamic efficiency of scientific research of HEIs China: Three stage DEA–Malmquist index approach. Sustainability, 13(15), 8207.

95.

Yang

G. L.

Fukuyama

Song

Y. Y.

(2018). Measuring the inefficiency of Chinese research universities based on a two-stage network DEA model. Journal of Informetrics, 12(1), 10–30.

96.

Zhang

Jiang

Chen

Streimikiene

Balezentis

(2022). Spreading knowledge and technology: Research efficiency at universities based on the three-stage MCDM-NRSDEA method with bootstrapping. Technology and Society, 68, 1–12.

97.

Zhang

Shi

(2019). Research about the university teaching performance evaluation under the data envelopment method. Cognitive Systems Research, 56, 108–115.

98.

Zhao

H. H.

Liu

Guo

X. G.

Gui

H. J.

(2022). Chinese provincial difference in the efficiency of universities’ scientific and technological activities based on DEA with shared input. Mathematical Problems in Engineering, 2022, 1–19.

99.

Zhu

Wan

Zhou

Pan

(2018). Fuzzy computation of teaching performance based on data envelopment analysis method. Cognitive Systems Research, 52, 351–358.

100.

Zhu

Yang

Gong

Zeng

(2023). Retraction note: Assessing the efficiency of innovation entities in China: Evidence from a nonhomogeneous data envelopment analysis and tobit. Electronic Commerce Research, 23(1), 175–205.