Analyzing higher education performance by entropy - TOPSIS method: A case study in Viet Nam private universities

Abstract

The development in both quantity and quality of the system of private universities over the past time has created great opportunities for people to study and improve their qualifications and working skills. It is also challenging for educational managers to find the right university development strategy in today's competitive environment. Therefore, evaluating the effectiveness of using private universities resource to analyze training performance is considered one of the issues. In this study, the Entropy method is used to determine the weight of the criteria. Then the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is used to determine the ranking order of the private universities. Next, Spearman's rank correlation coefficient was applied to evaluate the correlation between rank-ordered variables over the 2 years of analysis selection. Finally, ANOVA was used to compare the criteria between groups of universities. The combination of methods contributes to creating an objective environment in assessing the performance of each university. Thereby, universities can see the important criteria to pay attention to, creating a premise for universities to establish strategies for development and competition. At the same time, educational leaders have the right perspective in choosing key universities to invest in finance and facilities.

Introduction

Higher education plays a particularly important role in training high-quality human resources and is a prerequisite for rapid and sustainable development. Moreover, this team stays a pivotal role in technology transfer and scientific research to promote economic growth social development. Hence, the type of private university developed to meet the increasing demand for the high-level study of the people and contribute to the supply of high-quality human resources for the country's development. ¹

The recent development in size, quantity, and quality of private universities has created great opportunities for people to study and improve their qualifications and working skills. However, the management of private higher education institutions is still not reasonable. The conditions to ensure stability and improve the quality of teaching and learning of private universities, such as facilities, equipment, the organization, and management of training, the management staff, and the teaching staff are still inadequate. This leads to the poor quality of training, which has not met social needs and development requirements. Moreover, society does not yet trust the training quality of private universities. Some state management agencies and enterprises have not yet acknowledged and believed in the qualifications and abilities of graduates of private universities. ²

State management of higher education still has many shortcomings, especially the current policy of developing private universities is not synchronized and unreasonable. The establishment of universities still lacks thorough investigation and survey; lack of preparation of conditions to ensure the quality of teaching and learning. ³ The system of legal documents on private higher education is not complete and synchronous; the model of a private university, for-profit or non-profit, has not been clearly defined and built.

The state's mechanisms and policies have not really favored the development of private universities; policies on investment in finance, land, and facilities; The policy on developing teaching staff for private universities is still unreasonable and has not created equality between public and private higher education. Therefore, strengthening the state management of higher education and assessing the performance of private universities is necessary to amend and supplement policies to promote the development of private universities. Proper development and investment policies are an urgent requirement in the current period. ⁴

Previously, many studies on topics related to the problem assessed higher education performance in Vietnam. The well-known non-parametric method widely applied in operations research to estimate performance is mainly Data envelopment analysis (DEA). ^5–8 The advantage of DEA is handling multiple outputs and inputs without defining relationships between variables and does not require price information. ⁵ The studies did an empirical analysis of operational efficiency of Vietnamese higher education institutions, and ⁷ found a benchmark assessment of Vietnamese universities through teaching and research and revenue performance. In recent publications, other widely used methods to assess the quality of higher education are Analytical Hierarchy Process (AHP), ^9–12 Vise Kriterijumska Optimizajica I Kompromisno Resenje (VIKOR), ^13,14 Weight Aggregated Sum Product Assessment (WASPAS), ^15,16 Technique for Order of Preference by Similarity to Ideal Solution. ^17–19 Each method has its advantages and is applied by researchers following their purposes.

Both AHP and TOPSIS can evaluate individual test results, however, the AHP is only applicable when clear quantitative indicators are not available, and it is not objective and rigid. On the other hand, in the TOPSIS method, the decision variables are clearly quantified. In addition, they can be ranked and evaluated based on existing data. Compared to AHP's limitations and the other multi-criteria decision analysis methods, TOPSIS is considered a good MCDM method, Because of its simplicity, fast and ease of understanding of a systematic process. In addition, in its simple mathematical form, this method has good computational power and effectively measures the relative performance for each option. ^20,21 However, the TOPSIS method could not obtain the objective weights, so the Entropy was commenced. The coefficient of entropy method construction applies information theory knowledge. The greater the information contained in the indicator, the more critical the data variability (variance). The Entropy weighting method is commonly used to assign weights to indices. ²² It works well in discriminating alternatives due to the diversity with an attribute's susceptibility. This method, therefore, gives importance to the characteristics according to its capture variety. Entropy is a measure of uncertainty and diversity that captures the amount of information provided by an indicator. We normalize the input values in the entropy weight method and then calculate the entropy. ²² So, Entropy could be implemented for objective and accurate assessment of the research object. Therefore, we used entropy and TOPSIS methods to establish a model for evaluating private higher education performance.

The research problem of the study is as follows. Are the resources of private universities being used effectively in training in today's competitive environment, and how effective are private universities in training? Many existing reports synthesize research results of private universities ²³ and studies of the private model. ²⁴ In addition, these reports specify many recommendations for developing Vietnamese private universities ^25,26 . However, no studies give a specific method or specific quantitative techniques to implement the above solutions. This study aims to evaluate the performance of private sector universities by using the Entropy -TOPSIS method combined with ANOVA (Duncan Post hoc test) in-depth analysis together. First, in the study, the Entropy method is used to determine the weight of the criteria. Then, the TOPSIS is used to find out the ranking order of the private universities. Next, Spearman's rank correlation coefficient was applied to evaluate the correlation between rank-ordered variables over the 2 years of analysis selection. Finally, ANOVA (Duncan Post hoc test) was used to compare the criteria between groups of universities to see the efficiency of criteria or resources of universities. Thus, the study introduced a new combination of methods that contributes to creating an objective environment in assessing the performance of each university. Thereby, universities can see the important criteria to pay attention to, creating a premise for universities to develop strategies for development and competition. Furthermore, with the study, educational leaders can have the opportunity to know the right perspective in choosing key universities to invest in finance and facilities. The applicability of the study can be modified and then applied to other service areas.

The remainder of this study is presented as follows: Part 2 is a preliminary literature review on university performance and assessment methods. Next, part 3 is an analytical model that introduces the main methodology of the paper. Then, in part 4, methods of data collection and analysis are presented. Part 5, discussion and recommendation. And part 6, conclusion with limitations and future studies.

Literature review

University ranking

The ranking of academic institutions based on multidimensional performance indicators and methodology was first performed by US News in 1987. Although previously, the first ranking of educational institutions was based on the perception of university principals. This has prompted several rankings in the field with different themes in different countries. Rankings in these various subject areas have emerged from the press, media, private bodies, professional associations, and governments. ²⁷ There are many methods of ranking universities today. Reputable rankings worldwide all build their own set of criteria, leading to different final results. ²⁸

The first global university ranking belongs to Shanghai Jiaotong University. Shanghai Academic Ranking of World Universities (ARWU) is the world university ranking of Shanghai Jiaotong University. ²⁹ Since 2003, ARWU has introduced the world's top 500 schools annually based on a transparent approach and third-party data. ARWU developed four main criteria, including quality of education (10%), quality of faculty (40%), scientific research (40%), and academic productivity per capita (10%).

Following is the Times Higher Education (THE) QS ranking in 2004. In partnership with Quacquarelli Symonds (QS), Times Higher Education published the joint THE-QS World University Rankings from 2004 to 2009.2010 onwards, THE and QS publish the world university rankings independently. ²⁷ The Quacquarelli Symonds World University Rankings (QS) continues to use a unified methodological framework, compiled using six measures: academic reputation (40%), employer reputation (10%), and faculty/student ratio (20%), number of scientific citations per faculty (20%), international faculty ratio (5%), and international student ratio (5%).

THE is a magazine for news and higher education issues, based in London, England. THE uses 13 calibrated indicators to provide a comprehensive and balanced comparison. The indicators are divided into five groups: teaching (learning environment), research (quantity, income, and reputation), scientific citations (research influence), international outlook (lecturers), students and research), income through knowledge transfer. This ranking is trusted by students, scientists, university leaders, business owners, and governments. Schools are excluded from THE's world rankings if they do not offer bachelor's degrees, or research results are less than 1000 articles between 2012 and 2016 (minimum 150 articles required per year). If 80% of activities do not fall within the 11 fields specified by THE, the school will not be graded. Educational institutions provide data for rankings and must commit to accuracy. In rare cases, the school will use estimates to avoid zero values if not provided by the school. Teaching criteria account for (30%) with metrics like reputation (15%)., faculty/student ratio (4.5%), doctorate/bachelor's ratio (2.25%), doctorate/lecturer ratio (6%), income (2.25%). ³⁰

Although controversial, the research results from these three rankings are all valuable references in the overall assessment of universities in the world. In addition, there are many studies with different university ranking methods, the most popular of which is the MCDM method. Study ³¹ applies the specific combined multi-criteria decision-making model (MCDM). The author analyzes the hierarchical process (AHP), (VIKOR) ranks 12 private universities in Taiwan. Fernando García suggested using multilevel models based criteria programming targeted to determine objectively the weight of the indicators used to measure the performance of the universities and ultimately to determine their position in the universities ranking. ³² J.M. Conejeroa and et al. used a multi-criteria approach including TOPSIS and a worst-best case scenario approach which was compared with Pearson's well-known global sensitivity analysis technique to rank Vocational and Education Programs in Extremadura (Spain) for 2009–2016. ³³ Seyhan Nisel ³⁴ provides a specific multi-criteria model using the VIKOR (Vise Kriterijumska Optimizacija I Kompromisno Resenje) model. This model ranks Turkish universities by academic performance as evaluation criteria such as number of articles, number of articles and citations, number of Ph.D. students, student to faculty ratio. Renuka Nagpal ranks university websites for usability using fuzzy AHP, and fuzzy TOPSIS approaches. ³⁵

The UPM - University Performance Metrics education system developed by Vietnam was announced at the Vietnam National University, Hanoi. This is the first time Vietnam has obtained a ranking system for universities and the Asian region. UPM is an assessment system sponsored by the Ministry of Education and Training of Vietnam to assess the responsiveness of universities in the era of the fourth Industrial Revolution (4IR). UPM is a product made by a research team of Vietnam National University, Hanoi. ³⁶ This system helps higher education institutions identify and manage strategic goals, aiming to reach the top 100 standards of Asian universities. Many criteria are associated with the benchmarks of research-oriented universities specified in Decree 99/2019/ND-CP of Vietnam guiding the Law on Higher Education implementation. ³⁷

Currently, the quality of higher education in Vietnam is of great concern to the whole society. In the rankings of leading universities globally and in Asia, there are almost no Vietnamese universities. By Times Higher Education magazine's 2018 Asian university quality rankings, Vietnam does not have a representative. On 7 June 2018, the British university ranking organization Quacquarelli Symonds (QS) announced the QS world ranking 2020. Vietnam National University ³⁸ Ho Chi Minh City is in group 701,750, while VNU is 701,750. Hanoi belongs to groups 801–1000. Higher education institutions in Vietnam have received this result with different attitudes, but it is certainly a good sign for the higher education system in Vietnam. However, the current problems in higher education in Vietnam are still limited and affect the output quality of universities. ³⁹

Status of private universities in Vietnam

Vietnam's private higher education system plays a particularly important role in training human resources, contributing to economic growth and social development. The development of private universities reduces the pressure on state budget expenditure and contributes to diversifying the education system to meet the increasing learning needs of society. In recent years, private universities have achieved impressive growth. If in 1993, Vietnam had the first private university, by the end of 2020, there were 61 schools (out of a total of 236 universities). Private universities account for about 26% of the total universities number in the country and are distributed in 29/63 provinces and cities. ⁴⁰

Firstly, in terms of scale: The network of higher education institutions in general and non-public universities, in particular, has been expanded and developed partly to meet the learning needs of the people and socio-economic development requirements of the country. The network of private universities has covered seven regions of the country, in line with the state's university development planning orientation, contributing to the rapid development of the scale of higher education. Compared with public universities, the number of private universities also increases every year and currently accounts for about 25.5% of the total number of universities in the country. The number of students is over 253 thousand (accounting for 13.16% of university students nationwide), the teaching staff is 20,500 people. ⁴¹

Secondly, in terms of international rankings for higher education institutions and training programs, only a few private universities were evaluated such as: in 2019, Duy Tan University was the first private university in Vietnam to be awarded which ranked in the 500 best universities in Asia; Nguyen Tat Thanh University is ranked fourth by QS Accreditation Organization; FPT University has one major that meets Accreditation Council for Business Schools and Programs - ACBSP (FPT University was rated five stars by QS for four important criteria in 2015).

Thirdly, about scientific research: the qualitative change in research and publication of scientific results at the international level has raised the position and ranking of many universities on the regional university rankings and in the world, such as Duy Tan University was ranked for the first time by the World University Rankings by Academic Achievement (URAP) ⁴¹ ; However, the reality shows that the implementation of training at private universities is still limited. Quite a few private universities have low-quality admissions. Many private universities have a lower number of permanent faculty members than visiting faculty. For the academic year 2019–2020, many private universities did not recruit enough targets, which has affected the stability and development of private universities. The teaching staff of private universities is both lacking and weak. Full-time teachers with university degrees still have a large influence on the quality of training. Although many private universities cannot arrange enough permanent teachers, the number of visiting lecturers still accounts for a large proportion. ⁴²

State-issued private university development policies have not been synchronized and consistent, not timely, have not fully developed and concretized standards. The conditions and capacity to ensure training quality have not created a driving force for the development of private universities. Meanwhile, the public higher education system is prioritized by the state. The private university system has not received adequate attention from the state. ³⁸

In general, the growth in both quantity and quality of the private university system over the past time has created great opportunities for people to study to improve their qualifications and working skills. However, the management of private higher education institutions is still not reasonable. The conditions to ensure stability and improve the training quality of private universities are still inadequate. Moreover, commercializing private higher education is posing concerns to society that require solutions to perfect private universities in the future. To do that requires managers and administrators of private universities to have a clearer view of private universities' strategy formulation and development. ⁴³

Application of entropy, TOPSIS method in MCDM

Different methods for decision-makers provide different information. Therefore, by combining these methods, we can make more refined decisions. Over the past decades, research and development in the field of Multi-Criteria Decision Making (MCDM) involving both quantitative and qualitative factors have accelerated and appears to continue to evolve. MCDM methodology is applied in many fields, including energy, ^20,44–46 supply chain, ^47–49 quality assurance, transportation, ⁵⁰ information and communication technology, human resources, ⁵¹ education ^18,19 and so on.

In recent publications, much attention has been paid to problem-solving evaluation weighting criteria. One of the most well-known approaches to determining the target attribute weights is the entropy-based approach. In MCDM, the larger the value of Entropy, a general measure of uncertainty, corresponds to the smaller the attribute weight, the less discriminating power of that attribute in decision making. ⁵²

Many models use methods such as TOPSIS, Fuzzy DEMATEL, COPRAS, ARAS combined with Entropy weights. For example, Goswami ⁵³ integrates Entropy with Additive Ratio Assessment (ARAS) and Complex Proportional Assessment (COPRAS) MCDM techniques that solve problems in selecting of material handling equipment. On the other hand, HaiLi's study proposes a machine tool selection method based on a new hybrid MCDM model. ⁵⁴ The study uses a comprehensive weighting technique that integrates subjective weights, obtained using the fuzzy decision-making evaluation and testing laboratory (FDEMATEL) with weights objective is obtained by entropy weighting (EW). The author then uses the later defuzzification VIKOR (LDVIKOR) given to rank the optional alternatives.

In another study, ⁵¹ Krylovas, in selecting human resources, the Entropy - KEMIRA method was proposed to rank the criteria and determine the weight when solving the problem of Multiple Criteria Decision Making (MCDM). Morteza Yazdani and colleagues ⁴⁴ used the integrated Shannon Entropy and EDAS model in the renewable energy assessment to analyze the conflict of five resources renewables to choose the optimal energy source. Shannon Entropy determines the importance of the criteria and assessment based on the Distance from Average Solution (EDAS) application to prioritize renewable energy sources.

Compared with other multi-criteria decision analysis methods such as MABAC, MAIRCA, RAFSI, VIKOR, COPRAS…, TOPSIS has some advantages, such as adequate raw data information and no strict restrictions on the number of indicators. ⁵⁵ TOPSIS is a ranking method that attempts to choose alternatives that simultaneously have the shortest distance from the positive ideal solution and the farthest distance from the negative ideal solution. ⁵⁶ TOPSIS makes full use of the attribute information and provides a cardinal ranking of the alternatives. ⁵⁷ The TOPSIS method is based on the closeness of a limited number of evaluation objects to the ideal solution. There are two ideal solutions, one is the positive ideal solution, and the other is the negative ideal solution. The best object should be the closest to the positive ideal solution and the farthest from the negative ideal solution. This study calculates the relative closeness to the ideal solution based on TOPSIS. In general, the advantages for TOPSIS include (i) simple, rationally comprehensible concept, (ii) good computational efficiency, (iii) ability to measure the relative performance for each alternative in a simple mathematical form. ^58,59

Because of its advantages, TOPSIS is considered a good MCDM method. In addition, in its simple mathematical form, this method has good computational power and effectively measures the relative performance for each option. ^20,21 In study, ⁶⁰ the best feature of the entropy method and the TOPSIS method was applied for probabilistic linguistics in multiple attribute group decision-making (MAGDM) in selecting industrial machinery and products suppliers. The other study ⁶¹ used adjusted entropy and TOPSIS to prioritize patients. The integrated method helps prioritize patients with COVID-19 and can detect the health status of asymptomatic carriers. With more diverse applications, the author compared different coatings (carboxymethyl cellulose (CMC), Persian gum (PG), low methoxyl pectin (LMP), and tragacanth gum (TG) coating) to improve shelf-life utilization of strawberries using TOPSIS entropy.

The application of the entropy and TOPSIS method in education studies will be mentioned in the next section.

University performance evaluation

Evaluating the performance of higher education has many studies with different methods around the world. ⁶² John M. LaVelle ‘s view holds that evaluators and evaluators have an important role in serving positive global ideals, especially in education. Using online search and curriculum analysis, the author provides a quick look at assessment education at the end of 2017 as well as a retrospective analysis of the advanced degrees in the assessment offered at universities through the ages. Simon Cadez, Vlado Dimovski and Maja Zaman Groff use a model that has been tested with multiple linear regression analysis to evaluate performance based on comparing the relationship between research and teaching effectiveness. The results show that research positively affects teaching quality, which alleviates concerns about the impact of detrimental research on teaching when assessing performance. ⁶³

Lan Ding and Yao Zeng used the same order prioritization techniques with ideal solution ¹⁷ combined with information entropy weight (IEW) to investigate the performance of 68 Chinese universities under the Ministry of Education (MOE) from 2002 to 2011. With this study, the author points out the inadequacy of schools' financial allocation mechanism in China. Compare the performance of schools in different regions based on three-way analysis, teaching, research, and development. ²⁹ Shyh-Hwang Lee develops an intellectual capital (IC) assessment model to evaluate the performance of universities in Taiwan. A fuzzy approach integrated with the hierarchical analytical process (AHP) method is applied to build and prioritize metrics (IC) to build evaluation models (IC). A conceptual framework combining IC with university assessment programs in Taiwan is proposed to develop IC components as a central intangible resource of universities focusing on its relationship with university performance. ⁶⁴ Another study involved a combination of DEA and TOPSIS, to compare and rank effective units, the author evaluated the distance education of 56 public schools in Turkey, with six input variables and four output variables. ¹⁹ The Fuzzy Vikor technique (VlseKriterijumska Optimizacija I Kompromisno Resenje) is also used to evaluate the teaching management system in higher education. The method is applied to generate the criteria from the learning management system proposed in Saudi Arabian universities. ¹⁴

In recent years, in Vietnam, the evaluation of the performance of universities has received more attention. The quality assurance and accreditation of higher education in Vietnam are marked by establishing the Department of Examinations and Accreditation (now the Department of Quality Management). The evaluation of quality universities is based on the criteria issued by the Ministry of Education and Training in Decision No. 65/2007/QD-BGDDT dated 1 November 2007. Accordingly, this assessment will be based on ten standards with 61 criteria, including mission and goals of the university; organization and management; education program; training activities; management staff, lecturers and staff; learner; scientific research, application, development, and technology transfer; international cooperation activities; libraries, learning equipment, and other facilities; finance and financial management. ⁶⁵

In addition to the Ministry of Education and Training's accreditation and training quality assessment activities, studies are also discussed by the authors. Carolyn-Dung. T. T. Tran and Renato A. Villano 2016 propose an empirical study on the performance of educational institutions in Vietnam by applying a two-stage semi-parametric data envelopment analysis method. The authors used this method to estimate the efficiency of educational institutions, including universities and colleges, besides examining the influence of factors affecting the performance of schools. ⁵ The authors continued evaluating the performance of Vietnamese private universities by using DEA with the bootstrap gap approach. ⁸ Phung Phi Tran et al. used DEA as an application to evaluate the resource allocation of universities, including teaching and research efficiency. The authors apply the elasticity of scale to distinguish the leading benchmark among Vietnamese universities. ⁷ Our recent research has used cluster analysis in assessing the operational performance of the public universities in Vietnam. By cluster analysis, clusters with the same concert are grouped and help the author compare and find the field cluster that is the benchmark against other collections. In addition, combined with ANOVA analysis, the author evaluates the criteria affecting the performance of university groups. ⁶⁶ Xian-Bei Liu et al. extracted 11 criteria and built a model through TOPSIS and Entropy method to establish a comprehensive higher education evaluation system. Then classify based on the development score, apply Hopfield neural network to evaluate the accuracy of the classification. This study has applied the model to many countries, and especially for in-depth analysis, the authors chose Vietnam for implementation. ¹⁸

In the above studies, the indicators showing the university's resources were used to evaluate the performance, but there was no comparative analysis between the criteria of the universities. Benchmarking must find out which criteria are met as benchmarks. Furthermore, before the resource analysis, the analysis criteria have not yet built the weights. One issue that needs to be mentioned is that there should be a classification between the universities to group the analysis. With the advantage of classifying good universities with high rankings, based on the criteria collected in this study, the author uses the Entropy method to determine the weights of the criteria. Then TOPSIS is used to find the ranking order of private universities. The results of the ranking analysis, based on the analysis of the requirements of teaching effectiveness. Spearman's rank correlation coefficient was applied to determine the degree of correlation between the 2 years of study. Finally, using ANOVA in-depth analysis to assess the influence of the criteria on the performance of the training activities of private universities.

Methodology

The Technique for Order of Preference by Similarity to Ideal Solution ¹⁷ method is one of the multiple-criteria decision-making (MCDM) - model methods. This model is based on fuzzy set theory to solve complex selection problems with many criteria and many choices. Besides, the MCDM method will quantify these standards, calculate the total score of the assessment participants according to the weight of each criterion and help decision-makers have a more solid and accurate basis. Many studies have applied MCDM with methods such as TOPSIS, AHP, DEA, ANP, etc.

TOPSIS method of Hwang and Yoon ⁶⁷ is a popular tool to solve decision-making problems with MCDM multi-standards. The main content of TOPSIS is to evaluate options by simultaneously measuring the distance from the options to the positive optimal solution (PIS) and the negative optimal solution (NIS). The selected alternative must have the shortest distance from the PIS and the furthest distance from the NIS. . TOPSIS is built on Entropy weights. This weight is based on probability theory to evaluate the probability of an event occurring, limiting the subjective effects that other methods have, such as the Delphi method and the Analytical Hierarchical Process (AHP). The process of the model is presented in the following steps (Figure 1):

Figure 1.

Research model.

Step 1

Selecting the objects.

Step 2

Determine the evaluation criteria.

Step 3

Determine the weight of the evaluation criteria.

a. The general expression of the evaluation matrix is as follows.

Suppose there are m objects, k criteria for evaluation. The evaluation criteria of the objects are presented as a matrix D = [rij] m.k

$$\mathbf{D=[}{\mathbf{r}}_{\mathbf{ij}}{\mathbf{]}}_{\mathbf{m\times k}}\mathbf{=}\begin{array}{c}\\ \begin{array}{c}{\mathbf{A}}_{\mathbf{1}}\\ {\mathbf{A}}_{\mathbf{2}}\\ \mathrm{...}\\ {\mathbf{A}}_{\mathbf{m}}\end{array}\end{array}{\begin{array}{c}\left[\begin{array}{cccc}{\mathbf{C}}_{\mathbf{1}}& {\mathbf{C}}_{\mathbf{2}}& \mathbf{\dots }& {\mathbf{C}}_{\mathbf{k}}\\ {\mathbf{x}}_{\mathbf{11}}& {\mathbf{x}}_{\mathbf{12}}& \mathbf{\dots }& {\mathbf{x}}_{\mathbf{1k}}\\ {\mathbf{x}}_{\mathbf{21}}& {\mathbf{x}}_{\mathbf{22}}& \mathbf{\dots }& {\mathbf{x}}_{\mathbf{2k}}\\ \mathbf{\dots }& \mathbf{\dots }& \mathbf{\dots }& \mathbf{\dots }\\ {\mathbf{x}}_{\mathbf{m1}}& {\mathbf{x}}_{\mathbf{m2}}& \mathbf{\dots }& {\mathbf{x}}_{\mathbf{mk}}\end{array}\right]\end{array}}_{\mathbf{m\times k}}\begin{array}{c}\mathbf{i=1,\; 2,}\mathrm{...}\mathbf{m}\\ \mathbf{j=1,\; 2,}\mathrm{...}\mathbf{,\; k}\end{array}$$

(1)

Where A1, A2 … Am are alternatives to be chosen, C1, C2…Ck are criteria by which alternative performance are measured, xij is a crisp value that alternative Ai holds concerning each criterion Cj.

• b. Normalization of research data.

Value rij is calculated as.

$$r_{ij}=\frac{x_{ij}}{\sqrt{{\displaystyle \sum _{i=1}^m{x}_{ij}^2}}}\hspace{0.17em}j=1,2,\mathrm{...},k$$

(2)

• c. Calculate the Entropy value

The attributes weights consider the relative importance in the decision-making process in a typical multi-criteria decision-making (MCDM) approach. We cannot assume that each evaluation criterion is of equal importance because the evaluation of criteria includes many different opinions and meanings 68. Therefore, there are two weighting methods: subjective and objective. The subjective methods are to determine the weights only according to the whim or judgment of the decision makers. Then apply some mathematical techniques to calculate the overall rating of each decision-maker. Some popular subjective weighting methods are pairwise-comparison-based SWARA, KEMIRA, SIMOS, FUCOM, and DEMATEL, etc. However, although the subject method has the advantage of integrating information from experienced decision-makers, such communication may sometimes favor a specific criterion because of the decision maker’s past belief, thus leading to biased results. ⁶⁹ A part from this, decision-makers who do not have complete knowledge about the decision problem under consideration may not furnish the needed initial information. ⁷⁰ Besides, the process of delivering such information may become complex when the MCDM problem involves many criteria.

Difference from subjective methods, objective methods do not require initial information or judgment from the decision-makers ⁷¹ ; they merely assess the structure of the data available in the decision matrix to determine the weights. ⁷² These methods eliminate possible bias associated with subjective evaluation, thus increasing objectivity. ⁷³

The objective methods determines weights by solving mathematical models automatically without any consideration of the decision maker’s preferences, for example, the entropy method, multiple objective programming, etc. To ensure that the evaluation result will be affected by the weighting approaches, subjective weighting and objective weighting methods are both utilized in the comparison. The use of subjective weighting is based on decision maker’s expertise and judgment, nevertheless the objective weighting is based on mathematical computation. The approach with objective weighting is particularly applicable for situations where reliable subjective weights cannot be obtained. ⁷⁴

Researchers have proposed many objective weighting measures. Shannon’s entropy concept ⁷⁵ is well suited for measuring the relative contrast intensities of attributes to represent the average intrinsic information transmitted to the DM. The criteria weights are not estimated to ensure that each criterion's inter-dependence is not affected by the evaluation results. Shannon's entropy analysis is known as a measurement tool to calculate the criteria weights and measures uncertainty in knowledge formulated in probability theory. ⁷⁶ Wang et al. extend Shannon’s concept of using entropy measure as the weighting calculation method. Entropy weight is a parameter that describes how much different alternatives approach one another regarding a particular attribute. The greater the entropy value, the smaller the entropy weight, the smaller the other options in this specific attribute, and the less information the distinguishing feature provides, and the less important this attribute becomes in the decision-making process.

The evaluation criteria of this research are quantitative measured data. Therefore, objective weighting methods, namely, Entropy is used to determine evaluation criteria weights. Refer to the Entropy concept to calculate the weight of each criterion; Entropy is expressed as (the Entropy value of the criterion)

$$E_j=-c{\displaystyle \sum _{i=1}^m{r}_{ij}\hspace{0.17em}\log r_{ij}j}=1,2,\mathrm{...},k$$

(3)

where $c=\frac{1}{\log m}$ is a constant, make sure $0\le E_j\le 1$

• d. Calculate the Entropy weight.

$$w_j=\frac{1-E_j}{{\displaystyle \sum _{j=1}^k\left(1-E_j\right)}}j=1,2,\mathrm{...},k$$

(4)

Step 4

Processing TOPSIS Method

$$\begin{array}{l}V={\left[r_{ij}\right]}_{m\times k}{\left[w_j\right]}_{k\times 1}={\left[\begin{array}{cccc}{r}_{11}& r_{12}& \dots & r_{1k}\\ r_{21}& r_{22}& \dots & r_{2k}\\ \dots & \dots & \dots & \dots \\ r_{m1}& r_{m2}& \dots & r_{mk}\end{array}\right]}_{m\times k}{\left[\begin{array}{c}{w}_1\\ w_2\\ \mathrm{...}\\ w_k\end{array}\right]}_{k\times 1}\\ \begin{array}{c}\end{array}={\left[v_{ij}\right]}_{m\times k}={\left[\begin{array}{cccc}{v}_{11}& v_{12}& \dots & v_{1k}\\ v_{21}& v_{22}& \dots & v_{2k}\\ \dots & \dots & \dots & \dots \\ v_{m1}& v_{m2}& \dots & v_{mk}\end{array}\right]}_{m\times k}\begin{array}{c}i=1,2,\mathrm{...},m\\ j=1,2,\mathrm{...},k\end{array}\end{array}$$

(5)

• a. Constructing standardized weight matrix

where $v_{ij}=r_{ij}·w_j^{\begin{array}{l}\\ \end{array}}$ .

• b. Seek positive ideal solution and negative ideal solution.

  The positive ideal solution (PIS) V+ is presented as.

$$V^{+}=[\underset{i}{\max }\left(v_{i1}\right),\underset{i}{\max }\left(v_{i2}\right),\mathrm{...},\underset{i}{\max }\left(v_{ik}\right)]=(v_1^{+},v_2^{+},\mathrm{...},v_k^{+})$$

(6)

The negative ideal solution (PIS) V- is presented as.

$$V^{-}=[\underset{i}{\min }\left(v_{i1}\right),\underset{i}{\min }\left(v_{i2}\right),\mathrm{...},\underset{i}{\min }\left(v_{ik}\right)]=(v_1^{-},v_2^{-},\mathrm{...},v_k^{-})$$

(7)

• c. Calculate the separation measure positive-ideal one $D_i^{+}$ and negative-ideal one $D_i^{-}$ by us ing the n-dimension Euclidean distance. Each alternative separation from the ideal solution is identified as

$$D_i^{+}=\sqrt{{\displaystyle \sum _{j=1}^k(v_{ij}-v_j^{+}}{)}^2}\begin{array}{ccc}& & \end{array}i=1,2,\mathrm{...},m$$

(8)

$$D_i^{-}=\sqrt{{\displaystyle \sum _{j=1}^k(v_{ij}-v_j^{-}}{)}^2}\begin{array}{ccc}& & \end{array}i=1,2,\mathrm{...},m$$

(9)

• d. Calculate the relative closeness coefficient to the ideal solution $\left(C{C}_i\right)$ .

$$C{C}_i=\frac{D_i^{-}}{D_i^{+}+D_i^{-}}\hspace{0.17em}\hspace{0.17em}i=1,2,\mathrm{...},m$$

(10)

Where $0\le C{C}_i\le 1$

• e. Rank the preference order according to the descending order of $C{C}_i$

So based on CCi to rank the research objects, the object with the larger Ci is better.

Step 5

Apply Spearman's rank correlation coefficient

Theorem 1

Spearman rank correlation formula. Depending on whether or not there is a relationship in the rank (the same rank is assigned to two or more observations), the Spearman correlation coefficient can be calculated using one of the following formulas.

$$\rho =1-\frac{6{\displaystyle \sum d_i^2}}{n\left(n^2-1\right)}$$

(11)

where: di is the difference between a pair of ranks. n is the number of observations.

Step 6

Group the options.

Based on the ranking results, we group the fields to get groups of values compatible.

Step 7

ANOVA in-depth analysis comparing standards

Data collection and analysis.

Methods of data collection

This article focuses on private universities with publicly reported data based on the Regulations on Public Implementation of Education and Training Institutions of the National Education System (Issued with Circular No. 36/2017/TT-BGDĐT dated 28 December 2017, of the Minister of Education and Training). Combined with examining relevant statistics with the Vietnamese Ministry of Education, we have chosen to analyze 45 private universities (see Table 1) in Vietnam with data collected over two academic years, 2018–2019 and 2019–2020. We selected the data of two consecutive school years to have a more objective assessment. For example, the number of students enrolled in this school year is crowded, but they may reduce in the next school year. The steps for applying the proposed model are as follows:

Table 1.

List of 45 private universities.

University full name	Abbreviation	University full name	Abbreviation
Quang trung university	QTU	Ba ria vung tau university	BVU
Vietnam Northern university	VNU	University of economics – Technology of binh duong	ETBD
Tan tao university	TTU	Dai Nam university	DNAU
The saigon international university	SIU	Lac hong university	LHU
Luong the vinh university	LTVU	Da Nang university of Architecture	DAU
Gia dinh university	GDU	Dong Nai technology university	DNTU
Hung vuong university	HVU	Mekong university	MKU
Kinh bac university	KBU	Dong A university	DOAU
University of the Pacific	POU	Sai gon technology university	SGTU
Yersin university	YERSIN	Thang long university	TLU
University of Phan thiet	UPT	University of economics – Finance city. HCM	UEF
East Asia university of technology	EAUT	Thanh do university	THDOU
Hoa binh university	HOABINH	Hoa sen university	HSU
Eastern university of technology	MUT	Tay do university	TDU
Huu Nghi university of technology and management	HNUTM	Ho chi minh city university of foreign languages and information technology	HUFLIT
Long an university of economics and industry	LAEIU	Hong bang university	HBU
Buon ma thuot university	MBTU	Van hien university	VHU
Financial and banking university	FBU	Da Nang FPT university	FPT
Thanh dong university	THDU	Van lang university	VLU
Eastern international university	EIU	Nguyen Tat thanh	NTT
Dong do university	DODOU	Duy tan university	DTU
Phenikaa university	Phenikaa	Hutech university	HUTECH
Phuong dong university	PDU	—	—

Note: Follow each university code.

Step 1

Selecting Private Universities

In this study, the proposed model is applied to select, group, and evaluate 45 private universities in Vietnam. The private universities in Vietnam are gradually changing and developing to meet learners' needs and benefit society. Therefore, the evaluation and appropriate strategy for the educational leaders will be necessary to overview each school's operation and training organization.

Step 2

Define the evaluation criteria

The research data are collected through public school reports to the Ministry of Education and Training annually. Based on the publicly available standards, data in the report combined with the set of 10 higher education quality assessment standards issued on 1 November 2007, the Minister of Education and Training signed the Regulations on standards for assessing the quality of higher education, ⁷⁷ the group of authors choosing standards in the evaluation process include:

Academic staff (C1), Non-academic staff (C2), Facilities construction use for training, scientific research (m2) (C3), Number of students (C4), Graduated students (C5),% Student get job after one year (C6), Revenue from tuition (C7). See Table 2 for details.

This study selects the criteria suitable for the operational model process: teaching effectiveness and revenue efficiency. ^7,66 For teaching effectiveness, the input of this activity is Academic staff, Non-academic staff, and Facilities. The output is three factors: number of students enrolled at the school, number of graduates, and % of students having a job. These three factors are also inputs to the university's revenue efficiency, see Figure 2.

Table 2.

The selected criteria for analysis.

COD	Requirement criteria	Explanation
Publication of conditions to ensure the quality of education
C1	Academic staff	The academic staff knows as professors, assistant professors, lecturers, and experts
C2	Non-academic staff	Workforce working in student management offices, training rooms, corporate relations departments, international cooperation departments, accounting, human resources, examinations and accreditation, library, and security
C3	Facilities construction use for training, scientific research (m2)	An area unit with one square meter of international standard (m2) such as lecture hall, classrooms, libraries, laboratory areas, workshops
Publication of the actual quality of education
C4	Number of students	The number of students currently enrolled, registered, and entered in courses in the current year
C5	Graduated students	Total number of students graduating during the current year
C6	% Student get job after one year	The proportion of students who have a job after one year of measuring the effectiveness of training, skills, and knowledge gained after graduation is employed
Publication of financial revenues and expenditures
C7	Revenue from tuition	Revenue from tuition is composed of tuition collected from current students in the university

Figure 2.

Teaching effectiveness and Revenue efficiency.

Step 3

Determine the weight of the evaluation criteria.

a. Present the original decision matrix.

In this study, there are 45 universities (alternatives: A_i, i= 1, 2, 3…45), seven criteria (criteria Cj, j=1, 2, 3…,7) for evaluation. The original matrix is set in Table 3 for 45 universities and seven criteria.

b. Normalization of research data.

Each criterion has its heterogeneous unit that can be used for a general analysis. In applying TOPSIS, it is characteristic that the raw data normalization is suitable for eliminating outliers with different units of measure and scale in several statistical analysis methods. The initial decision matrix should be standardized using equations (1) and (2), and the normalized matrix is shown in Table 4 for the academic year 2018–2019 and Table 5 for the academic year 2019–2020. For further clarification, an instance is shown as follows

$$\begin{array}{l}{r}_{1.1}=\frac{x_{1.1}}{\sqrt{{\displaystyle \sum _{i=1}^m{x}_{1.1}^2}}}\\ r_{1.1}=\frac{60}{\sqrt{{\left(60\right)}^2+{\left(79\right)}^2+{\left(80\right)}^2+\mathrm{...}+{\left(807\right)}^2+{\left(1375\right)}^2+{\left(1257\right)}^2}}=\frac{60}{2756.3329}=0.0217681\end{array}$$

• b. Calculate the Entropy weight (eequations (3) and (4))

According to equations (3) and (4) we can calculate the Entropy weight. Looking the Table 6 to see the Entropy weight of seven criteria and example as below

$$\begin{array}{l}{E}_1=-c{\displaystyle \sum _{i=1}^m{r}_{1.1}}\log r_{1.1}\\ E_1=-\frac{1}{\log m}{\displaystyle \sum _{i=1}^m{r}_{1.1}}\log r_{1.1}\\ E_1=\frac{-1}{Log(45)x[(0.0217681xlog(0.0217681)+\mathrm{...}+(0.4560407xlog(0.4560407)]}\\ E_1=\frac{-1}{-9.4014018}=2.4697\\ W_1=\frac{1-E_j}{{\displaystyle \sum _{j=1}^k(1-E_j)}}=\frac{1-2.4697}{(1-2.4697)+(1-2.4389)+\mathrm{...}+(1-1.8522)}\\ W_1=\frac{-1.4697}{-10.30400}=0.14263\end{array}$$

Table 3.

Performance of 45 private universities.

No	COD NAME	Academic staff (C1)	Non-academic staff (C2)	Construc-tion use for training, scientific research (C3)	Number of students (C4)	Graduated student in recently year (C5)	% Student get Job after 1 year (C6)	Revenue from tuition fee (C7)
1	QTU	60	22	10,120	238	410	75	8.74
2	VNU	79	15	20,200	321	160	72.06	2.95
3	TTU	80	23	47,364	444	88	90.9	41.09
4	SIU	76	35	25,518	546	32	100	17.8
5	LTVU	87	18	16,078	577	150	80	6
6	GDU	90	25	16,714	666	250	93.39	7.1
7	HVU	75	20	7637	1100	250	85	1.98
8	YERSIN	95	35	9074	1352	160	94	11
9	POU	131	27	5491.9	1437	428	96.33	21
10	UPT	133	35	9054.27	1825	295	96.47	28.5
30	MKU	245	85	22,196	5252	385	97.75	77.73
31	DNTU	276	95	34,024	5300	994	90.4	78
32	LHU	430	96	42,971.29	5435	1210	99	197.6
33	SGTU	276	80	30,694	6271	1179	96.23	128.838
34	TLU	327	68	33,598	6942	693	87.13	168
35	THDOU	157	57	43,110	7842	893	100	81.2
36	VHU	474	75	26,050	8051	2800	87	40.1
37	HSU	351	105	30,527	8104	1429	88.7	256.253
38	TDU	608	66	76,129	8263	580	72	280
39	HBU	536	110	88,711	9400	1200	93	199.8
40	HUFLIT	395	89	30,448	11,413	1020	94	301
41	FPT	568	140	47,511	13,272	666	98.35	341.7
42	VLU	613	95	45,825	13,528	1770	94.2	490
43	DTU	807	325	55,574.54	16,751	5039	94.4	400
44	NTT	1375	350	66,332	21,183	1925	80	690.231
45	HUTECH	1257	189	93,281	23,395	3509	92	783

Note: Collected data from the report of each university

Table 4.

Data Normalization Year 2018–2019 (see full Table 16 in Appendix).

No	COD NAME	C1	C2	C3	C4	C5	C6	C7
1	QTU	0.02177	0.03351	0.03844	0.00474	0.04717	0.12594	0.00605
2	VNU	0.02866	0.02285	0.07673	0.00640	0.01841	0.12101	0.00204
3	TTU	0.02902	0.03504	0.17992	0.00885	0.01013	0.15264	0.02846
43	DTU	0.29278	0.49510	0.21111	0.33390	0.57979	0.15852	0.27701
44	NTT	0.49885	0.53319	0.25197	0.42225	0.22149	0.13434	0.47800
45	HUTECH	0.45604	0.28792	0.35434	0.46634	0.40375	0.15449	0.54225

Table 5.

Data Normalization Year 2019–2020 (see full Table 17 in Appendix).

No	COD NAME	C1	C2	C3	C4	C5	C6	C7
1	QTU	0.03060	0.03663	0.03846	0.00290	0.01854	0.12542	0.00482
2	VNU	0.02953	0.04249	0.07676	0.00784	0.01095	0.12208	0.00189
3	TTU	0.02882	0.04102	0.17999	0.00876	0.01262	0.15017	0.02017
43	NTT	0.49173	0.51718	0.25208	0.36737	0.20383	0.12793	0.42416
44	DTU	0.28963	0.48348	0.21120	0.36956	0.58423	0.15786	0.25315
45	HUTECH	0.44725	0.32525	0.35449	0.45770	0.40207	0.15385	0.50313

Table 6.

Entropy weight of each criterion.

No	Year	C1	C2	C3	C4	C5	C6	C7
1	Year 2018–2019	0.1426361	0.1396452	0.1586634	0.1281730	0.1218067	0.2263679	0.0827076
2	Year 2019–2020	0.1436931	0.1425977	0.1553486	0.1314698	0.1236888	0.2223441	0.0808579

Step 4

Processing TOPSIS Method

a. Constructing normalized weight matrix

From the equation $v_{ij}=r_{ij}·w_j^{\begin{array}{l}\\ \end{array}}$ ; we have the result in Table 7 and Table 8 and the calculation as an example below $v_{11}=r_{11}.w_1=0.021768\times 0.14263=0.0031049$

b. Seek positive ideal solution and negative ideal solution

Base on the equations (6) and (7), the positive ideal solution (PIS) V⁺ and the negative ideal solution (PIS) V^- are presented in Table 9.

c. Calculate the separation measure positive-ideal one $D_i^{+}$ and negative-ideal one $D_i^{-}$

From equation (8) $D_i^{+}=\sqrt{{\displaystyle \sum _{j=1}^k(v_{ij}-v_j^{+}}{)}^2}$ we have the positive-ideal $D_i^{+}$ as an example.

$\begin{array}{l}{D}_1^{+}=\sqrt{{\left(0.0031049-0.0711542\right)}^2+{\left(0.0046802-0.0744572\right)}^2+\mathrm{...}+{(0.0005006-0.0448481)}^2}\\ D_1^{+}=0.14757409\end{array}$ From equation (9) $D_i^{-}=\sqrt{{\displaystyle \sum _{j=1}^k(v_{ij}-v_j^{-}}{)}^2}$ we have the negative -ideal $D_i^{-}$ as an example.

$\begin{array}{l}{D}_1^{-}=\sqrt{{\left(0.0031049-0.0031049\right)}^2+{\left(0.0046802-0.0031910\right)}^2+\mathrm{...}+{(0.0005006-0.0001134)}^2}\\ D_1^{-}=0.00855720\end{array}$ And the result was presented in Table 10 (see full Table 10 in Appendix).

d. Calculate the relative closeness coefficient to the ideal solution $\left(C{C}_i\right)$ .

According to equation (10), the closeness coefficient CCi was calculated and presented in Table 11.

The CC₁ as an example $c{c}_1=\frac{D_1^{-}}{D_1^{+}+D_1^{-}}=\frac{0.00855720}{0.00855720+0.14757409}=0.054807722$

As the results are evident in Table 11, we favor the initial results consistent with their relative proximity to the ideal solution. Therefore, the rankings of the universities are shown as follows:

Table 7.

Normalized weight matrix 2018–2019 (see full Table 18 in Appendix).

No	COD NAME	C1	C2	C3	C4	C5	C6	C7
1	QTU	0.00310	0.00468	0.00610	0.00061	0.00575	0.02851	0.00050
2	VNU	0.00409	0.00319	0.01217	0.00082	0.00224	0.02739	0.00017
3	TTU	0.00414	0.00489	0.02855	0.00113	0.00123	0.03455	0.00235
43	DTU	0.04176	0.06914	0.03350	0.04280	0.07062	0.03588	0.02291
44	NTT	0.07115	0.07446	0.03998	0.05412	0.02698	0.03041	0.03953
45	HUTECH	0.06505	0.04021	0.05622	0.05977	0.04918	0.03497	0.04485

Table 8.

Normalized weight matrix 2019–2020 (see full Table 19 in Appendix).

No	COD NAME	C1	C2	C3	C4	C5	C6	C7
1	QTU	0.00310	0.00468	0.00610	0.00061	0.00575	0.02851	0.00050
2	VNU	0.00409	0.00319	0.01217	0.00082	0.00224	0.02739	0.00017
3	TTU	0.00414	0.00489	0.02855	0.00113	0.00123	0.03455	0.00235
43	NTT	0.04176	0.06914	0.03350	0.04280	0.07062	0.03588	0.02291
44	DTU	0.07115	0.07446	0.03998	0.05412	0.02698	0.03041	0.03953
45	HUTECH	0.06505	0.04021	0.05622	0.05977	0.04918	0.03497	0.04485

Table 9.

PIS and NIS solutions.

Year	Ideal solution	C1	C2	C3	C4	C5	C6	C7
Year 2018–2019	V+	0.07115	0.07445	0.05622	0.05977	0.07062	0.03801	0.04484
	V-	0.00310	0.00319	0.00289	0.00060	0.00044	0.02280	0.00011
Year 2019–2020	V+	0.07065	0.07374	0.05506	0.06017	0.07226	0.03718	0.04068
	V-	0.00403	0.00480	0.00283	0.00038	0.00102	0.02230	0.00011

Table 10.

The distance between the selections to the ideal solution. (see full Table 20 in Appendix)

No	COD NAME	Year 2018–2019		COD NAME	Year 2019–2020
		D+	D-		D+	D-
1	QTU	0.14757409	0.00855720	QTU	0.14723190	0.00655481
2	VNU	0.14757053	0.01055459	VNU	0.14533931	0.01040032
3	TTU	0.14201065	0.02838940	TTU	0.14068123	0.02754061
43	NTT	0.04671870	0.11905866	NTT	0.05220300	0.12082269
44	DTU	0.04781756	0.12753148	DTU	0.04369327	0.12000571
45	HUTECH	0.04098086	0.12678835	HUTECH	0.03614385	0.12577613

Table 11.

Closeness coefficient CCi and Ranking the University.

No	COD NAME	CCi	Ranking year 2018–2019	CCi	Ranking year 2019–2020
1	QTU	0.054807722	45	0.042622746	45
2	VNU	0.066748339	41	0.066780165	41
3	TTU	0.166604418	25	0.163716027	24
4	SIU	0.122425953	32	0.120045204	32
5	LTVU	0.066397094	42	0.065091004	43
6	GDU	0.094392046	38	0.091870146	38
7	HVU	0.065940083	43	0.064750778	44
8	YERSIN	0.089737277	39	0.089511372	39
9	POU	0.098999829	36	0.101955809	36
10	UPT	0.101766631	34	0.101101129	37
11	EAUT	0.101435029	35	0.102305368	35
12	HOABINH	0.063064427	44	0.065318647	42
13	KBU	0.12322236	31	0.119602279	33
14	MUT	0.268736308	13	0.269946035	13
15	HNUTM	0.116789888	33	0.113507672	34
16	LAEIU	0.137951013	29	0.137529463	28
17	EIU	0.277003355	12	0.278377578	12
18	MBTU	0.097014255	37	0.125383384	30
19	FBU	0.075696587	40	0.085023767	40
20	Phenikaa	0.132053668	30	0.136331905	29
21	DODOU	0.225210474	17	0.22564269	17
22	DNAU	0.158395652	27	0.165004715	23
23	ETBD	0.179659277	23	0.207514747	20
24	BVU	0.151187129	28	0.146567635	27
25	THDU	0.160383694	26	0.121969917	31
26	DOAU	0.200672286	20	0.162550242	25
27	PDU	0.170611655	24	0.147160837	26
28	UEF	0.20672511	19	0.249096117	14
29	DAU	0.198680859	21	0.20031417	21
30	MKU	0.183935658	22	0.18046447	22
31	DNTU	0.225613268	16	0.2171198	19
32	LHU	0.286361053	11	0.298500728	11
33	SGTU	0.232142527	15	0.233295476	15
34	TLU	0.222613514	18	0.225441212	18
35	THDOU	0.232813577	14	0.225989329	16
36	VHU	0.329473407	8	0.307706258	8
37	HSU	0.287669178	10	0.306373947	10
38	TDU	0.354575965	7	0.360584043	7
39	HBU	0.408564424	5	0.414051348	5
40	HUFLIT	0.302682418	9	0.306391442	9
41	FPT	0.387853364	6	0.379281805	6
42	VLU	0.42124547	4	0.446754237	4
43	DTU	0.718184084	3	0.733087713	2
44	NTT	0.727300685	2	0.698293372	3
45	HUTECH	0.755730737	1	0.776779563	1

Step 5

Apply Spearman's rank correlation coefficient

By the equation (11).

$$\begin{array}{l}\rho =1-\frac{6{\displaystyle \sum d_i^2}}{n\left(n^2-1\right)}=1-\frac{(0+0+1+\mathrm{...}+1+0)}{45({45}^2-1)}\\ \rho =1-\frac{6.192}{91080}=0.98735\end{array}$$

The result of spearman’s rank correlation coefficient between the year 2018–2019 and 2019–2020 showed that, over the past 2 years, the ranking of private universities has changed but not significantly, see Table 12.

Table 12.

Spearman rank correlation spreadsheet. (see full Table 21 in Appendix).

No	COD NAME	CCi	CCi	Ranking (2018–2019)	Ranking (2019–2020)	d	d 2
1	QTU	0.054807722	0.042622746	45	45	0	0
2	VNU	0.066748339	0.066780165	41	41	0	0
3	TTU	0.166604418	0.163716027	25	24	1	1
43	DTU	0.718184084	0.733087713	3	2	1	1
44	NTT	0.727300685	0.698293372	2	3	−1	1
45	HUTECH	0.755730737	0.776779563	1	1	0	0

Step 6

Group the options.

Based on the ranking results, private universities are grouped as follows, see Table 13.

Table 13.

Grouping of universities.

Year	Name of university	Number of universities	University grouping
2018–2019	HUTECH, NTT, DTU, VLU, HBU	5	Group 1
	FPT, TDU, VHU, HUFLIT, HSU, LHU, EIU, MUT, THDOU, SGTU	10	Group 2
	DNTU, DODOU, TLU, UEF, DOAU, DAU, MKU, ETBD, PDU, TTU, THDU, DNAU, BVU, LAEIU, Phenikaa	15	Group 3
	KBU, SIU, HNUTM, UPT, EAUT, POU, MBTU, GDU, YERSIN, FBU, VNU, LTVU, HVU, HOABINH, QTU	15	Group 4
2029–2020	HUTECH, DTU, NTT, VLU, HBU	5	Group 1
	FPT, TDU, VHU, HUFLIT, HSU, LHU, EIU, MUT, UEF, SGTU	10	Group 2
	THDOU, DODOU, TLU, DNTU, ETBD, DAU, MKU, DNAU, TTU, DOAU, PDU, BVU, LAEIU, Phenikaa, MBTU	15	Group 3
	THDU, SIU, KBU, HNUTM, EAUT, POU, UPT, GDU, YERSIN, FBU, VNU, HOABINH, LTVU, HVU,QTU	15	Group 4

Step 7

ANOVA in-depth analysis comparing standards

After ranking the universities according to the teaching effectiveness, groups of schools were divided, and four (4) groups were formed each academic year. In the next step, ANOVA analysis is applied to find the criteria with statistically significant differences. Then, to discover which performance was better at the same cluster level, the Duncan Post hoc test was performed. Results are presented in Tables 14 and 15.

a. General performance through seven analytical criteria

Non-significant differences Criteria.

Of the seven analytical criteria, % of students who had a job after one year was non-significant differences criteria; all groups had similar quality levels for 2 years; specifically, the ANOVA results as follows (F = 1.004, p = 0.402), (F = 0.753, p = 0.526) (see Tables 14 and 15) respectively.

Significant differences criteria

One-way ANOVA results show significant differences for the remaining six (6) criteria over the two school years 2018–2019, 2019–2020 respectively as follows: the Academic Staff (F =39.465, p < 0.001), (F = 41.655, p < 0.001); the Non-Academic Staff (F = 24.284, p < 0.001), (F = 27.789, p < 0.001); the Facilities construction use for training-scientific research (m2) (F = 24.977, p < 0.001), (F = 23.242, p < 0.001); the Number of Students (F = 47.313, p < 0.001), (F = 51.608, p < 0.001); the Graduated Student in recently year (F = 16.622, p < 0.001), (F = 16.479, p< 0.001); the Revenue from Tuition (F = 35.352, p < 0.001), (F = 39.975, p < 0.001) (see Tables 14 and 15); There exists a significant performance gap between 4 (four) clusters. This means that the quality of education of private universities has changed over 2 years.

ANOVA analysis results show that the values analyzed between the 2 years have different performances. Training assurance conditionality is additional, resulting in different training quality results. Therefore, educational managers need to pay attention to these criteria to operate teaching activities more effectively. The ANOVA theoretically supports identifying statistically distinct criteria for at least one pair of groups; however, it does not indicate differences in specific location between groups. The Duncan Post hoc test is applied in the next step to determine the group performance differences in these individual criteria.

b. Point out universities group performance.

The Duncan Post hoc test shows the general assessment of the private schools' training performance (see Tables 14 and 15). When applying this method, the levels of the criteria are indicated. Thus, there is a clear comparison between groups of fields. Since benchmarks are built, the low-performing groups can be based on standards to achieve better productivity. A detailed description of the differences of the 6 (six) spending is shown as follows:

Academic staff: The teaching team holds an important position in education and imparts knowledge to students. If their teaching works, they give the university credibility. During both academic years, with the highest rankings, group 1 universities achieved optimal performance across groups. Group A2 earned a higher degree than group A3, but there was no significant difference between these two groups. Group A4 has the lowest performance value, so the universities in this group need to improve teaching staff to improve the quality of training. There was no change in the ranks of the groups for the following school year. While group 1 and group 2 increased in performance, groups 3 and 4 did not see significant changes.

In this criterion, group A1 is the benchmark for A2 and A3 in the first school year. Likewise, A2 and A3 are the benchmarks for A4. In the second year of school, B1 is the standard for B2. In contrast, B2 is the standard of B3 and B4.

Non-academic staff: This team supports a lot of activities in the university besides the academic unit. The group with the highest performance is still group 1. During the two school years, group 1 was the benchmark for groups 2, 3, and 4.

Facilities construction use for training, scientific research (m²): This is necessary for the training activities. In the 2018–2019 and 2019-2020 school years, universities still maintain the same facilities, in which group 4 has lower performance than groups 2 and 3. Although groups 2 and 3 are not significantly different, they are the benchmark for group 4. In this criterion, group 1 has the highest performance, group 2, and the remaining groups follow.

Number of students: The number of students represents the university's enrolment ability. Students will enroll if the school has good training quality. In addition, training programs are tailored to students' competencies. Over the 2 years of study, the groups with the number of students have increased. There was a significant difference between groups, specifically group 1 performed better than group 2, then group 3, then group 4. Therefore, A1, B1 is the benchmark for A2, B2. A2, B2 are the benchmarks for A3, B3, and A3, and B3 is the benchmark for A4 and B4.

Graduate student in a recent year: The number of graduates shows the training quality of the institution when they ensure the correct output for students. In addition, it also demonstrates the student's learning ability. In the 2018–2019 school year, group A1 performed the best with the highest number of graduates compared to the rest. Groups A2 and A3 achieve the average value. However, A2 has better performance. The lowest is group A4. From that, we can say, A1 is the standard for groups A2 and A3. Besides, A2 and A3 are the standards for A4. During the following school year, performance groups increased but not significantly. Similar to Year 1, B1 is still the standard for B2 and B3. B2, B3 are the standard for B4.

Revenue from tuition: The revenue from tuition fees also demonstrates the effectiveness of the training activity. An increase in the number of students leads to an increase in tuition revenue. In this criterion, group 1 still holds the highest position. This is inevitable because, in other criteria, group 1 has high efficiency. Next comes group 2. Then comes groups 3 and 4. Thus, A1 is the benchmark for A2. A2 is the standard for A3 and A4. Likewise, B1 is the standard for B2. B2 is the standard for B3 and B4.

The Duncan Post hoc test helps to see the differences in performance between university groups and criteria. Making a comparison between groups of universities will help leaders know the university's position. From there, they make a development effort. A comparison between the criteria shows which groups of universities perform well and which criteria perform poorly for improvement.

After applying the research model to evaluate Viet Nam private universities, the following finding showed:

Based on the entropy approach, different criteria were determined for the target weights, the results of which are shown in Table 6. For the criteria affecting teaching effectiveness are shown, Academic staff (0.142) (0.143), Non-academic staff (0.139) (0.142), Facilities construction (0.158) (0.155), Number of students (0.128), (0.131) Graduated students (0.121) (0.123), % Student get job (0.226) (0.222), and Revenue from tuition (0.082) (0.080) for the two academic years 2018–2019 and 2019–2020, respectively. The criteria are all weighted insignificant in the next school year. Educational activities have changed but not strongly between the years. Private universities consider the impact of the criteria to operate.

The finding from calculating the closeness coefficient for the ideal solution gave the value of 45 private universities. From that result, the ranking of the fields is done. The university with the highest closeness coefficient value (0.7557) of the year 2018–2019; (0.7767) of the year 2019–2020 ranked first and the lowest (0.0548) of the year 2018–2019 (0.0426) of the year 2019–2020 entered in the last place.

The ranking correlation coefficient results between 2018–2019 and 2019–2020 in Table 11 show that the rankings of private universities have changed but not significantly in the past 2 years. The values of the criteria did not change much over one school year. Based on the coherence coefficient values and ratings for private universities regarding teaching effectiveness and revenue performance in Table 11, universities were classified into four groups in Table 13. The results show that five universities ranked from first to fifth place in group 1, 10 universities ranked from sixth to 15th place in group 2, 15 universities organized from 16th to 30th place in group 3, and the remaining 15 universities are in group 4.

ANOVA analysis is applied to find the criteria with statistically significant differences. In which, % of students who had a job after one year was non-significant differences criteria when this rate is statistically similar in universities. The remaining criteria are all important differences criteria. ANOVA analysis results show that the values analyzed between the 2 years have different performances.

Then, to discover which performance was better at the same group level, the Duncan Post hoc test was performed. Results are presented in Tables 14 and 15. Seeing that, group 1 with a set of high-ranked universities all have high-valued criteria as the benchmark for the remaining groups. Group 2 with the universities reaching the average value also becomes the benchmark for groups 3 and 4.

Table 14.

ANOVA and duncan post hoc test result for the year 2018–2019.

Criteria	Group means in each criterion	ANOVA result	Duncan post hoc results
Academic staff	A1 = 917.6 A2 = 377.90 A3 = 243.60 A4 = 111.87	F = 39.465 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	A4 < A3, A2 A3, A2 < A1
Non-academic staff	A1 = 213.80 A2 = 77.80 A3 = 61.07 A4 = 28.33	F = 24.284 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	A4 < A3, A2 A3, A2 < A1
Facilities construction use for training, scientific research (m2)	A1 = 69944.708 A2 = 48720.529 A3 = 25575.538 A4 = 12915.278	F = 24.977 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	A4, A3 < A2 A2 < A1
Number of students	A1 = 16851.40 A2 = 7353.10 A3 = 4075.33 A4 = 1446.27	F = 47.313 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	A4 < A3 A3 < A2 A2 < A1
Graduated student in recently year	A1 = 2688.60 A2 = 1029.40 A3 = 799.73 A4 = 300	F = 16.622 > FCrit (3, 41, 0.05) =2.832 Sig = 0.000***	A4 < A3, A2 A3, A2 < A1
% Student get job after 1 year	A1 = 90.72 A2 = 91.98 A3 = 87.79 A4 = 85.30	F = 1.0004 < FCrit (3, 41, 0.05) = 2.832 Sig = 0.402	Non-significant differences
Revenue from tuition	A1 = 512.606 A2 = 172.514 A3 = 78.741 A4 = 20.984	F = 35.352 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	A4, A3 < A2 A2 < A1

Group 1 = A1; group 2 = A2; group 3 = A3; group 4 = A4, ^∗∗∗ p <0.001.

Table 15.

ANOVA and duncan post hoc test result for the year 2019–2020.

Criteria	Group means in each criterion	ANOVA result	Duncan post hoc results
Academic staff	B1 = 924.60 B2 = 408 B3 = 243.60 B4 = 122.07	F = 41.655 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	B4, B3 < B2 B2 < B1
Non-academic staff	B1 = 224 B2 = 86.10 B3 = 60.40 B4 = 33.07	F = 27.789> FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	B4 < B3, B2 B3, B2 < B1
Facilities construction use for training, scientific research (m2)	B1 = 69944.708 B2 = 47576.036 B3 = 26493.866 B4 = 12625.011	F = 23.242 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	B4, B3 < B2 B2 < B1
Number of students	B1 = 18682.20 B2 = 7973.30 B3 = 4835.87 B4 = 1778	F = 51.608 > FCrit (3, 41, 0.05) =2.832 Sig = 0.000***	B4 < B3 B3 < B2 B2 < B1
Graduated student in recently year	B1 = 2741.20 B2 = 1138.70 B3 = 788.33 B4 = 333.53	F = 16.479 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	B4 < B3, B2 B3, B2 < B1
% Student get job after 1 year	B1 = 90.22 B2 = 92.03 B3 = 88.11 B4 = 86.41	F = 0.753 < FCrit (3, 41, 0.05) =2.832 Sig = 0.526	Non-significant differences
Revenue from tuition	B1 = 563.226 B2 = 199.254 B3 = 80.952 B4 = 20.917	F = 39.975 > FCrit (3, 41, 0.05) = 2.832 Sig = 0.000***	B4, B3 < B2 B2 < B1

Group 1 = B1; group 2 = B2; group 3 = B3; group 4 = B4, ^∗∗∗ p <0.001.

Discussion and recommendation

Viet Nam private and public universities have differences in management systems. That affects the quantity and quality of enrollment, the number of students, tuition fees, facilities, and the quality of training. Therefore, for the analysis to have objective results, this study chooses to analyze private universities separately.

University groups have insignificant changes in the adjacent academic years. Universities still maintain a level of development in teaching – training effectiveness. However, the development between private universities is not uniform. Universities in large cities and have been established a long time tend to be more developed and benchmarked for those universities in the province with lower criteria. Therefore, when choosing private schools to enroll in, students tend to choose a more prominent and quality one.

As mentioned in the literature review, there are many different methods used to analyze performance and decision-making. Recent studies on the evaluation of higher education performance in Vietnam are mainly based on the DEA method. In this method, the authors analyze input and output variables to examine which determinants significantly affect their performance. For benchmarking studies, universities are classified and ranked by region to select and compare groups of universities. In this study, the influencing criteria were analyzed, and the universities were ranked first. Then the comparison of the criteria is made later.

More importantly, through the results by determining the different levels of quality performance between groups and the criteria differ significantly, the authors give objective recommendations to educational managers as follows:

Firstly, it is related to increasing the size of the private university system. For the university, it is necessary to identify essential criteria for improvement. The number and quality of academic staff need to increase to match the university's training volume and contribute to the operation, scientific research, and technology transfer. Besides, the appropriate selection of non-academic staff is also one of the school leaders' requirements because this team can support a lot in the university's administration. Since all the departments have smooth coordination, the university's executive operations became well-executed. The conditions to ensure the quality of training must include the facilities. The university focuses on constructing a convenient classroom system, libraries, research rooms, and practice rooms to motivate teachers and students in learning and teaching. The above three criteria are also inputs to produce useful outputs that promote scientific research, maintaining and improving the quantity and quality of the international publication, and large numbers of students and the number of graduates on track. Thanks to the grouping after ranking results and the benchmarking formula, state education managers can select private universities according to the development strategy to invest in finance or land to expand the scale of the university's purpose.

Secondly is regard to planning and investing in the development of many crucial private universities. The high-performance groups' results were seen as the benchmark for the lower performing schools showed that applying the support model (Grouping-ANOVA-Duncan post hoc test) can help us easily assign the focus groups and under-standard groups. Educational administrators can use this approach to review new potential universities and select universities with high planning and development productivity.

Third, it is necessary to promote the competitiveness of private universities. From the benchmark results, university leaders can view the university's actual training activities for others in the same field or region to make a competitive strategy in the right direction. Based on the statistical analysis, the different significant criteria were indicated. At this time, the university will compare factors that should be promoted to increase productivity, focusing on investment or vice versa.

Conclusion

In the paper, we have established a higher education performance evaluation system by introducing a research model, including the Entropy weight-based TOPSIS method that calculated the weight and entropy of seven criteria. Then, we combined Spearman's assessment and applied deep ANOVA analysis with Duncan's post hoc test to compare the effectiveness of criteria and find out the benchmark. Finally, to make the model more realistic and popular, we analyzed 45 Vietnamese private universities, presenting data for two consecutive academic years.

Through the analysis results in the two academic years, 2018–2019 and 2019–2020, 45 private universities are ranked based on teaching capacity and revenue efficiency. Four groups of universities were collected based on the rankings. These groups possess important criteria, such as Academic staff, Non-academic staff, Facilities for training - scientific research (m2), Number of students, Graduate student in recent years, Revenue from tuition. After being a statistic, the fundamental differences in productivity between these groups can be easily indicated for each criterion. Then, analyzing Duncan's post hoc test helped see the groups that perform better and those that are not effective. Based on that result, under the same criterion, the groups with good performance are recommended to be standard for the lower performing groups. In addition, this standard is also the target for the following school years.

The advantages of the research method in this paper are reflected in several aspects such as specific data analysis process, weighted criteria, and subjects ranked for favorable classification. In addition, the criteria are compared and evaluated against each other. The application of this research model is a useful method to help administrators make decisions scientifically, objectively and limit subjective factors. Education leaders can generally evaluate which universities are performing well, recognize the weaknesses of the private education system based on benchmark analysis. Furthermore, they can see a clear goal to develop in the coming evaluation years. The contribution of this method also helps university leaders have an objective view of the actual training activities of different universities in the same field or region. They can formulate strategies development evenly or improve weak criteria. In addition, this method is also strongly applied in several other fields, such as selecting investment projects, assessing the reliability of the information, and evaluating the quality of products.

On the other hand, this paper also has shortcomings in the analysis. With the proposed model, the grouping of universities is based on the ranking results. This grouping can lead to the wrong universities cluster placement. The study analyzes the training effectiveness of 45 private universities in Vietnam based on seven selected criteria. However, to evaluate operational efficiency, it is advisable to analyze more scientific research effectiveness. Because scientific research activities between universities are not uniform, some universities do not have any international scientific research; therefore, the analytical results are not available when entering the Entropy weight analysis. Furthermore, the study does not analyze all private universities. The number of universities selected is based on fully published data. The research model only evaluates based on the data obtained from the report. Therefore, it does not analyze the factors affecting the ranking, such as reputation, time of establishment, location of construction.

This study was conducted to measure the training performance of Vietnamese private universities. This topic can be considered in public schools in Vietnam and expanded in different countries in future studies. More comprehensive performance measurement studies could be made by including new criteria such as research activity and technology transfer, international academic studies, student satisfaction, international cooperation. The new criteria will create new input and output variables. Qualitative variables such as student satisfaction can use fuzzy TOPSIS, or other fuzzy MCDM methods can be formatted. In future studies, other multi-criteria decision-making methods can be used instead of the TOPSIS method. The cluster method can be applied to group exactly the position of universities. Besides, future research can fully investigate the system of universities when universities simultaneously strictly report publicly to the Ministry of Education and Training. In addition, universities currently do self-assessment reports in more detail, so future research can combine to select robust data with new metrics.