Sage Journals: Discover world-class research

Abstract

This study was conducted to determine the variables that play a role in the efficiency of Health Management departments in Turkey and the ranking of them in order of importance. These variables were determined by systematic analysis. The Prisma method was used in the systematic analysis approach. Input and output variables used in studies assessing the efficiency of higher education institutions in the literature were listed. The ranking of these variables was determined by Analytical Hierarchy Process (AHP) method. The questionnaire used for the AHP analysis and the judgments of 127 academicians working in Health Management Departments in universities in Turkey were assessed. The first three input variables found as a result of the AHP were “the number of registered undergraduate students per faculty member”, “the number of faculty members”, and “the number of other academic staff”. The most important three output variables included “the number of articles searched on SCI, SSCI, SCI-E”, “the number of papers presented in international congresses”, and “the count of publications published in international peer-reviewed journals”. To achieve positive developments in the efficiency of Health Management Departments, it is recommended to carry out studies to increase the number of qualified publications.

Keywords

analytical hierarchy process efficiency health management systematic analysis

Introduction

Health institutions have a complex and dynamic structure. Within this structure, health managers should carry out the service delivery process in the best way possible and be able to distribute the available resources in a balanced way.¹ On the other hand, in addition to producing solutions to constantly changing problems today, these institutions have to adapt to external factors such as, globalization, legal and political factors, demographic changes, social and economic factors, technological advances, and occupation-specific regulatory conditions, which affect the provision of healthcare services.² Health managers, who shape the organization by making important decisions that affect the overall performance of health institutions, such as recruitment and education of personnel, acquisition of technologies related to the institution, additional services that can be offered, and appropriate distribution and allocation of financial resources,¹ are also under pressure for rational use of resources and reducing costs. In short, health managers are expected to do more with fewer possibilities, to produce measurable results and evidence-based management practices, and to increase the efficiency of the healthcare delivery process.³ Competent health managers are needed to meet all these conditions.

A qualified health management education is needed to train competent health managers. Health Management programs are opened in many universities in Turkey to realize this need. While 17 universities provided undergraduate education on Health Management in 2010, this number increased to 78 in 2020.⁴ Along with this quantitative increase, it is necessary to question the success of the teaching, learning, and research activities of the departments. In this context, it is very important to determine the variables that play a role in the efficiency of Health Management departments. Reflecting on this, this study was conducted to determine variables that play a role in the efficiency of Health Management departments in Turkey and the ranking of these variables in order of importance.

With this study, it will be possible to help decision makers and managers about the variables that should be given priority for an effective education. Academicians, who are one of the most significant stakeholders in health management, were polled for their perspectives, which offered objective criteria for evaluating the efficiency of the departments. On the other hand, no similar study was found in the literature about variables that are important in assessing the efficiency of Health Management departments. These features make up the notable original aspect of the study.

Theoretical framework

Higher education institutions are social and cultural institutions that play a critical role in society, economy, culture, innovation, and international communication.⁵ Financial difficulties in today’s global trends and the need to use scarce resources more efficiently have made it important to assess the performance of higher education institutions.⁶ Studies that determine which aspects should be improved in increasing the efficiency and productivity of higher education institutions are valuable.⁷ However, it is necessary to first determine the variables that play a role in the efficiency to do this.

Since the main activities of higher education institutions are teaching and doing scientific research,⁸ “teaching efficiency” and “research efficiency” are considered as the main criteria in the assessment of the efficiency of universities.⁹

Labor is an important input variable in universities, which are labor-intensive organizations.¹⁰ According to Tyagi et al. (2009) and Barra and Zotti (2016), this variable is the human resource parameter used by all departments for teaching and research activities.^11,12 When the literature is examined, it is seen that the “number of academic staff” variable is frequently used in the measurement of the efficiency of educational institutions. Similarly, Halkos et al. (2012) stated that the number of academic staff was widely used in the literature.¹³

Another variable frequently used in studies is the “number of students”. According to Tran and Villano (2017), the variable of the number of students is the users of concrete input resources in the teaching and research process of universities.¹⁴ According to De La Torre et al. (2017), the two most important variables representing human capital regarding inputs are the number of students and the number of academic staff.⁹

As can be seen, many variables are used to determine the efficiency of higher education institutions¹⁵ that produce a large number of outputs using various inputs. Although there is no clear rule regarding the determination of these variables, the variables used in studies in the past can be a guide for decision-makers.

A simultaneous assessment of many criteria and decision-making is involved in the efficiency of higher education institutions. For this purpose, multi-criteria decision-making (MCDM) techniques have been developed. One of the MCDM methods commonly used in the literature is the AHP method.

The AHP method, developed by Saaty in the 1980s, takes into account many factors simultaneously. Basically, the method makes a paired comparison of all criteria with each other related to the problem. In this method, first of all, the participants evaluate the priorities of the criteria/subcriteria in pairs according to their relative importance. To do this, a paired comparison matrix of the criteria is created. In this matrix the diagonals are equal to 1. Also a paired comparison matrix are positive and reciprocal. For example, if a_ij is shown as the pairwise comparison value of the i_th criterion and the j_th criterion, the a_ji value is obtained from the equation (1)/a_ij. Judgments represented by numbers from the 1–9 scale developed by Saaty is used for paired comparisons. According to this scale, the classification and values assigned are as follows: equal importance (1); moderate importance of one over another (3); essential or strong importance (5); very strong importance (7); extreme importance (9); and intermediate values between the two adjacent judgments (2, 4, 6, 8). If activity i has one of the above numbers assigned to it when compared with activity j, then j has the reciprocal value when compared with i. In the next step, the geometric mean of the values obtained as a result of each judgment is taken. The obtained pairwise comparison matrix is normalized. The priorities vector is calculated by taking the mean of the rows of the normalized matrix.

It is important to calculate the consistency ratio (CR) in the AHP method. With this value, it is checked whether the pairwise comparison judgment is consistent. It is desirable that the consistency ratio be less than 0.10. The fact that this value is less than 0.10 indicates that the inconsistencies in the judgments in the pairwise comparison matrix are at an acceptable level. When this value is exceeded 0.10, judgments often need to be reexamined.¹⁶ The overall consistency of these judgments is obtained as a ratio of sums of weighted consistency index (CI) to the corresponding sums of weighted random consistency indix (RI). The CI of a matrix of comparisons is given by CI = (λ_maks – n )/(n – 1). λ_maks is the largest or principal eigenvalue of A^′ = (a^′_ij) the perturbed value of A = (a_ij) with a^′_ji = l/a_ij forced. n is the number of elements being compared. There is a table of the RI value for each n. This table and details of calculations are included in the study by Saaty (1987).¹⁶

The AHP method was preferred in ranking the priority levels of the variables in this study.

Methods

The aim of the study

This study was conducted to determine the variables that play a role in the efficiency of Health Management departments in Turkey and the ranking of these variables in order of importance. The variables that determine the efficiency of Health Management Departments were ranked by the AHP method, starting from the most important.

To identify the variables, the literature was examined and systematic analysis was conducted. As a result of the systematic analysis, the variables used in studies assessing the efficiency of higher education institutions in the literature were listed. Accordingly, the judgments of the academicians were evaluated as expert opinion in order to rank the variables according to their importance. With these judgments, pairwise comparison matrices were created. All pairwise comparison matrices were combined into a single matrix by taking the geometric mean of the values for each criterion.

In line with the stated objectives, the study consists of two phases. These phases indicated as follows:

• Identifications of the variables related to the efficiency of Health Management departments by systematic analysis (PRISMA Method)

• Ranking the variables according to their priority with the AHP method in line with academicians judgments.

Population and sample of the study

In the first phase of the study, the variables that play a role in the efficiency of higher education institutions were listed by systematic analysis. The Prisma method was used in the systematic analysis approach. In this context, Web of Science (WOS), one of the most comprehensive databases, was used, similar to the studies of Clermont et al. (2015) and Witte and Lopez-Torres (2017),^17,18 and the articles published between 2014 and 2019 were searched. However, considering that the education system of each country may be different from each other, it is thought that it is important to examine parameters specific to Turkey. For this reason, besides WOS, Turkey YOK National Thesis Center Database¹⁹ was also searched.

The identification, screening, eligibility, and included stages of the review process for the PRISMA method are indicated in Figure 1.

Figure 1.

Flow chart of selection of studies reviewed according to the PRISMA guidelines.

There were 64 studies (52 articles, 12 theses) included in the systematic analysis according to the determined criteria. The findings of 64 studies were examined, and variables that could play a role in the efficiency of higher education institutions were determined. In this way, the variables that can be used in the efficiency of Health Management departments were listed.

In the second phase of the study, the ranking of the variables for the determination of the efficiency of the Health Management departments was identified in line with the academician’s judgments. All academicians working in Health Management departments that provide formal undergraduate education in Turkey made up the population of the study. No sampling procedure was employed; instead, it was aimed to reach the whole population. The total number of academicians was determined as 454. Of the total academicians, 127 responded to the questionnaire, and the judgments of 127 academicians were assessed using the AHP method.

While it is necessary to study with a sample large enough to represent the main population in survey/scale studies, as with other statistical methods, it is not necessary to administer the AHP method to a large number of people.²⁰ There is a consensus in the literature that AHP does not require a large sample.²¹ Although there is no study on the appropriate sample size for the AHP method, it can be administered to a small number of participants. According to Kil et al. (2016), the AHP method was applied in studies with small sample sizes by taking the opinions of a small number of experts (such as five experts, 18 experts, or 25 experts).²² For these reasons, it is thought that the number of experts reached in the study is sufficient for the method.

Data collection

In the systematic analysis phase of the study, the data were obtained through a comprehensive literature review. In the second phase of the study, primary data obtained from academician’s judgments between March 2019 and April 2019 were used.

Ethical considerations

Ethical approval for the study was taken (issue: 64075176–900-E.158; date: January 02, 2019) from Eskisehir Osmangazi University, the Committee for Research and Publication Ethics in the Social Sciences and Humanities.

Data analysis

In the first phase of the study, the Prisma method was used as a systematic analysis method. According to the inclusion criteria, we included studies which

• were specific to a university or higher education institution,

• included efficiency measurement with data envelopment analysis, which is the most widely used method in efficiency analysis in the literature,

• were written in English or Turkish,

• had an accessible full-text version,

• had been published (articles) in a peer-reviewed journal,

• had been published (articles) between 2014 and 2019 (year requirement was not sought for theses), and

• were research articles/theses.

A total of 64 studies meeting these criteria were evaluated. First of all, the input and output variables used in these studies were listed. In addition, by looking at the variables in the theses made in Turkey, the variables specific to Turkey were added to the list. By looking at this list, the common in almost all studies and most used input and output variables were determined. Thus, the short list was created. The data obtained from these shortlisted variables were determined as the variables that play a role in the efficiency of Health Management Departments and were included in the AHP analysis.

In the second phase of the study, the AHP method was employed to rank the variables that can be used in the efficiency of Health Management Departments in order of importance. MS Excel was used in the calculations related to the AHP method.

Results

Results of the systematic analysis

The variables used in the efficiency of Health Management departments are listed in Table 1.

Table 1.

Variables playing a role in the efficiency of health management departments.

Input variables	Output variables
Number of faculty members (Prof. Dr, associate professor Dr, Assist. Professor Dr)	KPSS 1 (public personnel selection exam) – General culture general aptitude score
Number of other academic staff (instructors, research assistants, specialists, lecturers)	Number of articles searched on SCI, SSCI, SCI-E^a
Number of registered undergraduate students/faculty members	Number of articles in international peer-reviewed journals
Number of computer labs	Number of articles in national peer-reviewed journals
2018 YKS (higher education institutions test) maximum points	Number of papers presented in international congresses
Total number of courses taught	Number of papers presented in national congresses
2018 quota occupancy rate (%)	Number of completed projects
Number of foreign students	Total number of citations/Number of academic staff
How long the department has been active (years)	Number of alumni

^aSCI: Science Citation Index; SCI – E: Science Citation Index-Expanded; SSCI: Social Sciences Citation Index.

Some of the studies examined with the systematic analysis used different classifications such as “the number of professors and associate professors”, while some others took into account “the total number of academic staff”. Considering the academic staff structure in Turkey, we discussed the number of faculty members and the number of other academic staff separately in this study.

The variable of “the number of students” was frequently used in the assessment of higher education efficiency in the studies. There are foreign students in the Health Management departments in Turkey. Therefore, we also included this variable in the analysis. Although some of the studies used the variable of “the number of graduate students”, this variable was not included in the current study due to the lack of postgraduate education in some of the Health Management departments in Turkey.

In some of the studies, “university entrance exam scores” were considered as an input variable. However, university entrance exams differ from country to country. Considering that a similar variable could be used in accordance with the education system in Turkey, “the YKS maximum points” was included in the analysis.

Studies measuring the efficiency of universities used spatial variables related to “the area in m²” as input variables. Instead of this variable, we thought it would be appropriate to use the variable “number of computer laboratories”, which Health Management departments often need to use, especially in applied courses. We also thought that “the total number of courses taught” in Health Management departments and “the quota occupancy rate” variables could also be considered as input variables. Health Management departments, which have made big progress especially in recent years, have opened in many universities, and their number is increasing day by day. For this reason, we thought that “the number of years” in which the department has been operating may also affect the efficiency as an input variable.

The majority of the studies examined with the systematic analysis used some financial indicators such as total current expenditures/expenses, research funding, government funding, grants, and budget appropriations. However, these variables were not included in the analysis in the current study since these data on Health Management departments in Turkey could not be obtained.

The most commonly used output variable found in the studies was the “number of publications” representing scientific production. This variable was used as a total number in some studies and as different classifications in some studies. In addition, the variable of “the number of citations” was also used. In the current study, the variable of “the number of publications” was examined under different categories. Also, we thought that it would be appropriate to include “the number of citations per academic staff” in the list of output variables in assessing the efficiency of Health Management departments.

One of the output variables used in the theses reviewed in the Turkey YOK National Thesis Center Database was “the Public Personnel Selection Exam (KPSS in Turkish) scores”. Accordingly, “KPSS -1 General Culture General Aptitude Score” variable was taken as an output variable in the current study.

Finally, the variables of “the number of alumni” and “graduate students” were also used in the studies examined. In this study, only “the variable of the number of alumni” was considered as one of the important factors in determining the efficiency since not all of the Health Management departments have postgraduate education in Turkey.

Results related to the AHP method

Input and output variables that play a role in the efficiency of Health Management departments and are listed in Table 1, were subjected to AHP analysis to determine their ranking in order of importance. The comparison matrix for the input variables is given in Table 2.

Table 2.

Comparison matrix of “input variables”.

Inputs^a	1.	2.	3.	4.	5.	6.	7.	8.	9.
1	1	3.113074	0.566774	4.797113	1.607853	2.434089	2.153700	4.915889	2.489800
2	0.321226	1	0.568676	3.724361	1.441528	1.935925	1.753513	4.702414	2.137631
3	1.764369	1.758471	1	4.464866	1.900036	2.263812	2.516471	4.865618	2.544188
4	0.208459	0.268502	0.223971	1	0.447010	0.447041	0.691203	2.058327	0.808263
5	0.621947	0.693708	0.526306	2.237087	1	1.502654	1.620689	3.871676	2.046437
6	0.410831	0.516549	0.441733	2.236930	0.665489	1	1.063089	3.286132	1.544038
7	0.464317	0.570284	0.397382	1.446752	0.617022	0.940655	1	2.829177	1.398984
8	0.203422	0.212657	0.205524	0.485832	0.258286	0.304309	0.353460	1	0.506838
9	0.401639	0.467808	0.393053	1.237222	0.488654	0.647652	0.714804	1.973019	1
Weights	0.203	0.142	0.216	0.049	0.120	0.091	0.083	0.032	0.065
CR = 0.016026233 < 0.10

^a1: Number of Faculty Members (Prof. Dr., Associate Professor Dr., Assist. Professor Dr.); 2: Number of Other Academic Staff (Teaching staff, Research Assistant, Specialist, Lecturer); 3: Number of Registered Undergraduate Students/Number of Faculty Members; 4: Number of Computer Labs; 5: 2018 YKS Maximum Point; 6: Total Number of Courses Taught; 7: 2018 Quota Occupancy Rate (%); 8: Number of Foreign Students; 9: How long the department has been operating (years).

The values in the matrix in Table 2 are the values of the pairwise comparison matrix formed by taking the geometric mean of the judgments of the academicians about input variables. With the help of this matrix, the weights of the input variables were calculated and indicated in the Table 2. The results were consistent as the consistency ratio (CR) was less than 0.10 (Table 2).

The visual of the distribution for the weights of the input variables is given in Figure 2.

Figure 2.

The distribution for the weights of the input variables.

Weights include values that show the importance of variables relative to each other. According to this, the variable, “the number of registered undergraduate students per faculty member” had the highest priority. This was followed by “the number of faculty members” and “the number of other academic staff”, respectively. The least important variable was “the number of foreign students” (Figure 2).

The comparison matrix for the output variables is given in Table 3.

Table 3.

Comparison matrix of “output variables”.

Outputs^a	1.	2.	3.	4.	5.	6.	7.	8.	9.
1	1	0.252748	0.282143	0.331223	0.294502	0.387481	0.342927	0.281219	1.268590
2	3.956517	1	4.319677	4.780255	1.720489	5.380989	4.677720	1.846171	4.588855
3	3.544306	0.231499	1	3.518566	0.968715	4.264419	3.089993	1.067737	3.624368
4	3.019115	0.209194	0.284207	1	0.422699	2.965724	1.044103	0.497540	2.813557
5	3.395567	0.581230	1.032295	2.365747	1	3.612114	2.680919	1.264245	3.999241
6	2.580775	0.185839	0.234499	0.337186	0.276846	1	0.360515	0.405437	2.144834
7	2.916077	0.213779	0.323625	0.957760	0.373006	2.773811	1	0.652492	2.982616
8	3.555944	0.541662	0.936560	2.009889	0.790986	2.466475	1.532585	1	3.663462
9	0.788276	0.217919	0.275910	0.355422	0.250047	0.466237	0.335276	0.272966	1
Weights	0.038	0.278	0.154	0.080	0.156	0.051	0.081	0.128	0.034
CR = 0.043274164 < 0.10

^a1: KPSS 1 – General culture general aptitude score; 2: Number of articles in journals searched on SCI, SSCI, SCI-E ; 3: Number of articles in international peer-reviewed journals; 4: Number of articles in national peer-reviewed journals; 5: Number of papers presented in international congresses; 6: Number of papers presented in national congresses; 7: Number of completed projects; 8: Total number of citations/Number of academic staff; 9: Number of alumni.

The values in the matrix in Table 3 are the values of the pairwise comparison matrix formed by taking the geometric mean of the judgments of the academicians about output variables. With the help of this matrix, the weights of the output variables were calculated and indicated in the Table 3. The results were consistent as the consistency ratio was less than 0.10 (Table 3).

The visual of the distribution for the weights of the output variables is given in Figure 3.

Figure 3.

The distribution for the weights of the output variables.

The most important output variable was “the number of articles in journals searched on SCI, SSCI, and SCI-E databases”. This variable was followed by “the number of papers presented in international congresses” and “the number of articles in international peer-reviewed journals”, respectively. The least important variable was “the number of alumni” (Figure 3).

Discussion

Higher education institutions use multiple inputs to produce more than one output.^15,23 This situation necessitates conducting studies to distribute resources appropriately and achieve efficiency and productivity in resource use. In this context, this study was conducted to determine the variables that determine the efficiency of the Health Management departments that offer formal undergraduate education in universities operating in Turkey and the ranking of these variables in order of importance. In the study, in addition to the literature, the judgments of people who were thought to be experts in their field were assessed with the AHP method to identify input and output variables. This is the original aspect of the study.

In the literature, there are studies in which the variables used in determining the efficiency of higher education institutions are assessed by the AHP method. For example, in the study in which the efficiency of higher education institutions in India was examined, Sahney and Thakkar (2016) used the AHP method to weight the variables and utilized the views of a total of 30 experts selected from higher education institutions.²⁴ In a similar study, Ruiz et al. (2015) assessed the efficiency of universities in Spain and used the AHP method by taking the opinions of 13 experts to determine the variables.²⁵ Another study was conducted by Thanassoulis et al. (2017: 432) by taking the opinions of 120 experts.²⁶ Also, Altamirano-Corro and PenicheVera (2014) used AHP and VZA methods together to assess the efficiency of graduate programs at a university in Mexico.²⁷

There are some studies in the literature on the education service of Health Management departments. For example, Shakhi et al. (2013) assessed the opinions of the students of the Health Services Management department about the quality of education. The survey method was used in this descriptive study.²⁸ Beigzadeh et al. (2014) aimed to investigate the challenges of the Health Services Management program at the undergraduate level. It is a qualitative study with in-depth and structured interviews with academics, instructors, and alumni.²⁹ In the study conducted by İnce and Söyük (2020), the academic performances of Health Management departments in Turkey were assessed using Copras, Moora, and Topsis methods, and universities were ranked.³⁰ Although there were studies on the efficiency of the departments in the literature, no study on determining and prioritizing the variables to be used in the assessment of the efficiency of the Health Management departments was found.

The current study has several limitations. These limitations included the use of Web of Science (WOS) and YOK national thesis center databases to determine the variables, the selection of the range of years 2014 2019 as the scope of the search, and the exclusion of variables from the analysis for which data could not be reached. In addition, only the judgments of the academicians who answered the questionnaire were evaluated. The non-respondents bias is another limitation of the study.

Conclusion

As a result of the study, “the number of registered undergraduate students per faculty member” was found to be the variable with the highest priority. This variable was followed by the number of faculty members and the number of other academic staff, respectively. This showed how important human resources were in providing an effective education service and research activity.

Of the output variables, “the number of articles in journals searched on SCI, SSCI, SCI-E databases” was found to be the most important variable. This was followed by “the number of papers presented in international congresses” and “the number of publications in international peer-reviewed journals”, respectively. In the light of this information, it can be said that the variable that most affected the efficiency of the departments was the number of publications. In line with this finding, it is recommended that departments determine strategies to publish more research in their improvement plans.

We think that the findings obtained from this study will guide the decision-makers, the administrations of departments, faculties, and universities, teaching staff, students already studying or considering to study in the department in the future, and, in short, the entire target audience. With the study, some information was presented to help Health Management departments to find appropriate solutions to develop their efficiency. Accordingly, studies can be conducted to support Health Management department managers and academic staff to determine new strategies, promote and sustain the efficiency of the departments.

We recommend that future research include activities such as community service studies, social responsibility projects, e-learning activities of departments in the list of variables determining the efficiency, measure internal and external stakeholder satisfaction and add this variable to the analysis, and make inter-department comparisons by using different MCDM methods other than the AHP. On the other hand, in order to provide an effective health management education, a balanced distribution of the number of faculty member, other academic staff and students is also very important. In this context, it is recommended that departments consider the number of faculty member and other academic staff when determining student quotas. In this way, they can determine the number of sufficient student quotas by calculating the ratio of registered undergraduate students per faculty member.

Supplemental Material

Supplemental Material - Assessment of variables determining the health management departments’ efficiency with analytical hierarchy process

Supplemental Material for Assessment of variables determining the health management departments’ efficiency with analytical hierarchy process by Gozde Yesilaydin and Emma Menderes Tarcan in Health Services Management Research

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Eskisehir Osmangazi University Scienctific Research Projects Coordination Unit under grant (project) number 201842D26.

Supplemental Material

Supplemental material for this article is available online.

ORCID iD

Gozde Yesilaydin

References

Buchbinder

Thompson

. Career Opportunities in Health Care Management. Sudbury, MA: Jones and Bartlett Publishers, 2010.

Bezuidenhout

. External factors influencing health services management. In: Booyens

(ed) Introduction to Health Services Management. Kenwyn: Juta Company, 2008.

Stefl

. Common competencies for all healthcare managers: the healthcare leadership alliance model. J Healthcare Manage 2008; 53(6): 360–373.

Higher Education Programs and Quotas Guide; 2010, www.oysm.gov.tr (2020, accessed 12 October 2020).

Yang

Fukuyama

Song

, et al. Measuring the inefficiency of Chinese research universities based on a two-stage network DEA model. J Informetrics 2018; 12(1): 10–30.

Tran

CDT

Villano

. Input rigidities and performance of Vietnamese Universities. Asian Econ J 2017; 31(3): 253–273.

Fuentes

Fuster

Lillo Banuls

, et al. A three-stage DEA model to evaluate learning teaching technical efficiency: key performance ındicators and contextual variables. Expert Syst Appl 2016; 48: 89–99.

Szuwarzyński

. Benefit of the doubt approach to assessing the research performance of Australian universities. Higher Educ Q 2019; 73(2): 235–250.

De La Torre

Agasisti

Perez-Esparrells

, et al. The relevance of knowledge transfer for universities’ efficiency scores: an empirical approximation on the Spanish public higher education system. Res Eval 2017; 26(3): 211–229.

10.

Berbegal-Mirabent

. The influence of regulatory frameworks on research and knowledge transfer outputs: an efficiency analysis of Spanish public universities. J Eng Technol Manage 2018; 47: 68–80.

11.

Tyagi

Yadav

Sing

, et al. Relative performance of academic departments using DEA with sensitivity analysis. Eval Program Plann 2009; 32(2): 168–177.

12.

Barra

Zotti

. A directional distance approach applied to higher education: an analysis of teaching-related output efficiency. Ann Public Coop Econ 2016; 87(2): 145–173.

13.

Halkos

Tzeremes

Kourtzidis

. Measuring public owned university departments’ efficiency: a bootstrapped DEA approach. J Econ Econom 2012; 55(2): 1–24.

14.

Tran

CDT

Villano

. Measuring efficiency of Vietnamese public colleges: an application of the DEA-based dynamic network approach. Int Trans Oper Res 2018; 25(2): 683–703.

15.

Johnes

. Efficiency and mergers in English higher education 1996/97 to 2008/9: parametric and non-parametric estimation of the multi‐input multi‐output distance function. Manchester Sch 2014; 82(4): 465–487.

16.

Saaty

. The analytic hierarchy process-what it is and how it is used. Mathl Model 1987; 9(3–5): 161–176.

17.

Clermont

Dirksen

Dyckhoff

, et al. Returns to scale of business administration research in Germany. Scientometrics 2015; 103(2): 583–614.

18.

Witte

López-Torres

. Efficiency in education: a review of literature and a way forward. J Oper Res Soc 2017; 68(4): 339–363.

19.

Turkey YOK National Thesis Center Database, https://tez.yok.gov.tr/UlusalTezMerkezi/ (accessed 20 May 2019).

20.

Özdağoğlu

. Bulanık AHP yaklaşımında duyarlılık analizleri: yeni bir hammadde tedarikçisinin çözüme eklenmesi. İstanbul Ticaret Üniversitesi Fen Bilimleri Dergisi 2008; 7(13): 51–72.

21.

Schmidt

Aumann

Hollander

, et al. Applying the analytic hierarchy process in healthcare research: a systematic literature review and evaluation of reporting. BMC Med Inform Decis Making 2015; 15(1): 112.

22.

Kil

Lee

Kim

, et al. Utilizing the analytic hierarchy process to establish weighted values for evaluating the stability of slope revegetation based on hydroseeding applications in South Korea. Sustainability 2016; 8(1): 1–17.

23.

Quiroga-Martínez

Fernández-Vázquez

Alberto

, et al. Efficiency in public higher education on Argentina 2004–2013: institutional decisions and university-specific effects. Latin Am Econ Rev 2018; 27(1): 1–18.

24.

Sahney

Thakkar

. A comparative assessment of the performance of select higher education ınstitutes in India. Qual Assur Educ 2016; 24(2): 278–302.

25.

Ruiz

Segura

Sirvent

, et al. Benchmarking and target setting with expert preferences: an application to the evaluation of educational performance of Spanish Universities. Eur J Oper Res 2015; 242(2): 594–605.

26.

Thanassoulis

Dey

Petridis

, et al. Evaluating higher education teaching performance using combined analytic hierarchy process and data envelopment analysis. J Oper Res Soc 2017; 68(4): 431–445.

27.

Altamirano-Corro

Peniche-Vera

. Measuring the institutional efficiency using DEA and AHP: the case of a Mexican University. J Appl Res Technol 2014; 12: 63–71.

28.

Shakhi

Hossinpour

Maharloo

, et al. Students' views about the educational quality of health services management department in AJUMS. Educ Develop Judishapur 2013; 4(3): 83–89.

29.

Beigzadeh

Okhovati

Mehrolhasani

, et al. Challenges of the bachelor program of health services management: a qualitative study. J Health Adm 2014; 17(55): 29–42.

30.

İnce

Söyük

. Ranking of academic performance of healthcare management departments by using multiple criteria decision-making methods. Int J Health Manage Strateg Res 2020; 6(3): 443–461.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.51 MB