Performance evaluation of schools’ math education from a cultural,social and economic point of view by data envelopment analysis modeling

Abstract

This paper aimed at investigating the impacts of some cultural, social and economic factors of the students’ families on the students’ education and attitude toward mathematics. This study has been carried out through the use of data envelopment analysis. Data envelopment analysis is a linear programming method that its main purpose is to compare and evaluate the efficiency of a number of similar decision-making units, and to compare and analyze them, which have different values of consumed inputs and produced outputs. In this study, the inputs are the various aspects of cultural, social and economic conditions (such as family income, physical and mental health condition of the family members, and parents’ occupation) of the families of 300 sixth-grade elementary school students in district 4 of Tehran, and the outputs are the extent of achievement in the mathematics test and the interest of the students of the mentioned schools to the activities related to the mathematics course and moral discipline. After the completion of the investigations, the effective and ineffective units were identified and the results were completely interpreted.

Keywords

Mathematics education cultural integration data envelopment analysis

Introduction

Education should enable us to respond to the opportunities and challenges we face as quickly as the change in the world in which we live and work. These changes involve some parts of school curriculum. Mathematics education has always been investigated as one of the main educational topics. The most important role of mathematics is the development of thinking. The general goal of mathematics teaching is to develop good mental habits in every student so that they can handle every type of problem as much as possible. Hence, mathematics education is relevant to the completion and implementation of an appropriate curriculum, as well as issues and topics related to teaching and learning mathematics.

Studies show that both internal factors (such as intelligence, memory and motivation) and external factors (such as classroom conditions, students’ social status and the extent of student’s interaction with his teacher) affect teaching.

Bucher and Manning¹ attributed the school’s atmosphere, which is the heart and soul of the school, to the parameters such as intimacy, tolerance and flexibility, and consider it as one of the external, important and influential factors in education. Springer et al.² and Brook et al.³ considered classroom environment as one of the students’ important learning factors. However, unlike them, Marcoulides et al.⁴ found a weak relationship between school culture and mathematical attitudes in his study in the context of Cyprus. Clearly, as school culture can influence student’s learning, family culture as the first platform for student’s growth plays a significant role in student’s learning. The extent to which family culture can have impact on student’s educability and enthusiasm regarding different sciences has been one of the topics studied by researchers in the field of education. Koutsoulis and Campbell⁵ considered parents as one of the key factors in developing and evolving positive attitudes in children toward their education. Tymms⁶ found a weak relationship between the two variables of family socioeconomic status and attitudes toward mathematics.

In a study by Marco Leeds (2005), it was found that the relationship between beliefs, values and the process of mathematics education is low. Pahlavan and Kajbaf⁷ investigated the effect of socioeconomic status of 500 public high school students’ families in Isfahan city on mathematics learning. They used the correlation method and implementation in TIMSS (Trends in International Mathematics and Science Study) scale in their study. Their findings indicated a significant and direct effect of the process of mathematics education on the attitude toward mathematics, the significant and reverse effect of the school culture’s structures and the family’s socioeconomic status on the process of mathematics education, as well as the significant and direct effect of the constructs of beliefs and values on the process of mathematics education and the attitude toward mathematics in the model.⁷

This study examines the effect of some cultural, social and economic factors of the students’ families such as the family income, physical and mental health condition of the family members, and parents’ occupation on the result of mathematics test, discipline and interest of students in performing mathematical activities in the classroom. In this study, the data were collected on the 300 sixth-grade male students from three primary schools in district 4 of Tehran, and the effective and ineffective units were identified and the results were analyzed through the use of data envelopment analysis (DEA) method. The following section represents an introduction to DEA.

Data envelopment analysis

Data envelopment analysis indicates a concept of the calculation of the efficiency levels’ evaluation within a group belonged to the organization. The efficiency of each unit is computed based on the comparison to a number of units with the highest performance.⁸ This technique is based on a linear programming approach which main purpose is to compare and measure the efficiency of a number of similar decision-making units, which have different number of consumed inputs and produced outputs. These units can be branches of a bank, schools, hospitals, and so on. Comparing and measuring efficiency means how good the decision-making unit has used its sources in the way of production, compared to the other decision-making units.

The development of the non-parametric estimation method of the DEA took place in the Farrell’s⁹ study in 1957. In this estimation method, in order to measure the efficiency of a firm, the performance of that firm was compared to the performance of the best existing firms. This method was developed in 1978 by the study of Charnes et al.¹⁰ and addition of the possibility of incorporating multiple inputs and multiple outputs. Moreover, in 1993, Andersen and Petersen¹¹ introduced a new method for ranking the efficient units. Since the initial presentation of this method up to the present time, the method of DEA has been expanded and many researchers have been turned to this subject. In 2003, Banker et al. conducted a study, based on which the analysis of the procedure of the implemented changes in technical and professional efficiency of the Texas public schools took place. The results of this study indicated a direct relationship between the inefficiency and the cost of variables.¹²

The first DEA model was called CCR. The basis for the formation of this model is the definition of efficiency as a ratio of an output to an input. In other words, in the CCR model, for the calculation of technical efficiency, instead of using the ratio of an output to an input, the ratio of the weighted sum of outputs to the weighted sum of inputs is used. In the CCR model, it is assumed that the returns to scale is constant for all firms. Constant returns to scale means that all industries have been produced on an optimal scale and the size of the organization has no effect on its efficiency.

Multiplier form of CCR with the input nature

\begin{matrix} Max \sum_{r = 1}^{s} u_{r} y_{r^{0}} \\ s . t . \\ \sum_{r = 1}^{s} u_{r} y_{r^{0}} = 1, \sum_{r = 1}^{s} u_{r} y_{rj} - \sum_{i = 1}^{m} v_{i} x_{ij} \leq 0 \\ \forall j = 1, \dots, n, u_{r}, v_{i} \geq 0, \forall r = 1, \dots, s, \forall i = 1, \dots, m \end{matrix}

In the CCR model, i = {1, …, m} and r = {1, …, s} respectively represent the inputs and outputs of each industry and j = {1, …, n} represents decision-making units. The parameters y_ij and x_ij represent the inputs and outputs, respectively. The u_r and v_i also respectively represent the decision-making variables for the weights of inputs and outputs.

The BCC model with input nature is explained below based on the definitions of decision-making variables, parameters and factors provided by Banker, Cooper and Charnes for the CCR model. The BCC output-oriented model is called a multiplier model. The value of w in the mode of achieving the optimal solution shows the type of the model’s return to scale. That is, if w is positive, returns to scale value is an increasing amount; if it is negative, returns to scale value is a decreasing value; and if it is zero, returns to scale value is a constant value.

Multiplier form of BCC with the input nature

\begin{matrix} Max \sum_{r = 1}^{s} u_{r} y_{r^{0}} + w \\ s . t . \\ \sum_{i = 1}^{m} v_{i} x_{r^{0}} = 1 \\ \sum_{r = 1}^{s} u_{r} y_{rj} + w - \sum_{i = 1}^{m} v_{i} x_{ij} \leq 0 \\ \forall j = 1, \dots, n, u_{r}, v_{i}, w \geq 0, \forall r = 1, \dots, s, \forall i = 1, \dots, m \end{matrix}

Statistical population/sample size

The population of this study included a total of 2800 sixth-grade elementary school male students in district 4 of Tehran during the academic year 2018–2019. Since access to the information of all students in different schools was very time consuming, using cluster sampling method, from the total of 25 primary schools, 3 primary schools were selected, namely Talayedaran primary school (school 1) with 106 sixth-grade students, Komeil primary school (school 2) with 160 sixth-grade students, and Ammar primary school (school 3) with 140 sixth-grade students. From these students, taking the sample size formula into account and considering the low relative error, 320 students were randomly selected and considered as the sample of the study. However, after data cleaning and removing the outliers, the data from 300 returned questionnaires were analyzed.

Procedure

For the purpose of sampling, the district 4 of Tehran Education department was divided into three parts of north, central and southern, and a school was randomly selected from each region. After determining the population and the sample and the stages of implementation, the questionnaires were completed by school counselors and then the students’ teachers. Each questionnaire contained six items as inputs and three items as outputs for a student.

The scores considered for the inputs and outputs are based on the results of the studies by Andersen and Petersen,¹¹ Banker et al.,¹² Martin et al.,⁸ and Koutsoulis and Campbell.⁵

The first input is the student’s father’s educational level. The results were divided into five categories with different scores. In this categorization, score 4 was allocated to the illiterate ones, score 8 was allocated to the ones with literacy at reading and writing, score 12 was allocated to the ones with diplomas, score 16 was allocated to the ones with associate degree and bachelor degree and score 20 was allocated to the ones with master’s degree and above. The second input is the amount of student’s mother’s educational level which was scored the same as above.

In Tables 1 –5, scores for different inputs of the model are represented. Inputs in Table 1 represent the financial conditions of the student’s family. Tables 2 and 3 indicate the economic and cultural conditions of the student’s family, and finally, Tables 4 and 5 reflect the physical, emotional and moral health of the family.

Table 1.

Assigned scores to the input 3.

Input 3	Which of the following options represents the monthly income of your family?
Option	Allocated score	Option	Allocated score
Over 5-million toman	20	Between 1- and 2-million toman	8
Between 3.5- and 5-million toman	16	Under 1-million toman	4
Between 2- and 3.5-million toman	12

Table 2.

Assigned scores to the input 4.

Input 4	Which of the following options defines your father’s job?
Option	Allocated score	Option	Allocated score
Employee of the education or the related organizations	20	Worker	8
Employee of the organizations other than education	16	Unemployment	4
Self-employment	12

Table 3.

Assigned scores to the input 5.

Input 5	Which of the following options defines your mother’s job?
Option	Allocated score	Option	Allocated score
Employee of the education or the related organizations	20	Worker	8
Employee of the organizations other than education	16	Unemployment	4
Self-employment	12

Table 4.

Assigned scores to the input 6.

Input 6	Who is your guardian?
Option	Allocated score	Option	Allocated score
Parents	20	Grandfather or other relatives	8
Only mother	16	Guardian-less	4
Only father	12

Table 5.

Assigned scores to the input 7.

Input 7	Which of the options below defines the physical and mental status of your family?
Option	Allocated score	Option	Allocated score
All family members are in complete physical and mental health	20	Addiction of a family member other than the parents	8
Organ failure or life-threatening illness of a family member	16	Addiction of the parents	4
Death of a family member	12

The result of the student’s mathematics test in the last year’s final assessment, the extent of the student’s interest in solving math problems and participation in classroom activities, and ultimately the student’s ethical discipline are the outcomes of the evaluation model; the scores of which are shown respectively in Tables 6 –8. Moreover, in Tables 9 and 10, the mean and the variance of the scores for each input and output for the students of the schools 1-3 and the whole participants are specified, and their bar graphs and radar charts are plotted for a better comparison.

Table 6.

Assigned scores to the output 1.

Output 1	Which of the options below represent the result of your mathematics test in the last year?
Option	Allocated score	Option	Allocated score
Very good	20	acceptable	10
Good	15	Need more efforts	5

Table 7.

Assigned scores to the output 2.

Output 2	In your opinion, how much is the student’s level of interest in solving math problems and engaging in classroom activities?
Option	Allocated score	Option	Allocated score
Very high	20	Too low	8
High	16	Low	4
Average	12

Table 8.

Assigned scores to the output 3.

Output 3	How do you evaluate your student’s discipline?
Option	Allocated score	Option	Allocated score
Excellent	20	As expected	8
Very good	16	Less than expected	4
Good	12

Table 9.

Mean score of students in different schools.

	Input 1	Input 2	Input 3	Input 4	Input 5	Input 6	Input 7	Output 1	Output 2	Output 3
School 1	17.56	16.52	16.16	14.04	14.64	18.88	18.04	18.70	18.40	17.72
School 2	14.84	14.24	12.76	11.88	14.56	18.72	17.84	18.30	14.04	13.08
School 3	13.48	11.16	7.44	10.55	13.76	18.28	17.92	18.50	10.44	11.32
Total	15.29	13.97	12.12	12.16	14.32	18.63	17.93	18.50	14.30	14.04

Table 10.

Standard deviation of the students’ scores in different schools.

	Input 1	Input 2	Input 3	Input 4	Input 5	Input 6	Input 7	Output 1	Output 2	Output 3
School 1	3.16	2.53	3.28	3.16	14.64	2.45	3.80	3.04	2.70	2.70
School 2	3.46	1.97	2.52	3.30	14.56	2.58	4.20	3.04	2.79	3.94
School 3	3.71	2.41	2.15	3.17	13.76	3.69	4.53	3.48	2.84	3.93
Total	3.90	3.30	4.60	3.56	14.32	2.99	4.14	3.21	4.37	4.56

As it can be seen, the mean of the inputs 1 to 5 in school 1 is higher than school 2, and in school 2 is higher than school 3. The relatively low standard deviation of these inputs for all three schools reflects their low dispersion. The mean score of input 3 for school 3 is very low, and due to the low standard deviation of this input, it is concluded that the students in this school have very poor economic conditions. Inputs 6 and 7, which are about the quality of guardianship and the physical health of the students’ families, indicate appropriate mean score in the three schools.

Output 1, which is related to the results of the students’ mathematics test, has a very high mean score and low standard deviation in the three schools. However, outputs 2 and 3 which reflect the students’ classroom activities and discipline in school 1 are higher than school 2, and in school 2 are higher than in school 3. This shows that the economic situation of the students’ families is directly related to the students’ discipline and motivation to participate in classroom activities in mathematics class.

Now we examine the efficiency of the schools through the use of DEA method. Efficiency stands for not losing the resources, and it is derived from the ratio of the total output to the total input. An efficient unit is a unit that with regard to the inputs’ conditions had a desired output. On the other hand, an inefficient unit is a unit which compared to the other units and with regard to its potential input could have a better output but its output had not been a desired one. In DEA models, the solution to improve inefficient units is to reach the efficiency boundary. The efficiency boundary consists of the units with efficiency amount of 1; therefore, each unit with an efficiency amount greater than or equal to 1 is called efficient and otherwise is inefficient. Table 11 lists the number of efficient and inefficient units in three schools and in total.

Table 11.

Number of the efficient and inefficient units.

	School 1	School 2	School 3	Total
Number of the efficient units	95	87	57	145
Number of the inefficient units	5	13	43	155

As can be seen, 95% of the students of school 1, 87% of the students of school 2% and 57% of the students of school 3 are efficient.

Those decision-making units which are evaluated efficient in DEA models all have the efficiency score of 1, so no distinction can be made between them. This is while most of decision makers are seeking a complete ranking of units, so that they can better evaluate their efficiency and take steps toward further improvement. Table 12 indicates the number of the ranks higher than, equal to and lower than 1 of each of the schools separately and in general through the efficiency values derived from the BCC output-oriented approach. The higher the ranking, the better the school students’ performance.

Table 12.

Ranking.

	School 1	School 2	School 3	Total
Number of students with ranks lower than 1	5	13	43	150
Number of students with ranks equal to 1	74	70	40	108
Number of students with ranks higher than 1	21	17	17	42

As it can be seen, the number of students with the efficiency greater than one is approximately equal in all three schools. School 1 has the highest number of students with rank 1, and school 3 has the highest number of students with the rank lower than 1.

Conclusion

In this paper, a comparative comparison was made among the three schools for boys in district 4 of Tehran in terms of efficiency. The findings of this study indicated the direct impact of various aspects of the economic, cultural, social and physical and mental health condition of the students’ families on the efficiency of their schools.

As can be seen, 95% of the students of school 1, 87% of the students of school 2% and 57% of the students of school 3 are efficient, showing that students of school 1 had a very good performance. School 1 had proper inputs and could have outputs with high mean scores. This school had been able to create a good motivation for the students whose families were in an appropriate condition in terms of the various economic, cultural, social and physical and mental health aspects, so that the students would be disciplined, would participate in mathematics activities and achieve a good result in the mathematics test. The mean score of the inputs and outputs of the students in school 2 was lower than school 1, and the number of students in this school was lower than school 1. School 3 did not have appropriate inputs and outputs compared to the other two schools, which shows that the number of inefficient students in this school is very high.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Mohsen Rostamy-Malkhalifeh

References

Bucher

Manning

. Creating safe school. Clear 2005; 79(1): 55–60.

Springer

Morganfield

Diffily

. Actual versus preferred classroom experience among secondary teachers and their students. Am Second Educ 2007; 35(2): 17–35.

Brook

Fisher

Richards

, et al. Californian science students perceptions of their classroom learning environments. Educ Res Eval 2006; 12(1): 3–25.

Marcoulides

Heck

Papanastasiou

. Student perceptions of school culture and achievement: testing the invariance of a model. Int J Educ Manag 2005; 12(2): 140–152.

Koutsoulis

Campbell

. Family processes affect students’ motivation and science and math achievement in Cypriot high schools. Struct Equ Modeling 2001; 8(1): 108–127.

Tymms

. A test of the big fish in the little pond hypothesis: an investigation into the feelings of seven-year-old pupils in school. Sch Eff Sch Improv 2001; 12(2): 161–181.

Pahlavan

Kajbaf

. Mediating role of classroom processes and school culture on the relationship between socioeconomic status, and value and beliefs about math learning with math attitude. Q J Educ Innov 2012; 10(40): 149–168.

Martin

Kocher

Sutter

. Measuring efficiency of German football teams by DEA. Austria: University of Innsbruck, Australia. 2000, pp. 4–5.

Farrell

. The measurement of productive efficiency. J R Stat Soc Ser A G 1957; 120: 253–290.

10.

Charnes

Cooper

Rhodes

. Measuring the efficiency of decision making units. Eur J Oper Res 1987; 2(6): 429–444.

11.

Andersen

Petersen

. A procedure for ranking efficient units in data envelopment analysis. Manag Sci 1993; 39(10): 1261–1264.

12.

Banker

Janakiramang

Natarajan

. Analysis of trends in technical and allocative efficiency: an application to Texas public school districts. Eur J Oper Res 2003; 154: 477–491.