Sage Journals: Discover world-class research

Abstract

This study developed a fuzzy logic and Gagné learning hierarchy (FL-GLH) for assessing mathematics skills and identifying learning barrier points. Fuzzy logic was used to model the human reasoning process in linguistic terms. Specifically, fuzzy logic was used to build relationships between skill level concepts as inputs and learning achievement as an output. Gagné learning hierarchy was used to develop a learning hierarchy diagram, which included learning paths and test questions for assessing mathematics skills. First, the Gagné learning hierarchy was used to generate learning path diagrams and test questions. In the second step, skill level concepts were grouped, and their membership functions were established to fuzzify the input parameters and to build membership functions of learning achievement as an output. Third, the inference engine generated fuzzy values by applying fuzzy rules based on fuzzy reasoning. Finally, the defuzzifier converted fuzzy values to crisp output values for learning achievement. Practical applications of the FL-GLH confirmed its effectiveness for evaluating student learning achievement, for finding student learning barrier points, and for providing teachers with guidelines for improving learning efficiency in students.

Keywords

Fuzzy logic Gagné learning hierarchy mathematics skills learning barrier points

Introduction

Both educators and students can use learning assessment methods to find learning barrier points and to evaluate learning achievement. Ma and Zhou¹ presented an integrated fuzzy set approach to assess the outcomes of student-centered learning and used fuzzy set principles to represent the imprecise concepts for subjective judgment and applied a fuzzy set method to determine the assessment criteria and their corresponding weights. In Grigoriadou et al.,² integrating fuzzy logic concepts and multi-criteria decision-making in a learning assessment system provided a more complete and accurate description of expert knowledge as well as increased flexibility in assessment of learning achievement in students. Chen³ developed an intelligent personalized learning system that used a genetic algorithm to determine the best learning paths according to the incorrect answers of individual learners. Ketterlin-Geller and Yovanoff⁴ used cognitive diagnostic assessments to provide detailed and precise information about cognitive processes in students who had difficulty learning mathematics. Tay and Lim⁵ investigated a fuzzy inference system as an alternative to the conventional addition or weighted addition in the criterion-referenced assessment. A fuzzy-based model was presented, and the monotonicity and sub-additivity properties were investigated. Ozyurt et al.⁶ proposed a structure and improvement processes for computerized adaptive testing systems. The adaptive assessment system evaluated students according to their actual ability rather than according to their test grades as in a conventional system. Hauswirth and Adamoli⁷ proposed a pedagogical approach in which students are asked a series of questions to facilitate learning from their mistakes. Hwang et al.⁸ proposed a group decision approach for developing a concept-effect relationship model with the cooperation of multiple domain experts. Low-achievement students taught by the group decision approach had significantly better learning achievement compared to those taught by the conventional approach. Jee et al.⁹ proposed a procedure that comprised sufficient conditions, fuzzy rule reduction, and monotonicity-preserving similarity reasoning. The framework reduced the number of fuzzy rules gathered from an assessor with a proposed fuzzy rule selection approach. Chai et al.¹⁰ proposed a fuzzy peer assessment methodology that considered vagueness and imprecision of words used throughout the evaluation process in a cooperative learning environment. Yang et al.¹¹ proposed a two-tier test-based learning method for enhancing learning outcomes in computer-programming courses in a web-based learning environment. Samarakou et al.¹² presented a fuzzy-logic-based model for the diagnosis of the so-called students’ learning profile. The fuzzy logic module was coupled with an interactive open learning environment that incorporated the text comprehension theory by Denhière and Baudet, the dialogue theory of Collins and Beranek, and the learning styles theory of Felder and Silverman. Lai et al.¹³ described applications of the assessment triangle in two studies. Although comparisons of the evaluation results in the studies revealed mixed results, the evidence generally agreed that the progression in student performance should be modeled as a loose network of concepts rather than as a strict hierarchy. Wilkins and Norton¹⁴ used a hierarchy of fraction scheme that charted the progression from understanding part-whole concepts to understanding concepts of fractions. Tang et al.¹⁵ developed an inheritance coding with Gagné-based learning hierarchy approach to building systems for assessing mathematics skills and diagnosing student learning problems. A practical application showed that it can find student learning barrier points and provide learning suggestions for teachers and students. Although researchers have developed several methods of analyzing learning barrier points in students,^15–19 the best diagnostic method for finding learning barrier points in elementary students who struggle with mathematics still needs further study.

The goal of this research was to develop a reasonably objective and accurate method of evaluating mathematics skills and finding learning barrier points in students. The traditional assessment method is to set a standard that the student must achieve to demonstrate successful learning. Test scores above and below the standard are interpreted as learning success and learning failure, respectively. However, the use of a cutoff to judge learning failure and success is questionable. For example, if the passing score for a test is 60, then a score of 60 is interpreted as a learning success whereas a score of 59, a 1-point difference, is interpreted as a learning failure. Thus, the traditional method of judging student achievement is inappropriate.

To address this issue, this study used fuzzy logic concepts to integrate human semantics and thinking forms in evaluations of student learning effectiveness. The fuzzy logic theory expands the absolute membership relationship, which is one or the other, and uses the concept of membership function to determine the degree of belonging. When one set occupies a large proportion, the membership of the set is relatively high. The objective was to reduce the role of subjectivity and provide a more reasonable and objective measure of learning abilities and weaknesses.

To support efficient remedial teaching, this study combined fuzzy logic and Gagné learning hierarchy (FL-GLH) to enable accurate evaluation of student learning ability and learning barrier points. Practical applications of the FL-GLH confirmed its accuracy and effectiveness for finding student learning barrier points, which can aid teachers and students in improving learning efficiency.

Building a learning path diagram and generating testquestions by Gagné learning hierarchy theory

In the hierarchical analysis of intellectual skills advocated by Gagné,²⁰ component skills are task-analyzed in a part-to-whole sequence in order of their importance and in a “bottom-up” fashion. The hierarchical task-analysis is followed by increasingly complex combinations of the parts.²¹ According to the Gagné learning hierarchy theory, intellectual skills are acquired cumulatively, that is, mastery of higher-level skills requires prior mastery of lower-level skills. Since intellectual skills are acquired in a hierarchical order, successful instruction requires the student to acquire lower-level skills before progressing to higher-level skills.^20,22,23

Based on the results of previous research by the authors,¹⁵ this study applied Gagné learning hierarchy theory in the development of learning path diagrams and test questions for assessing mathematics skills and finding learning barrier points. Assessing knowledge acquisition by the students required the teachers to apply their expertise in designing test questions for each level. For example, even though they had different learning paths, integers and fractions were both related to certain learning processes. Therefore, the learning hierarchy diagrams for integers and for fractions were integrated to enhance student learning. Figure 1 shows the learning hierarchy diagrams for integers and fractions. The a-path and b-path are the paths for integers and fractions, respectively, and the numbers in front of the a and b represent the learning level.

Figure 1.

Gagné learning hierarchy diagram for integers and fractions.

Steps of assessing mathematics skills by fuzzy logic

A fuzzy logic²⁴ system comprises a fuzzifier, membership functions, a fuzzy rule base, a fuzzy inference engine, and a defuzzifier. First, the fuzzifier uses membership functions to fuzzify the given input parameters and outputs. The fuzzy inference engine then generates fuzzy values by applying fuzzy rules based on fuzzy reasoning. Finally, the defuzzifier converts the fuzzy values into crisp output values. For example, fuzzy reasoning can be described as two-input, one-output fuzzy logic. The fuzzy rule base consists of a group of if–then control rules with two inputs, x₁ and x₂, and one output, y. When x₁ and x₂ have three membership functions each, a representative rule set of nine fuzzy if-then rules can be stated as

R^{n} : if x_{1} is A_{i} and x_{2} is B_{j} then y is Y_{k},

(1)

where Rⁿ (n = 1, 2, …, 9) denotes the n-th implication. The x₁ and x₂ are the input values, and y is the output value. Subset A_i, subset B_j (i, j∈ 1, 2, 3), and subset Y_k (k∈ 1, 2, 3, 4, 5) are fuzzy subsets defined by the corresponding membership functions µ_Ai, µ_Bj, and µ_Yk.

The max-min compositional operation^25,26 yields a fuzzy output that complies with these rules. This study used the center of gravity defuzzification method to transform the fuzzy inference output into a non-fuzzy value.

The four steps of using fuzzy logic to assess mathematics skills were as follows. The first step was grouping skill level concepts and establishing their membership functions to fuzzify the input parameters. The second step was building membership functions of learning achievement as an output. The third step was building fuzzy rules according to the expertise of the teacher. The final step was using the fuzzy inference engine to generate fuzzy values by applying fuzzy rules based on fuzzy reasoning of the min-max compositional operation. Then, the center of gravity method was used to perform defuzzification, that is, to convert fuzzy values into crisp output values for learning achievement. The detailed steps of the assessment were as follows.

Grouping skill level concepts and establishing their membership functions to fuzzify the input parameters

In Figure 1, the a-path is an example of 11 selected skill level concepts (i.e. 1a–11a) divided into four groups. Thirty questions were designed to test the eleven skill level concepts. Table 1 shows the test questions (Q_k) for the four groups (F_i) of skill level concepts (C_j), where k = 1, …, 30, i = 1, …, 4, and j = 1, …, 11. A test question that included the skill level concept was coded as 1. All other test questions were coded as 0.

Table 1.

Four groups of skill level concepts and associated test questions for the integer unit.

	F₄		F₃			F₂			F₁
	C₁₁	C₁₀	C₉	C₈	C₇	C₆	C₅	C₄	C₃	C₂	C₁
Q₁											1
Q₂											1
Q₃										1
Q₄										1
Q₅									1
Q₆									1
Q₇								1
Q₈								1
Q₉							1
Q₁₀							1
Q₁₁						1
Q₁₂						1
Q₁₃					1
Q₁₄					1
Q₁₅				1
Q₁₆				1
Q₁₇			1
Q₁₈			1
Q₁₉			1
Q₂₀			1
Q₂₁		1
Q₂₂		1
Q₂₃		1
Q₂₄		1
Q₂₅	1
Q₂₆	1
Q₂₇	1
Q₂₈	1
Q₂₉	1
Q₃₀	1

The first part F₁ used six questions to test three concepts (C₁ integer comparisons, C₂ integer addition and subtraction, and C₃ sorting and equal). The second part F₂ used six questions to test three concepts (C₄ integer multiplication, C₅ integer division, and C₆ integer addition, subtraction, multiplication and division). The third part F₃ used eight questions to test three concepts (C₇ integer mixed operations, C₈ division and fraction, and C₉ skilled for integer mixed operations). The fourth part F₄ used 10 questions to test two concepts (C₁₀ multiples and common multiples and C₁₁ factors and common factors).

The four inputs to the fuzzy logic analysis were F₁, F₂, F₃, and F₄. Each input had two fuzzy subsets (“Poor” and “Good”), which were extracted by teachers and measured the degree of membership of a given value to the fuzzy subset. For example, the first part F₁ used six questions to test three concepts. The teacher thought it was “Poor” for learning if students have learned less than two test questions, and it was “Good” for learning if students have learned more than four test questions. The third part F₃ used eight questions to test three concepts. The teacher thought it was “Poor” for learning if students have learned less than three test questions, and it was “Good” for learning if students have learned more than six test questions. Therefore, the trapezoidal type was used as the fuzzy logic membership function for the study. The trapezoidal membership function is a curve that defines how each point in the input space is mapped to a membership value (or degree of membership, µ) between 0 and 1. Figure 2 show trapezoidal membership functions for F₁, F₂, F₃, and F₄, respectively.

Figure 2

Two trapezoidal membership functions for F₁, F₂, F₃, and F₄.

Building membership functions for learning achievement as an output for evaluating student learning achievement

The trapezoidal membership functions were also used in five fuzzy subsets (“Poor,”“Fair,”“Average,”“Good,” and “Excellent”) assigned to learning achievement. Figure 3 shows the trapezoidal membership functions for learning achievement.

Figure 3.

Five trapezoidal membership functions for learning achievement.

Building fuzzy rules according to teacher expertise

The four inputs and one output were used to calculate various membership degrees of the fuzzy sets. Table 2 shows the 16 fuzzy rules that were directly derived according to the expertise of the teacher. For example, fuzzy rule 1 (R₁) was IF F₁ is “Poor” and F₂ is “Poor” and F₃ is “Poor” and F₄ is “Poor” THEN learning achievement (LA) is “Poor”; fuzzy rule R₁₁ was IF F₁ is “Good” and F₂ is “Good” and F₃ is “Poor” and F₄ is “Poor” THEN LA is “Average.”

Table 2.

Fuzzy rule table for assessing learning achievement.

	F₁	F₂	F₃	F₄	LA
R₁	Poor	Poor	Poor	Poor	Poor
R₂	Poor	Good	Poor	Poor	Fair
R₃	Poor	Poor	Good	Poor	Fair
R₄	Poor	Poor	Poor	Good	Fair
R₅	Good	Poor	Poor	Poor	Fair
R₆	Poor	Good	Good	Poor	Average
R₇	Poor	Poor	Good	Good	Average
R₈	Poor	Good	Poor	Good	Average
R₉	Good	Poor	Good	Poor	Average
R₁₀	Good	Poor	Poor	Good	Average
R₁₁	Good	Good	Poor	Poor	Average
R₁₂	Poor	Good	Good	Good	Good
R₁₃	Good	Good	Good	Poor	Good
R₁₄	Good	Good	Poor	Good	Good
R₁₅	Good	Poor	Good	Good	Good
R₁₆	Good	Good	Good	Good	Excellent

LA: learning achievement.

Figure 4 shows the fuzzy logic system of four inputs, one output, and 16 rules for finding barrier points for learning skill level concepts for integers.

Figure 4.

Fuzzy logic system of four inputs, one output, and 16 rules for finding student learning barrier points.

Building a range of crisp output values for evaluating learning achievement

The Mamdani method²⁵ was applied to transform the fuzzy inference output into a non-fuzzy value, that is, the value of learning achievement. A high value for learning achievement indicated good performance characteristic. Mamdani method transforms the fuzzy output into a crisp value by setting membership functions of inputs (Figure 2) and an output (Figure 3) and by using the min-max inference method. The Mamdani method was also used to integrate the experience of the teacher in the inference mechanism in the consequent part as shown in Table 2. The min-max operation for the case of a four-input-single-output system is shown below.

μ_{E} (LA) = \max [\min (μ_{Ai} (F_{1}), μ_{Bj} (F_{2}), μ_{Ck} (F_{3}), μ_{Dl} (F_{4}))]

(2)

where the terms A_i, B_j, C_k, D_l (i, j, k, l = 1, 2), and E are the linguistic terms of the precondition part with membership functions µ_Ai(F₁), µ_Bj(F₂), µ_Ck(F₃), µ_Dl(F₄), and µ_E(LA).

Deffuzification was performed by using center of gravity method to convert the fuzzy values into a crisp output value for learning achievement.

LA_value = \frac{\sum_{i = 1}^{n} NSL C_{i} μ_{E} (L A_{i})}{\sum_{i = 1}^{n} μ_{E} (L A_{i})}

(3)

where the term n is the number of skill level concepts and NSLC is number of skill level concepts.

For evaluating student learning achievement and finding learning barrier points, the respective ranges of crisp output values for learning achievement among skill level concepts were computed by fuzzy logic inference.

For example, Table 1 shows that the four inputs to the fuzzy logic analysis were F₁, F₂, F₃, and F₄. The first part F₁ contained three concepts and six questions. The second part F₂ contained three concepts and six questions. The third part F₃ contained three concepts and eight questions. The fourth part F₄ contained two concepts and 10 questions. When [F₁, F₂, F₃, F₄] was equal to [0, 0, 0, 0], the crisp output value for learning achievement was 0.67. When [F₁, F₂, F₃, F₄] was equal to [6, 0, 0, 0], the crisp output value for learning achievement was 3. When [F₁, F₂, F₃, F₄] was equal to [6, 6, 0, 0], the crisp output value for learning achievement was 5.5. When [F₁, F₂, F₃, F₄] was equal to [6, 6, 8, 0], the crisp output value for learning achievement was 8. When [F₁, F₂, F₃, F₄] was equal to [6, 6, 8, 10], the crisp output value for learning achievement was 10.33. Table 3 shows the respective ranges of crisp output values for evaluating learning achievement.

Table 3.

Ranges of crisp output values for evaluating learning achievement.

Input	Skill level concepts,number of questions	Ranges of crisp outputvalues for learningachievement, LA_value
F₁	C₁–C₃, 6	0.67 ≤ LA_value ≤ 3
F₂	C₄–C₆, 6	3 < LA_value ≤ 5.5
F₃	C₇–C₉, 8	5.5 < LA_value ≤ 8
F₄	C₁₀–C₁₁, 10	8 < LA_value ≤ 10.33

Practical applications of the FL-GLH in finding student learning barrier points

The effectiveness of the FL-GLH for finding student learning barrier points and evaluating learning achievement was evaluated in a practical implementation. Table 1 shows the test questions (Q_k) for the four groups (F_i) of skill level concepts (C_j), where k = 1, …, 30, i = 1, …, 4, and j = 1, …, 11. A test question that included the skill level concept was coded as 1. All other test questions were coded as 0.

The FL-GLH was used to assess mathematics skills learning achievement in 68 students. The results obtained by the FL-GLH for ten representative students in the integer unit subject are shown below. The FL-GLH results were then compared with those obtained by a conventional scoring method. The 30 test questions had a value of 100 points. Learning achievement is conventionally assessed by the number of correct answers.

Of the 30 test questions, Student 1 correctly answered eight questions (Q₁, Q₅, Q₈, Q₁₀, Q₁₅, Q₁₈, Q₂₂, and Q₂₉). The traditional interpretation would be that the student understood 27% of the tested concepts. However, Figure 5 shows that the learning achievement analysis by FL-GLH indicated that [F₁, F₂, F₃, F₄] was equal to [2, 2, 2, 2], and the crisp output value for learning achievement was 0.67. Based on the range of crisp output values in Table 3, student 1 did not understand any of the tested concepts. The teacher concluded that student 1 needed an improved understanding of concepts C₁–C₁₁ and inferred that the student answered test questions by guessing. This example demonstrates that the FL-GLH assessed the ability to answer questions correctly and identified learning barrier points. In contrast, the traditional scoring method overestimates learning achievement based on the number of correct answers and cannot find learning barrier points.

Figure 5.

Evaluating the learning achievement of [2, 2, 2, 2] by FL-GLH.

Of the 30 test questions, Student 2 correctly answered nine questions (Q₁, Q₂, Q₄, Q₇, Q₈, Q₁₃, Q₁₆, Q₁₉, and Q₂₈). The traditional interpretation would be that the student understood 30% of the tested concepts. However, Figure 6 shows that, according to the learning achievement analysis by the FL-GLH, [F₁, F₂, F₃, F₄] was equal to [3, 2, 3, 1], and the crisp output value for learning achievement was 2.16. According to the range of crisp output values in Table 3, student 2 understood the concepts in interval F₁ (concepts C₁–C₃), approximately concepts C₂ and C₃. The interpretation by the teacher was that student 2 needed an improved understanding of concepts C₃–C₁₁. According to the Gagné learning hierarchy theory, the student answered two questions correctly in interval F₂, but understanding of the concepts was still poor. Additionally, the student may have guessed the correct answers for three questions in interval F₃ and for one question in interval F₄. The traditional scoring method overestimates learning achievement based on the number of correct answers.

Figure 6.

Evaluating the learning achievement of [3, 2, 3, 1] by FL-GLH.

Of the 30 test questions, Student 3 correctly answered 10 questions (Q₁, Q₂, Q₄, Q₆, Q₇, Q₈, Q₉, Q₁₅, Q₂₀, and Q₂₉). The traditional interpretation would be that the student understood 33% of the tested concepts. However, Figure 7 shows that, according to the learning achievement analysis by FL-GLH, [F₁, F₂, F₃, F₄] was equal to [4, 3, 2, 1], and the crisp output value for learning achievement was 3.69. Based on the range of crisp output values in Table 3, student 3 understood the concepts up to interval F₂ (concepts C₄–C₆), approximately concepts C₄ and C₅. The interpretation by the teacher was that student 3 needed an improved understanding of concepts C₅–C₁₁. According to Gagné learning hierarchy theory, the student may have correctly guessed the answers for two questions in interval F₃ and for one question in interval F₄. The results obtained by the FL-GLH were similar to the results obtained by using the number of correct answers to judge the ability of the student to answer test questions.

Figure 7.

Evaluating the learning achievement of [4, 3, 2, 1] by FL-GLH.

Of the 30 test questions, student 4 correctly answered 26 questions (Q₁–Q₅, Q₇–Q₁₁, Q₁₃–Q₂₀, and Q₂₁–Q₂₈). The traditional interpretation would be that the student understood 87% of the tested concepts. Figure 8 shows that, according to the learning achievement analysis by FL-GLH [F₁, F₂, F₃, F₄] was equal to [5, 5, 8, 8], and the crisp output value for learning achievement was 9.58. Based on the range of crisp output values in Table 3, student 4 understood the concepts up to interval F₄ (concepts C₁₀–C₁₁). The interpretation by the teacher was that student 4 needed an improved understanding of concept C₁₁. The teacher inferred that student 4 made careless errors on the test questions in intervals F₁ and F₂. The FL-GLH results were consistent with the results of traditional scoring methods. Notably, in terms of assessing the ability to answer the test questions, the FL-GLH was comparable to an assessment based on the number of correctly answered questions.

Figure 8.

Evaluating the learning achievement of [5, 5, 8, 8] by FL-GLH.

Of the 30 test questions, student 5 correctly answered 24 questions (Q₁–Q₅, Q₇–Q₁₂, Q₁₃–Q₁₉, Q₂₁–Q₂₃, and Q₂₈–Q₃₀). The traditional interpretation would be that the student understood 80% of the tested concepts. Figure 9 shows that, according to the learning achievement analysis by FL-GLH, [F₁, F₂, F₃, F₄] was equal to [5, 6, 7, 6], and the crisp output value for learning achievement was 8.78. Based on the range of crisp output values in Table 3, student 5 understood the concepts up to interval F₄ (concepts C₁₀–C₁₁), particularly concept C₁₀. The interpretation by the teacher was that student 5 needed an improved understanding of concepts C₁₀–C₁₁. The teacher inferred that student 5 made careless errors in answering test questions in intervals F₁ and F₃.

Figure 9.

Evaluating the learning achievement of [5, 6, 7, 6] by FL-GLH.

Of the 30 test questions, student 6 correctly answered 15 questions (Q₁–Q₆, Q₇–Q₁₁, Q₁₅, Q₁₈, Q₂₀, and Q₂₇). The traditional interpretation would be that the student understood 50% of the tested concepts. Figure 10 shows that, according to the learning achievement analysis by FL-GLH, [F₁, F₂, F₃, F₄] was equal to [6, 5, 3, 1], and the crisp output value of learning achievement was 5.5. Based on the range of crisp output values in Table 3, student 6 understood the concepts up to interval F₂ (concepts C₄–C₆) and around concept C₆. The interpretation by the teacher was that student 6 needed an improved understanding of concepts C₇–C₁₁. According to the Gagné learning hierarchy theory, the student may have guessed the correct answers for three questions in interval F₃ and one question in interval F₄. The traditional scoring method overestimates learning achievement based on the number of correct answers.

Figure 10.

Evaluating the learning achievement of [6, 5, 3, 1] by FL-GLH.

Of the 30 test questions, student 7 correctly answered 20 questions (Q₁–Q₆, Q₇–Q₁₂, Q₁₃–Q₁₇, Q₂₁, Q₂₅, and Q₂₈). The traditional interpretation would be that the student understood 67% of the tested concepts. Figure 11 shows that, according to the learning achievement analysis by FL-GLH, [F₁, F₂, F₃, F₄] was equal to [6, 6, 5, 3], and the crisp output value for learning achievement was 6.89. Based on the range of crisp output values in Table 3, student 7 understood the concepts up to interval F₃ (concepts C₇–C₉) and around concept C₈. The interpretation by the teacher was that student 7 needed an improved understanding of concepts C₈–C₁₁. According to the Gagné learning hierarchy theory, the student may have guessed the correct the answers to three questions in interval F₄. The FL-GLH results were consistent with the results obtained by using the number of correct answers to judge the ability of the student to answer test questions.

Figure 11.

Evaluating the learning achievement of [6, 6, 5, 3] by FL-GLH.

Of the 30 test questions, student 8 correctly answered 23 questions (Q₁–Q₆, Q₇–Q₁₂, Q₁₃–Q₁₈, Q₂₁, Q₂₃, Q₂₆, Q₂₈, and Q₃₀). The traditional interpretation would be that the student understood 77% of the tested concepts. Figure 12 shows that, according to the learning achievement analysis by FL-GLH, [F₁, F₂, F₃, F₄] was equal to [6, 6, 6, 5], and the crisp output value of learning achievement was 7.8. Based on the range of crisp output values in Table 3, student 8 understood the concepts up to interval F₃ (concepts C₇–C₉) and around concept C₉. The interpretation by the teacher was that student 8 needed an improved understanding of concepts C₉–C₁₁. According to the Gagné learning hierarchy theory, the student answered five questions correctly in interval F₄, but understanding of the concepts was still poor. The FL-GLH results were consistent with the results obtained by using the number of correct answers to judge the ability of the student to answer test questions.

Figure 12.

Evaluating the learning achievement of [6, 6, 6, 5] by FL-GLH.

Of the 30 test questions, student 9 correctly answered 25 questions (Q₁–Q₆, Q₇–Q₁₂, Q₁₃–Q₁₉, and Q₂₁–Q₂₆). The traditional interpretation would be that the student understood 83% of the tested concepts. Figure 13 shows that, according to the learning achievement analysis by FL-GLH [F₁, F₂, F₃, F₄] was equal to [6, 6, 7, 6], and the crisp output value for learning achievement was 8.78. Based on the range of crisp output values in Table 3, student 9 understood the concepts up to interval F₄ (concepts C₁₀–C₁₁) and around concept C₁₁. The interpretation by the teacher was that student 9 needed an improved understanding of concept C₁₁. The teacher inferred that student 9 made careless mistakes in the answers to test questions in interval F₃.

Figure 13.

Evaluating the learning achievement of [6, 6, 7, 6] by FL-GLH.

Of the 30 test questions, student 10 correctly answered 29 questions (Q₁–Q₆, Q₇–Q₁₂, Q₁₃–Q₂₀, Q₂₁–Q₂₈, and Q₃₀). The traditional interpretation would be that the student understood 97% of the tested concepts. Figure 14 shows that, according to the learning achievement analysis by FL-GLH [F₁, F₂, F₃, F₄] was equal to [6, 6, 8, 9], and the crisp output value for learning achievement was 10.3. Based on the range of crisp output values in Table 3, student 10 understood the concepts up to interval F₄ (concepts C₁₀–C₁₁) and around C₁₁. The teacher inferred that student 10 made careless errors in the answers to test questions in interval F₄. The FL-GLH results were very similar to those obtained by traditional scoring methods.

Figure 14.

Evaluating the learning achievement of [6, 6, 8, 9] by FL-GLH.

Discussion

The above results obtained in practical applications indicate that the FL-GLH can accurately find student learning barrier points and can provide teachers and students with suggestions for improving learning efficiency. The experimental sites were elementary schools in Taiwan.

The FL-GLH for assessing student mathematics skills has the following four notable requirements.

The first requirement is that the test questions for each level of a mathematics unit must be carefully designed according to Gagné learning hierarchy theory and according to the expertise of the elementary school teacher in assessing student learning achievement. In a previous work by the authors,¹⁵ test questions used for mathematics assessment revealed an expert validity of 0.8, a Cronbach alpha value of 0.8 for internal reliability, test–retest reliability of 0.91, and parallel-form reliability of 0.9.

The second requirement is that the test questions for each level of a mathematics unit must be selected and grouped by the learning hierarchy path.

The third requirement is that the membership functions for fuzzifying the inputs and the output must be selected and designed. Fuzzy rules for assessing learning achievement must also be built according to the expertise of the teacher.

The fourth requirement of the fuzzy logic system for assessing mathematics skills is that the min-max inference method must be used to transform the fuzzy inference output, and the center of gravity method must be used to convert the fuzzy values into crisp output values for learning achievement.

Conclusions

By integrating fuzzy logic and Gagné learning hierarchy theory, the FL-GLH provided a human-like reasoning method for assessing the mathematics skills of students and finding their learning barrier points. Fuzzy logic was used to model the human reasoning process and to build relationships between skill level concepts and learning achievement. Gagné learning hierarchy was used to develop the learning paths and test questions. The main contribution of this study is the development of an FL-GLH that can accurately identify learning barrier points and can provide teachers and students with suggestions for improving learning ability in a real-world classroom. Practical applications in elementary schools showed that, compared to the traditional method of using test scores to assess mathematics skills, the FL-GLH is superior in terms of accuracy in assessing student learning achievement. Since the learning barrier points of students were validated by their mathematics teachers, we conclude that the proposed FL-GLH has excellent performance in mathematics skills assessment and in finding learning barrier points. In the future work, the authors will combine education experts to design the learning hierarchy diagrams for different learning units with their fuzzy sets and to gather consistence fuzzy rules for improving students learning ability in a real-world classroom.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the Ministry of Science and Technology, Taiwan, R.O.C., under Grant Numbers MOST 104-2410-H-153-011, MOST 106-2410-H-153-013, MOST 107-2410-H-153-008-MY2, and MOST 109-2221-E-153-005-MY3.

ORCID iD

Jinn-Tsong Tsai

Author biographies

Wei-Ling Tang is currently a professor of the Department of Education at the National Pingtung University, Taiwan, R.O.C. Since receiving Ph. D. from National Taiwan Normal University in 1996, she has taught graduate students at the National Taitung Teachers College, following her career of a junior high school teacher, and an assistant researcher in Ministry of Education. For extending research experiences, she has been a visiting scholar in the Institute of Education, University of Sheffield, UK and then in the Department of Curriculum and Instruction, University of Maryland at College Park, USA. She specializes in Teacher education, Curriculum and Instruction. She has published many research papers in educational academic journals and books. She is also an editorial member of TSSCI journals in Taiwan. She has won many research awards from the National Science Council/ Ministry of Science and Technology of Taiwan. Her research interests include e-learning, problem/project-based learning, remedial instruction, computational thinking, and STEAM education for prospective elementary teachers.

Jinn-Tsong Tsai received the B.S. and M.S. degrees in Mechanical and Electro-Mechanical Engineering from National Sun Yat-Sen University, Taiwan, in 1986 and 1988, respectively, and the Ph.D. degree in Engineering Science and Technology from National Kaohsiung University of Science and Technology, Taiwan, in 2004. He is currently a Distinguished Professor with the Department of Computer Science, National Pingtung University, Pingtung, Taiwan. From July 1988 to May 1990, he was a Lecturer with the Vehicle Engineering Department, Chung Cheng Institute of Technology, Taiwan. From July 1990 to August 2004, he was a Researcher and the Chief of the Automation Control Section, Metal Industries Research and Development Center, Taiwan. From September 2004 to July 2006, he was an Assistant Professor with the Medical Information Management Department, Kaohsiung Medical University, Kaohsiung, Taiwan. From August 2006 to July 2010, he was an Assistant Professor; from August 2010 to January 2014, he was an Associate Professor; and from February 2014 to July 2018, he was a Professor with the Department of Computer Science, National Pingtung University, Pingtung, Taiwan. His research interests include computational intelligence, machine learning, robust optimization, and intelligent control and systems.

Yao-Mei Chen received the B.S. degree in Nursing from Kaohsiung Medical University in 1988 and the M.S. degree in Gerontological Nursing in 1995 and the Ph.D. degree in Nursing Administration in 2008 from University of Maryland Baltimore in USA. She is currently an Assistant Professor of the School of Nursing, Kaohsiung Medical University (KMU), Kaohsiung, Taiwan and a senior specialist in the Superintendent Office in the affiliated medical center, Kaohsiung Medical University Hospital (KMUH). From August 2001 to July 2018, she had adjunct positions of nursing director and deputy in both KMUH and its branch hospital (Kaohsiung Municipal Siaogang Hospital, KMSH) in Taiwan. From August 1995 to July 2008, she was an instructor of School of Nursing in KMU, Taiwan. From August 1996 to July 2001, she was appointed as a head nurse and supervisor of Nursing Department in KMUH; she worked as a case manager of stroke team in 1996 and as a registered nurse from August 1988 to July 1993. She served for many professional associations, currently as a vice president of Taiwan Nurses Association (TWNA) and the president of Taiwan Medical Case Management Association (TMCMA). She is a certified lecturer of Leadership for Change (LFC) Program Taiwan, endorsed by International Council of Nurses (ICN). Her research interests include nursing work environment, work stress and health, quality improvement and patient safety, human resource management, manpower and staffing, professional development, education and training, case management, and technology application in healthcare.

References

Zhou

. Fuzzy set approach to the assessment of student-centered learning. IEEE Trans Educ 2000; 43: 237–241.

Grigoriadou

Kornilakis

Papanikolaou

,et al. Fuzzy inference for student diagnosis in adaptive educational hypermedia. In: Proceedings of Methods and applications of artificial intelligence, Second Hellenic Conference on AI, SETN, Thessaloniki, Greece, 2002, pp. 191–202. Berlin, Heidelberg: Springer.

Chen

. Intelligent web-based learning system with personalized learning path guidance. Comput Educ 2008; 51: 787–814.

Ketterlin-Geller

Yovanoff

. Diagnostic assessments in mathematics to support instructional decision making. Pract Assess Res Eval 2009; 14: 1–11.

Tay

Lim

. A fuzzy inference system-based criterion-referenced assessment model. Exp Syst Appl 2011; 38: 11129–11136.

Ozyurt

Baki

. Architecture and design process of the individualized assessment system integrable to distance education. Turk Online J Dist Educ 2012; 13: 212–225.

Hauswirth

Adamoli

. Teaching Java programming with the Informa clicker system. Sci Comput Program 2013; 78: 499–520.

Hwang

Panjaburee

Triampo

,et al. A group decision approach to developing concept effect models for diagnosing student learning problems in mathematics. Br J Educ Technol 2013; 44: 453–468.

Jee

Tay

. A new fuzzy criterion-referenced assessment with a fuzzy rule selection technique and a monotonicity-preserving similarity reasoning scheme. J Intellig Fuzzy Syst 2013; 24: 261–279.

10.

Chai

Tay

Lim

. A new fuzzy peer assessment methodology for cooperative learning of students. Appl Soft Comput 2015; 32: 468–480.

11.

Yang

Hwang

Yang

SJH

,et al. A two-tier test-based approach to improving students’ computer-programming skills in a web-based learning environment. Educ Technol Soc 2015; 18: 198–210.

12.

Samarakou

Prentakis

Mitsoudis

,et al. Application of fuzzy logic for the assessment of engineering students. In: Proceedings of 2017 IEEE global engineering education conference, Athens, 2017, pp.646–650. Piscataway, NJ: IEEE.

13.

Lai

Kobrin

DiCerbo

,et al. Tracing the assessment triangle with learning progression-aligned assessments in mathematics. Meas Interdisc Res Perspect 2017; 15: 143–162.

14.

Wilkins

Norton

. Learning progression toward a measurement concept of fractions. Int J STEM Educ 2018; 5: 1–11.

15.

Tang

Tsai

Huang

. Inheritance coding with Gagné-based learning hierarchy approach to developing mathematics skills assessment systems. Appl Sci 2020; 10.

16.

Díaz

Olaya

. An engineering view for social systems: agency as an operational principle for designing higher education access policies. Syst Pract Action Res 2017; 30: 627–649.

17.

Algraini

McIntyre-Mills

. Human development in Saudi education: a critical systemic approach. Syst Pract Action Res 2018; 31: 121–157.

18.

Henshaw

. Systems thinking for systems making: joining systems of thought and action. Syst Pract Action Res 2018; 32(2): 1–34.

19.

Malik

. Improvements in introductory programming course: action research insights and outcomes. Syst Pract Action Res 2018; 31: 637–656.

20.

Gagné

. The conditions of learning and theory of instruction. New York, NY: Holt, Rinehart & Winston.

21.

Reigeluth

Curtis

. Learning situations and instructional models. In: Gagné

(ed.), Instructional technology: foundations. New York, NY: Routledge, 2010.

22.

Gagné

Wager

Golas

,et al. Principles of instructional design. 5th ed. Belmont, CA: Thomson/Wadsworth, 2005.

23.

Takker

Subramaniam

. Knowledge demands in teaching decimal numbers. J Math Teacher Educ 2019; 22: 257–280.

24.

Ross

. Fuzzy logic with engineering applications. New York, NY: McGraw-Hill Inc, 1995.

25.

Mamdani

Assilian

. An experiment in linguistic synthesis with a fuzzy logic controller. Int J Man Mach Stud 1975; 7: 1–13.

26.

Zimmermann

. Fuzzy set theory and its applications. London: Kluwer, 1985.