Cognitive diagnosis of high school students’ set knowledge based on the DINA model

Abstract

In recent years, scholars have increasingly focused on the cognitive diagnostic model, which is widely used in educational and psychological assessments. This study carries out a cognitive diagnosis of high school students’ understanding of set knowledge via the deterministic inputs, noisy “and” gate model (the DINA model). The results indicate that the cognitive attributes and attribute hierarchies established by this study are reasonable. It is observed that 44.21% of students exhibit proficiency in all attributes, which is inseparable from the fact that set knowledge is fundamental knowledge, and 89.25% of the students’ mastery patterns belong to the ideal mastery patterns, which indicates that the distribution of high school students’ cognitive model on set knowledge is relatively concentrated. The research results also further illustrate that cognitive diagnosis can compensate for the shortcomings of traditional tests and provide a more comprehensive assessment approach. Through the results of cognitive diagnosis, teachers and students can better understand the situation of set learning and conduct more effective teaching and learning.

Keywords

cognitive diagnosis mastery pattern set the DINA model middle school education

1. Introduction

Mathematics learning is the process of acquiring mathematical knowledge, skills, and abilities; developing individual qualities; and cultivating rational thinking (Munzar et al., 2021). Concept learning is the core of mathematics learning. The purpose of learning mathematics is to build students’ knowledge of mathematical concepts (Lestiana et al., 2016). The concept of set is the foundation of modern mathematics, as well as one of the core and essential components of mathematics (Fischbein & Baltsan, 1998). The idea of set permeates into primary and secondary school knowledge, such as equations, inequalities, and functions. Learning about set knowledge helps students correctly understand the logical relationships between concepts, improve their ability to use mathematical language, and realize the transition from intuitive knowledge to abstract knowledge (Rahul et al., 2021).

As a primitive concept, set knowledge is characterized by high abstraction and multiple symbolic terms. However, students are prone to cognitive barriers in learning mathematical concepts, which has received increasing attention from scholars. According to the literature review, the research on set mainly focuses on the teaching and learning of set knowledge. The studies on students’ learning of set mainly concentrate on the learning objectives (Guan, 2016), learning difficulties (Sheng et al., 2023), and coping strategies of set knowledge (Alfiyansah et al., 2020; Fischbein & Baltsan, 1998; Hodgson, 1996). Some studies focus on the learning level of set knowledge. Yu (2017) proposed that the three levels of learning set knowledge are understanding, transfer, and innovation of knowledge. Li (2020) expressed that the concept of set, the relations of set, and the operations of set are a coherent and logical whole. Besides, the research on set teaching from the perspective of teachers mainly focuses on the analysis of teachers’ abilities (Zhang, 2020), the arrangement of textbooks (Bingolbali et al., 2021; Wang, 2019), and the teaching design of set knowledge (Jiang, 2020; Cheng, 2021). Studies have shown that there are often some problems in the learning and teaching of set knowledge, but these problems cannot be explicitly reflected in testing. In most schools’ math tests, students are typically provided with only a total score. Although some traditional measurement theories have the advantage of providing a macro-level assessment of students’ overall ability levels, such as classical test theory (CTT) and item response theory (IRT), they cannot reflect the specific knowledge structure and substantive content of students’ psychological traits (Scherer & Siddiq, 2019; Li et al., 2020).

Large-scale international assessment programs (Trends in International Mathematics and Science Studies – TIMSS and Program for International Student Assessment) inspired the introduction or expansion of national assessment programs (Wu et al., 2020a; OECD, 2013). In China, Mathematics Curriculum Standards for High School (2017 Edition, 2020 Revision) points out that mathematics education has the responsibility to foster virtue through education and enhance students’ well-rounded development (MOE of PRC, 2017), which proposes that we should focus not only on students’ learning results but also on the learning process. This is particularly emphasized in high school mathematics teaching assessment (Jung et al., 2023; Wang et al., 2018a). The single score in traditional tests no longer suffices for the purpose of evaluative improvement (Arican & Kuzu, 2020), indicating a need for educational evaluation reforms.

In this context, cognitive diagnostic models (CDMs), as advanced psychometric models, can combine the cognitive process and the measurement method (Wu, 2019; Kuo et al., 2016) and thus can compensate for the deficiency of single outcomes generated via CTT and IRT (Ravand & Robitzsch, 2018). Therefore, CDMs have gained increased attention in educational and psychological assessments recently (Tatsuoka, 2002; Wang et al., 2012).

Cognitive diagnosis (CD) is a new generation of test theory that combines the new generation of educational measurement with psychology (Ding et al., 2011; Henson & Douglas, 2005; Leighton & Gierl, 2007). People usually refer to the diagnostic evaluation of the individual cognitive process, processing skills, or knowledge structure as CD.

Cognitive diagnosis fully absorbs the rich achievements and unique research paradigm of cognitive psychology on the internal mechanism of human cognitive processing. It develops psychometric models known as CDMs. One of the advantages of CDMs is their ability to identify the strengths and weaknesses in a set of attributes, providing diagnostic information for learning and instruction (Bradshaw & Levy, 2019; Chiu et al., 2018; Choi et al., 2015). There are nearly 60 CDMs (Tu et al., 2008). Among these, the deterministic inputs, noisy “and” gate model (the DINA model) (Junker & Sijtsma, 2001; Chen & Chen, 2016) is a fully noncompensatory CDM. It assumes that to answer an item correctly, the subject must master all the attributes measured by that item. Lacking any one of these attributes will result in an incorrect response or a very low probability of answering the item correctly. The DINA model is highly preferred by researchers because of its simple interpretation, good model–data fit, and relatively better classification accuracy (Aryadoust, 2021; Loibl et al., 2020; DeCarlo, 2012).

The theory and application of the DINA model are the main areas of academic study, and the improvement and higher-order creation of the DINA model are the mainstays of theoretical research. For example, de la Torre (2009) described a method of DINA parameter estimation in detail to reduce the time of MCMC parameter estimation. de La Torre (2011) constructed the G-DINA model by relaxing some assumptions. Tu et al. (2012) extended the DINA model to a multilevel scoring model.

In the aspect of application research, the DINA model is used to diagnose children's psychological development, academic ability tests, and so on (Tu et al., 2012). Most of them are applied in mathematics. For example, Xu et al. (2023) used 67 released mathematical items for the fourth grade in TIMSS-2011 as assessment tools, selected the DINA model as the CD model, and formed the evaluation framework of students’ mathematical competency. Similarly, Zhang et al. (2023) used the DINA model to diagnose the mathematical ability of fourth-grade students in three rural Yi primary schools and found that there was individual variability with 21 different types of cognitive error patterns identified, which can inform the implementation of targeted remediation for teaching and learning.

Through the review of the literature, it has been found that research on CD and set knowledge has received attention from scholars. However, there is a lack of research on students’ mastery of set knowledge using the DINA model, both domestically and internationally. This study aims to use cognitive diagnostic assessment technology to measure students’ mastery of set knowledge and provide remedial teaching and learning suggestions for teachers and students based on the diagnostic results. Precisely, following the general steps of CD, this study is guided by the following three research questions: (a) How can cognitive attributes and hierarchical relationships be reasonably determined? (b) What is the mastery pattern of set knowledge for high school students? (c) What suggestions can the results of CD provide for teaching and learning?

2. Materials and methods

2.1 Participants

Current pedagogical studies are gradually shifting towards the student-centered direction (Chapman, 2013), emphasizing the importance of practical application in the classroom. In the high school comprehensive curriculum experimental textbooks used in China, the set knowledge is the learning content of the first unit of high school (ICTR, 2019), and it is the starting point for students to engage with high school mathematics.

In recent years, a province in eastern China has implemented comprehensive reforms aimed at categorized examinations, holistic assessments, and diversified admissions (Bian, 2015), giving students greater flexibility and autonomy in the classroom and examination. These reforms are still applicable from the perspective of the experience of the international curriculum (Sinnema et al., 2020). Meanwhile, some scholars have found that in terms of content setting and cognitive level, the degree of consistency between this province's college entrance examination papers and the Curriculum Standard is relatively high (Zhang & Pei, 2019), so this study was conducted in this region.

In this study, 247 senior students from 5 regular classes were selected by stratified sampling method in an ordinary high school in this selected province as participants and were distributed 247 test papers. The selected high school is a local ordinary high school. The school's class placement policy and teacher assessment results indicate that there are both top and weak students and more intermediate-level students among the participants, which is more representative.

The test was conducted after the students had learned set knowledge and lasted 30 min, under the teacher's supervision to ensure that the students completed it independently. After the test was finished, the papers were collected and corrected. There were 242 valid questionnaires, with a recovery rate of 97%. Then, the data from the formal test were analyzed and organized.

2.2 Testing tools

Compared to traditional assessment methods, cognitive diagnostic methods at a more granular level can reveal the underlying knowledge structure behind scores, allowing for fine-grained instructional evaluation and targeted remedial guidance. In this study, the concise and easily interpretable DINA Model was selected. With the help of SPSS (http://www.spss.com.cn) software and the CD platform (flexCDMS, http://www.psychometrics-studio.cn), the test results of the self-compiled test items of set knowledge were analyzed to obtain diagnostic results.

2.3 The determination of set cognitive attributes

Cognitive attributes describe the internal cognitive processing involved in individual problem-solving, referring to the knowledge structure and processing skills that participants should possess, and there is a certain logical order or hierarchical relationship between each cognitive attribute (Chen et al., 2008; Leighton et al., 2004). Reasonably establishing the cognitive attributes and hierarchical relationships is the primary prerequisite for obtaining correct diagnostic results.

By reviewing the literature about set attributes, adequate references have been provided for this study on the hierarchical setting of the cognitive attributes of set. Furthermore, based on the attribute framework in the Third International Math and Science Study–Revised (Tatsuoka et al., 2004) and the analysis of mathematics curriculum standards, exam outlines, teaching materials, and supporting books, the cognitive attributes and hierarchical relationships of the set have been preliminarily determined. After consulting several frontline teachers and refining the framework through detailed modifications, the cognitive attributes and hierarchical relationships of the set knowledge have been determined, as shown in Table 1.

Table 1.
The cognitive attributes of “set” knowledge for high school students.

Cognitive attributes Code Content explanation

Concept of set A1 Be able to understand the specific meaning of the set according to certainty, difference, and disorder of elements in the set; be able to understand the relationship between elements and set, and can denote it with corresponding symbols. (e.g., $1 \in Z, 1.5 \notin Z$ )

Representation of set A2 Be able to represent sets by enumeration, illustration, and description. (e.g., represent the set of all even numbers by description: ${x | x = 2 n, n \in N}$ )

The basic relations of set A3 Be able to understand the relationships of inclusion and equivalence among sets and can denote them with natural, graphical (Venn diagrams, number axis), and symbolic language. (e.g., denote the subset relationship with symbolic language: $A \subseteq B$ )

The basic operations of set A4 Be able to understand and compute unions, intersections, and complements of sets and can represent them by natural, graphical (Venn graphs, number axis), and symbolic language. (e.g., denote the intersection of sets A and B by natural language: the set composed of all elements that belong to both set A and set B)

Cognitive attributes	Code	Content explanation
Concept of set	A1	Be able to understand the specific meaning of the set according to certainty, difference, and disorder of elements in the set; be able to understand the relationship between elements and set, and can denote it with corresponding symbols. (e.g., $1 \in Z, 1.5 \notin Z$ )
Representation of set	A2	Be able to represent sets by enumeration, illustration, and description. (e.g., represent the set of all even numbers by description: ${x \| x = 2 n, n \in N}$ )
The basic relations of set	A3	Be able to understand the relationships of inclusion and equivalence among sets and can denote them with natural, graphical (Venn diagrams, number axis), and symbolic language. (e.g., denote the subset relationship with symbolic language: $A \subseteq B$ )
The basic operations of set	A4	Be able to understand and compute unions, intersections, and complements of sets and can represent them by natural, graphical (Venn graphs, number axis), and symbolic language. (e.g., denote the intersection of sets A and B by natural language: the set composed of all elements that belong to both set A and set B)

The hierarchical relationships of the four cognitive attributes are shown in Figure 1. It can be clearly seen from the figure that attribute A1 is the prerequisite of attribute A2, and attribute A2 is the prerequisite of attributes A3 and A4. However, the relationship between attributes A3 and A4 is unordered, presenting an unstructured juxtaposition.

Figure 1.

The set cognitive attributes and their hierarchical relationships.

2.4 Compilation of test items

First, compiling test items requires determining the adjacency matrix and the reachable matrix with reference to the direct and indirect relations among attributes, which are both 0–1 matrices of K rows and K columns (K refers to the number of attributes). If there exists a direct relation between cognitive attributes, the corresponding element in the adjacency matrix is denoted as “1,” otherwise “0”; If there exists any of the direct, indirect, or self-relation between cognitive attributes, the corresponding element in the reachable matrix is denoted as “1,” otherwise “0.”

From the hierarchical relationships among attributes, the adjacency matrix can be obtained as follows:

(\begin{matrix} A 1 & A 2 & A 3 & A 4 \\ A 1 & 0 & 1 & 0 & 0 \\ A 2 & 0 & 0 & 1 & 1 \\ A 3 & 0 & 0 & 0 & 0 \\ A 4 & 0 & 0 & 0 & 0 \end{matrix})

(1)

The reachable matrix can be obtained based on the adjacency matrix and the Boolean algorithm

R = (A + I)^{n}

. When the value of R is stable and constant, then R is the reachable matrix as follows:

= (\begin{matrix} A 1 & A 2 & A 3 & A 4 \\ A 1 & 1 & 1 & 1 & 1 \\ A 2 & 0 & 1 & 1 & 1 \\ A 3 & 0 & 0 & 1 & 0 \\ A 4 & 0 & 0 & 0 & 1 \end{matrix})

(2)

The ideal mastery pattern, represented as a K-column 0–1 matrix, is determined by the hierarchical relationships among attributes. If the subject has mastered a certain attribute, it is denoted by the element “1”; otherwise, it is marked as “0.” Considering the hierarchical relationships among the four cognitive attributes of set knowledge, there are 16 possible mastery patterns, of which only six of these mastery patterns satisfy the hierarchical relationships among attributes, as shown in Table 2.

Table 2.

The ideal mastery pattern.

Mastery pattern	A1	A2	A3	A4
1	0	0	0	0
2	1	0	0	0
3	1	1	0	0
4	1	1	1	0
5	1	1	0	1
6	1	1	1	1

Second, the Q-matrix needs to be obtained. The Q-matrix is typically a 0–1 matrix of J rows (J refers to the number of test items) and K columns (K refers to the number of attributes measured by the test) (Tatsuoka, 2012), which describes the relationships between test items and attributes, indicating which attributes are required to answer each item correctly. The Q-matrix can connect the observable responses and unobservable attribute mastery patterns, which is the theoretical basis for developing cognitive diagnostic tests (Chen et al., 2015; Li & Suen, 2013; Luo et al., 2015).

The Q-Matrix is established based on the ideal mastery patterns, with columns representing four cognitive attributes of this study and rows corresponding to the item numbers in the test paper. Typically, testing items should ensure that each attribute is measured at least three times (Tu et al., 2012). In this study, due to the relatively few attributes of the set knowledge, more items are included in the Q matrix to ensure quantities of items in the final test and multiple evaluations of each attribute. Due to the fact that attributes A3 and A4 contain many knowledge points, the number of items should be set more in order to examine the required cognitive attributes exhaustively. The Q-Matrix is shown in Table 3.

Table 3.

The Q-matrix.

	A1	A2	A3	A4
1	1	0	0	0
2	1	0	0	0
3	1	1	0	0
4	1	1	0	0
5	1	1	1	0
6	1	1	1	0
7	1	1	1	0
8	1	1	0	1
9	1	1	0	1
10	1	1	0	1
11	1	1	0	1
12	1	1	1	1
13	1	1	1	1
14	1	1	1	1
15	1	1	1	1

Third, based on the Q-matrix and the principles of test paper compilation, test items are selected from widely recognized matching practice items. After selecting multiple items that conform to the assessment mode of the Q-matrix, two experienced mathematics teachers are invited to review and refine these items to avoid ambiguity and ensure the accuracy of the items. This test paper contains 15 items, including two single-choice items, two multiple-choice items, 10 fill-in-blank items, and one free-response question. To minimize the possibility of students correctly answering items by chance, there is a deliberate limitation on the number of choice items, and the setting of multiple choice is also for this consideration. In accordance with the requirements of CD, the attributes involved in the test items increase sequentially from the beginning to the end of the test paper, and all items are scored 0–1, with 1 point for each and fifteen points in total.

2.5 The analysis of test item quality

Before conducting the large-scale test, we carry out a small-scale test to verify the discrimination and reliability of the test paper, as well as the rationality of the attribute hierarchy structure. In other words, we need to do a pilot test (Clement et al., 2008; Van Teijlingen & Hundley 2002). This study chose 20 students from a high school in the selected province for the pilot testing.

In CD, item discrimination is a crucial factor in assessing item quality and influencing the accuracy of attribute classification (Wang et al., 2018b). It is defined as the probability that subjects who have mastered all the attributes of items answer correctly [P(1)] minus the probability that subjects answer correctly without mastering one or more required attributes [P(0)], which typically ranges from −1 to 1. According to the CTT standard, the test paper is superior when the discrimination is larger than 0.4. In this test paper, only the discrimination of items 1 and 2 is 0, while the remaining items all meet the CTT standard, indicating that this test paper meets the standards in terms of discrimination. The discrimination of items 1 and 2 is 0 because they are the most basic items and it is reasonable for everyone to master them in a small-scale test. However, this study aims to learn about students’ mastery of set knowledge and it is essential to investigate attribute A1. Both items 1 and 2 focus on the assessment of Attribute A1, so they are reserved.

Reliability refers to the stability of test results (Twycross & Shields, 2004). In CD, two kinds of reliability indices are developed on the basis of the standard reference test. One is the attribute retest consistency reliability index; the other is the classification consistency reliability index. For this test, the average retest consistency reliability is 0.8693, which exceeds the threshold of 0.7. Additionally, the classification consistency reliability index also surpasses this benchmark, indicating that the test demonstrates good reliability. This paper adopts the Hierarchical Consistency Index (HCI) test (Cui et al., 2006) to verify the rationality of hierarchical relationships among cognitive attributes. The final value range of HCI is [−1,1], of which 1 represents perfect fit and −1 represents complete nonfit. Generally, an HCI exceeding 0.7 is considered to represent an excellent model-data fit. The average HCI value of the subjects in this study is 0.7699, indicating a qualified fit. Therefore, the construction of hierarchical relationships among the set attributes in this paper is deemed reasonable.

3. Results

3.1 Parameter estimation

The item response function of the DINA model is defined as $P (x_{i j} = 1 ∣ α_{i}, q_{j}, η_{i j}) = (1 - s_{j})^{η_{i j}} g_{j}^{(1 - η_{i j})}$ , where $s_{j}$ represents the probability that the subject i has mastered all the required attributes but answers wrong (slip probability); $g_{j}$ represents the probability that the subject i answers correctly without fully mastering all required attributes (guess probability) (Luo, 2019; Wu et al., 2020b). These two parameters indirectly reflect the quality of the test. The lower the values of these two parameters, the better the test model is considered to be. When both parameters are below 0.4, the test is considered to have a good effect. FlexCDMs calculate the estimated parameters of each item, and all are below 0.4, indicating good overall quality for each item in the test paper (Table 4).

Table 4.
Item parameter estimation.

Item g s

Item 1 0.0001 0.0111

Item 2 0.2686 0.0094

Item 3 0.2679 0.1361

Item 4 0.0834 0.3454

Item 5 0.3208 0.0842

Item 6 0.0773 0.2024

Item 7 0.2663 0.2663

Item 8 0.2513 0.1646

Item 9 0.2338 0.1348

Item 10 0.2539 0.0745

Item 11 0.2691 0.1161

Item 12 0.2799 0.1235

Item 13 0.2395 0.2003

Item 14 0.3836 0.2877

Item 15 0.0239 0.3627

Item	g	s
Item 1	0.0001	0.0111
Item 2	0.2686	0.0094
Item 3	0.2679	0.1361
Item 4	0.0834	0.3454
Item 5	0.3208	0.0842
Item 6	0.0773	0.2024
Item 7	0.2663	0.2663
Item 8	0.2513	0.1646
Item 9	0.2338	0.1348
Item 10	0.2539	0.0745
Item 11	0.2691	0.1161
Item 12	0.2799	0.1235
Item 13	0.2395	0.2003
Item 14	0.3836	0.2877
Item 15	0.0239	0.3627

3.2 The statistics of attribute mastery situation for all subjects

In the test, subjects who have mastered a particular attribute should be able to answer all related items correctly. Consequently, the number of items a subject answers correctly regarding an attribute is directly related to the likelihood of their mastery of that attribute. Figure 2 illustrates the attribute mastery situation of all subjects.

Figure 2.

Radar chart of the attribute mastery situation for all subjects.

As shown in Figure 2, the maximum value of all attributes is 1, indicating that some individuals have fully mastered the attribute. Regarding the mean values, there are differences among the four attributes. Subjects master the first two attributes well, as the mean probabilities of mastering these attributes are both greater than 0.9, which is very close to 1. That is because both attributes A1 and A2 are considered essential attributes, namely the concept and representation of the set, which are easy for students to grasp. The probability of mastery attribute A3 is above 0.7, indicating that the overall subjects grasp this attribute well. But that is lower than the probabilities of mastering attributes A1 and A2, possibly because attribute A3 represents the fundamental relations of the set, which involves multiple knowledge points. If the subjects have not yet understood any of the basic relations of the set, then it will lead to a low probability of mastery attribute A3. For example, the figures (Figures 3–5) show the responses of three subjects to items 5, 6, and 7, all of which relate to attribute A3.

Figure 3.

Subject 1.

Figure 4.

Subject 2.

Figure 5.

Subject 3.

The mean value of the probability of subjects mastering attribute A4 is lower than 0.7, indicating that subjects have relatively poor mastery of attribute A4. Attribute A4 represents the basic operations of the set, which has consistently posed a learning challenge for students, as noted in previous studies. If students do not grasp any operation, the probability of mastery attribute will be low. For instance, Figures 5 and 6 show the responses of a subject to items 8 and 11, both of which are the investigation of attribute A4 (Figure 7).

Figure 6.

Subject 4.

Figure 7.

Subject 5.

3.3 The classification of subjects’ attribute-mastery patterns

Generally, the actual reaction pattern of subjects corresponds to one of the ideal mastery patterns. In this circumstance, the attribute-mastery pattern matches the corresponding ideal mastery pattern. However, due to students’ guesses or misjudgments during the problem-solving process, the actual response pattern does not align with the ideal mastery pattern. In such cases, the maximum likelihood estimation method needs to be adopted to evaluate the attribute-mastery pattern of subjects. The ideal reaction pattern with the maximum similarity to the actual one is estimated, and the corresponding attribute-mastery pattern is the attribute-mastery pattern of subjects.

Some subjects exhibit the same attribute-mastery pattern, allowing for classification based on their attribute-mastery patterns. The proportion of different attribute-mastery patterns can be obtained, as shown in Table 5.

Table 5.
The proportion of subjects’ attribute-mastery patterns.

Mastery pattern Number of people Proportion (%) Overlay ratio (%)

1111 107 44.21 44.21

1110 51 21.07 65.28

1101 36 14.88 80.16

1100 17 7.02 87.18

1001 7 2.89 90.07

1011 5 2.07 92.14

0100 3 1.24 93.38

0110 3 1.24 94.62

0001 3 1.24 95.86

1000 3 1.24 97.1

0000 2 0.83 97.93

0010 1 0.41 98.34

1010 1 0.41 98.75

0111 1 0.41 99.16

0101 1 0.41 99.57

0011 1 0.41 99.98

Mastery pattern	Number of people	Proportion (%)	Overlay ratio (%)
1111	107	44.21	44.21
1110	51	21.07	65.28
1101	36	14.88	80.16
1100	17	7.02	87.18
1001	7	2.89	90.07
1011	5	2.07	92.14
0100	3	1.24	93.38
0110	3	1.24	94.62
0001	3	1.24	95.86
1000	3	1.24	97.1
0000	2	0.83	97.93
0010	1	0.41	98.34
1010	1	0.41	98.75
0111	1	0.41	99.16
0101	1	0.41	99.57
0011	1	0.41	99.98

From this, it can be seen that the subjects’ attribute-mastery patterns have the following characteristics:

The number of students with the attribute-mastery pattern of “1111” accounts for the most significant proportion, 44.21%, nearly half. That indicates almost half of the subjects have a complete grasp of set knowledge and can answer all the test items correctly, which is inseparable from the fact set knowledge itself is a relatively basic knowledge point.

Among all the subjects, a total of 93 subjects fail to master only one attribute, accounting for 38.43%. As for these subjects, they only need to strengthen learning and practice of the corresponding attribute directly.

Among all the subjects, 10 types of attribute-mastery patterns do not belong to any ideal mastery pattern, with 26 persons accounting for 10.74%. Although this group of students does not account for a large proportion, it is necessary to know about their learning situation and teachers’ teaching situation in detail. For example, for subjects with the attribute-mastery pattern of “1011,” only attribute A2 has not been mastered. Question 4 requires students to use the enumeration method to list items’ answers. Students may easily substitute only the situation in the question stem rather than carry out in-depth operations, leading to incomplete enumeration and subsequent mistakes.

4. Discussion

Cognitive diagnosis is more comprehensive and can make up for the shortcomings of traditional tests. It not only focuses on the results but also on the processes, which means teachers can more clearly understand the different attribute mastery patterns under the same score of students.

On the one hand, the test results indicate that students have achieved satisfactory mastery overall. However, it is worth noting that the mastery level of A3 and A4 attributes is comparatively lower than that of A1 and A2. In many previous studies, the basic relations and operations of the set were also considered difficult (Alfiyansah et al., 2020; Xie & Li, 2011; Zhu & Liu, 2003). Some errors could arise from students’ use of natural language (Allen et al., 2015), graphic language, and symbolic language (Hodgson, 1996; Konyalioglu et al., 2005). The development of set thinking and the mastery of set language differ from students’ original thinking and language habits, which require long-term practice to form and develop.

On the other hand, subjects whose attribute mastery patterns do not belong to the ideal mastery patterns may struggle not due to a lack of understanding but due to calculation issues and other factors, which may have an impact on the experimental conclusion. Further research should exclude issues stemming from students’ carelessness or inattention. This is where we can refer to the ‘Explanatory Mixture Item Response Theory Model’ proposed by Ulitzsch et al. (2022). This model can be used to better understand the conditions under which carelessness occurs and to design research studies that can inhibit the incidence of carelessness. Second, a lack of a solid grasp of specific knowledge in junior high school can also lead to errors. In junior high school, students have been exposed to set-related knowledge, such as the set of natural numbers, rational numbers, the solution set of one-variable inequality, and the definition of a circle (Ma & Ouyang, 2019). All these provide knowledge support for the systematic study of the concept of set, the basic relationship between sets, and the operations of sets in high school. If students are not familiar with these contents, they may make mistakes.

Based on the results of this research, some suggestions can also be provided. After obtaining the diagnostic results of students’ set knowledge, teachers can take effective measures for remedial teaching.

In this study, the proportion of students who did not master attribute A4 is the highest. Hence, teachers need to strengthen the explanation and practice of attribute A4 in future exercise and review classes. For example, they can pose a few simple questions about A4 in class, and let the students solve them by using natural language, graphic language, and symbolic language in groups. For graphic language, teachers can use Geogebra to show the problem-solving process so that students can understand more easily. These ways could help students broaden their minds and deepen their understanding of new knowledge. Teachers should follow the cognitive law of thinking from intuition to abstraction, thereby enabling students to establish links between newly acquired information and their existing knowledge base (Acharya, 2017). For example, when constructing the essential relationships and basic operations of sets, starting from reality and existing relevant experience, teachers should analogize the real number size relationship and basic operations to make it easier for students to understand. What's more, teachers should provide targeted instruction to the small subset of students who have yet to grasp the fundamental attributes A1 and A2. When assigning homework, teachers should arrange it in layers. This stratified teaching method classifies students based on their internal psychological processing, helps them learn knowledge points and fill in gaps, and strives to make every student get the best development in the level of learning that is most suitable for them (Shi et al., 2020).

In this study, students’ mastery and scoring of set attributes vary, and precise teaching is also required for each type of student's learning. This teaching method defines instructional targets, monitors daily performance, and uniformly presents performance data (White, 2005). For example, online platforms such as Dingtalk and Classin are among the best ways to assign homework. With the help of modern educational technology, the condition of students’ answers can be clearly presented through visual data. This method can complement cognitive diagnostic results and provide powerful suggestions for teachers’ remedial teaching measures.

Learning mathematics is learning a formal language with specific meanings (Li et al., 2023; Kuo, 2019). The high level of abstraction of set leads to a lower level of mastery of the attributes that require a deeper understanding of the cognitive structure. Teachers should guide students in performing language conversion using natural, graphical, and symbolic language to maximize problem-solving efficiency. Furthermore, teachers must guide students to understand mathematical thinking and help them establish a mathematical concept system.

For students, it is crucial to fully utilize the advantages of CD and understand their shortcomings in the set knowledge. The following are remedial learning suggestions for students.

Cognitive diagnosis allows students to determine where to focus their attention and effort (Kanar & Bell, 2013). Some students may use explicit criteria to focus on precisely what needs to be done to reach a desired level of achievement rather than learning the material thoroughly (Panadero & Jonsson, 2013). For instance, students who have not mastered A4 can improve their set operation ability by targeted practice.

The study shows some students still have foundational problem. So for them, they should follow the teacher's thinking in class. To ensure a smooth transition between junior high and senior high school thinking, most teachers will use concrete examples to illustrate knowledge. If students try to think proactively and use the context to clarify the new content which are about to learn, they would get a new knowledge more thoroughly (Liu, 2020). After the lesson, students can use mind maps to sort out their knowledge and strengthen their memory based on the teacher's summary in class (Tan, 2022). In this manner, they can clarify the differences and connections between the details of set elements, elements, classification, etc., and consolidate their knowledge more efficiently.

5. Limitation

The sample size in this study is small for a large-scale test, and the division of set knowledge's attributes is not detailed enough; both aspects can be improved in subsequent research. Meanwhile, if conditions permit, the sample should consider including students from different districts or with different family environment to enhance the generalizability of the findings. Future studies can be carried out by extending the range of sampling areas to obtain more universal conclusions. Additionally, all teaching suggestions and learning recommendations in this study are based on the data and diagnostic results, and it remains debatable whether they can be effectively applied to teachers and students. Only when remedial measures and pedagogical suggestions are implemented in subsequent studies will it be known whether they are effective for students.

6. Conclusion

In general, based on the DINA model, this study conducts CD on mathematical set knowledge and analyzes the response data of subjects. Students exhibit a relatively concentrated mastery pattern for set knowledge, with 89.25% of subjects classified as ideal mastery patterns. 44.21% manifest a mastery pattern of “1111,” 21.07% exhibit a mastery pattern of “1110,” 14.88% demonstrate a mastery pattern of “1101,” and 7.02% present a mastery pattern of “1100.” The proportion of students mastering other patterns is relatively small, accounting for no more than 3%. Cognitive diagnosis can obtain a relatively accurate status of students’ knowledge and provide more comprehensive evaluation methods. This study guides teachers to use CD to reflect on their teaching methods, adjust teaching content, provide remedial teaching, and guide students to identify areas of inadequate mastery based on their cognitive diagnostic results and practice them in a targeted manner.

Footnotes

Acknowledgements

The authors extend their thanks for the support of Humanities & Social Science of Chinese Ministry of Education.

Contributorship

Zhangtao Xu rewrite, reviewed, edited, and supervised the research. Qi Mi conducted the research and drafted the manuscript. Xinyue Wu visualize and rewrite the manuscript. Zhaoyang Yin visualize and rewrite the manuscript. All authors read and approved the final manuscript.

Consent for publication

All authors agreed with the content and that all gave explicit consent to submit and that we obtained consent from the responsible authorities at the institute where the work has been carried out, before the work is submitted.

Data availability statement

The original contributions presented in the study are included in the article/supplementary material; further inquiries can be directed to the corresponding author.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article was funded by—A study on teachers’ teaching knowledge in the implementation of the “double reduction” policy of the Ministry of Education's humanities and social science planning in 2022 (22YJA880068). Humanities and Social Science Fund of Ministry of Education of China.

Informed consent

Informed consent was obtained from all subjects involved in the study.

ORCID iD

Zhangtao Xu

Author biographies

Qi Mi received the MS degree from Central China Normal University in Wuhan, China, in 2021. Her research interests lie in teaching evaluation and assessment.

Xinyue Wu is currently working toward the MS degree with the School of Mathematics and Statistics, Central China Normal University in Wuhan, China. Her research interests lie in teaching evaluation and assessment.

Zhaoyang Yin is currently working toward the MS degree with the School of Mathematics and Statistics, Central China Normal University in Wuhan, China. Her research interests lie in teaching evaluation and assessment.

Zhangtao Xu is currently a professor with the School of Mathematics and Statistics, Central China Normal University in Wuhan, China. His research interests lie in teaching evaluation and assessment.

References

Acharya

B. R.

(2017). Factors affecting difficulties in learning mathematics by mathematics learners. International Journal of Elementary Education, 6(2), 8–15. https://doi.org/10.11648/j.ijeedu.20170602.11

Alfiyansah

A. F.

Fitriani

Narimoati

Handayani

(2020). Identify maths learning difficulties on the set subject. Proceedings of the International Conference on Science and Engineering, 3(1), 319–322. https://doi.org/10.14421/icse.v3.520

Allen

L. K.

Snow

E. L.

McNamara

D. S.

(2015). Are you Reading my mind? Modeling students’ Reading comprehension skills with natural language processing techniques. In Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, 246–254. https://doi.org/10.1145/2723576.2723617

Arican

Kuzu

(2020). Diagnosing preservice teachers’ understanding of statistics and probability: Developing a test for cognitive assessment. International Journal of Science and Mathematics Education, 18(4), 771–790. https://doi.org/10.1007/s10763-019-09985-0

Aryadoust

(2021). A cognitive diagnostic assessment study of the listening test of the Singapore–Cambridge general certificate of education O-level: Application of DINA, DINO, G-DINA, HO-DINA, and RRUM. International Journal of Listening, 35(1), 29–52. https://doi.org/10.1080/10904018.2018.1500915

Bian

(2015). A comparative study on the depth of Zhejiang and Shanghai programmes for the new round of college entrance examination reform. China Exam, 12(02), 3–7. (In Chinese). https://doi.org/10.19360/j.cnki.11-3303/g4.2015.02.001

Bingolbali

Demir

Monaghan

J. D.

(2021). Knowledge of sets: A didactic phenomenon. International Journal of Science and Mathematics Education, 19(6), 1187–1208. https://doi.org/10.1007/s10763-020-10106-5

Bradshaw

Levy

(2019). Interpreting probabilistic classifications from diagnostic psychometric models. Educational Measurement: Issues and Practice, 38(2), 79–88. https://doi.org/10.1111/emip.12247

Chapman

(2013). Investigating teachers’ knowledge for teaching mathematics. Journal of Mathematics Teacher Education, 16(4), 237–243. https://doi.org/10.1007/s10857-013-9247-2

10.

Chen

(2016). Exploring Reading comprehension skill relationships through the G-DINA model. Educational Psychology, 36(6), 1049–1064. https://doi.org/10.1080/01443410.2015.1076764

11.

Chen

Y. H.

Gorin

J. S.

Thompson

M. S.

Tatsuoka

K. K.

(2008). Cross-cultural validity of the TIMSS-1999 mathematics test: Verification of a cognitive model. International Journal of Testing, 8(3), 251–271. https://doi.org/10.1080/15305050802262217

12.

Chen

Liu

Ying

(2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110(510), 850–866. https://doi.org/10.1080/01621459.2014.934827

13.

Cheng

(2021). A teaching model for accurate teaching of high school mathematics—taking “sets” as an example. Correspondence of the Teaching of Mathematics, 43(27), 57–58. (In Chinese).

14.

Chiu

C. Y.

Sun

Bian

(2018). Cognitive diagnosis for small educational programs: The general nonparametric classification method. Psychometrika, 83(2), 355–375. https://doi.org/10.1007/s11336-017-9595-4

15.

Choi

K. M.

Lee

Y. S.

Park

Y. S.

(2015). What CDM can tell about what students have learned: An analysis of TIMSS eighth grade mathematics. Eurasia Journal of Mathematics, Science and Technology Education, 11(6), 1563–1577. https://doi.org/10.12973/eurasia.2015.1421a

16.

Clements

D. H.

Sarama

J. H.

Liu

X. H.

(2008). Development of a measure of early mathematics achievement using the rasch model: The research-based early maths assessment. Educational Psychology, 28(4), 457–482. https://doi.org/10.1080/01443410701777272

17.

Cui

Leighton

J. P.

Gierl

M. J.

Hunka

(2006). A person-fit statistic for the attribute hierarchy method: The hierarchy consistency index. In Annual Meeting of the National Council on Measurement in Education, San Francisco, CA.

18.

DeCarlo

L. T.

(2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36(6), 447–468. https://doi.org/10.1177/0146621612449069

19.

De La Torre

. (2009). DINA Model and parameter estimation:A didactic. Journal of Educational and Behavioral Statistics, 34(1), 115–130. https://doi.org/10.3102/1076998607309474

20.

De La Torre

. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179–199. https://doi.org/10.1007/s11336-011-9207-7

21.

Ding

Wang

Yang

(2011). Design of cognitive diagnostic test blueprint. Journal of Psychological Science, 34(02), 258–265. (In Chinese). https://doi.org/10.16719/j.cnki.1671-6981.2011.02.015

22.

Fischbein

Baltsan

(1998). The mathematical concept of set and the ‘collection’ model. Educational Studies in Mathematics, 37(1), 1–22. https://doi.org/10.1023/A:1003421206945

23.

Guan

(2016). Leading teaching, set thinking. Journal of Mathematics(China), 66(01), 24–25 + 30. (In Chinese).

24.

Henson

Douglas

(2005). Test construction for cognitive diagnosis. Applied Psychological Measurement, 29(4), 262–277. https://doi.org/10.1177/0146621604272623

25.

Hodgson

(1996). Students’ ability to visualize set expressions: An initial investigation. Educational Studies in Mathematics, 30(2), 159–178. https://doi.org/10.1007/BF00302628

26.

Institute of Curriculum and Textbook Research(ICTR). (2019). High School Textbook Mathematics (Compulsory Volume 1). People's Education Press. (In Chinese).

27.

Jiang

(2020). Research on high school mathematics teaching under the infiltration of the theory of multiple intelligences theory [Unpublished master's thesis]. Yunnan Normal University.

28.

Jung

Lim

S. A.

Fan

(2023). Teacher’s factors affecting students’ math class engagement: The mediating effect of math self-efficacy. Educational Psychology, 43(8), 929–946. https://doi.org/10.1080/01443410.2023.2267809

29.

Junker

B. W.

Sijtsma

(2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272. https://doi.org/10.1177/01466210122032064

30.

Kanar

A. M.

Bell

B. S.

(2013). Guiding learners through technology-based instruction: The effects of adaptive guidance design and individual differences on learning over time. Journal of Educational Psychology, 105(4), 1067. https://doi.org/10.1037/a0033831

31.

Konyalioglu

A. C.

Ipek

A. S.

Isik

(2005). The role of visualization approach on student’s conceptual learning. International Journal for Mathematics Teaching and Learning, 1(3), 1–10. https://bit.ly/3BLaLQv

32.

Kuo

B. C.

(2019). Chinese And mathematics. Educational Psychology, 39(5), 569–571. https://doi.org/10.1080/01443410.2019.1617987

33.

Kuo

B. C.

Chen

C. H.

Yang

C. W.

Mok

M. M. C.

(2016). Cognitive diagnostic models for tests with multiple-choice and constructed-response items. Educational Psychology, 36(6), 1115–1133. https://doi.org/10.1080/01443410.2016.1166176

34.

Leighton

Gierl

M. E.

(2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press.

35.

Leighton

J. P.

Gierl

M. J.

Hunka

S. M.

(2004). The attribute hierarchy method for cognitive assessment: A variation on tatsuoka's rule-space approach. Journal of Educational Measurement, 41(3), 205–237. https://doi.org/10.1111/j.1745-3984.2004.tb01163.x

36.

Lestiana

H. T.

Rejeki

Setyawan

(2016). Identifying students’ errors on fractions. Journal of Research and Advances in Mathematics Education, 1(2), 131–139. https://doi.org/10.23917/jramathedu.v1i2.3396

37.

(2020). Meta guidance: A learning guidance paradigm based on literacy and unit. Journal of Mathematics Education, 29(05), 64–68. (In Chinese).

38.

Suen

H. K.

(2013). Constructing and validating a Q-matrix for cognitive diagnostic analyses of a Reading test. Educational Assessment, 18(1), 1–25. https://doi.org/10.1080/10627197.2013.761522

39.

Sun

Zhou

Wang

(2023). Controlled attention, but not temporal storage of working memory correlates to math problem solving. Educational Psychology, 43(8), 895–913. https://doi.org/10.1080/01443410.2023.2254522

40.

Zhou

Huang

Gao

Yang

(2020). Assessing kindergarteners’ mathematics problem solving: The development of a cognitive diagnostic test. Studies in Educational Evaluation, 66(03), 100879. https://doi.org/10.1016/j.stueduc.2020.100879

41.

Liu

. (2020). Problem-led independent inquiry teaching design for high school students–taking “the meaning of set and its representation” as an example. Secondary Mathematics Teaching Reference, 29(18), 23–25. (In Chinese).

42.

Loibl

Leuders

Dörfler

(2020). A framework for explaining teachers’ diagnostic judgements by cognitive modeling (DiaCoM). Teaching and Teacher Education, 91(05), 103059. https://doi.org/10.1016/j.tate.2020.103059

43.

Luo

(2019). Fundamentals of cognitive diagnostic evaluation. Beijing Normal University Press. (In Chinese).

44.

Luo

Gao

Peng

(2015). A simple cognitive diagnosis method based on Q-matrix theory. Acta Psychologica Sinica, 47(2), 264. https://doi.org/10.3724/SP.J.1041.2015.00264

45.

., & Ouyang

. (2019). The concept of set’ teaching design, teaching reflection and review. Secondary Mathematics Teaching Reference, 28(22), 17–23. (In Chinese).

46.

Ministry of Education of the People's Republic of China (MOE of PRC). (2017). Mathematics curriculum standards for ordinary high schools (2017 Edition). People's Education Press. (In Chinese).

47.

Munzar

Muis

K. R.

Denton

C. A.

Losenno

(2021). Elementary students’ cognitive and affective responses to impasses during mathematics problem solving. Journal of Educational Psychology, 113(1), 104. https://doi.org/10.1037/edu0000460

48.

Organization for Economic Co-operation and Development (OECD.). (2013). PISA 2012 results: Excellence through equity: Giving every student the chance to succeed (Volume II). OECD Publishing.

49.

Panadero

Jonsson

(2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9(02), 129–144. https://doi.org/10.1016/j.edurev.2013.01.002

50.

Rahul

Kumar

G. S.

Nidhi

(2021). An overview on sets in mathematics. An International Multidisciplinary Research Journal, 11(10), 107–114. https://doi.org/10.5958/2249-7137.2021.02077.2

51.

Ravand

Robitzsch

(2018). Cognitive diagnostic model of best choice: A study of Reading comprehension. Educational Psychology, 38(10), 1255–1277. https://doi.org/10.1080/01443410.2018.1489524

52.

Scherer

Siddiq

(2019). The relation between students’ socioeconomic status and ICT literacy: Findings from a meta-analysis. Computers & Education, 138(11), 13–32. https://doi.org/10.1016/j.compedu.2019.04.011

53.

Sheng

Dong

(2023). Teaching research on mathematical thinking methods in set knowledge in high school. Correspondence of the Teaching of Mathematics, 45(21), 41–44. (In Chinese).

54.

Shi

Cheung

E. S.

Cheung

A. C.

(2020). The impact of stratified teaching on the academic performance of Chinese middle school students: A meta-analysis. Science Insights Education Frontiers, 7(1), 735–760. https://doi.org/10.15354/sief.20.re012

55.

Sinnema

Nieveen

Priestley

(2020). Successful futures, successful curriculum: What can Wales learn from international curriculum reforms? The Curriculum Journal, 31(2), 181–201. https://doi.org/10.1002/curj.17

56.

Tan

. (2022). Emphasis on the guidance of thinking, feel the true knowledge of mathematics—taking the teaching of the chapter of “set” as an example. Teaching Reference of Middle School Mathematics, 31(33):23–24. (In Chinese).

57.

Tatsuoka

(2002). Data analytic methods for latent partially ordered classification models. Journal of the Royal Statistical Society Series C: Applied Statistics, 51(3), 337–350. https://doi.org/10.1111/1467-9876.00272

58.

Tatsuoka

K. K.

(2012). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In Cognitively diagnostic assessment (pp.327–359). Routledge.

59.

Tatsuoka

K. K.

Corter

J. E.

Tatsuoka

(2004). Patterns of diagnosed mathematical content and process skills in TIMSS-R across a sample of 20 countries. American Educational Research Journal, 41(4), 901–926. https://doi.org/10.3102/00028312041004901

60.

Cai

Dai

(2008). A review on cognitive diagnostic models under modern test theory. Psychological Exploration, 28(02), 64–68. (In Chinese)

61.

Cai

Ding

(2012). Cognitive diagnosis: Theory, methods and applications (Vol. 09). Beijing Normal University Press.

62.

Twycross

Shields

(2004). Validity and reliability – What’s it all about? Paediatric Nursing, 16(10), 36. https://doi.oprg/10.7748/paed.16.10.36.s22

63.

Ulitzsch

Yildirim-Erbasli

S. N.

Gorgun

Bulut

(2022). An explanatory mixture IRT model for careless and insufficient effort responding in self-report measures. British Journal of Mathematical and Statistical Psychology, 75(3), 668–698. https://doi.org/10.1111/bmsp.12272

64.

Van Teijlingen

Hundley

(2002). The importance of pilot studies. Nursing standard (through 2013), 16(40), 33. https://doi.org/10.7748/ns.16.40.33.s1

65.

Wang

(2019). Learning as a language, developing mathematical abstract literacy and logical reasoning literacy—Introduction to chapter 1 “set and commonly used logical terms” of “mathematics in ordinary high school textbooks”. Teaching Reference of Middle School Mathematics, 28(25), 12–13 + 24. (In Chinese).

66.

Wang

Chang

H. H.

Douglas

(2012). Combining CAT with cognitive diagnosis: A weighted item selection approach. Behavior Research Methods, 44(01), 95–109. https://doi.org/10.3758/s13428-011-0143-3

67.

Wang

Jou

Huang

C. C.

(2018a). An investigation on teaching performances of model-based flipping classroom for physics supported by modern teaching technologies. Computers in Human Behavior, 84(07), 36–48. https://doi.org/10.1016/j.chb.2018.02.018

68.

Wang

Song

Ding

(2018b). An item discrimination index and its application in cognitive diagnostic assessment on a classification-oriented view. Journal of Psychological Science, 41(02), 475–483. (In Chinese. https://doi.org/10.16719/j.cnki.1671-6981.20180234

69.

White

O. R.

(2005). Precision teaching . In Hersen

Sugai

Horner

. (Eds.), Encyclopedia of behavior modification and cognitive behavior therapy (pp. 1433–1437). Sage Publications, Inc.

70.

H.-M.

(2019). Online individualised tutor for improving mathematics learning: A cognitive diagnostic model approach. Educational Psychology, 39(10), 1218–1232. https://doi.org/10.1080/01443410.2018.1494819

71.

Y. J.

Carstensen

C. H.

Lee

(2020a). A new perspective on memorization practices among east Asian students based on PISA 2012. Educational Psychology, 40(5), 643–662. https://doi.org/10.1080/01443410.2019.1648766

72.

Liang

Yürekli

Becker

B. J.

Paek

Binici

(2020b). Exploring the impact of Q-matrix specifications through a DINA model in a large-scale mathematics assessment. Journal of Psychoeducational Assessment, 38(5), 581–598. https://doi.org/10.1177/0734282919867535

73.

Xie

(2011). Some noteworthy issues in the set “description method”. Journal of Liaoning Normal University (Natural Science Edition), 34(01), 1–5. (In Chinese).

74.

Sun

Kong

(2023). Cognitive diagnostic analysis of students’ mathematical competency based on DINA model. Psychology in the Schools, 60(09), 3135–3150. https://doi.org/10.1002/pits.22916

75.

(2017). A framework for evaluating mathematical core literacy. Journal of Mathematics Education, 26(02), 19–23 + 59. (In Chinese).

76.

Zhang

(2020). What and how to prepare for “preknowledge”. Journal of Mathematics (China), 70(08), 1–14.

77.

Zhang

Pei

(2019). A study on the consistency of mathematics test papers with the curriculum standard in the context of the new college entrance examination reform—taking the 2017-2018 National II paper and Zhejiang paper as examples. Journal of Mathematics Education, 28(04), 55–60. (In Chinese).

78.

Zhang

Shi

(2023). Cognitive diagnosis and algorithmic cultural analysis of fourth-grade Yi students’ mathematical skills in China: A case study of several primary schools in Puge county, Liangshan prefecture. Frontiers in Psychology, 14(01), 1110950. https://doi.org/10.3389/fpsyg.2023.1110950

79.

Zhu

Liu

(2003). A study on the development characteristics of set concepts among junior high school students. Journal of Mathematics Education, 12(02), 55–58. (In Chinese).

	A1	A2	A3	A4
1	1	0	0	0
2	1	0	0	0
3	1	1	0	0
4	1	1	0	0
5	1	1	1	0
6	1	1	1	0
7	1	1	1	0
8	1	1	0	1
9	1	1	0	1
10	1	1	0	1
11	1	1	0	1
12	1	1	1	1
13	1	1	1	1
14	1	1	1	1
15	1	1	1	1

	A1	A2	A3	A4
1	1	0	0	0
2	1	0	0	0
3	1	1	0	0
4	1	1	0	0
5	1	1	1	0
6	1	1	1	0
7	1	1	1	0
8	1	1	0	1
9	1	1	0	1
10	1	1	0	1
11	1	1	0	1
12	1	1	1	1
13	1	1	1	1
14	1	1	1	1
15	1	1	1	1

	A1	A2	A3	A4
1	1	0	0	0
2	1	0	0	0
3	1	1	0	0
4	1	1	0	0
5	1	1	1	0
6	1	1	1	0
7	1	1	1	0
8	1	1	0	1
9	1	1	0	1
10	1	1	0	1
11	1	1	0	1
12	1	1	1	1
13	1	1	1	1
14	1	1	1	1
15	1	1	1	1