Abstract
Formulating is a crucial skill for transforming real-life situations into mathematical representations and reconnecting mathematical abstractions to real contexts. This skill plays an essential role in mathematical literacy, problem-solving, modelling, and STEM practices. This study aims to examine high school students’ formulating skills and to propose a framework for evaluating them. The Formulating Test developed for this research was administered to 561 students to conduct validity and reliability analyses. After establishing the test’s measurement properties, the formulating skills of an additional 431 students from different high school types were analysed in relation to the proposed framework, school type, grade level, and gender. The findings indicate that students’ formulating skills are generally below the expected level, consistent with previous research. Moreover, results revealed that (1) students performed better in the mathematical sense-making dimension than in other dimensions, (2) school type had a significant effect on formulating skills, (3) grade level had a small effect, and (4) gender differences were not significant. These findings highlight the need for more comprehensive studies to support the development of formulating skills. Overall, this research provides an important foundation for strengthening formulating skills in future studies and educational practices.
Plain Language Summary
This study examines high school students’ ability to transform real-life situations into a mathematical structure and to adapt mathematical generalisations to real life, that is, their ability to formulate. Formulating involves understanding a situation, selecting the necessary information, and transforming it into a mathematical problem. To assess this skill, a special 15-question test was developed using PISA problems related to the formulating process within the research framework. The test was first adapted for a group of students, then administered to 431 students from different high schools, and the results were analysed. The findings indicate that students are successful at translating given situations into mathematical problems; however, they struggle to generate new real-life problems using mathematical thinking. This situation reveals that students need more support in constructing their own problems and structuring their mathematical thinking. These results indicate that students’ ability to formulate real-life problems and structure their mathematical thinking needs further development. Therefore, it is recommended that more emphasis be placed on modelling activities, STEM applications, and open-ended problem-setting exercises to support skill development.
Keywords
Introduction
Today, mathematics is seen as a prerequisite course and a fundamental aspect of literacy for being a stakeholder in a modern society (National Council of Teachers of Mathematics [NCTM], 1989). It is now necessary to move away from memorising procedures in mathematics and focus on mathematical thinking, reasoning, and problem-solving (Cai & Howson, 2012). With this evolving perspective, mathematics is not merely a collection of abstract operations but a fundamental literacy area that individuals need throughout their lives. In other words, mathematics is a tool that provides individuals with basic skills such as analytical thinking, problem-solving, and reasoning, which STEM education and critical thinking focus on, rather than a discipline that individuals only see as abstract and logical (Maass et al., 2019).
Mathematics plays a crucial role in many fields, including science, technology, engineering, architecture, robotics, coding, art, agriculture, and medicine. It is effectively used for everyday analysis, prediction, inference, and interpretation. The use of mathematics is not limited to quantitative fields such as science, technology, engineering, and medicine, but also plays an active role in the social sciences. In the social sciences, mathematical modelling and analysis are utilised in numerous contexts, ranging from human behaviour to societal concerns, and from economic expectations to political processes (Coleman, 1990; Hedström, 2005). Mathematics provides a scientific basis for drawing conclusions, formulating policy, enhancing the quality of research processes, and making predictions in the social sciences. In this context, mathematics is at the core of communication, reasoning, correlation, and problem-solving in the social sciences as much as in the natural sciences. In this regard, cognitive processes such as decision-making, problem-solving, reasoning, and interpretation in daily life heavily rely on mathematical thinking. Developing these skills is essential for an individual’s success in both social and academic settings (Niss & Højgaard, 2019). Therefore, contemporary mathematics education aims not only to enhance students’ computational abilities but also to equip them with the skills needed to analyse real-life situations, convert these situations into mathematical models, and reinterpret the results. In this framework, mathematics curricula focus on developing individuals’ critical and creative thinking, reasoning, problem-formulating, and solution-generating skills, enabling them to become mathematically literate (Leung et al., 2014; Tesfamicael & Enge, 2024).
In recent years, the concept of mathematical literacy has received considerable attention, particularly through the Organisation for Economic Co-operation and Development’s (OECD) Programme for International Student Assessment (PISA). PISA evaluates students’ abilities to formulating, employing, and interpreting mathematical processes, emphasizing how effectively they can translate real-life problems into mathematical frameworks (OECD, 2023). The ability to formulating is the foundation of mathematical thinking, a critical problem-solving strategy, and an essential component of the modelling process. Research shows that students often struggle specifically in the formulating stage (Tasarib et al., 2025). These findings highlight the need for a more comprehensive approach to teaching the skill of formulating in educational programs.
Background
Formulating is a fundamental thinking skill that allows individuals to structure abstract concepts and develop logical reasoning processes for problem-solving (Mason et al., 2010). The ability to generalise and abstract relationships between events and objects forms the foundation of mathematical thinking. However, converting these relationships into systematic mathematical expressions requires specific formulating skills (Tall, 2013).
For instance, recognising the Fibonacci sequence in nature demonstrates mathematical thinking, while translating it into a symbolic expression requires formulation. In the problem-solving process, formulating is essential for both developing strategies and generating solutions (Polya, 1945; Schoenfeld, 1985). Consequently, formulating plays a crucial role in helping students understand the problems they face, develop paths to solutions, and construct mathematical structures (Luna & Díaz, 2023).
Formulating skill is at the core of mathematics education. It is considered an important component across many approaches, such as problem-solving, mathematical modelling, realistic mathematics education, STEM education, and mathematical literacy (Figure 1). The problem-solving literature emphasises that the process involves formulating the problem and generating a solution (Santos-Trigo, 2024), which aligns with the stages proposed by Polya and Schoenfeld. In mathematical modelling studies, it has also been noted that the formulating process plays a decisive role in modelling real-life situations (Berry & Houston, 1995; Stillman et al., 2007). Similarly, the mathematical literacy framework emphasises individuals’ ability to express, reason, and solve daily-life problems mathematically; it states that formulating skills are at the core of this transformation (MoNE, 2023; OECD, 2013, 2023).
Mathematical processes involving the formulating.
Polya (1945) and Schoenfeld’s (1985) framework for problem solving provides a starting point for understanding the place of formulating in mental processes, but it does not include cognitive aspects such as the formation of the problem situation, how this situation corresponds to the mathematical world, how problems are constructed, and a detailed explanation of the formulating process. Similarly, while Krutetskii’s (1976) analytical descriptions of mathematical abilities are valuable, this model is also limited in explaining the processes of formulating presented in the context of today’s realistic mathematics education, modelling, STEM, and mathematical literacy. Therefore, this study aims to reposition the skill of formulating within the framework of contemporary research, based on classical theoretical approaches.
In the context of realistic mathematics education, formulating skills are used as an effective tool in horizontal and vertical mathematisation processes (Artut & Bal, 2016; Zulkardi, 2000). In the context of STEM education, formulating is a critical stage in integrating science, technology, engineering and mathematics disciplines, defining scientific research questions, and identifying engineering design problems (Kennedy & Odell, 2014). This process enables the transfer of problems in real-life contexts to the mathematical plane and the development of solutions by integrating interdisciplinary content. Similarly, in the ‘computational thinking’ approach, formulating the problem is defined as the first step in generating solutions through a computer (Wing, 2006); in this context, studies conducted within the framework of computational thinking-based mathematical problem-solving activities reveal how necessary the formulating skill is even at the primary education level (Ng & Cui, 2021).
Additionally, research indicates that students’ ability to solve mathematical problems and to engage in mathematical modelling and problem posing largely depends on their formulating skills (Leikin et al., 2025; Santos et al., 2024). Several cognitive challenges have been identified, particularly in understanding the problem context, recognising relationships, and creating appropriate mathematical expressions (Aguirre et al., 2024). Consequently, formulating is closely linked to the development of higher-order thinking skills, such as reasoning, generalisation, and abstraction (Niss & Højgaard, 2019).
Formulating and Proposing a Framework
Mathematical formulating is a high-level cognitive process that enables individuals to transform contextual situations into mathematical structures logically and symbolically. This process relies on fundamental cognitive skills such as abstraction, establishing relationships, recognising structures, and generalisation. Krutetskii (1976) states that formulating is based on students’ ability to recognise structures and relationships within content, while the OECD (2013) defines formulating as an individual’s ability to structure a contextual situation mathematically, thereby explaining its position in mathematical literacy. Within these perspectives, formulating is seen as a fundamental theoretical structure at the core of mathematical thinking.
Formulating process model.
Formulating skills include (a) Mathematical sense-making: making mathematical sense of situations given in a real-world context (OECD, 2023), (b) Structuring and generalising: reaching generalisations in the process of transforming mathematical expressions into a systematic structure (Tall, 2013), (c) Interpretation of Mathematical Abstractions: interpretation of abstract concepts in mathematics to real life world contexts (Blum & Niss, 1991), and (d) Real-life problem creation: transforming the applications of mathematical abstractions in real life into problem situations (Figure 2).
(a)
(b)
(c)
(d)
Summary of the Formulating Process Framework.
Rationale
NCTM (2000) emphasises that problem-solving, reasoning, relating, and communicating should be central to mathematics education. According to the Ministry of National Education (MoNE, 2018), students’ understanding of mathematics in real-life contexts is essential for problem-solving and higher-order thinking skills. These perspectives highlight that today’s mathematics education requires not only the development of procedural skills but also higher-order thinking skills that involve transforming real-life situations into mathematical structures, understanding abstract concepts, and developing new problem-solving scenarios. In this context, the ability to formulate is central to problem-solving processes, along with mathematical literacy (OECD, 2023), which is necessary for transforming real-life situations into mathematical structures, and modelling (Blum & Leiss, 2007), which helps concretise abstract concepts.
The OECD’s (2023) PISA 2022 results show that students in Türkiye experience severe difficulties in mathematical literacy, particularly in formulating, and perform below the average of participating countries. This situation also reveals that teaching practices expected by the curriculum, such as mathematical reasoning, generalisation, and modelling, have not yet been sufficiently developed in practice. Therefore, this issue requires immediate attention. Furthermore, the difficulties students face in translating real-life situations into mathematical structures point to a systemic instructional gap that requires urgent intervention.
The limited research on high school students’ formulating skills highlights the absence of comprehensive scales and frameworks for assessing this ability. Therefore, examining high school students’ formulating skills by focusing on the difficulties they encounter can shed light on current learning outcomes and open the door to designing curricula and teaching processes in this direction. While the literature has identified a lack of scales and theoretical models aimed at improving formulating skills, especially at the high school level, a systematic framework to address this shortcoming has yet to be proposed. Many existing studies often examine isolated components of formulating within the context of mathematical literacy or primarily focus on middle school students. This approach leaves a conceptual and empirical gap regarding the development of formulating skills in older students. The existing literature on formulating has significant gaps, highlighting the urgent need for studies that conceptualise formulating as a multidimensional construct and provide a solid empirical basis for its assessment.
Related Literature and Aim of the Study
Mathematics teaching should provide students with opportunities to approach events from different perspectives, to reflect on their thinking, and to evaluate their procedures critically (Cai & Howson, 2012). Such reflections play a critical role, especially in the process of formulating new problems (Toh et al., 2023). However, studies reveal that students experience various difficulties in formulating problems (Levenberg & Shaham, 2014; Pérez et al., 2017). Ufer et al. (2009) stated that students have difficulty formulating mathematical proofs even at later stages of their schooling, while Pérez et al. (2017) reached similar results at the international level and emphasised that students are unable to formulate mathematical proofs independently. In addition, Ramdhani et al. (2019) stated that students struggle to express generalisations symbolically and develop rules based on patterns.
These difficulties in the formulating process are evident across different age and ability groups. Levenberg and Shaham (2014) found that approximately half of gifted students in grades 4–6 could not formulate verbal problems. Yang and Lin (2015) stated that Taiwanese students lack mathematical literacy, especially when formulating real-life problems, and cannot distinguish between relevant and irrelevant information. Edo et al. (2013), in their study focusing on high-level PISA problems, stated that students had difficulty identifying mathematical structures (patterns, relationships, orders) and formulating mathematical representations of these structures. Similarly, Dewantara et al. (2015) found that formulating was the process in which seventh-grade students made the most errors. Wildani (2020) reported that 47.3% of ninth-grade students’ errors were related to the formulating process. These findings suggest that formulating is a central area of difficulty in mathematical literacy and problem-solving.
Most studies in the literature indicate that high school students experience particular difficulties in formulating. However, research on the proficiency of high school students in the formulating process is limited (Blum & Leiss, 2007). This study offers a unique contribution to mathematics education by proposing an empirically validated multidimensional framework for high school students’ formulating skills. Unlike previous research, which generally treats formulating skills as a single-step cognitive process, this study conceptualises them through four distinct and interrelated dimensions. It operationalises these within a carefully developed assessment tool. The study not only proposes a new theoretical framework but also provides robust empirical evidence from a large, diverse sample, addressing the need for systematic tools and models in this under-researched area. In this regard, the primary objective of the research is to determine the levels of this skill among high school students and to examine how variables such as school type, grade level, and gender affect it.
Formulating is critical to mathematical achievement. Therefore, identifying the factors influencing this process can inform strategic planning to improve students’ competencies in this area. In addition, considering the relationship between formulating and basic concepts such as mathematical literacy, problem solving and modelling, the study’s findings are also expected to guide students’ academic achievement and career development. In line with the data obtained, the aim is to review the curricula, restructure the teaching methods, and contribute to increasing the success of international assessment applications such as PISA.
Research Problem
The fundamental problem of this research: What is the level of high school students’ formulating skills?
Within this scope, answers will be sought to the following sub-research questions (RQ):
Hypotheses
Method
In this study, the survey method was used to determine the levels of high school students’ formulating. Survey design is a non-experimental research approach that quantitatively describes individuals’ attitudes, opinions and tendencies through a sample selected from a specific population (Creswell, 2013). The primary purpose of survey studies is to understand the population’s characteristics and make statistical predictions accordingly (Fowler, 2014). This method, which is generally based on data collection tools such as questionnaires and interviews (Johnson & Christensen, 2012), can be applied cross-sectionally or longitudinally, depending on the time of data collection (Creswell, 2013). This study’s cross-sectional survey design was preferred since the data were collected simultaneously.
As the research aimed to reveal the current state of students’ formulating skills, a cross-sectional survey design was adopted using data collected in a single session. The research structure also incorporates comparative survey features, and comparisons were made of students’ formulating performance by school type, class level, and gender. The research is theoretically structured as a descriptive study focusing on formulating, in line with the OECD’s mathematical literacy framework.
Participants
This study was conducted with students from Anatolian High School, Science High School, Fine Arts High School, and Social Sciences High School in a city centre in the Marmara Region of Türkiye, with permissions obtained from the Ministry of National Education (MoNE). The provincial centre where the research was conducted has two Science High Schools, one Social Sciences High School, one Fine Arts High School, and over ten Anatolian High Schools. In the study, at least one school was selected from each type of school, and the applications were carried out in schools authorised for the research by the provincial directorate of national education. The study was conducted at the high school level because research on formulating skills generally focuses on the middle school level, and there are not enough studies on this age group. In addition, according to Piaget’s cognitive development theory, high school students are in the abstract operations period, and it is accepted that high-level skills such as abstract thinking, problem-solving, and reasoning develop during this period. Since formulating is closely related to these skills, the high school level was determined as an appropriate target group. The fact that PISA is also aimed at the 15-year-old age group is another reason to conduct the study at this level.
There were 431 students from four different high school types: Science High School (108), Social Sciences High School (110), Fine Arts High School (108), and Anatolian High School (105). The 431 students participating in the study were selected from schools authorised to conduct the study under research permits issued by the Ministry of National Education. They comprised students from schools that voluntarily participated, providing the researcher with convenience in conducting the study. The selection of four different types of schools is based on student statistics cited in the National Education Statistics Formal Education 2023/2024 Report, published by MoNE (2024). According to the report, approximately 2 million of the approximately 2.16 million students enrolled in public high schools, excluding open education high school students, attend these four types of high schools. The selected school types cover the types of high schools attended by approximately 93% of students enrolled in secondary education institutions other than open education high schools. Another reason for selecting these school types with different academic orientations was to obtain comparable groups. The sample was selected using convenience sampling, with participants identified from easily accessible individuals (L. Cohen et al., 2005). In this context, the schools participating in the research were selected from institutions easily accessible to researchers. Students at these schools volunteered for the study. In addition, students who were not included in the scale development process were included in the study, and data collection was conducted independently of scale development.
To determine the study sample, statistical methods were used to ensure it was sufficiently representative of the target population, based on the MoNE’s (2024) report, which indicates that approximately 2.16 million students are enrolled in high schools. Calculations were made assuming a 95% confidence level, a 5% margin of error, and an estimated
Additionally, a post-hoc power analysis was conducted to evaluate the statistical adequacy of the study’s sample size of 431 participants (α = .05). The analyses indicate that the power is above 0.99 for medium effect sizes (Cohen’s
Demographic Information of the Participants.
According to Table 2, of the 431 students who participated in the study, 226 were female (52.4%), and 205 were male (47.6%). When the distribution by gender is analysed in terms of school types, 60.2% of Science High School students are female and 39.8% are male; 49.1% of Social Sciences High School students are female and 50.9% are male; 52.8% of Fine Arts High School students are female and 47.2% are male; 47.6% of Anatolian High School students are female and 52.4% are male. These distributions show that a balanced sample was obtained with respect to gender overall.
When the participants were analysed by grade level, it was seen that a similar number of 9th, 10th, 11th and 12th grade students from each school type participated in the study. 108 students from Science High School, 110 students from Social Sciences High School, 108 students from Fine Arts High School and 105 students from Anatolian High School were included in the study. In general, 9th grade students consisted of 107 (24.8%), 10th grade students 106 (24.6%), 11th grade students 107 (24.8%) and 12th grade students 111 (25.8%). This shows that a balanced distribution was achieved across grade levels, with a similar number of students from each grade represented in the sample.
Data Collection Tool
Within the research framework, the Formulating Test (FT) was used to determine high school students’ level of formulating skills.
Formulating Test (FT)
The Formulating Test (FT), which was developed to determine the formulating levels of high school students, was created from PISA mathematical literacy problems focusing on formulating skills and made available by OECD.
Formulating is considered a fundamental component of the PISA mathematical literacy framework, and mathematical literacy problems are classified according to processes such as formulating, employing, interpreting, and reasoning (OECD, 2023). In this context, the preference for problems that directly focus on formulating skills in the creation of the FT increases the test’s suitability for its purpose.
Moreover, the fact that expert teams develop PISA problems ensures that these problems meet high standards of validity and reliability. This important factor makes using these problems in the FT meaningful and valuable. The development process of the FT is visualised in Figure 3.
FT development process.
During the development process of the FT, the necessary permissions were first obtained from OECD for using the problems prepared by OECD in the test, and from the MoNE General Directorate of Measurement, Evaluation and Testing Services for using these problems in Turkish.
After obtaining permission, the mathematical literacy problems used by the OECD in the PISA pilot and primary applications, declassified, made publicly available, and directly related to formulating skills, were analysed in detail. As a result of these analyses, a problem pool containing appropriate problems centred on the formulating process was created.
While determining the relationship of the problems with formulating skills, the content areas and contexts in which they are located, the reports published by OECD (2013) were utilised. Information about the problems used in the draft FT—problem code, name, content area and context—is presented in Table 3.
Information on the Problems in FT.
Validity and Reliability of the Formulating Test
FT developed to determine the formulating levels of high school students consists of items scored on a binary (0–1) scale. As these items are not based on the assumption of continuous variables, they may fail to meet the normality and linearity assumptions required for traditional exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). Therefore, the content validity of the items in the FT was ensured through expert opinion; construct validity was assessed using item statistics. This approach is consistent with the measurement and assessment literature on performance-based tests (DeVellis & Thorpe, 2022; Haynes et al., 1995; Lawshe, 1975).
To ensure the content validity of the draft FT, expert opinions were sought by established criteria (Cronbach & Meehl, 1955; Messick, 1995; Thorndike et al., 1991). The aim was to confirm that the test items adequately represented the measured domain. For this purpose, a form was prepared to collect expert evaluations on the suitability of the 15 problems included in the FT for measuring high school students’ formulating skills. Four field experts—three professors and one assistant professor—evaluated the problems using a 5-point scale. All experts assigned the maximum score of 5, indicating that the problems were appropriate regarding purpose, content, and student level.
To determine the feature’s coverage power relative to the research’s scale items, an expert opinion process must be conducted (Allen & Yen, 2002; McMillan & Schumacher, 2010). This method, widely used in measurement and evaluation, is considered an important step in ensuring content validity (Haynes et al., 1995; Lawshe, 1975). In this context, the opinions of four faculty members (3 Prof. Dr., 1 Asst. Prof.) who are experts in their fields were sought to determine which dimensions the problems in the FT belong to within the framework of the formulating process model. The experts were asked to classify each problem in the FT within the framework of the formulating process model into the dimensions of Mathematical Sense-Making, Structuring and Generalising, Interpretation of Mathematical Abstractions, and Real-Life Problem Creation, and to state their reasons. Each problem was evaluated within the sub-dimension shared by at least three of the four experts. For example, Table 4 shows that in question 1, all experts classified the problem under the sub-dimension of ‘Mathematical sense-making’. In contrast, in question 2, three out of four experts classified the problem under the sub-dimension of ‘Interpretation of mathematical abstractions’. Table 4 presents the dimensions of the formulating process to which the problems in the FT belong, along with examples of the justifications for these classifications.
Classification of FT Based on Expert Opinions in the Context of Subdimensions.
As a result of the expert evaluations conducted, it was determined that six problem fall under ‘mathematical sense-making’ (1, 2, 4, 12, 14, 15), four problem under ‘structuring and generalising’ (3, 6, 10, 13), four problem under ‘interpretation of mathematical abstractions’ (5, 8, 9, 11), and one problem under ‘real-life problem creation’ (7).
Based on expert feedback, the test was deemed suitable for its intended scope and target group, and a pilot study was subsequently conducted. The pilot was administered to 26 tenth-grade students from a high school in the Marmara region of Türkiye. Students were asked to report unclear expressions, spelling errors, or visual issues. The pilot study confirmed that the test items were comprehensible, error-free, and suitable in content and application time. As a result, the full implementation and reliability analysis stages were initiated.
Following permissions from the MoNE, the finalised FT was administered to 561 students from various high school types (Science, Anatolian, Fine Arts, and Social Sciences) across the Marmara and Aegean regions. Reliability was assessed using internal consistency analysis, which evaluates the extent to which test items measure a single construct (Johnson & Christensen, 2012). For dichotomously scored items (1 = correct, 0 = incorrect or blank), the Kuder–Richardson Formula 20 (KR-20) was used to calculate internal consistency (Kuder & Richardson, 1937; Ntumi et al., 2023).
Item scoring in FT followed OECD and MoNE PISA guidelines, where correct answers received 1 point, and incorrect or unanswered items received 0. The KR-20 coefficient was computed based on item difficulty, item discrimination, and overall test consistency (Başol et al., 2013). These statistics, including item difficulty and discrimination indices for each test item, along with the overall KR-20 reliability coefficient, are presented in Table 5.
FT Internal Consistency Analysis Results.
When Table 5 is analysed, it is observed that the item difficulty index of the 15 problems in FT ranges from 0.10 to 0.83. According to Walsh and Betz (1995), the ideal item difficulty range is 0.10–0.90. According to Adıgüzel and Özdoğru (2013),
Focusing on the problems in the FT within the framework of item discrimination, it is seen that the discrimination levels of the problems vary between .31 and .84. According to Towns (2014), in the item discrimination index analysis, if Discrimination Value is >0.40, the item is considered as an excellent item, and if it is between 0.20 and 0.40, it is considered as a good item. Within this framework, four items in the FT are satisfactory, and 11 are excellent in terms of discrimination.
A KR-20 reliability analysis was conducted to assess the reliability of FT. Based on the data analysis of 561 high school students, the KR-20 reliability coefficient was calculated to be 0.84 (α = .84). According to Wallen and Fraenkel (2013), KR-20 reliability coefficient of .70 or above is required to obtain a reliable score. According to the What Works Clearinghouse (WWC) standards, the internal consistency coefficient for educational studies should be greater than .60 (WWC, 2022). Atalay et al. (2015) stated that a KR-20 value between 0.60 and 0.70 is considered acceptable, values between 0.70 and 0.90 are considered good, and values above 0.90 are considered excellent. In this framework, the reliability value obtained is nearly perfect. As a result of the analyses, it is evident that FT is a valid and reliable test for measuring students’ formulating levels.
Data Analyses
The study used FT to determine high school students’ level of formulating skills. The students’ responses to the FT were scored as 1 for correct answers and 0 for incorrect or blank answers, following the scoring method used for PISA problems. The data obtained in this context were transferred to the IBM Statistical Package for Social Sciences for Personal Computers (SPSS 26.0) programme, and statistical analyses were carried out within the research problem framework.
Item response theory and Bloom’s cognitive domain theory were considered in evaluating the FT scores. A criterion-based evaluation approach was adopted to determine the formulating levels, and achievement levels were classified according to objective criteria (Anderson & Krathwohl, 2001; Lok et al., 2016). In this context, high-level students (11–15 points) are expected to show analysis, synthesis and evaluation skills by solving complex problems. Medium-level students (6–10 points) generally answer easy and medium problems correctly and show deficiency in complex problems; this group shows competence in comprehension and application steps. Low-level students (0–5 points) are inadequate in formulating skills and show deficiencies in fundamental knowledge and understanding.
Before the statistical analyses, the data were examined for compliance with normal distribution. This examination aims to determine whether the tests’ assumptions are met and to obtain more valid results by using appropriate alternative tests in the event of possible violations (Can, 2014). Violating the normality assumption may undermine the validity of the results by reducing statistical power (Wells & Hintze, 2007). In educational research, this assumption is assessed using descriptive statistics, graphical methods, and hypothesis tests (Demir et al., 2016). The study examined the normality assumptions by evaluating skewness and kurtosis coefficients using descriptive methods. According to Tabachnick and Fidell (2013), coefficients between −1.5 and +1.5 indicate that the data are normally distributed. In the study, before determining whether there was a significant difference in formulating skills across school type, grade level, and gender variables, the skewness and kurtosis values of the data groups were examined. It was observed that all groups fell within this range, and the assumption of normality was accepted.
The analysis was conducted in accordance with the research’s sub-problems. First, a repeated-measures ANOVA was conducted, and Mauchly’s Sphericity test was performed to examine performance differences across the four sub-dimensions of the formulating model, assuming normality. A Bonferroni correction was used to adjust for multiple comparisons. ANOVA was preferred to determine differences across school type and grade level variables, as the data were normally distributed; however, because the variances were non-homogeneous, Welch ANOVA was used, followed by the Games-Howell post hoc test. Finally, an independent-samples
In addition, the focus was on the effect size in determining the extent to which the type of school and grade level significantly affect the formulating skills of high school students. Effect size is a statistical value that expresses the strength of the relationship between variables (Hedges, 2008) and shows the extent to which the results obtained from the sample deviate from the null hypothesis (J. Cohen, 1994). Today, many educational journals require reporting of effect size in addition to
Ethical Considerations
This research was conducted with the approval from Ministry of National Education and the relevant school administration. The study was conducted with the permission of the Ethics Committee of the Faculty of Science and Engineering at Balikesir University (Protocol No: E-19928322-302.08.01-32076).
Before implementing the applications, participants’ parents were informed and provided consent via the consent form. The students’ statements regarding their voluntary participation in the research were obtained, and they were informed that they could withdraw from the research at any time without giving any reason.
To minimise potential risks, it has been stated that the research was not conducted for evaluation or grading purposes and will not affect students’ academic standing. Data was collected anonymously to ensure participant confidentiality. This study aims to contribute to a better understanding of students’ formulation skills in mathematics education. In this regard, it is believed that the expected scientific and social benefits of the research outweigh the risks that participants may encounter.
Findings
The findings section is structured according to the four sub-problems examined in the research. The results are presented under subheadings, with the relevant analysis methods applied for each sub-problem. This aims to ensure that both the statistical findings and their interpretations are consistent with the problem statement.
Finding on the Comparison of High School Students’ Success Achievement Levels According to the Formulating Process Framework Sub-Dimensions?
Firstly, within the scope of RQ1, it was examined whether there was a significant difference in the success of high school students in the sub-dimensions of the formulating model. The scores obtained by students in FT for each sub-dimension were calculated. Then, multiple ANOVA analyses were used to determine whether there were significant differences in the scores of high school students across the sub-dimensions. The results of the multiple ANOVA analyses are presented in Table 6, along with descriptive findings:
Multiple ANOVA Analysis Results According to Sub-Dimensions in the Formulating Model.
A repeated-measures ANOVA was conducted to examine significant differences in the scores of high school students across the sub-dimensions of the formulating model. Considering Mauchly’s sphericity test, a significant difference was found (χ2(5) = 297.77,
Sub-Dimension Pairwise Comparisons (Bonferroni Corrected).
When examining the descriptive statistics conducted to determine significant differences in high school students’ scores across sub-dimensions, it was observed that students scored highest on the mathematical sense-making sub-dimension (
Findings on the Comparison of High School Students’ Formulating Skills According to School Types
Within the scope of RQ2, it was examined whether high school students’ formulating skills differed by school type. Data from the FT administered to students at Anatolian High School, Fine Arts High School, Science High School, and Social Sciences High School were analysed to determine whether the levels differed significantly by school type. In this regard, ANOVA findings comparing students’ FT scores by school type are presented in Table 8.
ANOVA Analysis Table of FT Scores According to School Type.
When the ANOVA results in Table 8 were analysed, it was found that the variances of the students’ FT scores across school types were not homogeneously distributed (
According to the analyses, significant differences were found between Science High School and Social Sciences and Fine Arts High Schools; between Social Sciences High School and Fine Arts and Anatolian High Schools; and between Anatolian High School and Fine Arts High School. When the mean FT scores were analysed, it was observed that Science High School Students (
When the mean scores were evaluated by school type, the formulating skill levels of students from Science High School, Anatolian High School, and Social Sciences High School were classified as ‘medium’. In contrast, the levels of Fine Arts High School students were classified as ‘low’. These findings show that the development of formulating skills can vary significantly across school types and that school type is an important factor in developing these skills. In addition, the calculated eta squared value (η2 = .46), which is above .14, indicates a high effect size level. This supports the idea that school type is a determinant variable in formulating skills.
Findings on the Comparison of High School Students’ Formulating Skills According to Grade
Within the scope of RQ3, it was examined whether high school students’ formulating skills differed by grade level. The results of the ANOVA analysis, which aim to determine significant differences in formulating skills among high school students by grade level, are presented in Table 9.
ANOVA Analysis Table of FT Scores According to Grade.
According to the ANOVA results in Table 9, the variances of high school students’ FT scores by grade level were not homogeneously distributed (
As a result of the Games–Howell post hoc analysis, which determined which grade levels the difference was between, significant differences were found between the 9th and 10th, and 11th and 12th grades, in favour of the upper grades, respectively. In addition, a significant difference was found between the 11th and 12th grades in favour of the 12th. Generally, the highest level of formulating skill was observed in the 12th grade, and the lowest level in the 9th grade. This finding shows that students’ formulating skills improve as the grade level increases.
When the mean FT scores according to grade level were analysed, it was found that 10th, 11th and 12th grade students (
Findings on the Comparison of High School Students’ Formulating Skills According to Gender
Within the scope of RQ 4, it was analysed whether students’ formulating skills differed by gender. To determine whether there is a significant difference by gender, the independent-samples
As a result of the
Findings on the İnteraction Effect of School Type, Grade Level, and Gender Variables on Formulating Skills
Within the scope of RQ 5, the interaction effect of school type, grade level, and gender on formulating skills was examined. To reveal the relationships among variables, reduce the probability of Type 1 error, and determine whether the combined effects of school type, grade level, and gender on secondary school students’ formulation skills are significant, a three-way between-groups factorial ANOVA was utilised. The results of the between-groups factorial ANOVA are presented in Table 11:
Factorial ANOVA Results for FT Scores by School Type, Grade Level, and Gender.
According to the factorial ANOVA results, school type was found to have a significant effect on students’ formulation skills,
The analysis results indicate that the interaction between school type and grade level is statistically significant,
Discussion
Formulating skill is employed in the processes of mathematical expression of situations in a real-life context (OECD, 2023), abstracting these expressions by transforming them into a systematic structure (Tall, 2013), interpreting and applying abstract concepts to real-life context (Blum & Niss, 1991) and transforming these applications into problem situations. Formulating skill is an important skill not only in mathematical literacy problems but also in the mathematisation process in realistic mathematics education (Artut & Bal, 2016; Demirdöğen & Kaçar, 2010; Zulkardi, 2000), mathematical modelling process (Ang, 2010; Berry & Houston, 1995; Blomhøj & Jensen, 2007; Mason, 1988; Stillman et al., 2007; Tuna et al., 2013; Voskoglou, 2006) and problem-solving process (Temel & Altun, 2022). So, what is the status of high school students in formulating, which is critical for mathematical skills and concepts?
The research revealed that high school students’ formulating skills have a multidimensional structure and that students demonstrate different levels of performance in the sub-dimensions of this skill. Students’ higher scores in the mathematical sense-making dimension indicate that they are relatively more successful in understanding mathematical situations and establishing connections between mathematics and real life. However, the lowest performance is observed in the real-life problem-creation dimension, indicating that students have difficulty transforming their mathematical knowledge into real-life contexts and creating original problems. This supports the finding that formulating was the process in which students struggled most in their mathematical literacy in PISA 2012 and 2022 (OECD, 2014, 2023). Furthermore, another notable finding is that students were more successful at interpreting mathematical abstractions than at structuring and generalising. This parallels the findings of the study by Ellis et al. (2022), which found that students struggled when asked to create a new generalisation; however, they were more successful at relating an existing generalisation to different contexts or extending it to a new domain. Similarly, these results are consistent with Mason’s (2005) study, which revealed that students experience difficulties in producing generalisations.
The findings from the FT indicate that students’ formulating skills in the 10th, 11th, and 12th grades at Science High Schools, Anatolian High Schools, and Social Sciences High Schools are at a medium level. In contrast, those of 9th-grade students and students at Fine Arts High Schools remain low. These results once again indicate that students generally experience difficulties in the formulating process. Similar findings from studies conducted in Türkiye (Karaduman et al., 2023; Özgen, 2019; Ülger et al., 2020) support the difficulty in the formulating process. At the same time, the low correct-answer rates on some problems in the FT (problems 3, 7, 11, etc.) also indicate that students struggle, especially in situations requiring higher-level mathematisation. In this context, it is necessary to develop and expand teaching practices that focus on the formulating process to improve students’ mathematical literacy.
Türkiye’s PISA 2022 results show significant differences in mathematics literacy scores across high school types. Science High Schools achieved the highest performance, while Social Sciences High Schools, Anatolian High Schools, Fine Arts High Schools, and Vocational and Technical Anatolian High Schools showed lower performance, respectively (MoNE, 2023). This situation is consistent with the findings reported by Suna et al. (2020) on differences between school types using PISA 2003–2018 data. The results of the study show that Science High School students are more successful in formulating skills than students from other school types. In contrast, Fine Arts High School students are at the lowest level, consistent with PISA results and reports by Suna et al. (2020). This consistency suggests that school type is a structural determinant of mathematical performance, particularly in formulating skills.
In addition, the fact that formulating skills remain at a medium level even in schools with longer mathematics lesson times and greater programme intensity indicates that this skill is not sufficiently reinforced in teaching. In other words, attributing the difference between school types solely to curriculum intensity is insufficient. The fact that even students in Türkiye’s leading secondary schools have only medium-formulating skills supports the criticism, long emphasised in international assessments such as PISA (OECD, 2014, 2023), that mathematical modelling and mathematisation processes are not sufficiently addressed in teaching. Furthermore, despite the increase in content intensity, the continued limitation of students’ ability to transform real-life situations into mathematical structures indicates that teaching practices remain content-focused rather than process-focused.
The low performance of Fine Arts High School students is consistent with the two fundamental problems highlighted by Döğer and Gayretli (2019) regarding this type of school: (1) the limited time allocated to mathematics teaching in this type of school, and (2) students’ negative attitudes towards numerical subjects. However, the findings go beyond this situation and, when considered alongside studies (Yu & Li, 2022) showing that representation and spatial reasoning skills are critical for both artistic production and formulating processes, suggest that Fine Arts High School students may have a potential cognitive advantage. The fact that this advantage does not translate into performance indicates that the teaching design specific to this type of school does not activate this potential. Therefore, it is thought that the low performance of Fine Arts High School students is related not only to inadequate mathematics education but also to the structure of current teaching models, which do not incorporate interdisciplinary cognitive approaches. This criticism coincides with the reasons why STEAM-based approaches are gaining increasing importance in the international literature (Preciado-Babb et al., 2013).
The study revealed that formulating skills increase with grade level, with 9th-grade students demonstrating the lowest performance and 12th-grade students the highest. This finding is consistent with Wildani’s (2020) findings, which indicate that 9th-grade students in particular experience difficulty in converting verbal problems into mathematical structures. This increase in formulating across grade levels can be attributed to the development of students’ cognitive capacity and academic competence. When evaluated in the context of Piaget’s cognitive development theory, 9th-grade students are transitioning from concrete to formal operations. Therefore, it is expected that students will struggle with higher-level skills such as abstraction, representation creation, and reaching generalisations, such as formulating (Piaget, 1952). However, this explanation is open to criticism because it assumes that cognitive development progresses with age. This is because skills such as mathematical modelling, reasoning, and problem solving are shaped not only by age but also by educational experience and teaching quality (Harel & Sowder, 2007; Schoenfeld, 1992). Therefore, the increase at the grade level should be associated not only with developmental maturation but also with students gaining more experience in mathematical modelling, reasoning, representation, and problem solving in later years. In this context, the low performance at the 9th-grade level can be seen as a reflection of both the cognitive transition period and the lack of adequate learning opportunities to develop mathematical modelling, problem-solving, representation, and formulating skills at this grade level.
The finding that gender has no significant effect on formulating skills indicates that this skill is independent of gender. This finding is consistent with Hyde’s (2005)‘gender similarity hypothesis’, which suggests that gender differences in mathematical ability and similar cognitive domains are generally low. Furthermore, the study by Else-Quest et al. (2010), which provides empirical evidence that gender has no significant effect on mathematical performance in many contexts, is consistent with the present findings. In this respect, the study aligns with the literature, which shows that formulating skills are more closely related to learning experiences and teaching processes than to individual differences.
Conclusion
The results of this study indicate that high school students’ formulating skills are generally inadequate, particularly in the processes of structuring, generalising and real-life problem creation. It was found that formulating skills are independent of gender but vary with factors such as grade level and school type. It was observed that school types with programmes that place greater emphasis on mathematics provide more support for formulating skills. In contrast, in school types where mathematics teaching is limited, these skills remain at a low level. Results at the grade level show that students’ formulating skills develop over the years, but 9th-grade students require more support in developing them.
Recommendations
Recommendations for Educational Applications
The research findings indicate that current teaching programmes are not sufficiently effective in developing formulating skills. Therefore, teaching programmes need to be restructured to include more applications and activities that enable students to transform real-life situations into mathematical structures and generalise them. It is critically important to systematically integrate activities involving representation creation, modelling, problem setting, and structured discussion processes into the programmes. Such activities will not only serve as cognitive tasks that develop students’ formulating skills but will also help them gain experience in interdisciplinary problem-solving.
Interventions aimed at developing formulating skills within the framework of findings should not be limited to adjustments to the teaching programme. Incorporating mathematical modelling, problem-solving, realistic mathematics education, and STEM applications into the curriculum is not sufficient on its own. Teaching processes must be redesigned to activate students’ representation and transformation skills, enabling them to convert everyday situations into mathematical structures. STEM activities and mathematical modelling are considered critical, particularly for contributing to the abstraction and structuring processes students need when dealing with complex problems.
Schools with limited mathematics teaching hours, such as Fine Arts High Schools, should incorporate interdisciplinary teaching practices that integrate art and mathematics. Developing STEAM-focused teaching designs that consider interdisciplinary cognitive levels for such schools can enhance students’ formulating skills. Such activities can advance students’ formulating skills, particularly their representational and spatial reasoning abilities. Furthermore, developing pedagogical strategies to transform negative attitudes towards these subjects in schools with few numerical courses can improve students’ performance in mathematical tasks.
The study’s results indicate that 9th-grade students require more support in their formulating skills than students in other grades. In this context, students starting high school must be supported in their mathematics lessons with teaching strategies that strengthen the processes of establishing mathematical relationships, abstraction and mathematisation. Supporting students through modelling activities that facilitate their transition from concrete situations to abstract concepts and incorporating applications that increase representation diversity into teaching programmes can support the development of the formulating process.
Recommendations for Policy and Programme Development
Differences in the mathematical content between school types at the secondary level can be one factor affecting students’ formulating skills. Particularly in schools with intensive mathematical content, students’ better performance demonstrates the impact of the quality of the teaching programme. Therefore, policymakers need to develop an inclusive programme that focuses on students’ mathematical literacy and formulating skills in these schools, considering the needs of different school types. When redesigning the teaching process for schools with less mathematical content, programme developers are advised to focus on strengthening interdisciplinary integration (mathematics-arts, vocational subjects-mathematics) within a mathematical context. In this regard, it is recommended that teaching content be supported by STEAM-based applications and interdisciplinary connections to develop formulating skills.
The findings also convey a message that teacher training needs to be reviewed. One of the fundamental duties of teachers is to guide students and involve them in the learning process to develop their learning skills (Fullan et al., 2017). In this regard, enhancing teachers’ competence in pedagogical methods that support students’ formulating skills, such as mathematical representation, modelling, mathematisation, problem formulation, and higher-order thinking, may enable more effective development of students’ formulating skills in classroom applications. Therefore, it is recommended that in-service training programmes for teachers be restructured to include comprehensive educational content focused on exemplary practices and interdisciplinary connections that will develop students’ formulating skills.
Recommendations for Researchers
One of the key findings of this study is that students experience difficulties in their formulating skills. In this context, it is recommended that future research focus on the sources of the difficulties students encounter in formulating problems. Within this scope, the reasons for the difficulties students experience in the formulating stage when transferring complex problems into the mathematical world and converting generalisations and abstractions into real-life applications can be investigated in depth through qualitative studies.
Another issue that needs to be addressed in the future is how to develop students’ formulating skills. In this context, researchers can examine the impact of processes that can be incorporated into teaching, such as mathematical literacy problems, modelling-based activities, and problem-solving strategies, on the development of formulating skills. In this regard, it is recommended that modelling activities based on real-life scenarios and instructional processes developed within the framework of different problem-solving strategies be systematically implemented in the early years of high school and that longitudinal studies examining the effects of different teaching approaches on formulating be conducted. Such studies will reveal the development process of the skill and more clearly identify effective intervention strategies.
Another issue that has emerged in the research and awaits further examination in future studies is the structural and pedagogical mechanisms underlying differences between school types. There is a need not only for reporting performance-based differences but also for explanatory models and model development studies that relate these differences to their causes.
Limitations
The FT developed within the framework of the study was constructed with questions that measure formulating skills within the scope of PISA mathematical literacy problems. However, the fact that the vast majority of the problems in the FT cover processes ‘Mathematical sense-making’, ‘Structuring and generalising’ and ‘Interpretation of mathematical abstractions’ of the formulating model in Figure 2 is considered a methodological limitation of the study. The fact that there is only one problem in the real-life problem creation dimension is another limitation of the study. For these reasons, it is suggested that tests that will address the formulating process more broadly and include different problem types should be developed in future research.
Another limitation of the study is that the participants were only students from Science High School, Social Sciences High School, Fine Arts High School and Anatolian High School. This may limit the generalizability of the findings to students studying in different types of high schools. Future studies with a broader range of students may provide a more comprehensive assessment of formulation skills.
The study examined formulating skills measured at a specific time, and this cross-sectional approach provides a limited perspective on students’ developmental processes over time. Therefore, conducting longitudinal studies in future studies will help analyse the changes in students’ formulating skills more comprehensively. Studies based on long-term follow-up may also provide more in-depth information about which learning processes develop these skills.
Finally, the data collection tools and methods used in the study also have some limitations. The data were collected only through FT. In future studies, using more diverse data collection methods could enable formulating skills to be assessed through written and oral performances. In addition, studies that combine quantitative and qualitative research methods could provide a more in-depth understanding of students’ formulating skills.
Footnotes
Acknowledgements
This study utilized generative tools, such as Grammarly, to enhance language proficiency.
Ethical Considerations
Prior to the research, the necessary application permits were obtained in accordance with the letter numbered E-49614598-605.01-25591877 from the Ministry of National Education’s Strategy Development Directorate. Within the scope of this permit, approval was obtained by applying to the relevant school administration. The study was also approved by the Ethics Committee of the Faculty of Science and Engineering at Balıkesir University (Ethics Committee Protocol No: E-19928322-302.08.01-32076).
Consent to Participate
Informed consent was obtained from the parents of the students participating in the study; the students themselves also declared that they participated voluntarily. Participants were informed that they could withdraw from the study at any time without providing any reason. During the study, participants’ identity information was kept confidential in accordance with ethical guidelines, and the data obtained were used solely for scientific purposes.
Consent for Publication
The author takes full responsibility for the claims presented in this article and has given explicit permission for its submission to Sage Open.
Funding
The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Research Fund of Balıkesir University. Project Number: 2021/003.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The data that support the findings of this study are available from the author upon reasonable request.