Sage Journals: Discover world-class research

Abstract

The vast development and expansion of educational technology has led to the deconstruction of various barriers and challenges that the traditional approaches were incapable of overcoming. However, a field that still holds a great deal of interest and needs to be strengthened concerns the practices that surround mathematics education. Digital learning tools can provide a partial solution to this problem. In this study, we introduce a curriculum-driven digital practicing tool which accommodates diverse learning styles and educational needs. The representative sample of the pilot included 135 third-grade primary school students, randomly split in two groups, who participated in the 8-week experiment. The findings suggest that the digital practicing path can facilitate the development of subject mastery and increase the accuracy of arithmetic fact calculations. In addition, the use of learning analytics tools can facilitate the knowledge acquisition process and prevent learners from developing misconceptions toward the learning subject.

Keywords

mathematics primary education learning analytics technology-enhanced learning

Mathematics is an essential skill to learn (Kazanidis & Pellas, 2019). In addition to providing the necessary tools for the everyday life, they also form the base on which many other subjects and skills are built upon (Kelly, 2006; Roy et al., 2017). Educational research in mathematics places the role of the teacher, the design of the task, and the learning environment among the most influential factors that determine learners’ success (Geiger et al., 2010). However, despite the efforts that teachers and instructors put into making the subject less dry and more attractive, students are still facing serious problems and require a significant amount of support to overcome the obstacles they face (Doabler & Fien, 2013). An example of a problem that pupils usually encounter concerns the number of exercises they solve correctly (Chinn, 2013). In the traditional context, however, the teacher can provide only a limited amount of instruction or support to students, whereas the time gap between practicing and provision of feedback is usually long (Cunha et al., 2018). As a result, students usually find themselves in a position that disallows them from iterating their tasks in full prior to progressing to the next topic (Welder, 2006).

The benefits that the integration of information and communications technology (ICT) brings in the educational context are numerous, but the most important is identified on its potential to mediate the gap between the interacting stakeholders. In greater detail, digital learning environments enable teachers to monitor students’ progress and, therefore, provide them with immediate feedback and/or assistance as needed (Brečko et al., 2014; Pérez-Sanagustín et al., 2017). At the same time, students can regulate their own learning strategies and needs in accordance to their intrinsic motivation (Acedo & Hughes, 2014; da Silva Figueira-Sampaio et al., 2009). Nevertheless, a common problem in digital education concerns the lack of software solutions that are explicitly designed and developed in accordance to the needs of the particular curriculum of each educational institution (Kucirkova, 2014).

The framework for mathematics learning that Reinhold et al. (2020) propose describes the fundamental dimensions that should be taken into account when designing interactive learning environments. The parameters, briefly mentioned here, are (a) the integration of the particular curriculum to the digital content, (b) the provision of diverse instruction, and (c) the design of the technology, which should be capable of providing scaffolding for advancement and reflection. In an effort to validate the proposed framework, the authors conducted different experiments using portable devices (i.e., tablet PCs) with the aim to facilitate curriculum teaching of fractions. As the empirical findings were positive and encouraging, it was suggested that additional studies to be done so as to further assess the framework from different learning dimensions and parameters (e.g., embodied activities, adaptivity, automated/immediate feedback).

The Present Study

Motivated by the aforementioned recommendation, in this work, we investigate the educational potential of a digital learning tool—which has been fundamentally designed and developed under the notion of the presented framework—in the context of the elementary school mathematics curriculum. In addition, we compare the skills and the competencies that students develop during the intervention and explore the elements that make such platform successful. The former is realized through the use of the built-in learning analytics (LA) tool that captures and records learners’ interactions with the platform and the involved stakeholders (i.e., instructors, other students).

The aforementioned objectives guide the focus of our investigation and used as a guide to provide preliminary answers to the following research question:
To what extent curriculum-driven educational software improve pupils’ mathematic mastery and arithmetic fluency, compared with the traditional didactic approach?
The remaining of this article is organized as follows. The next section briefly reviews the open research problems and conceptualizes the areas that this study examines. This is followed by a section that details the features of the learning platform and the educational context in which the intervention was conducted as well as the methods, the processes, and the tools used for the data collection and analysis. Next, the primary data are depicted and are discussed in the penultimate section. The final section concludes the article by providing an answer to the research question and recommendations for future studies.

Background and Related Work

The accelerating pace at which ICT has advanced over the past few decades has affected many aspects of our society with almost immediate effects (Grabe & Grabe, 2007). Nevertheless, the educational policies are still adapting at a rather slow pace, while the traditional instructional approaches manifest in the classrooms (Albert & Kim, 2013; Ayinde, 2014). The reasons that underpin the aforementioned issues are manifold. Hiebert and Grouws (2007) identify the root of the problem in the lack of robust theories related to classroom teaching. Others link the traditional, old fashioned, teacher-dependent techniques—which are still dominant in many educational systems—with the inadequacy of the curriculum to facilitate the development of subject mastery and literacy in a coherent, consistent, and progressive manner (Contreras, 2014; Daniel, 2017; Maaß & Artigue, 2013; Ozdamli et al., 2013). Another inadequacy concerns the lack of personalized learning strategies and, therefore, the provision of support to those individuals who are in need. Despite the fact that personalized learning is regarded as a highly contributing factor toward learners’ educational advancement (Deed et al., 2014; Johnson, 2004), the difficulty to implement this model in practice makes the current situation even more problematic (Courcier, 2007). A proposed solution to mitigate the aforementioned issues considers the embodiment of ICT applications in the traditional classroom (Drigas et al., 2014; Ghavifekr et al., 2012).

Traditional Education and ICT

Τhe evolutionary emergence of ICT sparked researchers’ interest to explore, assess, and address how technology influences, enhances, or enriches learning (Cheung & Slavin, 2012; Erdogdu & Erdogdu, 2015; Ghavifekr et al., 2014; Luu & Freeman, 2011; Yuen & Hew, 2018). Nevertheless, the less prominent outcomes have divided the opinion of the scientific community as to whether (educational) technology genuinely contributes to the advancement of knowledge. To this end, the supporters of the technology-enhanced learning approach argue over the dynamic and multidimensional nature of the available e-Learning tools to sustain motivation (Hussain et al., 2011), improve the teaching and learning quality (Bingimlas, 2009; Hamidi et al., 2011), and facilitate the attainment of distinct learning outcomes (Bower, 2017). On the other hand, the more skeptical adopters proclaim that such technologies have the potential to bring better learning outcomes only if the interventions are designed and developed under the guidelines of established learning theories and models (Mayes & de Freitas, 2007). This claim is aligned with the conclusions drawn by other researchers who suggest that, despite the increase of the digital educational tools, there is no significant impact on learners’ experience or advancement (Geiger et al., 2010; Lameras & Moumoutzis, 2015).

The argument of the debate, however, is not concerning only the contradictory results. As Postholm (2007) advocates, studies related to educational technology are usually reporting empirical observations and fail to disclosure the theories that have been utilized to conceptualize the research context or the frameworks that informed the methodological techniques. As a result, a portion of researchers has subjected the outcomes of such studies under strict critique (Bennett & Oliver, 2011; Bulfin et al., 2014; Jones & Czerniewicz, 2011; Markauskaite & Reimann, 2014). Theorized and evidence-based research is, indeed, essential to understand the elements and the variables that contribute toward students’ learning growth. Accomplishing this goal will enable educators to track students’ needs and policy makers to transform the curriculum planning (Hiebert & Grouws, 2007).

Nevertheless, for these objectives to be achieved, a deeper and more thorough understanding related to students’ personal learning style, needs, preferences, and misconceptions is required. In that view, however, the traditional classroom setting does not provide fertile ground for the collection of broad-scale data due to the practical limitations that such practices involve. An approach to deal with this challenge is by utilizing digital learning tools that have been fundamentally developed under the principles of well-established learning theories and advanced in accordance to the guidelines of enacted instructional design frameworks and models (Kucirkova, 2014; Pepin et al., 2017).

Digital Instructional Design for Mathematics

Research related to mathematics education demonstrates a supportive stance toward the augmentation of the traditional practices through the use of digital tools (Bray & Tangney, 2017; Bussi et al., 2000). To this end, researchers (e.g., Artigue, 2002; Oates, 2011; Olive et al., 2009) distinguish the added value of the digital learning approach into two broad categories: (a) the pragmatic, which regards the efficiency of the tool to facilitate the speed and the accuracy of the computations and (b) the epistemic, which concerns the knowledge that learners acquire and the ways they develop deep understanding and subject mastery.

Nevertheless, in order for such tools to be effective, special attention should be paid on the design elements of the digital learning environment. Not surprisingly, different studies highlight different variables. According to Rodríguez-Aflecht et al. (2018), the technology should be flexible enough so as to account the different approaches that learners may utilize toward the solution finding and capable of facilitating the reflection process upon completion (Heirdsfield, 2011). Others (e.g., Arnab et al., 2015; Mulligan & Mitchelmore, 2013; Threlfall, 2009; Verschaffel et al., 2009) focus on the practical elements that are responsible for the mental and cognitive engagement of learners (e.g., the utilization of gamification techniques to integrate the curriculum, the visual representation of the exercises, the escalation of the difficulty level). In any case, researchers and educators share the same opinion as far as the aim of such technologies is concerned by emphasizing on the need for change from the teacher-centered learning model to the student-centric learning approach where learners can control their goals and monitor their progress (Hoyles & Lagrange, 2010; Wright, 2010). The prerequisite for this change, however, involves the active involvement and continue participation of the teachers for the design of the exercises in accordance to the curriculum changes (Brezovszky et al., 2019). An approach to achieve this outcome, at least on the national level, is suggested by Laakso et al. (2018) who emphasize on the importance of cross-school cooperation and content sharing.

Self-Regulated Learning

Enabling learners to monitor and regulate their learning is a critical factor for educational success (Panadero & Järvelä, 2015). As part of the 21st century skills movement, self-regulated learning is thought to be among the most important competences that students need to develop (Shute, 2011; Sottilare et al., 2014; Zimmerman, 2008). In the literature, self-regulated learning is described as an active and goal-oriented process driven by learners’ thoughts, feelings, and strategic actions (Panadero & Järvelä, 2015). To this end, students who regulate their learning needs and carefully plan their learning tasks are considered to be more successful and confident than others (Zimmerman & Schunk, 2011). However, this is not generally the norm as students are not generally successful at self-regulation (Azevedo & Aleven, 2013).

In computer-based learning environments, learners’ behaviors can be traced and monitored in an unobtrusive fashion (Greene & Azevedo, 2010). These traces can be then utilized to provide students with personalized scaffolds or feedback (Molenaar et al., 2019). However, computer-based learning environments are often described as static technologies as they are not sensitive to students’ learning characteristics (Walker et al., 2014), a shortcoming that also affects the inadequacy of such tools to adapt to individuals’ capacity to regulate their learning (Zimmerman & Schunk, 2011).

An approach to overcome these obstacles considers the employment of students’ online learning traces (e.g., information selection, learning strategies, task planning) as the means to make inferences about their learning needs and patterns (e.g., Azevedo & Aleven, 2013; Walker et al., 2014). These traces are, subsequently, used to predict students’ self-regulatory processes and, thus, academic success (Molenaar et al., 2019).

Automated Assessment

Examining students’ knowledge and skills is a core element of the educational system (Pellegrino, 2014). Assessments, as a process to aid learning, can take various forms (e.g., formative, informative, summative) and levels of complexity (e.g., correctness of the exercise, style of the proposed solution, authenticity of the content; Black & Wiliam, 1998; Joy et al., 2005). However, the practical issues that surround this process (e.g., repetition of the task, extensive use of human resources) have led to its (partial) automation (Biggam, 2010). The benefits of automatic assessment are numerous (e.g., continuous diagnostic information, accelerated results, better use of human resources; Barana & Marchisio, 2016) but so are the drawbacks (e.g., lack of dynamic analysis, low accuracy or precision of correctness in problem-solving exercises; Bey et al., 2018). With that in mind, the full potential of this approach—at least for the time being—is realized mainly in the context of practicing exercises (Pelkola et al., 2018).

Automated Feedback

Educators are often combining multiple approaches to construct their feedback (e.g., explanatory, guided, corrective, suggestive, epistemic) which is then distributed to students during the different phases of the learning process (e.g., task planning, progression check, task completion; Alvarez et al., 2012; Guasch et al., 2013). Nevertheless, according to Irons (2007), the effectiveness of feedback on learning decreases as the response time increases. Automated generated feedback can mitigate this issue by offering learners the opportunity to monitor and regulate their performance. In doing so, students’ intrinsic motivation to complete the task increases, and their confidence to progress is boosted (Schaap, 2011). Nevertheless, the field of automated feedback is still relatively unexplored, especially when it comes to the typologies or the strategies that are utilized to identify individuals’ needs (Sedrakyan et al., 2020). Therein, additional research toward this direction is needed to maximize the potential of targeted feedback and ensure that learners’ personal goals and objectives are fulfilled.

Learning Analytics

Researchers have attempted to define LA from different perspectives and angles. Some consider them as an innovative approach to collect student-generated data, which can be subsequently utilized to provide personalized learning experiences (Junco & Clem, 2015; Xing et al., 2015), while others focus on the patterns that can be developed—in accordance to students’ learning behaviors—so as to inform the future developmental decisions of the learning environment (Drachsler & Kalz, 2016; Rubel & Jones, 2016). Long and Siemens (2011) propose a definition that rounds this topic from both perspectives. The authors describe LA as a method to collect longitudinal educational data and a process that utilizes the collected data to optimize learning and the environment in which it occurs. Regardless of the chosen definition, the significance of LA as the means to identify diverse learning needs and improve the present educational practices is undeniable, though their full potential is yet to be explored and understood.

Research Design

The Digital Learning Environment

The LA team at the University of Turku, in collaboration with Finnish school teachers, has created for the online platform Eduten Playground a digital learning path that covers the first nine educational grades (ages 6 to 15). Eduten Playground (in Finland known as ViLLE) is an exercise-based learning environment for mathematics, programming, and languages practicing (Laakso et al., 2018). There are currently almost 150 different exercises, based on the Finnish curriculum, with full support for automatic assessment and immediate feedback. Educators can also create and share additional content or exercises through the teacher’s dashboard function. Each lesson consists of 25 to 30 exercises that contain 10 to 15 mathematical subtasks (Figure 1, top-left frame). The exercises, as well as the whole curriculum, can be easily customized by the teachers so as to meet the structure of the national curriculum and students’ needs (Figure 1, bottom frame). The learning paths utilize gamification elements, such as virtual trophies and awards, aligned to the educational level and learners’ age (Figure 1, top-right frame).

Figure 1.
Teachers’ Dashboard for Task Assigning With Adjustable Difficulty Levels.

Figure 2 illustrates a sample of the different instructional design elements that have been used for this experiment along with a brief explanation that is provided later.

Figure 2.
Example Exercises of the Digital Learning Path for Mathematics.

The forest (top-left snap): a traditional exercise where the students have to fill in the missing number. A total of 10 questions, with varied difficulty levels, are presented to students, and the answers are checked by the system automatically. In cases where an incorrect submission has been made, the correct value is presented before moving to the next set of questions.

The kingdom (top-right snap): a problem-solving exercise, which combines numbers and symbols, where the students have to deduce which number each of the symbols represents. The aim of this exercise is to train and evaluate pupils’ analytical thinking. A total of 25 exercises are available, the difficulty of which is increasing gradually. To prevent guessing, different symbols may or may not have the same value.

The riddle (middle snap): At the end of each learning subject, a complex mathematical challenge becomes available. In order for these bonus exercises to be unlocked, the student should have a cumulative sum of 95% on all the previous tasks (school/homework). Likewise the previous cases, the difficulty of these exercises increases incrementally.

The clock (bottom-left snap): a drag and drop exercise where the students have to connect the different formats of time. Feedback is provided to students upon the submission of the responses.

The racer (bottom-right snap): a gamified exercise that is used to train students’ fluency in mathematic calculations. Each round has 20 questions, and a total of 5 mistakes (virtual lives) are allowed. The speed of the virtual car is increasing progressively, but students can accelerate or decelerate it without, however, being able to immobilize it completely.

In addition, the platform presents a comprehensive LA feature that allows teachers to collect, in real-time, important information about their students’ actions. The performance statistical data—such as the number of completed exercises, the response times, the scores—are presented in a visualized way so as to facilitate the monitoring process of students’ progression in a timely and effortless manner (Figure 3, bottom frame).

Figure 3.
Dashboard for Monitoring Students’ Progress and Misconceptions.

Moreover, the embedded artificial intelligence engine detects and highlights students’ misconceptions in different mathematical topics and enables educators to provide additional mentoring or support to those students who are in need (Figure 3, top frame). The real-time detection of misconceptions has been compared with the Finnish standardized test in mathematics, and the results shown that it can detect them equally reliably (Lokkila et al., 2015).

Research Context

The present study was conducted in the context of a joint effort between the University of Turku, Finland and the Ministry of Education of the United Arab Emirates. The research team of the Centre for LA coordinated the conduct of the intervention in different ways (e.g., by providing training and technical support to the local schoolteachers, overseeing the experiment process) and further performed the data collection and interpretation.

For the needs of this experiment, two elementary schools in Dubai were volunteered to participate with their third-grade pupils. Each school joined the initiative with two classes, and each class was assigned randomly a different group role (i.e., control and treatment). Students who were in the control groups continued undertaking their mathematics classes following the traditional way, while the treatment groups had one of their weekly mathematics classes transformed into an electronic lesson using the digital exercises from the Eduten Playground platform. This electronic lesson was the only difference between the treatment and the control group and, therefore, the independent variable to measure the effects of this intervention.

Research Method

To contextualize the study from different viewpoints and perspectives, students’ competence in mathematics was examined prior to the start and after the completion of the experiment using two identical mathematical tests. The knowledge-testing exercises were designed from the researchers of the University of Turku, in collaboration with the participating teachers, prior to the initiation of the intervention. The conduct of the experiment was commenced right after the completion of the pretests and concluded 8 weeks later with the distribution of the posttests. All the tests were answered using pen and paper to avoid handing an advantage to the treatment group. Students were also requested to disclosure their name in order to satisfy requests for withdrawal. Figure 4 provides an overview of the experimental process.

Figure 4.
Overview of the Experiment Process.

Research Material

The mathematics performance test was based on the same norm that other standardized tests have and aimed at evaluating students’ skills in predefined topics of mathematics. The test consisted of seven sections, and each section contained a varied number of subexercises that were assessed as correct (1 point) or incorrect (0 points). The maximum score from the whole test was 36 points, and students had 45 minutes to complete it. Accordingly, the arithmetic fluency test challenged students’ readiness to complete as many as possible calculations (e.g., addition, subtraction, multiplication, division) within a finite timeframe (90 seconds). For this test, 160 exercises were presented to students and each correct answer was worth 1 point.

Results Presentation and Discussion

In total, 160 students presented parental consent to participate in the study. However, only those pupils who answered both the pre- and the postintervention test were considered in the data analysis process, and thus, the sample of the study was reduced to 135. Of these, 70 represented the control group and the rest (65) the treatment group (Figure 5).

Figure 5.
The Sample of the Study.

To preserve the experimental validity and ensure that the student cohorts had equivalent prior knowledge in mathematics, the preintervention scores were statistically examined using Levene’s test for homogeneity of variance. The homogeneity test indicated that the results did not achieve statistical significance, and thus, additional tests for statistical comparison could be performed. Prior to proceeding, it should be noted that the test scores have been scaled (lowest: 0 and maximum: 1) so as to ease the presentation and discussion process.

Pupils’ Prior and Background Knowledge

Both student cohorts of each school showed similar capabilities in terms of mathematic competence (Table 1) and arithmetic fluency (Table 2). However, in order for this observation to be validated, a two-tailed independent-samples t test was performed. The test confirmed the initial assumption (p > .05), and thus, we can confidently conclude that all the cohorts were equally knowledgeable and skillful prior to the conduct of the intervention.

Table 1.
Preintervention Test Results of Pupils’ Mathematic Performance.

No. Group M Min Max Mdn SD p

School 1 Control (n= 18) 0.49 0.14 0.72 0.51 0.17 .76

School 1 Treatment (n=17) 0.50 0.19 0.83 0.47 0.19

School 2 Control (n = 52) 0.51 0.17 0.93 0.53 0.18 .54

School 2 Treatment (n = 48) 0.49 0.00 0.93 0.47 0.18

Table 2.
Preintervention Test Results of Pupils’ Arithmetic Fluency.

No. Group M Min Max Mdn SD p

School 1 Control 0.34 0.18 0.5 0.35 0.09 .87

School 1 Treatment 0.34 0.17 0.65 0.31 0.12

School 2 Control 0.29 0.11 0.63 0.26 0.11 .96

School 2 Treatment 0.29 0.03 0.56 0.27 0.11

Regarding the mathematics tests per se, the relatively small distribution of students’ scores and the mean values indicate that the majority of the pupils were average-versed. Not surprisingly, though, both schools had both high (School 1: 17%, School 2: 19%, n = 35) and low (School 1: 15%, School 2: 13%, n = 100) achieving pupils. As high achievers, we classify those students who answered correctly more than two of three of the given exercises, while under the low-achievers category, we consider those who scored less than 33% (total score). A noteworthy case concerns an individual who did not provide any correct answer.

Likewise, the mathematics mastery test, a similar picture is drawn with regard to pupils’ capacity in performing arithmetic calculations. In this case, however, the scores are considerably lower (below average). Nevertheless, this outcome can be attributed only partially to students’ skills as the difficulty to complete the test is also a decisive factor (see Research Material section).

Pupils’ Constructed Knowledge

Upon completion of the intervention, pupils’ mathematical and arithmetic fluency skills were reevaluated. As expected, an overall positive difference in the performance of all the groups is observed with the treatment group having the leading edge in both tests (Tables 3 and 4). However, when comparing the impact of the conventional teaching method to the digital learning approach, the postintervention t test yielded statistically significant results only for the mathematic mastery test and only for the cohorts of School 2. Nevertheless, as the sample size of School 1 is relatively small (n = 35), any statistical conclusions should be treated with caution.

Table 3.
Postintervention Test Results of Pupils’ Mathematic Performance.

No. Group M Min Max Mdn SD p

School 1 Control (n = 18) 0.55 0.22 0.8 0.61 0.15 .32

School 1 Treatment (n = 17) 0.60 0.36 0.89 0.61 0.15

School 2 Control (n = 52) 0.58 0.17 0.86 0.58 1.82 .037*

School 2 Treatment (n = 48) 0.65 0.3 0.89 0.65 1.83

Table 4.
Postintervention Test Results of Pupils’ Arithmetic Fluency.

No. Group M Min Max Mdn SD p (fluency) Errors (average) p (errors)

School 1 Control 0.43 0.18 0.71 0.425 0.17 .81 2.05 .001*

School 1 Treatment 0.44 0.22 0.71 0.42 0.17 0.16

School 2 Control 0.35 0.09 0.71 0.33 0.14 .62 1.25 .04*

School 2 Treatment 0.36 0.14 0.76 0.35 0.14 0.68

As far as the arithmetic fluency test results are concerned, special attention should also be given to the difference between the number of incorrect answers that the pupils of each cohort had. As Table 4 indicates, the treatment groups of both schools made considerably and statistically significant fewer mistakes than their fellow students. As a result, a positive impact of this approach—at least—on the accuracy and precision of the calculations can be observed.

To this end, in an effort to increase the statistical power and, therefore, the validity of this experiment, we combine the scores of the individual cohorts from both schools and treat them homogenously (Table 5). Under this consideration, the impact of the intervention on pupils’ mathematic knowledge development is further validated, while the added value of this tool on students’ arithmetic skills remains limited or, at best, inconclusive.

Table 5.
Pretest and Posttest Results Combined.

No. Group
Mathematics performance
Arithmetic fluency

Schools 1 and 2 M p M p

Preintervention Control 0.52 .33 0.30 .92

Treatment 0.50 0.30

Postintervention Control 0.64 .02* 0.36 .57

Treatment 0.57 0.38

Given the short-term duration of the intervention, a direct comparison related to pupils’ knowledge acquisition and skills’ enhancement was deemed noteworthy. The treatment groups had more radical advancement during the course of the experiment. Subsequently, it can be speculated that a greater—and possibly statistically significant—improvement may as well be observed in the case of the arithmetic fluency development. This assumption is further grounded after considering the findings from a similar study which was conducted in the context of three schools in Lithuania (Kurvinen et al., 2019).

Conclusions and Future Work Direction

In the present study, we contrasted the educational potential of a curriculum-driven digital tool with the learning outcomes that students in the traditional classroom context achieved. From the presented evidences, it can be concluded that the construction of mathematical knowledge and understanding—as with any other subject—is not a privilege for the few gifted or talented students but a cognitive skill that can be developed and improved under the presence of certain conditions (e.g., diverse collection of gamified exercises, automated assessment and feedback tools, large variety of instructional design methods/techniques, and so on).

Eduten Playground brings together various elements that can promote learners’ self-efficacy and regulation of learning, increase motivation, and strengthen performance (Figure 6). In addition, the freedom offered to teachers to create and share their own exercises—tailored to their needs and teaching approach—differentiates the aforementioned platform from other available solutions that are more restrictive when it comes to content customization and personalization. To this end, the inclusion of an LA dashboard further enables teachers to identify students who are not making adequate progress and, thus, are in need of additional support.

Figure 6.
Overview of the Eduten Playground Features and the Links Between the Interacting Stakeholders.

Notwithstanding the foregoing, the significantly improved learning gains that students demonstrated, despite the short duration of the intervention, makes apparent that, for such a tool to reach its full potential, a longer time investment is needed so as to compliment further students’ effort and, thus, educational achievements. The takeaway message from this work suggests that the inclusion of such a tool, from as early as the primary school level, can greatly prevent the development of wrong conceptions and set the foundations for continuous and sustainable knowledge development.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Fire Fellowship Grant provided by KAUTE Foundation (Finland) for the needs of the project: Adaptive Electronic Study Path in Mathematics.

No.	Group	M	Min	Max	Mdn	SD	p
School 1	Control (n= 18)	0.49	0.14	0.72	0.51	0.17	.76
School 1	Treatment (n=17)	0.50	0.19	0.83	0.47	0.19
School 2	Control (n = 52)	0.51	0.17	0.93	0.53	0.18	.54
School 2	Treatment (n = 48)	0.49	0.00	0.93	0.47	0.18

No.	Group	M	Min	Max	Mdn	SD	p
School 1	Control	0.34	0.18	0.5	0.35	0.09	.87
School 1	Treatment	0.34	0.17	0.65	0.31	0.12
School 2	Control	0.29	0.11	0.63	0.26	0.11	.96
School 2	Treatment	0.29	0.03	0.56	0.27	0.11

No.	Group	M	Min	Max	Mdn	SD	p
School 1	Control (n = 18)	0.55	0.22	0.8	0.61	0.15	.32
School 1	Treatment (n = 17)	0.60	0.36	0.89	0.61	0.15
School 2	Control (n = 52)	0.58	0.17	0.86	0.58	1.82	.037*
School 2	Treatment (n = 48)	0.65	0.3	0.89	0.65	1.83

No.	Group	M	Min	Max	Mdn	SD	p (fluency)	Errors (average)	p (errors)
School 1	Control	0.43	0.18	0.71	0.425	0.17	.81	2.05	.001*
School 1	Treatment	0.44	0.22	0.71	0.42	0.17	0.16
School 2	Control	0.35	0.09	0.71	0.33	0.14	.62	1.25	.04*
School 2	Treatment	0.36	0.14	0.76	0.35	0.14	0.68

No.	Group	Mathematics performance	Arithmetic fluency
Preintervention	Control	0.52	.33	0.30	.92
Treatment	0.50	0.30
Postintervention	Control	0.64	.02*	0.36	.57
Treatment	0.57	0.38

Footnotes

ORCID iD

Athanasios Christopoulos

Author Biographies

Athanasios Christopoulos, PhD, Computer Science, has over 7 years of hands-on experience in teaching Computer Science & Engineering courses in HEIs. During his service he was also responsible for the support and guidance of several cross-university collaborations with educational institutions located in developing countries. Besides his teaching duties, his research activities led to the formation of a Hybrid Virtual Learning model on the basis on which different instructional design techniques were explored and evaluated. The educational outcomes of his work have been published in several peer-reviewed scientific journals and conference proceedings. Dr Christopoulos is currently working for the Centre for Learning Analytics, University of Turku (Finland) as a Postdoctoral Researcher where he investigates matters related to educational technology advancement and Artificial Intelligence techniques for educational data mining.

Henri Kajasilta, MSc, Applied Mathematics, is the main statistician of the Centre of Learning Analytics, University of Turku, where he works with diverse and large datasets of educational data. To date, he has been part in several projects which involved different Learning Analytics themes that aimed to utilise the learners´ data to its full potential. In addition, his responsibilities also include teaching and developmentof innovations to measure students´ learning performance.

Tapio Salakoski, PhD, Tech., is a Professor of Computer Science at University of Turku, where he received his Ph.D. in 1997. His main research focus has been in methodology development using machine learning and other intelligent techniques. He is leading a multidisciplinary research group studying various task domains, including problems related to human learning and computing education research.

Mikko-Jussi Laakso, PhD, Tech., leads the Centre for Learning Analytics, which has a key role in executing the research project. He has published some 100 international refereed journal papers, conference papers, and other publications. Laakso has been working in the field of educationaltechnology and learning, especially automatic assessment accompanied with immediate feedback, software visualisation, and learning management systems. Laakso held a visiting scholar position (January 2009 – June 2011) at several universities in Melbourne, Australia; namely Monash University, Melbourne University and RMIT University. Laakso has also worked in the ministry of education in United Arab Emirates (UAE), has built a national data centre, and participated in developing teacher education in UAE.

References

Acedo

Hughes

(2014). Principles for learning and competences in the 21st-century curriculum. Prospects, 44(4), 503–525.

Albert

L. R.

Kim

(2013). Developing creativity through collaborative problem solving. Journal of Mathematics Education at Teachers College, 4(2), 32–38.

Alvarez

Espasa

Guasch

(2012). The value of feedback in improving collaborative writing assignments in an online learning environment. Studies in Higher Education, 37, 387–400.

Arnab

Lim

Carvalho

M. B.

Bellotti

de Freitas

Louchart

, De Gloria, A. (2015). Mapping learning and game mechanics for serious games analysis. British Journal of Educational Technology, 46(2), 391–411.

Artigue

(2002). Learning mathematics in a CAS environment: The genesis of a reflection about instrumentation and the dialectics between technical and conceptual work. International Journal of Computers for Mathematical Learning, 7(3), 245–274.

Ayinde

O. M.

(2014). Impact of instructional object based card game on learning mathematics: Instructional design nettle. Middle Eastern & African Journal of Educational Research, 2014(8), 4–18.

Azevedo

Aleven

(2013). International handbook of metacognition and learning technologies (Vol. 26). Springer.

Barana

Marchisio

(2016). Ten good reasons to adopt an automated formative assessment model for learning and teaching mathematics and scientific disciplines. Procedia-Social and Behavioral Sciences, 228, 608–613.

Bennett

Oliver

(2011). Talking back to theory: The missed opportunities in learning technology research. Research in Learning Technology, 19(3), 179–189.

10.

Bey

Jermann

Dillenbourg

(2018). A comparison between two automatic assessment approaches for programming: An empirical study on MOOCs. Journal of Educational Technology & Society, 21(2), 259–272.

11.

Biggam

(2010). Using automated assessment feedback to enhance the quality of student learning in universities: A case study. In M. D. Lytras, P. O. De Pablos, D. Avison, J. Sipior, Q. Jin, W. Leal, L. Uden, M. Thomas, & S. Cervai, D. Horner (Eds.), International conference on technology enhanced learning (pp. 188–194). Springer.

12.

Bingimlas

K. A.

(2009). Barriers to the successful integration of ICT in teaching and learning environments: A review of the literature. Eurasia Journal of Mathematics, Science and Technology Education, 5(3), 235–245.

13.

Black

Wiliam

(1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74.

14.

Bower

(2017). Technology integration as an educational imperative. In M. Bower (Ed.), Design of technology-enhanced learning: Integrating research and practice (pp. 1–16). Emerald Publishing Limited.

15.

Bray

Tangney

(2017). Technology usage in mathematics education research–A systematic review of recent trends. Computers & Education, 114, 255–273.

16.

Brečko

B. N.

Kampylis

Punie

(2014). Mainstreaming ICT-enabled innovation in education and training in Europe system level: Policy actions for sustainability, scalability and impact at system level (JRC Scientific and Policy Reports). Joint Research Centre. http://doi.org/10.2788/52088

17.

Brezovszky

McMullen

Veermans

Hannula-Sormunen

M. M.

Rodríguez-Aflecht

Pongsakdi

, … Lehtinen

(2019). Effects of a mathematics game-based learning environment on primary school students’ adaptive number knowledge. Computers & Education, 128, 63–74.

18.

Bulfin

Henderson

Johnson

N. F.

Selwyn

(2014). Methodological capacity within the field of “educational technology” research: An initial investigation. British Journal of Educational Technology, 45(3), 403–414.

19.

Bussi

M. G. B.

Chiappini

Paola

Reggiani

Robutti

(2000). Teaching and learning mathematics with tools. Research and Teacher Training in Mathematics Education in Italy, 2003, 138–169.

20.

Cheung

A. C. K.

Slavin

R. E.

(2012). How features of educational technology applications affect student reading outcomes: A meta-analysis. Educational Research Review, 7(3), 198–215.

21.

Chinn

(2013). Is the population really woefully bad at maths? Mathematics Teaching, 232, 25–28.

22.

Contreras

(2014). Where is the treasure? Ask interactive geometry software! Journal of Mathematics Education at Teachers College, 5(1), 35–40.

23.

Courcier

(2007). Teachers’ perceptions of personalised learning. Evaluation & Research in Education, 20(2), 59–80.

24.

Cunha

Rosário

Núñez

J. C.

Nunes

A. R.

Moreira

Nunes

(2018). Homework Feedback Is …: Elementary and middle school teachers’ conceptions of homework feedback. Frontiers in Psychology, 9, 32.

25.

Daniel

M. F.

(2017). Primary school curriculum to foster thinking about mathematics. In M. A. Peters (Ed.), Encyclopedia of educational philosophy and theory (pp. 29–40). Springer.

26.

da Silva Figueira-Sampaio

dos Santos

E. E. F.

Carrijo

G. A.

(2009). A constructivist computational tool to assist in learning primary school mathematical equations. Computers & Education, 53(2), 484–492.

27.

Deed

Lesko

T. M.

Lovejoy

(2014). Teacher adaptation to personalized learning spaces. Teacher Development, 18(3), 369–383.

28.

Doabler

C. T.

Fien

(2013). Explicit mathematics instruction: What teachers can do for teaching students with mathematics difficulties. Intervention in School and Clinic, 48(5), 276–285.

29.

Drachsler

Kalz

(2016). The MOOC and learning analytics innovative cycle (MOLAC): A reflective summary of ongoing research and its challenges. Journal of Computer Assisted Learning, 32, 281–290.

30.

Drigas

A. S.

Ioannidou

R. E.

Kokkalia

Lytras

M. D.

(2014). ICTs, mobile learning and social media to enhance learning for attention difficulties. Journal of Universal Computer Science, 20(10), 1499–1510.

31.

Erdogdu

(2015). The impact of access to ICT, student background and school/home environment on academic success of students in Turkey: An international comparative analysis. Computers & Education, 82, 26–49.

32.

Geiger

Faragher

Goos

(2010). CAS-enabled technologies as ‘agents provocateurs’ in teaching and learning mathematical modelling in secondary school classrooms. Mathematics Education Research Journal, 22(2), 48–68.

33.

Ghavifekr

Afshari

Salleh

(2012). Management strategies for E-learning system as the core component of systemic change: A qualitative analysis. Life Science Journal, 9(3), 2190–2196.

34.

Ghavifekr

Razak

A. Z. A.

Ghani

M. F. A.

Ran

N. Y.

Meixi

Tengyue

(2014). ICT integration in education: Incorporation for teaching & learning improvement. Malaysian Online Journal of Educational Technology, 2(2), 24–45.

35.

Grabe

(2007). Integrating technology for meaningful learning (5th ed.). Houghton Mifflin.

36.

Greene

J. A.

Azevedo

(2010). The measurement of learners’ self-regulated cognitive and metacognitive processes while using computer-based learning environments. Educational Psychologist, 45(4), 203–209.

37.

Guasch

Espasa

Alvarez

I. M.

Kirschner

P. A.

(2013). Effects of feedback on collaborative writing in an online learning environment. Distance Education, 34(3), 324–338.

38.

Hamidi

Meshkat

Rezaee

Jafari

(2011). Information technology in education. Procedia Computer Science, 3, 369–373.

39.

Heirdsfield

A. M.

(2011). Teaching mental computation strategies in early mathematics. Young Children, 66(2), 96–102.

40.

Hiebert

Grouws

D. A.

(2007). The effects of classroom mathematics teaching on students’ learning. In F. Lester (Ed.), Second handbook of research on mathematics teaching and learning (Vol. 1, pp. 371–404). Information Age.

41.

Hoyles

Lagrange

J. B.

(2010). Mathematics education and technology: Rethinking the terrain. Springer.

42.

Hussain

A. J.

Morgan

Al-Jumeily

(2011). How does ICT affect teachings and learning within school education. In A. Hussain, H. Tawfik, A. K. Nagar, & O. Abuelmaatti (Eds.), Developments in E-systems engineering (DeSE) (pp. 250–254). IEEE.

43.

Irons

(2007). Enhancing learning through formative assessment and feedback. Routledge.

44.

Johnson

(2004). Personalised learning: New directions for schools? New Economy, 11(4), 224–228.

45.

Jones

Czerniewicz

(2011). Theory in learning technology. Research in Learning Technology, 19(3), 173–177.

46.

Joy

Griffiths

Boyatt

(2005). The boss online submission and assessment system. Journal on Educational Resources in Computing, 5(3), 2.

47.

Junco

Clem

(2015). Predicting course outcomes with digital textbook usage data. The Internet and Higher Education, 27, 54–63.

48.

Kazanidis

Pellas

(2019). Developing and assessing augmented reality applications for mathematics with trainee instructional media designers: An exploratory study on user experience. Journal of Universal Computer Science (J.UCS). Special Issue: Immersive Learning Technologies: Research and Future Directions, 25(5), 489–514.

49.

Kelly

C. A.

(2006). Using manipulatives in mathematical problem solving: A performance-based analysis. The Montana Enthusiast, 3(2), 184–193.

50.

Kucirkova, N. (2014). iPads in early education: separating assumptions and evidence. Frontiers in psychology, 5, 715.

51.

Kurvinen, E., Kaila, E., Kajasilta, H., & Laakso, M. J. (2019, May). Teachers’ Perceptions of Digital Learning Path in Mathematics, Languages and Programming. In 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) (pp. 643–648). IEEE.

52.

Laakso

M. J.

Kaila

Rajala

(2018). ViLLE – Collaborative education tool: Designing and utilizing an exercise-based learning environment. Education and Information Technologies, 23(4), 1655–1676.

53.

Lameras

Moumoutzis

(2015). Towards the gamification of inquiry-based flipped teaching of mathematics a conceptual analysis and framework. In Proceedings of the 2015 international conference on interactive mobile communication technologies and learning (IMCL) (pp. 343–347). IEEE.

54.

Lokkila

Kurvinen

Kaila

Laakso

M. J.

(2015). Automatic recognition of student misconceptions in primary school mathematics. In L. Gómez Chova, A. López Martínez, I. Candel Torres (Eds.), Proceedings of the 7th International Conference on Education and New Learning Technologies (pp. 2267–2273). IATED.

55.

Long

Siemens

(2011). Penetrating the fog: Analytics in learning and education. Educause Review, 46(5), 31–40.

56.

Luu

Freeman

J. G.

(2011). An analysis of the relationship between information and communication technology (ICT) and scientific literacy in Canada and Australia. Computers & Education, 56(4), 1072–1082.

57.

Maaß

Artigue

(2013). Implementation of inquiry-based learning in day-to-day teaching: A synthesis. ZDM, 45(6), 779–795.

58.

Markauskaite

Reimann

(2014). Editorial: E-research for education: Applied, methodological and critical perspectives. British Journal of Educational Technology, 45(3), 385–391.

59.

Mayes

de Freitas

(2007). Learning and e-learning: The role of theory. In H. Beetham, & R. Sharpe (Eds.), Rethinking pedagogy for a digital age (pp. 33–45). Routledge.

60.

Molenaar

Horvers

Baker

R. S.

(2019). What can moment-by-moment learning curves tell about students’ self-regulated learning? Learning and Instruction. Advance online publication. https://doi.org/10.1016/j.learninstruc.2019.05.003

61.

Mulligan

J. T.

Mitchelmore

M. C.

(2013). Early awareness of mathematical pattern and structure. In L. D. English & J. T. Mulligan (Eds.), Reconceptualising early mathematics learning (pp. 29–45). Springer.

62.

Oates

(2011). Sustaining integrated technology in undergraduate mathematics. International Journal of Mathematical Education in Science and Technology, 42(6), 709–721. http://doi.org/10.1080/0020739x.2011.575238

63.

Olive

Makar

Hoyos

Kor

L. K.

Kosheleva

Strässer

(2009). Mathematical knowledge and practices resulting from access to digital technologies. In H. Celia, & L. Jean-Baptiste (Eds.), Mathematics education and technology-rethinking the terrain (pp. 133–177). Springer.

64.

Ozdamli

Karabey

Nizamoglu

(2013). The effect of technology supported collaborative learning settings on behaviour of students towards mathematics learning. Procedia-Social and Behavioral Sciences, 83, 1063–1067.

65.

Panadero

Järvelä

(2015). Socially shared regulation of learning: A review. European Psychologist, 20(3), 190–203. https://doi.org/10.1027/1016-9040/a000226

66.

Pelkola, T., Rasila, A., & Sangwin, C. (2018). Investigating Bloom's Learning for Mastery in Mathematics with Online Assessment. Informatics in Education, 17(2), 363–380.

67.

Pellegrino

J. W.

(2014). Assessment as a positive influence on 21st century teaching and learning: A systems approach to progress. Psicología Educativa, 20(2), 65–77.

68.

Pepin

Choppin

Ruthven

Sinclair

(2017). Digital curriculum resources in mathematics education: Foundations for change. ZDM, 49(5), 645–661. https://doi.org/10.1007/s11858-017-0879-z

69.

Pérez-Sanagustín

Hilliger

Alario-Hoyos

Delgado Kloos

Rayyan

(2017). H-MOOC framework: Reusing MOOCs for hybrid education. Journal of Computing in Higher Education, 29(1), 47–64.

70.

Postholm

M. B.

(2007). The advantages and disadvantages of using ICT as a mediating artefact in classrooms compared to alternative tools. Teachers and Teaching: Theory and Practice, 13(6), 587–599.

71.

Reinhold

Hoch

Werner

Richter-Gebert

Reiss

(2020). Learning fractions with and without educational technology: What matters for high-achieving and low-achieving students? Learning and Instruction, 65, 101264.

72.

Rodríguez-Aflecht

Jaakkola

Pongsakdi

Hannula-Sormunen

, … Lehtinen

(2018). The development of situational interest during a digital mathematics game. Journal of Computer Assisted Learning, 34(3), 259–268.

73.

Roy

Fueyo

Vahey

(2017). Supporting middle grades mathematics teachers and students: A curricular activity system used in an urban school district. RMLE Online, 40(2), 1–15.

74.

Rubel

Jones

(2016). Student privacy in learning analytics: An information ethics perspective. The Information Society, 32(2), 143–159.

75.

Schaap

(2011). Students’ personal professional theories in vocational education: Developing a knowledge base [PhD thesis]. Utrecht University.

76.

Sedrakyan

Malmberg

Verbert

Järvelä

Kirschner

P. A.

(2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior. 107. Article ID 105512. https://doi.org/10.1016/j.chb.2018.05.004

77.

Shute

V. J.

(2011). Stealth assessment in computer-based games to support learning. Computer Games and Instruction, 55(2), 503–524.

78.

Sottilare

R. A.

Graesser

. B

Goldberg

(Eds.). (2014). Design recommendations for intelligent tutoring systems: Volume 2-instructional management (Vol. 2). US Army Research Laboratory.

79.

Threlfall

(2009). Strategies and flexibility in mental calculation. ZDM, 41(5), 541–555.

80.

Verschaffel

Luwel

Torbeyns

Van Dooren

(2009). Conceptualizing, investigating, and enhancing adaptive expertise in elementary mathematics education. European Journal of Psychology of Education, 24(3), 335–359.

81.

Walker

Rummel

Koedinger

K. R.

(2014). Adaptive intelligent support to improve peer tutoring in algebra. International Journal of Artificial Intelligence in Education, 24(1), 33–61.

82.

Welder

R. M.

(2006). Prerequisite knowledge for the learning of algebra. Conference on Statistics, Mathematics and Related Fields, January, Honolulu, HI, United States.

83.

Wright

(2010). Orchestrating the instruments: Integrating ICT in the secondary mathematics classroom through handheld technology networks. Technology, Pedagogy and Education, 19(2), 277–284.

84.

Xing

Guo

Petakovic

Goggins

(2015). Participation-based student final performance prediction model through interpretable genetic programming: Integrating learning analytics, educational data mining and theory. Computers in Human Behavior, 47, 168–181.

85.

Yuen

A. H.

Hew

T. K. F.

(2018). Information and communication technology in educational policies in the Asian region. In J. Voogt, G. Knezek, R. Christensen, & K. W. Lai (Eds.), Handbook of information technology in primary and secondary education (pp. 1–20). Springer.

86.

Zimmerman

B. J.

(2008). Investigating self-regulation and motivation: Historical background, methodological developments, and future prospects. American Educational Research Journal, 45(1), 166–183.

87.

Zimmerman

B. J.

Schunk

D. H.

(2011). Self-regulated learning and performance: An introduction and an overview. In D. H. Schunk, & B. Zimmerman (Eds.), Handbook of self-regulation of learning and performance (pp. 15–26). Routledge.

No.	Group	Mathematics performance		Arithmetic fluency
Schools 1 and 2	Group	M	p	M	p
Preintervention	Control	0.52	.33	0.30	.92
Preintervention	Treatment	0.50	.33	0.30	.92
Postintervention	Control	0.64	.02*	0.36	.57
Postintervention	Treatment	0.57	.02*	0.38	.57

Limits and Virtues of Educational Technology in Elementary School Mathematics

Abstract

Keywords

The Present Study

Background and Related Work

Traditional Education and ICT

Digital Instructional Design for Mathematics

Self-Regulated Learning

Automated Assessment

Automated Feedback

Learning Analytics

Research Design

The Digital Learning Environment

Research Context

Research Method

Research Material

Results Presentation and Discussion

Pupils’ Prior and Background Knowledge

Pupils’ Constructed Knowledge

Conclusions and Future Work Direction

Declaration of Conflicting Interests

Funding

Footnotes

ORCID iD

Author Biographies

References