Abstract
BACKGROUND:
There is strong evidence that cognitive skills and executive functions are skills that children need in order to successfully learn in school. Although executive function disorders are not considered a learning disability, weaknesses in executive functioning are often observed in students with learning disabilities or ADHD. Cognitive games are a type of educational games which focus on enhancing cognitive functioning in children with different profiles of cognitive development, including students with neurocognitive and/or learning disabilities. Self-regulation and metacognitive skills also play an important role in academic performance.
OBJECTIVE:
In this work, we highlight the need of monitoring and supporting metacognitive skills (self-regulation) in the context of a cognitive training game. We propose a system for self-regulated cognitive training for children which supports metacognitive strategies allowing the child to reflect on their own progress, weaknesses and strengths, self-arrange the training content, and thus to promote their self-regulated learning skills.
METHODS:
We provide a narrative review of research in cognitive training, self-regulated learning and explainable recommendation systems for children in educational settings.
RESULTS AND CONCLUSIONS:
Based on the review, an experimental testbed is proposed to explore how transparency, explainability and persuasive strategies can be used to promote self-regulated learning skills in children, considering individual differences on learning abilities, preferences, and needs.
Keywords
Introduction
There is an increasing interest on the benefits of Serious Games (SG) and Games for Health (G4H) to children’s behavior for purposes beyond entertainment, e.g., education and healthcare, game-based learning, interventions to augment therapy and to promote health and well-being [1, 2, 3]. Recent works focus on game design features for cognitive games designed for children, with or without learning disabilities [4]. Motivational design theories have been proposed for game-based cognitive training in children, considering the relationship between user’s emotional state and cognitive skills, as well as the importance of intrinsic motivation and sustained engagement [5]. Such theories highlight basic user needs that should be considered during game design: Competence, Autonomy, and Relatedness. Another study proposed a set of G4H design features for children [6], namely Interactivity, Feedback, Agency or Control, Identity and Immersion. A main challenge is to measure the effects of the game-based training environment (and its design features) both to children’s motivation and training efficiency. For example, a recent study examined how a game-based training environment influences both motivational variables (e.g., willingness to play), as well as executive control during a cognitive training invention in children with ADHD [7]. Considering the long-term interactions of such gamified training systems, a proper selection of design features is required to ensure the positive effects on the intended training purposes [8].
In this paper, we highlight the importance of system transparency and user autonomy, as design features, in the context of a cognitive training game for children which supports metacognitive skills. We review recent works on cognitive and educational training and therapy games for children, as well as works which focus on self-regulated learning skills (SRL) and their impact on learning for children with or without learning disabilities. We also review research works in Explainable AI (XAI) and transparent tutoring agents, focusing on techniques to help the student understand and evaluate the agent’s underlying behaviours and decisions, as well as reflect on their own behavior [9]. We extract guidelines from this review, which are applied in a self-regulated cognitive training system for children. This system supports children in monitoring their performance and self-arranging their training regimen, thus enhancing their SRL skills. We emphasize on these skills as there are indications that SRL-based interventions for children with learning disabilities or ADHD can enhance several self-regulatory skills, such as self-monitoring and planning [10, 11].
Background and related work
Cognitive training for children
There is evidence that executive functioning plays an important role in learning during childhood and it relates to academic skills and competence in mathematics, reading comprehension, vocabulary and writing both in primary and secondary school ages [12]. Although executive function disorders are not considered a learning disability, weaknesses in executive functioning are often observed in the learning profiles of student with learning disabilities or ADHD [13, 14].
Digital cognitive (or brain) training games have been utilized as an educational tool both for typically and atypically developing children. A mathematical and cognitive training application has been proposed for preschool children with Autism Spectrum Disorders (ASD) to enhance their cognitive and social interaction skills [15]. The proposed system includes a set of game-like activities with different levels of difficulty and skills (e.g., counting, color and size recognition, etc.). The child is able to perform these activities in any preferred order and difficulty level. A main goal of such system is to establish an efficient and engaging long-term interaction.A research study focused on the feasibility of a game-based training program for elementary school students [16]. More specifically, they examined the effect of the training environment (school, home), the association of game metrics to cognitive measurements, as well as the relationship between training time and improvement in cognitive abilities. The results highlighted the importance of considering contextual information for the in-home program with regard to child’s participation and interaction with the system, e.g., training frequency and time, parent supervision, etc. The use of attention control to the game elements have shown positive learning outcomes to a mixed class of typically developing children and children with ASD [17, 18], while timely feedback and rewards during an educational game can lead to learning outcomes equal to one-to one training by an expert [19, 20].
Another study introduced a computer-based cognitive training program as an intervention tool to stimulate executive functions through enhancing planning in typically-developing children. The training program, Executive Function Cognitive Enhancement Program, is a set of tasks/games to help the child develop their planning skills [21]. Neurophysiological assessment tasks were used for pre-post evaluation of basic cognitive skills and executive functions. The results showed that the training program was most effective for students with lower pre-training assessment scores, indicating the need of cognitive training in children with lower cognitive skills. Commercial applications for cognitive training in students, such as ACTIVATE1
Self-regulated Learning (SRL) refers to the learner’s abilities to control and evaluate their own learning environment and learning strategies, monitor their performance and assess their strengths and weaknesses [24]. SRL skills allow a learner to self-assess and guide their own learning more effectively. Self-regulated learning consists of three essential components: cognition, metacognition and motivation. Metacognitve skills are related to “learning to learn” skills, including self-monitoring, self-efficacy and goal setting, which allow the learner to effectively self-assess their skills and guide their own learning process. Learners with high SRL skills are able to monitor their current and previous performance and progress, reflect on their skills and set appropriate goals during learning. A recent work has proposed the integration of an Open Learner Model (OLM) in an attention training game to help the users reflect on their progress and evaluate the system’s decisions regarding task and difficulty selection [25]. Open Learner Models have been mainly used in educational contexts and they refer to the system’s visual representation of the learners’ current understanding of a topic, or their level of competency, usually in the form of skill bars. Another work has proposed an OLM-based approach to assist students to develop self-regulated learning skills (SRL) in an educational game with a robotic tutor [26]. The results showed that personalized SRL scaffolding can increase student’s engagement with the system and support appropriate SRL-based behaviors, including self-monitoring and goal setting. Metacognitive skills are essential in any game-based learning, training or therapy system, e.g., educational systems, cognitive training, physical rehabilitation, behavior change systems, etc. Thus, there is a need to formally deign and integrate a metacognitive skills component in game-based learning environments to enhance self-regulated learning in its users [27].
Supporting and enhancing such skills can also benefit students with learning disabilities and/or cognitive impairment. In the context of a school-based intervention program for two female students with ASD [28], a research study focused on the effects of a social skills program which included a self-regulated learning phase (self-monitoring and goal setting). Another study investigated the effects of a self-regulation enhancement training program on neurocognitive and social skills in students with dyscalculia [29]. The results indicated that such training programs can enhance students’ neurocognitive and social skills, as measured with neurophysiological assessment tasks.
Persuasion and explainability in recommendations for children
Despite the fact that recommendation systems have been extensively and successfully applied and evaluated in numerous research and commercial applications, there are still open challenges in designing and evaluating such systems with children. A major challenge is the need for transparent and explainable recommendations to effectively communicate the content to a child [30, 31].
Recommendation systems have been proposed in classroom and educational settings. Xie and colleagues focused on how to integrate personalization techniques to a recommendation system for second language vocabulary learning, focusing on the diversity and readability of the recommendations [32]. Another study analyzed the challenges of a recommendation system for reading materials in a classroom setting which focused on the individual student differences both in preferences and comprehension abilities [33]. Social robots have been used in educational settings to investigate if recommendation explanations can improve cognitive, affective, and perceived learning in second language tutoring in children [34]. Another recent study focuses on how transparency and explainability of an educational recommendation system affects student’s task selection and performance [35]. Recommendation systems have also been applied as a therapy support system for children with ASD [36]. The proposed system is designed to recommend daily activities and therapy tasks for the child, allowing both parents and therapists to monitor the interaction with the system.
BrainHood: game environment and rules. There are four task rules, three difficulty levels and three speed values. Each parameter from the Rule category corresponds to a different addressed cognitive skill. Each combination of these elements results to a different game training specific skills.
Providing “appropriate” recommendations refers to what to recommend, as well as how and when to recommend an item. An online experiment on a real-world platform discusses the persuasive role of explanations suggesting that they are an essential piece of functionality of a recommendation system, since it enhances user’s perception and engagement [37]. Explanations can also enhance user’s decision making regarding whether to choose the recommended item, as well as to assess the potential benefits of following the recommendation [38]. Persuasive profiling is a method to personalize the persuasive strategies used by a system to influence its users [39]. Persuasive systems have been also used in social robots to investigate how users respond socially to persuasive social robots [40], focusing on users’ psychological reactance, liking, trusting beliefs and compliance. Their results showed that interactive social cues and proper timing in social praise can enhance the interaction. Another persuasive system has been proposed to support students with ASD in managing their daily activities and schedule [41]. Their findings showed that teenagers with ASD were likely to prefer receiving persuasive messages from the system rather than their teachers. The study highlights the need to transform traditional educational and training methods into the context of persuasive technology.
In this section, we present BrainHood; a cognitive training game where the child needs to control the archer and shoot the targets that appear on different positions on the screen, following a set of rules defined by a set of task parameters. The game is designed based on the: (a) number of available target/player positions, (b) types of targets, (c) player color, and (d) task rules. Each configuration of these parameters results in a different task which requires specific skills. There are three types of targets (red/green apples, left/right birds), two player colors (green, red), and seven possible positions/spots for targets/player. The purpose of such a parametric game design is to make sure that the possible configurations can cover a wide range of player abilities, skills and preferences. A screenshot of the game environment is shown in Fig. 1.
The child can select the parameters of the next round. The goal of the system is to help the child select appropriate tasks based on their skills and preferences, through the Open Learner Model, persuasive recommendations and explanations.
A prototype version of BrainHood is designed considering three types of task parameters: task rules, task difficulty and task speed. Task Rules. There are four different task types/rules
Each task configuration has a maximum possible score TS , which is proportional to the task complexity and difficulty. For example, configuration T
The main feature of the proposed system is the SRL component for a cognitive training application for children. The SRL component includes features for goal setting (set a target score for a session), self-efficacy (make accurate self-assessments of game skills), and task selection strategies (select the tasks that match their weaknesses and needs). An example of the task selection menu is shown in Fig. 2. The child can choose a task configuration and play the game, following the specific rules. Our research aims to investigate the persuasive roles of transparency and explainable recommendations and their influence on children’s SRL-related behaviors and engagement.
The goal of the proposed system is to enhance the effectiveness of the proposed cognitive training game by supporting the child’s SRL skills. Basic SRL skills relate to self-efficacy, goal setting, and task selection. The system architecture enables the child to monitor their progress, self-assess their weaknesses and strengths and decide which task(s) are more appropriate for them in order to reach their target goal, resulting to a self-guided training session. The system architecture is shown in Fig. 3. At the beginning of each session, the child can set their own target in the form of a total score that they need to reach by the end of the session. Each session consists of
A session consists of multiple rounds of task configurations. For each round, the child is asked to select the next task to perform. Explainable and persuasive recommendations are provided, along with the open learner model, to help the child identify their weaknesses and strengths, select the appropriate tasks and thus maximize the training benefits of the session.
Once a new session starts, the system retrieves the user profile, which includes information about user performance and preferences over the different task parameters, as well as other metrics e.g., training time, number of responses, response time, etc. A user profile is a collection of user models (task performance, task selection, etc.). It can also hold information about user preferences and susceptibility to specific persuasive strategies and explanations (persuasive profiling [39]). The database includes the set of all user profiles, as well as additional information about the users, e.g., frequency of training. Self-reports from both the child and supervisor users (e.g., teachers, parents) can be stored in the database. We will follow a data-driven approach to build prediction models (probability of task success/selection) considering individual differences in performance and preferences.
Different visualizations of Open Learner Models (OLM). OLM is used to help the child monitor their performance. OLM visualizes the overall performance of the child, as well as their performance on each task. Visualizations can include data from previous sessions, as well as personal best scores, to help the child check their progress, select an appropriate next task configuration and self-arrange the session.
From recommendation to persuasive explanation.
A basic feature of the system is the Open Learner Model (OLM). OLM visualizes the progress of the user after each round. More specifically, it shows information for each individual skill, as well as the average of the individual skill scores (total score). In addition, it shows the player’s personal best scores from previous sessions, as well as information from previous rounds. Following the guidelines from previous works that use OLM in educational/training applications, the OLM values are updated after each round with respect to each specific skills that were tested; if a round is not completely successful (i.e., some rules are violated), some skill bars can be increased, while others stay the same. Since each task configuration has its own maximum possible score, selecting the appropriate task configuration is essential towards reaching a target score. An example of different OLM visualizations is shown in Fig. 4. We will investigate different approaches for visualization (skill bars, pie charts, mastery grid) and information (e.g., relative vs. absolute scoring, social comparison), considering user profile (performance and preferences).
Recommendation system
The recommendation system has two main functionalities: (a) recommend a target score at the beginning of the session and (b) suggest appropriate task configurations for the next round within a session. The recommendation system takes into account user preferences (based on previous user selections), user abilities (based on previous task performance), as well as other user and contextual information (time of day, mood, self-reports, etc.). Today’s Goal. At the beginning of a new session, the child can set their own goal for the upcoming session in the form of a total score. The recommendation system takes into consideration previous selected target scores both form the child and other similar users from the database. Next Task. After each round, the system suggests possible task configuration adjustments (e.g., lower difficulty, remove rule, etc.), based on the target goal and how the child performed. The system can also evaluate a user’s selection through the recommendation system. Educational recommendation systems can provide suggestions based on probability models of mastering an activity/task and the possible gain of performing an activity [34]. Other approaches for task recommendation follow the flow theory, which has been used for difficulty adaptation in games both for entertainment and educational purposes [42, 43].
Persuasive explanations model
This module is responsible to deliver the recommendation output in an explainable and persuasive way in order to help the child set their goal and select appropriate tasks. An example is shown in Fig. 5, where the recommendation system outputs the top three recommendations and delivers them following a persuasive strategy (e.g., authority, social proof). The model personalizes the output considering the persuasion profile of the child and contextual information, e.g., training time and frequency. An interesting parameter would be the frequency (timing) of the recommendations during a session and how it affects the child’s decisions. The child can request further explanations on a recommendation. Requesting further explanations could be a behavior indicator related to SRL skills. Data from previous sessions and other users can be used to develop personalized persuasion profiles, which match persuasion/explanation strategies (e.g., authority, reward, social comparison) to user’s performance and preferences.
Future work and discussion
In this paper, we presented BrainHood; a self-guided cognitive training game for children, which supports SRL-related skills (self-monitoring, task selection, goal setting), through the use of an open learner model and persuasive and explainable recommendations. The goal is to motivate the user to customize their training session, evaluate their task selection and performance, providing them with personalized recommendations and prompts to ensure the effectiveness of the training session. The proposed system is especially developed to facilitate experiments that will answer the following questions:
How do transparency, persuasion and explainability affect children’s self-perception, task selection and task performance? More specifically, we are interested in identifying behavior patterns related to SRL skills, based on the use of the Open Learner Model (transparency) and the recommendations (persuasion and explainability). Our next step is to conduct a pilot study to evaluate the game prototype on a focused target population (e.g., 12–15 years old children). Collecting performance data over the different task parameters will help us understand the distribution of task selection strategies and task performance over the different configurations. Moreover, subjective data through built-in questionnaires will be collected (e.g., self-assessment, task preference, engagement levels). The large number of possible task configurations can cover a wide range of user abilities and preferences. Baseline models for the several modules (user profile, database, recommendation and explanations) will be developed in a data-driven way using the collected data, which will include behavioral data (game metrics, task choices, requests for explanations, responses to persuasive strategies, etc.) and subjective data through self-reports using built-in questionnaires. The dataset will be made available online for several research purposes, including user and behavior modeling, recommendation systems, game adaptation, and others. How to design and evaluate personalized persuasive recommendations and explanations to promote self-regulated learning skills in children? More specifically, we are interested in the relationship between different profiles of the child (e.g., cognitive or persuasion profile) and explanation/persuasion methods, towards developing a personalized persuasion strategy. There is evidence that a learner’s cognitive abilities relate to their susceptibility to persuasive strategies [44]. Moreover, design elements can be adapted to user preferences in order to increase engagement [45]. Techniques and theories of behavior change can inform the design of systems which support the development and transfer of SRL skills [46]. Our goal is to design and develop appropriate persuasive recommendations and explanations considering the individual differences and preferences of each child. A main challenge is to measure and evaluate the susceptibility of a child user to a persuasive strategy. The proposed system architecture supports the interaction between non-technical experts (e.g., teachers) and the system modules for monitoring and control purposes how explainability can enhance expert’s decision making, e.g., view and update user profiles, task recommendations and explanations.
Considering the aforementioned research objectives, our proposed evaluation plan includes three phases:
Game prototype evaluation and user experience. An exploratory qualitative and quantitative data analysis on game behavior, performance and survey data will be the outcome of our pilot study. Statistical analysis and visualizations will provide us with insights about the relationship of the collected data. This analysis will inform the design and evaluation of user modeling, clustering methods and prediction models to implement the recommendation system, based on user preferences and performance [47]. Offline evaluation of algorithms and models. The next step includes the evaluation of the user modeling and clustering methods to define user profiles and recommendations/explanations, based on the collected data. Multi-Dimensional Scaling (MDS) and unsupervised clustering (KNN) can be used in order to define a set of user clusters and the corresponding prediction models [48]. An example of a prediction model could be a Bayesian model as the probability of success for a given task given the previous performance. System integration and evaluation. This phase includes the evaluation of the system with real users. The trained user models and recommendation system will be deployed to an integrated system, aiming to address research questions related to the effect of personalized recommendations and persuasive explanations on both user experience and learning outcomes. Comparative studies will be conducted to measure the effect of model transparency, explainable recommendations and persuasive strategies on the child’s behavior in terms of the demonstrated self-regulated learning skills during the interaction.