Promoting Self-Regulation Progress and Knowledge Construction in Blended Learning via ChatGPT-Based Learning Aid

Abstract

This study combines ChatGPT, Apple’s Shortcuts, and LINE to create the ChatGPT-based Intelligent Learning Aid (CILA), aiming to enhance self-regulation progress and knowledge construction in blended learning. CILA offers real-time, convergent information to learners’ inquiries, as opposed to traditional Google search engine that provide divergent information. By addressing questions promptly, CILA minimizes interruptions during the performance phase of self-regulation progress. The tool records learners’ questions and answers, aiding self-reflection in self-regulation progress. We evaluated self-regulation progress using motivation, engagement, and self-efficacy as indicators. Findings show that CILA’s intervention effectively improves self-regulation progress and knowledge construction, offering benefits over divergent information in blended learning contexts with respect to amotivation, intrinsic motivation, and behavioral engagement. This research highlights the potential of incorporating large language models like ChatGPT in educational settings to support teachers and students.

Introduction

The digitization of traditional educational methods has propelled a major shift towards blended learning paradigms, a pedagogical approach which synergizes in-person instruction with online learning. This transformation has been largely driven by global events such as the COVID-19 pandemic, which necessitated educational institutions globally to rapidly adopt online teaching strategies to combat the crisis (Kaffenberger, 2021; Mali and Lim, 2021; Pokhrel and Chhetri, 2021). While such instructional transitions offer numerous advantages, such as flexibility and access to a vast range of digital resources, they concurrently present several obstacles for learners. One of the most notable challenges pertains to self-regulation, a critical determinant of academic success in blended learning scenarios (López-Fernández et al., 2021; Rasheed et al., 2020; Vanslambrouck et al., 2019).

Self-regulation refers to the process by which students actively supervise, regulate, and guide their learning activities using various cognitive and behavioral techniques. It’s an integral facet of blended learning that demands a higher degree of learner autonomy in navigating the learning process compared to traditional teaching methods (Zheng et al., 2020; Zimmerman, 2000). A fundamental challenge linked to self-regulation concerns the evolving role of teachers in facilitating learning, with implications for student motivation, engagement, and self-efficacy. These factors, collectively, shape the overall effectiveness of self-regulation in blended learning settings (Kim et al., 2019; Vanslambrouck et al., 2019).

In attempts to navigate these challenges, students often resort to digital tools like Google to acquire necessary information. However, the sheer volume of information available on such platforms can lead to cognitive overload, as learners grapple with discerning reliable and relevant sources. This often augments the demand for more focused and specific information, termed as “convergent information”, as opposed to the diverse and wide-ranging “divergent information” furnished by search engines such as Google (Shimodaira et al., 2006). Advances in artificial intelligence have led to the development of large language models like ChatGPT, which provide more precise and contextually relevant answers to learners’ inquiries, offering the sought-after convergent information.

To investigate the capabilities of such AI models in augmenting self-regulation and knowledge construction in blended learning, we have devised a ChatGPT-based Intelligent Learning Aid (CILA). Our study seeks to contrast the effects of CILA with traditional search engines like Google within a blended learning framework. The primary research questions guiding our study are:

1. How significantly does the supply of convergent information via CILA influence students’ self-regulation progression in blended learning compared to Google’s provision of divergent information?

2. How significantly does the supply of convergent information via CILA impact students’ knowledge construction in blended learning compared to Google’s provision of divergent information?

3. How significantly does the self-regulation process in blended learning affect knowledge construction?

Following the introduction, we first delve into a comprehensive literature review to contextualize our study within the larger body of research. Then, we discuss our proposed methodology before presenting our findings and discussing their implications. Finally, we conclude with a summary of our results, their potential applications, and prospects for future research.

Related Work

Blended Learning

In recent years, the COVID-19 pandemic has challenged the traditional face-to-face teaching mode, and blended learning has emerged as the mainstream teaching approach during this crisis (Kaffenberger, 2021; Pokhrel and Chhetri, 2021). Blended learning is an innovative teaching method that combines traditional face-to-face instruction with online learning (Hrastinski, 2019). In this mode, students can opt for in-class or online learning, gaining a more comprehensive and diverse learning experience. This approach merges real-world practicality with the convenience of digital technology, ultimately achieving the goal of enhancing students’ learning efficiency and satisfaction (Rasheed et al., 2020).

Blended learning integrates the advantages of traditional and online learning. Some studies have demonstrated that the implementation of blended learning can effectively improve students’ learning outcomes. For instance, Kundu et al. (2021) introduced blended learning in a fourth-grade classroom in India, and after nine weeks of instruction, discovered that blended learning significantly increased students’ learning engagement. When taught by educators familiar with blended learning, it can further encourage active learning among students. Martínez et al. (2019) recommended incorporating a vast array of already available online learning resources into blended learning to ensure course continuity. The strategies suffice in transforming traditional face-to-face teaching into blended instruction, thereby enhancing students’ willingness and performance in learning and reducing dropout rates. This demonstrates the benefits of blended learning for students.

Although the advantages of blended learning for students are evident, there remain challenges in its implementation within the educational field. Blended learning enables students to determine their learning methods and pace and provides them with suitable learning resources. Consequently, blended learning offers students more opportunities for self-directed learning to acquire a deeper understanding and develop skills. However, since blended learning provides students with greater opportunities for self-directed learning, learners may easily become overwhelmed or face difficulties and give up. According to a systematic review article by Rasheed et al. (2020), challenges that students may encounter in blended learning can be broadly classified into five categories: self-regulation, technological literacy and competency, student isolation, technological sufficiency, and technological complexity. The following elaborates on these five categories:

• Self-regulation challenges: Due to the high degree of autonomy offered by blended learning, students often devote most of their time to unrelated activities during remote learning, resulting in reduced time spent on coursework and leading to procrastination.

• Technological literacy and competency challenges: Blended learning relies heavily on the support of information and communication technology (ICT) tools; therefore, students may struggle to quickly master the ICT tools utilized in class.

• Student isolation challenges: In synchronous blended instruction, many students may lack motivation due to feelings of isolation, which could lead to difficulties in engaging with course content, submitting assignments, and increased anxiety.

• Technological sufficiency challenges: Blended learning necessitates the use of advanced ICT tools; thus, the widespread availability of the required hardware and software for blended instruction is a critical consideration.

• Technological complexity challenges: The varying degrees of difficulty in using diverse learning programs result in students spending more time learning how to navigate online resources.

In recent years, advancements and widespread adoption of Information and Communication Technology (ICT) tools have addressed challenges related to technology and student isolation (Kardipah and Wibawa, 2020; Montgomery et al., 2019). Nonetheless, the inherent nature of blended learning grants learners a high degree of autonomy, which consequently positions self-regulation as a primary challenge in contemporary blended learning environments. Thus, this study aims to explore potential solutions to the self-regulation challenge in blended learning.

Self-Regulation Progress

Self-regulation refers to learners being able to control, monitor, and regulate their own learning process independently to achieve learning goals (Pintrich, 2000; Zimmerman, 2008). The most well-known self-regulation model currently is proposed by Zimmerman (2000), which divides self-regulation into three phases. The first phase is the forethought phase, which is the earliest and most important phase in the learning process. Learners first analyze the learning task, set goals and develop methods to achieve them. During this process, learners often need to adjust their learning motivation to ensure they have enough motivation to complete the learning task. The second phase is the performance phase, which is the phase where learners actually engage in learning activities. Learners need to actively participate in learning activities, monitor and adjust their own learning behavior to ensure that they achieve the expected learning goals. The final phase is the self-reflection phase, which is the last phase in the learning process. Learners need to review and evaluate their own effectiveness and learning outcomes to gain more learning experience and knowledge.

Zimmerman’s model of self-regulated learning, with its emphasis on forethought, performance, and self-reflection phases, forms a solid foundation for understanding this complex process. However, as Panadero (2017) suggest, other models provide complementary viewpoints, enhancing our understanding of self-regulated learning from different angles. For instance, Boekaerts et al. (1999) underscore the significance of goals and emotions in self-regulated learning, presenting an emotional perspective often overlooked. Winne (2011) offers another lens, viewing the process as a sequence of iterative cycles where learners continuously monitor and adjust their strategies, illuminating the cyclical and adaptable nature of self-regulated learning.

Similar to Zimmerman’s approach, Pintrich (1995) emphasizes the role of motivation but places an increased focus on the context. His model draws attention to the influence of environmental factors in shaping self-regulation strategies. Efklides (2011), meanwhile, concentrates on the metacognitive elements of self-regulated learning, elucidating the learners’ awareness of their own cognition. Lastly, Järvelä et al. (2013) accentuate the collaborative aspect, revealing that learning isn’t an isolated endeavor but a process often improved through group interactions.

These multifaceted perspectives on self-regulated learning demonstrate that a robust model shouldn’t rely excessively on a single approach. Instead, it should integrate valuable insights from various models, ultimately enhancing and supporting the learner’s unique journey (Panadero, 2017). Thus, a successful application of self-regulation should incorporate Zimmerman’s foundational phases of forethought, performance, and self-reflection, but also consider emotions, context, metacognition, and collaboration, as outlined in the six SRL models above (Panadero, 2017). Indeed, this comprehensive integration is crucial for constructing an effective and encompassing self-regulated learning framework.

Consequently, effective self-regulation requires learners to maintain their motivation in the forethought phase (Pintrich, 2000; Zimmerman, 2000, 2008), to actively engage in learning activities in the performance phase (Bernardo et al., 2022; Doo and Bonk, 2020), and to deepen their self-efficacy through reflection and review of the learning process in the self-reflection phase (Rabin et al., 2020; Stephen et al., 2020). Furthermore, numerous studies have also confirmed the importance of learning motivation, learning engagement, and self-efficacy in self-regulation (Doo and Bonk, 2020; Vanslambrouck et al., 2019; Yu et al., 2022). Therefore, in this study, motivation, engagement, and self-efficacy will be used as indicators to measure the progress of self-regulation.

On the other hand, compared with traditional face-to-face teaching, learners in blended learning environments often receive less support and guidance. Therefore, how to effectively integrate self-regulation has become a key factor in the success or failure of blended learning (Kizilcec et al., 2017). Numerous studies have confirmed the benefits of self-regulation for blended learning. For example, Moreno-Marcos et al. (2020) demonstrated the positive effects of self-regulation behavior on predicting dropout in blended learning environments. Rasheed et al. (2021) developed a self-regulation scaffolding in blended learning environments to enhance learners’ engagement and significantly improve their academic performance.

However, due to the lack of support and guidance in blended learning environments compared with traditional face-to-face teaching, learners often face difficulties that cannot be resolved, leading to low motivation and engagement, as well as decreased self-efficacy and failure (Kim et al., 2019; Kizilcec et al., 2017; Wert et al., 2021). To improve the situation, students typically use Google search engine to solve problems and difficulties encountered (Isda et al., 2021; Mracek, 2019). Although the Google search engine can help learners address problems and difficulties, the information is often scattered and disorganized, requiring learners to interrupt learning to retrieve information, thereby affecting self-regulation progress. In view of this, this study designs a learning aid based on ChatGPT in blended learning and investigates whether the aid can effectively enhance students’ self-regulation progress and knowledge construction compared with traditional Google search engine.

Large Language Models

A large language model is a type of model trained on billions of parameters or weights, allowing it to learn from vast amounts of data and generate new sentences. It is commonly employed in natural language processing (NLP) and comprises three components: perception, understanding, and generation. The model initially processes the sentence through “tokenization” to convert it into a machine-readable format, then analyzes the sentence’s features, such as syntax and semantics, to optimize the training outcome. Finally, it generates human-readable and comprehensible natural language (Carlini et al., 2021).

The Transformer is the most widely used deep learning model for training language models, supplanting earlier NLP approaches that frequently utilized Long Short Term Memory (LSTM) and other Recurrent Neural Networks (RNN) models (Vaswani et al., 2017). Subsequently, two primary pre-training models emerged: the BERT model (Ettinger, 2020) and the GPT model (Radford et al., 2018). Large-scale pre-training has enabled GPT to excel, with its third generation incorporating training with billions of parameters, making it the network model with the highest number of parameters to date. The introduction of GPT-3.5, trained through instructional learning, has elevated model performance even further. ChatGPT is based on GPT-3.5 and incorporates more comprehensive reinforcement learning from human feedback (RLHF) training, reducing most ineffective (useless, biased) outputs and aligning the results more closely with human expectations.

The responses generated by ChatGPT can not only help organize key points but also function as a translation tool and perform basic program checks. Recently, ChatGPT has been employed in numerous instructional research studies, such as Baidoo-Anu and Owusu Ansah (2023), who investigated how ChatGPT fosters personalized and interactive learning in education. Kung et al. (2023) discovered that ChatGPT had a remarkable answering rate on the United States Medical Licensing Examination (USMLE), indicating its potential in medical education. Kasneci et al. (2023) demonstrated that ChatGPT can provide accurate explanations and guide learners step by step to comprehend the underlying reasons for problems, offering high school students personalized and effective learning experiences.

Considering ChatGPT’s exceptional performance, this study develops a learning aid for blended learning based on ChatGPT, providing students with immediate and helpful support and addressing their questions. The study also investigates whether this tool can aid students in knowledge construction during learning compared with the traditional Google search engine.

The Design of ChatGPT-Based Intelligent Learning Aid (CILA)

To effectively address learners’ questions in blended learning environments, we have integrated ChatGPT, Apple’s Shortcuts, and the instant messenger LINE to develop an intelligent learning aid specifically tailored for such settings. This learning aid leverages the large language model ChatGPT to address students’ queries and employs Apple’s Shortcuts to facilitate seamless communication among Apple devices, ChatGPT, and the instant messenger LINE. Ultimately, previously asked questions are archived in the LINE, allowing students to review and reflect on course content. The system workflow is illustrated in Figure 1. Subsequent sections provide comprehensive explanations of the workflow.

Figure 1.

The workflow of CILA.

Apple’s Shortcuts

The Shortcuts feature in Apple devices enables users to create custom programs for automated control of apps on their Apple devices. The fundamental components of Shortcuts are actions, each representing a single task, such as sending a message, playing music, or checking weather forecasts. Users can combine multiple actions to create custom shortcuts, allowing them to perform specific tasks. Moreover, the developed program facilitates voice calls through Apple’s voice assistant, Siri, enhancing user interaction. Owing to the similarity between Shortcuts program’s coding and visual programming, developers can effortlessly complete development without requiring programming training. Users can conveniently test and execute the developed programs on Apple devices, streamlining the development process.

Shortcuts can be created by dragging and combining various program blocks to achieve the desired functionality. The development interface is displayed in Figure 2. Users can set the name and logo of the program at the top of Figure 2. Uniquely, users only need to call the program’s name, allowing them to use Siri’s voice command to execute the program. On the right side of Figure 2, users can select available functions and application capabilities. Users can choose the required program blocks to complete the desired functionality of the program. On the left side of Figure 2 is the workspace for this program. Users can drag the chosen program blocks from the right side into the workspace to arrange the sequence and logic of the program’s execution, ultimately completing the program’s creation.

Figure 2.

Development interface of Shortcuts.

It is important to note that we adopt the Shortcuts method to integrate ChatGPT functionality, as opposed to the conventional Chatbot and ChatGPT integration. The rationale behind this is that when constructing a Chatbot, backend services must be deployed on a Serverless hosting platform, with Vercel being the most popular platform. This platform can only wait for a response time of 10 seconds, implying that if the question asked is more challenging or the results returned are more extensive, it will exceed 10 seconds, causing Vercel to be unable to send the response to the Chatbot. Consequently, the solution is incapable of answering overly complex or lengthy questions.

ChatGPT-Based Intelligent Learning Aid (CILA)

To address the challenge of answering complex or extensive questions, our research institute has developed CILA, a system that employs shortcuts to transmit data directly, circumventing the steps involved in serverless hosting platforms. Learners’ queries are directly submitted via POST requests to the ChatGPT web page, and the responses are retrieved through GET requests from the same page. Following this, the answers to the queries are displayed on the screen, and both the posed questions and their corresponding results are forwarded to LINE for documentation and preservation purposes.

ChatGPT

The ChatGPT parameters utilized in the CILA developed by this research institute are presented in Table 1. The objective of the model is to configure the large language model used, and in this study, the highly efficient “text-davinci-003” is employed as the development model for CILA. The max_tokens parameter is intended to set the maximum number of characters that can be generated. In this study, we set it to 2000 to enable CILA to answer questions as comprehensively as possible. The temperature parameter determines the sampling temperature for responses, with values ranging from 0 to 2; higher values result in more random output, and vice versa. In this study, we set it to .8 to provide CILA with greater diversity. The presence_penalty is a value between −2.0 and 2.0; positive values penalize new tokens based on their appearance in the text so far, thus increasing the model’s propensity to discuss new topics. In this study, we set it to .4 to make CILA’s answers more innovative.

Table 1.

The Configuration of ChatGPT in the CILA.

Parameter	Model	Max_tokens	Temperature	Presence_penalty
Value	Text-davinci-003	2000	0.8	0.4

The Construction of CILA

CILA, developed based on Apple’s Shortcuts, necessitates an Apple device running iOS 12 or higher for installation and execution. Furthermore, we provide the code for CILA to illustrate the system’s architecture, as depicted in Figure 3. To utilize the ChatGPT feature, users must register with their email on the OpenAI official website and acquire an authentication key (see Figure 3(a)). Figure 3(b) enables learners to input questions they encounter during the course into the dialogue box. Figure 3(c) transmits the authentication key, the student’s submitted question, and the ChatGPT parameters from Table 1 to ChatGPT for processing. ChatGPT initially verifies the accuracy of the authentication key; if correct, it answers the learner’s question based on the parameters outlined in Table 1 and returns the result to the dialogue box. Ultimately, to encourage self-regulation and reflection in learners during the learning process, we employ the instant messenger LINE as a platform to record all questions and answers. Learners can revisit the questions they posed after class for concept recollection and memory retention, serving as a foundation for post-class reflection (as displayed in Figure 3(d)). It is important to note that the instant messenger LINE is used solely as a platform for recording questions and answers, not as the chatbot mentioned previously.

Figure 3.

The code of CILA. (a) Authentication key of ChatGPT. (b) Enter and post the question to ChatGPT. (c) Process the question through ChatGPT. (d) Record the results in LINE app.

Methodology

Participants

In this study, we employed a quasi-experimental design in a mathematics class at a high school in central Taiwan. We enrolled two classes, with a total of 70 K-12 students (40 males and 30 females) participating. One class was designated as the Experimental Group (EG) (21 males and 14 females), while the other class was designated as the Control Group (CG) (19 males and 16 females). Both the participants and their parents were informed about the experiment and provided signed consent forms. In the EG, participants were given access to CILA, a tool developed by our research team to support their self-regulation progress during the experimental activities. In contrast, participants in the CG used the Google search engine to aid their self-regulation progress. To ensure fairness in the study, the same teacher instructed both the EG and CG, and all participants were provided with iPads to eliminate potential hardware limitations (Rasheed et al., 2020). The experimental activities were embedded into the mandatory mathematics course for K-12 students, maintaining the regular dynamics of the class schedule and academic focus. No additional classes were allocated, ensuring that the dynamics followed the standard teaching pattern for such activities. There were no extrinsic incentives, such as extra credit, provided to the students for their participation, as their performance in the experimental activities was included in the calculation of their semester scores. The lack of additional incentives was to prevent external motivation from influencing self-regulation progress. This integration of experimental activities into their regular score calculations encouraged students to engage more actively, fostering intrinsic motivation and further promoting their self-regulation progress.

Experimental Procedure

We employed a quasi-experimental design to investigate whether the CILA intervention could enhance knowledge construction and self-regulation progress in learners within blended learning environments. The experiment focused on the topic of vectors in mathematics and spanned ten days, following Zimmerman (2000) three-step self-regulation progress for activity design.

On the first day, the instructor introduced the course, emphasizing the principles and applications of vector knowledge. A pre-test was administered to assess the participants’ initial understanding of vector concepts. Subsequently, participants were guided through the process of creating their own learning schedules and objectives for self-study outside of class, representing the forethought phase of the learning process. This involved setting clear goals such as ‘mastering the basics of vector operations’ or ‘understanding the calculation of vectors in math’ and creating a schedule to achieve these goals while considering their other commitments. From the second to the ninth day, representing the performance phase, participants actively worked towards their set goals. They engaged in online learning activities during their free time, utilizing resources like online tutorials, webinars, digital textbooks, and interactive exercises to enhance their understanding of vectors. During class time, participants participated in group discussions, sharing insights, asking questions, and collaboratively solving problems. They also reported on their progress towards their goals, allowing for continuous tracking and adjustments of their learning paths as necessary.

On the 10th day, participants entered the self-reflection phase. They reviewed the content they had studied over the previous nine days and reflected on the knowledge and concepts they had acquired. They considered questions such as ‘What were the most challenging aspects, and how did I overcome them?’ and ‘How has my understanding of vectors evolved through this course?’ Finally, a post-test on vector knowledge was administered to assess the participants’ improvement and overall mastery of the topic. Additionally, a scale measuring motivation, engagement, and self-efficacy was administered alongside the post-test. The entire experimental procedure is illustrated in Figure 4.

Figure 4.

Experimental procedure.

In the EG, participants were permitted to use CILA to address difficulties and problems encountered during the performance phase (Day 2-Day 9), and they could learn about their past issues through the instant messenger LINE during the self-reflection phase (Day 10), which fostered recall and reflection on concepts. In the CG, participants were allowed to use the Google search engine to tackle difficulties and problems during the performance phase (Day 2-Day 9), and they had to recollect their own challenges during the self-reflection phase (Day 10), which similarly promoted recall and reflection on concepts. Table 2 displays the differences in self-regulation between the EG and CG at various phases.

Table 2.

Differences Between Groups in Self-Regulation at Different Phase.

	EG	CG
Forethought phase	Before starting to learn about mathematical vectors, learners should establish their own daily study goals and track their progress
Performance phase	When encountering problems in studying, learners can use CILA to solve the issues	When encountering problems in studying, learners can use Google search engine to solve the issues
Self-reflection phase	By reviewing the archived questions and answers in the instant messenger LINE, it enables reflection on the lessons and helps with recalling concepts	Through learners’ own memories, facilitate reflection on the classroom and the recollection of concepts

Assessment Tools

Vector Knowledge Pre- and Post-Tests

In this study, we utilized pre- and post-tests utilizing the Vector Knowledge framework to evaluate participants’ progress in the mathematics vector unit. The test consists of 20 multiple-choice questions, each worth 5 points. This test was developed for the study by two mathematics experts, both professional teachers with over a decade of teaching experience. The term ‘Knowledge Construction’ used in the study refers to the process by which individuals build up their understanding of a particular topic or subject area (van Kesteren and Meeter, 2020). It implies the transformation of information into knowledge through active cognitive processes, creating new understandings, ideas, or concepts integrated with the pre-existing cognitive structure (Gan et al., 2020). To maintain the appropriateness and validity of the test, we calculated an internal consistency (Cronbach’s α value) of .78. According to Nunnally (1978), this value represents a sufficiently high internal consistency, thus ensuring the effectiveness of our ‘Knowledge Construction’ measure.

Motivation Scale

The motivation scale used in this study was adapted from The Situational Motivation Scale (SIMS), which was proposed by Guay et al. (2000). The scale divides learning motivation into four dimensions: intrinsic motivation, identified regulation, external regulation, and amotivation. Intrinsic motivation is driven by internal factors such as personal interest, enjoyment, and satisfaction, rather than external rewards or pressure. Identified regulation is an external motivation that occurs when individuals perform an activity because they recognize its importance. External regulation involves activities driven by external factors such as rewards, punishment, or social pressure, rather than internal factors such as personal interest or enjoyment. Amotivation is neither an internal nor an external motivation, lacking a sense of purpose or an expectation of reward, and there is no possibility of changing the course of events. It can be considered similar to learned helplessness, in which individuals experience a sense of incompetence and an expectation of uncontrollability (Guay et al., 2000). The scale uses a five-point Likert scale and has been shown to have sufficiently high reliability and validity in past studies. After translating the scale into Chinese, we conducted a reliability analysis again. Table 3 shows the original and modified reliability of the scale. The results indicate that after modification, the reliability is all above .7, which is considered sufficiently high (Nunnally, 1978).

Table 3.

Reliability Analysis of Motivation Scale.

	Intrinsic Motivation	Identified Regulation	External Regulation	Amotivation
Initial reliability	.93	.84	.89	.87
Reliability after modification	.87	.81	.90	.81

Engagement Scale

The engagement scale used in this study was adapted from the Math and Science Engagement Scales proposed by Wang et al. (2016). The scale divides engagement into four dimensions: cognitive engagement, behavioral engagement, emotional engagement, and social engagement. Cognitive engagement refers to self-regulation and the use of necessary cognitive strategies to understand complex ideas. Behavioral engagement refers to participating in academic and classroom activities, exhibiting positive behaviors, and avoiding destructive behaviors. Emotional engagement is defined as the presence of positive emotional reactions to teachers, peers, and classroom activities, as well as a focus on and interest in learning content. Social engagement refers to the quality of social interaction with peers and the willingness to establish and maintain relationships during the learning process (Wang et al., 2016). The scale is in the form of a five-point Likert scale and has been proven to have sufficiently high reliability and validity in past research. To meet the purpose of this study, the scale was translated into Chinese and subjected to another reliability analysis. Table 4 shows the original reliability of the scale and the reliability after modification. The results indicate that after modification, the reliability of all dimensions is above .7, which is considered sufficiently high reliability (Nunnally, 1978).

Table 4.

Reliability Analysis of Engagement Scale.

	Cognitive Engagement	Behavioral Engagement	Emotional Engagement	Social Engagement
Initial reliability	.75	.82	.89	.74
Reliability after modification	.78	.81	.90	.73

Self-Efficacy Scale

The self-efficacy scale used in this study was adapted from the New General Self-Efficacy Scale proposed by Chen et al. (2001). It was revised from the General Self-Efficacy Scale proposed by Schwarzer and Jerusalem (1995) to address issues such as low content validity and multidimensionality. The scale defines self-efficacy as the belief in one’s ability to mobilize motivation, cognitive knowledge, and action plans to meet the specific demands of a given situation. In short, self-efficacy is related to confidence to some extent, and students believe that they have a certain level of confidence to perform well in the classroom. The scale adopts a five-point Likert scale and has been demonstrated to have sufficiently high reliability and validity in past research. We translated the scale into Chinese, and a reliability analysis was conducted, yielding a result of .88, indicating sufficient reliability (Nunnally, 1978).

Data Analysis

To address research question 1, we employ an independent samples t test to analyze whether there are significant differences in self-regulation progress between the groups. It should be noted that, based on the description of previous studies, motivation, engagement, and self-efficacy are used as the basis for determining self-regulation progress.

To clearly address research question 2, we must first verify homogeneity to ensure the effectiveness of subsequent analyses. After confirming homogeneity, ANCOVA is used to determine whether there is a significant difference between the EG and CG in the post-test of vector knowledge. The pre-test scores of vector knowledge are used as covariates to eliminate the influence of pre-test differences on significance. It should be noted that we use pre- and post-tests of vector knowledge as the basis for measuring knowledge construction.

To clearly address research question 3, we must first use the Shapiro-Wilk test to verify the normality of the residuals and the Durbin-Watson statistic to verify the independence of the residuals. After satisfying these tests, multiple regression analysis can be employed to explore whether self-regulation progress (motivation, engagement, and self-efficacy) can serve as predictors for knowledge construction (post-test vector knowledge).

Results

The Impact of CILA on Self-Regulation Progress

To investigate this research question, we use engagement, motivation, and self-efficacy as the basis for measuring self-regulation progress. An independent sample t test is employed to determine whether there are significant differences between the EG and the CG in terms of motivation, engagement, and self-efficacy across various dimensions.

The results of the independent samples t test analysis on motivation are shown in Table 5. According to Table 5, significant differences are observed between the EG and the CG in Intrinsic Motivation (t = 2.92, p < .01) and Amotivation (t = −2.05, p < .05). Based on the mean values, the EG demonstrates significantly higher Intrinsic Motivation compared with the CG, as well as significantly lower Amotivation. These findings suggest that the CILA designed in this study can effectively enhance learners’ intrinsic motivation in blended mathematics learning and effectively reduce their Amotivation.

Table 5.

Independent Sample t test for Motivation.

	Group	Mean	SD	T	p	Effect Size
Intrinsic motivation	EG	18.89	2.76	2.92	.005**	.697
	CG	17.0	2.56
Identified regulation	EG	18.83	3.40	1.21	.229	.290
	CG	18.0	2.43
External regulation	EG	17.86	1.48	1.09	.278	.262
	CG	17.4	2.17
Amotivation	EG	9.14	3.02	−2.05	.044*	−.490
	CG	10.7	3.16

Note. *p<.05, **p<.01, ***p<.001

The independent samples t test results for engagement are displayed in Table 6, revealing significant differences between the EG and the CG in cognitive engagement (t = 2.68, p < .01), behavioral engagement (t = 3.02, p < .01), and emotional engagement (t = 3.35, p < .01). Furthermore, the mean values indicate that the EG exhibits substantially higher levels of cognitive, behavioral, and emotional engagement compared with the CG. These findings suggest that the CILA developed in this study effectively enhances learners’ cognitive, behavioral, and emotional engagement within blended learning environments.

Table 6.

Independent Sample t test for Engagement.

	Group	Mean	SD	t	p	Effect size
Cognitive engagement	EG	19.63	3.27	2.68	.009**	.641
	CG	17.6	3.05
Behavioral engagement	EG	18.0	2.72	3.02	.004**	.721
	CG	16.2	2.33
Emotional engagement	EG	18.69	3.26	3.35	.001**	.801
	CG	16.1	3.16
Social engagement	EG	16.29	2.58	1.24	.220	.296
	CG	15.5	2.63

Note. *p<.05, **p<.01, ***p<.001

The results of the independent samples t test for self-efficacy are shown in Table 7. According to Table 7, there is a significant difference in self-efficacy between the EG and the CG (t = 2.24, p < .05). Furthermore, based on the mean values, the self-efficacy of the EG is significantly higher than that of the CG. In summary, addressing Research Question 1, the implementation of CILA effectively improves student engagement, motivation, and self-efficacy, which in turn enhances the self-regulation progress of learners in blended learning.

Table 7.

Independent Sample t test for Self-Efficacy.

	Group	Mean	SD	t	p	Effect size
Self-efficacy	EG	18.11	3.45	2.24	.029*	.535
	CG	16.2	3.70

Note. *p<.05, **p<.01, ***p<.001

The Impact of CILA on Knowledge Construction

In order to investigate whether there is a significant difference in vector knowledge between the EG and the CG after introducing CILA, we used pre-test and post-test scores of vector knowledge as the basis for measuring knowledge construction. An Analysis of Covariance (ANCOVA) was adopted, with pre-test scores of vector knowledge as the covariate and post-test scores as the independent variable. This approach allowed the study to examine differences in vector post-test scores between the EG and CG while reducing the impact of vector prior knowledge differences.

To assess the appropriateness of using ANCOVA for analysis, we first conducted a Levene’s test to verify the homogeneity of variances between the EG and CG. The results showed that the measured values did not have a significant effect (F = 2.17, p = .146 > .05), so the null hypothesis was not rejected. Therefore, ANCOVA analysis can be employed. As presented in Table 8, when considering the pre-test scores of vector knowledge, there is a significant difference in post-test scores between the EG and the CG (F = 9.89, p < .005). The mean scores of the two groups reveal that the performance of the EG (M = 67.6, SD = 16.47) is significantly higher than that of the CG (M = 54.9, SD = 16.45). Consequently, addressing Research Question 2, the results indicate that the CILA designed in this study can enhance learners’ knowledge construction more effectively than the Google search engine.

Table 8.

ANCOVA for Knowledge Construction.

	Sum of Squares	df	Mean Square	F	p	Partial η2
Pre-test score	4093	1	4093	30.03	<.001***	.309
Group	1349	1	1349	9.89	.002**	.129
Residuals	9133	67	136

Note. *p<.05, **p<.01, ***p<.001

The Relationship Between Self-Regulation Progress and Knowledge Construction in Blended Learning

To investigate this research question, we first need to perform a Shapiro-Wilk test to verify whether the sample follows a normal distribution. The results indicate that the sample is normally distributed (W = .981, p = .36 > .05). The Durbin-Watson statistic is used to test the independence of the residual terms, and the results show that the residuals are independent (DW = 1.69, p = .138 > .05). Therefore, we can use multiple linear regression to explore the relationship and predictability between self-regulation progress (engagement, motivation, and self-efficacy) and knowledge construction (post-test vector knowledge). The results are shown in Table 9. The predictors derived from self-regulation progress, which encompass motivation, engagement, and self-efficacy, consist of intrinsic motivation (IM), identified regulation (IR), external regulation (ER), amotivation (AM), cognitive engagement (CE), behavioral engagement (BE), emotional engagement (EE), social engagement (SE), and self-efficacy (SEF). The dependent variable is represented by the post-test scores of vector knowledge.

Table 9.

Multiple Linear Regression for Self-Regulation Progress on Knowledge Construction.

R 2	Adjusted R2	Component	Original Coefficients		Standardized Coefficients	t	p	VIF
			B	Std. Error
.885	.867	Constant	9.209	10.891		.846	.401
		IM	1.154	.402	.2214	2.869	.006**	3.10
		IR	.225	.248	.0455	.904	.370	1.32
		ER	−.581	.388	−.0739	−1.496	.140	1.27
		AM	−1.081	.282	−.2339	−3.835	<.001***	1.94
		CE	.719	.318	.1624	2.259	.028*	2.69
		BE	.883	.320	.1617	2.761	.008**	1.79
		EE	.890	.382	.2095	2.331	.023*	4.20
		SE	−.334	.260	−.0597	−1.284	.204	1.13
		SEF	.604	.275	.1520	2.197	.032*	2.49

Note. *p<.05, **p<.01, ***p<.001

According to Table 9, self-regulation progress of learners in the blended learning context has a significant impact on knowledge construction (F = 51.2, p < .001), confirming the effectiveness of this regression model. The adjusted R² value of .867 indicates that 86.7% of the variance in post-test scores for vector knowledge can be explained by this regression model. Specifically, IM, AM, CE, BE, EE, and SEF have a significant impact on the regression model (p < .05), while the others do not. Addressing Research Question 3, these results indicate that the self-regulation progress of learners in blended learning environments can effectively affect knowledge construction and significantly influence learners’ knowledge construction in the dimensions of Intrinsic motivation, Amotivation, Cognitive engagement, Behavioral engagement, Emotional engagement, and Self-efficacy.

Discussion

CILA was developed to enhance self-regulation progress and knowledge construction in blended learning. We need to understand how the functions provided by CILA can benefit these two aspects.

The Impact of CILA on Self-Regulation Progress

In this study, based on the framework proposed by Zimmerman (2000), the progress of self-regulation is divided into three phase: forethought, performance, and self-reflection. Furthermore, according to its definition and descriptions in previous literature, motivation, engagement, and self-efficacy are used as indicators to measure the self-regulation progress.

In Table 5, the intervention of CILA significantly increases learners’ intrinsic motivation and reduce their amotivation. This is because CILA allows students to no longer fear learning due to unresolved problems, thus enabling them to enjoy learning more and not feel learned helplessness. This result is consistent with previous research on the motivational benefits of timely intervention during learning (Liu, 2022; Ross et al., 2018). Since the experimental activity in this study is a compulsory math course, learners will not experience additional external rewards or pressure due to the presence or absence of CILA, which may be one of the main reasons for the lack of significant improvement in extrinsic motivation.

Table 6 shows that the intervention of CILA significantly enhances learners’ cognitive engagement, behavioral engagement, and emotional engagement. In terms of cognitive engagement, CILA can resolve learners’ difficulties in real-time through offering convergent information during the self-regulation performance phase, thus improving the progress of their knowledge construction and increasing their cognitive engagement. This result is consistent with previous research on the benefits of timely intervention during learning for cognitive engagement (Liu et al., 2022; Zhang et al., 2021). In terms of behavioral engagement, CILA’s intervention in the self-regulation performance phase prevents students from being forced to interrupt their learning due to difficulties, leading them to adopt more proactive behaviors in learning and improving their behavioral engagement. This result is similar to previous research on the benefits of timely intervention during learning for behavioral engagement (Lu et al., 2017; Yang and Ogata, 2023). In terms of emotional engagement, low motivation and attitudes caused by difficulties and setbacks have long been one of the main reasons for students’ dropout or failure in math courses (Luttenberger et al., 2018). Therefore, CILA’s intervention during the self-regulation performance phase can help students solve problems through providing convergent information in a timely manner, reduce their frustration, and ultimately enhance their emotional engagement. This result is consistent with previous research on the benefits of timely intervention for emotional engagement (Caspari-Sadeghi, 2022; Tao et al., 2023).

Table 7 shows that CILA ‘s intervention during the self-regulation performance phase positively impacts self-efficacy. By providing a record of past questions and answers during the self-regulation self-reflection phase, CILA encourages learners to further examine and review their strengths and weaknesses in the concepts and knowledge they have acquired in the classroom. This, in turn, leads to an increase in learners’ confidence and self-efficacy. This result is consistent with previous research on the benefits of reflective learning for self-efficacy (Hsia and Hwang, 2020; Menon and Azam, 2021).

The Impact of CILA on Knowledge Construction

As the role of teachers evolves, learners in blended learning environments are transitioning from passive knowledge absorption to actively seeking knowledge in order to complete the knowledge construction process (Capone, 2022). Consequently, effectively supporting learners’ self-regulation development in blended learning has become a critical aspect (Rasheed et al., 2020). The results presented in Table 8 indicate that the proposed CILA in this study significantly enhances learners’ knowledge construction within blended learning environments. CILA can quickly provide convergent information by integrating ChatGPT to address the challenges learners face during the self-regulation performance phase. In contrast to the traditional Google search engine, which offers a broad range of answers to learners’ queries, ChatGPT filters out irrelevant information to deliver targeted solutions, thereby expediting the resolution of learners’ issues in blended learning. With CILA’s assistance, learners no longer need to sift through extraneous information on their own, a process which can disrupt learning and hinder the performance phase of self-regulation development. This finding aligns with previous research emphasizing the significance of timely problem-solving for knowledge construction (Afini Normadhi et al., 2019; Tsai et al., 2020; Xu and Ouyang, 2022).

On the other hand, compared with traditional learners who can only reflect through self-recollection, CILA provides learners with a record of questions and answers that they have asked in the past. Learners can focus on reviewing and reflecting on unfamiliar concepts, thus enhancing their understanding and knowledge construction of the concepts, which can improve self-reflection phase in the self-regulation progress. This result is similar to previous research on the beneficial effects of reflective learning on knowledge construction (Lin et al., 2022; Lonka et al., 2021; Xie and Lin, 2016).

The Relationship Between Self-Regulation Progress and Knowledge Construction in Blended Learning

In Table 9, cognitive engagement, behavioral engagement, emotional engagement, intrinsic motivation, amotivation, and self-efficacy dimensions significantly impact knowledge construction in blended learning (p < .05). This result is similar to research on motivation (Bai et al., 2020; Ibrahim and Nat, 2019), engagement (Heo et al., 2022; Hui et al., 2019), and self-efficacy (Heo et al., 2022; Suryani et al., 2021) being helpful in blended learning. Moreover, amotivation, intrinsic motivation, and behavioral engagement have stronger predictive power (p < .01). The reason is that blended learning relies on a high degree of learner autonomy. In the absence of teacher control, learners need to have sufficiently high enthusiasm and interest in learning during the forethought phase and not develop learned helplessness. Additionally, they need to engage in more proactive learning behaviors in the performance phase to ensure good performance in blended learning. These findings not only further support why CILA can enhance students’ knowledge construction in blended learning but also align with past research on how to improve students’ performance in blended learning (Law et al., 2019; Zhang et al., 2020).

Conclusion

Considering the significance of self-regulation in blended learning and the exceptional performance of large language models, this study developed CILA, a system that integrates ChatGPT, Apple’s Shortcuts, and the instant messaging platform LINE to assist learners in enhancing their self-regulation progress and knowledge construction within blended learning environments. In comparison with the conventional approach of utilizing the Google search engine to address issues in blended learning, CILA offers more targeted responses to learners’ inquiries in real-time, thereby minimizing disruptions during the self-regulation performance phase of learning. Furthermore, CILA maintains a log of learners’ questions and answers, providing a foundation for recollection and evaluation during the self-regulation reflection phase. The results suggest that the implementation of CILA effectively improves self-regulation progress and knowledge construction in blended learning. Additionally, as students desire to resolve difficulties immediately to continue their learning process, it is evident that providing convergent information for students is more suitable for blended learning than divergent information. Consequently, this study presents insights into the integration of large language models, such as ChatGPT, in educational settings to support teachers and students.

This study indeed possesses a few notable limitations. Firstly, the research primarily targets high school mathematics courses as the core subject in blended learning. This could potentially generate biased predictions of the self-regulation progress’s impact on knowledge construction due to the students’ perceptions of mathematics. Secondly, the tool used in this study, CILA, is developed based on the architecture of GPT-3.5. One unaddressed limitation in this study is the occasional inaccuracy of GPT-3.5’s responses, introducing another variable into the intricate dynamics of self-regulated learning and knowledge construction. The implications of these inaccuracies, especially regarding how they might influence the results, represent an additional area of potential bias that this study did not fully address. Moreover, a crucial methodological constraint not mentioned before is the lack of a pre-test. Without conducting a pre-test, it becomes challenging to pinpoint the causes of any significant differences observed in the post-test results. This omission might confound the interpretation of the outcomes and overstate the intervention’s impact. Furthermore, since CILA is developed and designed based on Apple’s Shortcuts, all participants in this study were provided with iPads to ensure the usability of CILA. However, this method might limit the implementation in other contexts due to the reliance on Apple devices. Future research could work towards mitigating these limitations. Researchers could gather experimental data from a diverse range of subjects in blended learning to reduce the bias in regression models and thereby enhance the effectiveness of the models in predicting knowledge construction from the self-regulation progress. A deeper exploration into the accuracy of GPT-3.5 responses in educational settings could offer a more comprehensive understanding of this aspect. It would also be beneficial to conduct pre-tests to better gauge the effectiveness of the interventions. Moreover, adapting CILA into a web-based version would make it more accessible and easier to install for all learners, irrespective of the device they use.