Abstract
Distance learning is rapidly gaining ground globally. In this case study, we focused on professional (vocational) teacher education (PTE) student online activity in a blended learning context. We applied learning analytics (LA) to identify students’ (n = 19) online study patterns. Our key interest was in determining when and what kinds of online activity and behavior PTE students engage in during their studies. We applied quantitative content analysis to analyze the students’ behavior. Moodle’s event log enabled us to identify active hours and days, variation in use of learning materials, the impact of interventions, and stumbling blocks to student learning in the study unit. Based on our data, educator availability is an essential factor for good student engagement in digital learning environments. Interaction forums are important for PTE students effective learning. Monday and Tuesday afternoons are the most effective times for educators to be available for PTE students. There is a clear need for contact learning in professional teacher education, even when operating in digital learning environments. It plays an essential role in keeping students’ activity alive. It could be beneficial to plan a post-process for students who do not graduate as planned, including regular group meetings for supporting studies, receiving guidance, and meeting peers. PTE students’ behavior in a distance learning environment in the context of blended learning follows Zipf’s law, which models the occurrence of distinct objects in particular sorts of collections.
Introduction
Distance learning is rapidly gaining ground globally. In this case study we focus on professional (vocational) teacher education (PTE) student online activity in the context of blended learning, which combines online and face-to-face activities. Blended learning “is a series of content blocks sequenced” to create flexible modes of learning, and personalized learning trajectories (Boelens et al., 2018; Hofmann, 2018). Our interest is in how PTE students behave in a blended learning context as this knowledge can help educators enhance the quality of the learning process. Meta-analysis of 252 studies of learning analytics (LA) in higher education indicates that even though the identified potential of LA for improving quality of learning is high, there is not much transfer of the potential into higher education (Viberg et al., 2018). Moreover, there is an evident knowledge gap as research on LA in the context of blended learning in PTE is in its infancy. However, there seem to be a willingness to increase distance learning and teaching online also in PTE. Therefore, there is a need for teacher educators to learn how to keep the learning process alive and to find more tools to prevent students from dropping out and losing touch with their studies. LA can be a tool for teacher educators to enhance the learning process, but it could also help students to be concretely aware of their study process (Knobbout & Van Der Stappen, 2020).
In this study, we focus on LA from the teachers’ point of view. Professional teachers who work in vocational education are important actors in promoting the skills and competencies needed in future society. Purpose of this research is to equip educators with the information needed to guide their own vocational students in the blended learning environment. New possibilities for teaching, guiding and keeping in contact with students brought to light through LA could also demonstrate innovative ways of being a teacher and serve to attract more people to study to become a teacher.
Literature Review
Learning analytics (LA) reveals how students use educational materials. Learning analytics is the “measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Long & Siemens, 2011, p. 34). The main purpose of LA is to support learners, to enhance learning, and to support the development of teaching and teaching materials. LA makes visible student behaviors in digital learning environments. It helps educators to modify their teaching styles based on student performance (Hasan et al., 2019) and to improve the structure of online courses (Saqr et al., 2018). It also promotes early identification of students at risk of not submitting (or failing) their next assignment (Herodotou et al., 2019). LA can also be a way to identify students at risk of dropping out (Suhonen & Kinnari-Korpela, 2019). Moreover, LA can be used to predict final grades (Buenaño-Fernández et al., 2019). For example, according to Montgomery et al. (2019), access per day and overall access frequency are the strongest predictors of student success in the context of a flipped, blended learning music teacher education course.
According to a systematic literature review, research on learning analytics has focused on learning environment, learning process, and learning outcome (Knobbout & Van Der Stappen, 2020) as demonstrated in Table 1. In this case study we focus mainly on the learning process including online activity and behavior, self-regulated learning and engagement. From the self-regulated learning point of view, focusing on the learning process is essential (Matcha et al., 2020). Online study platforms underline student self-regulation (Zimmerman & Schunk, 2011), that is, students’ active involvement in maintaining their engagement by understanding the tasks involved (Rovers et al., 2019). The context of the present case study is Finnish PTE, in which students are independent and autonomous learners. They are responsible for their productivity by applying their metacognitive, motivational and behavioral processes to achieve learning and performance goals (Paris & Byrnes, 1989; Zimmerman, 2008). Self-regulated behavior that is systematically oriented toward achievement of learning goals is seen as an increasingly important predictor of positive learning outcomes (Matcha et al., 2020; Zimmerman, 1990). According to expectancy-value theory, the imaginable benefits of learning outcomes determine motivation and engagement. Things that are seen to be important are easier to learn (Wigfield & Eccles, 2000). According to the self-efficacy theory, a student’s positive perception of their own ability predicts good engagement (Bandura, 1997). Expectations of personal efficacy are derived from learning situations. There are four principal sources of information: performance accomplishments, vicarious experience, verbal persuasion, and physiological states (Bandura, 1977). According to the self-determination theory, fulfilment of the basic needs for competence, relatedness and autonomy is a predictor of positive learning outcomes (Deci & Ryan, 2000). Goal setting also contributes to student engagement. Appropriately challenging and jointly designed goals predict good learning outcomes (Locke et al., 1968).
Operational Definitions of Learning Affected by Learning Analytics Interventions.
Source. Adapted from Knobbout and Van Der Stappen (2020, p. 640).
The PTE program (60 credits) includes pedagogical studies, teaching practice, basic studies in education and free-choice pedagogical studies and is aimed at teachers who work or aim to work in vocational institutions or universities of applied sciences. Typically, PTE students hold a master’s degree and at least 3 years of work experience in the field of teaching. During the studies, students learn how to combine field-specific expertise and teachership as part of their professional teacher identity development. Online study platforms offer promising solutions for a dynamic learning process that is needed to keep work, family, and student life in balance. PTE joint study program is planned for 1 year, although students are entitled to complete the studies over 3 years. The students study 60 credits together in groups based on individual competency-based learning paths, according to which prior learning was recognized in addition to the course lectures and material made available. The PTE students can thus also demonstrate competencies that they had achieved elsewhere. In practical terms, recognition of prior learning means that the students have to integrate educational theory into their demonstration work. The demonstrations can be conducted either as part of their daily work as a teacher and assessed by their mentor, or on contact days when the other PTE students assess the demonstration (TAMK, 2020). Thus, the flexibility of the studies is very much needed in this respect.
Human activity and behavior is a complex phenomenon. It is characterized by many degrees of freedom. However, power laws span various disciplines including physics, biology, engineering, the earth sciences, economics, political science, sociology, and statistics (Clauset et al., 2009). According to Ectors et al. (2019) “the same power law distribution holds for a large number of events in different domains, ranging from sizes of earthquakes, annual income of companies, solar flares, to the number of citations received on papers, social media check-in data” (p. 1691).
In this research we focus on professional (vocational) teacher education (PTE) student online activity in a blended learning context. Our key interest is in determining when and what kinds of online activity and behavior PTE students engage in during their studies. This information could help educators to enhance their teaching practices, assess the impact of teacher interventions, and improve the structure of study units. Our specific research questions are:
When do students do online tasks?
What kinds of distance learning material are popular among students?
How does student activity relate to educators’ interventions?
Is there any power law to be identified in students’ behavior?
Material and Methods
The case study was based on a blended learning approach. We collected the data for the study via the online study platform during the learning process and its interventions. Moodle (Modular Object-Oriented Dynamic Learning Environment) was used as the platform for online-learning (Costello, 2013). Moodle also had a role to play as a source of individual learning paths according to individual time and space lines. Contact teaching was implemented via seven contact days for the whole group, and learning was mostly conducted in the students’ authentic learning environments outside the university. In addition, nine obligatory webinars were held for the students plus three webinars for students who had not studied educational science before. The pedagogical structure of the course was as follows:
Welcoming words with a picture describing the home base for studies
Personal study plan and learning diary (three credits)
Webinars and workshops
Information about contact days
Learning material for study unit 1 (seven credits)
Learning material for study unit 2 (eight credits)
Learning material for study unit 3 (seven credits)
Teaching practice (14 credits)
Information on demonstration of competencies
Basic studies in educational science (10 credits)
Development work (five credits)
Pedagogical approach to learning teaching methods
Free-choice pedagogical studies (six credits) were not included on the Moodle learning environment as these studies are offered to all TAMK teacher students or them can be studied according to students’ personal study plan.
All of the learning materials were able to be used in multiple ways. Lectures during the contact days were recorded to enable students who were unable to attend the contact days to keep up to date. Individual and collaborative tasks were given between and during contact days. Material developed during group discussions was posted on Moodle. This gave students who did not attend the contact days the possibility to study the material independently. Students unable to attend online or face-to-face meetings were also able to watch webinars or videos of the meetings on Moodle. The students wrote their learning diaries and teaching practice reports and completed their educational science exercises online on Moodle. The schedule for the contact teaching days was shared on Moodle, as was the material presented during these days. All information about the studies and study modules was also made available on Moodle. Teachers also informed the students which materials to study (e.g., before contact days) via email or WhatsApp. For the majority of students who were able to take part in group work, the material in Moodle was used for both independent and collaborative study. Some students participated less in group work for personal reasons related, for example, to work or family life. These students were able to use the online material more for independent study. For students more actively engaged in group work, the contact days provided opportunities for re-examining and deepening their learning and understanding of the material.
At the beginning of the course the students were informed about this study and all of the students gave written permission to use their data for the research. In Moodle, the students were required to accept a data security announcement and, by accepting it, they knew what kind of data will be made available in the Moodle log. Moodle stores actions and their timestamps individually for each student. Once collected, the data were pseudonymized so that no student identifications were included in the data processing. Thus, no participants could be recognized from the results and analysis. All IP information was also deleted.
The data consists of a Moodle event log based on the learning activities of a professional teacher education group of 19 students, including seven women and 12 men. The students started their studies in May 2018 and the joint study program was scheduled to end in May 2019. After the end of the course, 12 students qualified, three students graduated during the following summer, and four students continued their studies outside the group in the autumn. Three students dropped out for personal reasons soon after beginning the course. Approximately 250 students start their PTE studies at Tampere University of Applied Sciences (TAMK) each year.
We applied quantitative content analysis to analyze students’ activity and behavior (Holsti, 1969; Miles et al., 2014). Validity of the research was provided by developing a protocol that was theoretically valid and by establishing its validity empirically (Rourke & Anderson, 2004). Theoretically, we focus on the learning process including online activity and behavior (Table 1). Empirically the learners’ data in the Moodle log files provided the basic data for analysis. Variables were (a) time of log event, (b) object of log event, and (c) frequency of log event. The data consists of the log events and number of clicks—what material the students clicked, how many times they opened the material and at what time of day and week they logged on to the Moodle pages. The Moodle log shows all actions taken at the highest level of page hierarchy in Moodle’s structure, whereas deeper structures are not recorded. For example, opening a folder containing many links to videos is logged as an event, whereas opening individual videos is not logged. Therefore, all of the activities included in the analysis are at the highest level of the Moodle page hierarchy. We categorized the log data into eight categories according to 85 parameters.
Results
Our results reveal how the students behaved in the digital learning environment. The temporal distributions of student activity are presented in Figure 1. On the left, the daily sum of all log events for all students is shown as a function of the weekday. On the right, the activity at different times is presented. The activity is the sum of all log events in the Moodle log file for each day. The most active study day was Monday and study activity was highest at 13.00 to 14.00 in the afternoon on all days. We identified a linear drop in student activity over the average week from Monday to Sunday.
Study activity as a function of day (left) and time of day (right).
The teacher education Moodle platform had 85 different learning materials and links in eight categories (forums, assignments, folders, links, pages, files, webinars, and a seminar). The materials and links (n = 85), the number of clicks (range 4–901) and the mean number of clicks per category are presented in Table 2. The materials and links were not wholly comparable: for example, open discussion forums needed to be opened more often than webinar recordings, which were often only watched once. The openings at the top of the table show that basic information on the study unit was read several times: general information and news got the most clicks. According to the data, interaction forums and assignments are important to PTE students. The five least opened materials included webinars, a recording from a contact teaching day, and a guide material. The webinars and the final seminar of the learning period were the least opened materials.
Eight Categories of Learning Materials (n = 85), Number of Clicks, and Mean Clicks.
Figure 2 below illustrates Zipf’s law in the data. It shows the materials and links according to their popularity. Popularity is identified by number of clicks. The number of material and link openings (clicks) ranged between 4 and 901. Chat forums had the most openings (mean 466 openings), and webinar recordings the least (mean 14 openings) as well as a seminar recording (four openings). According to LA students’ behavior follows Zipf’s law, which models the occurrence of distinct objects in particular sorts of collections (Zipf, 1949).
Illustrating of Zipf’s law in the data based on the number of openings (4–901) per learning material (1–85) on the Moodle platform follows a power law (Zipf’s law).
Figure 3 shows overall activity as a function of date. Activity is the sum of all daily log events. The left side of the timeline shows activities and interventions. The official starting date of the studies has the highest activity rate. This is expected, as all of the students logged into Moodle and familiarized themselves with its contents on the same day. Activity remains subsequently rather high throughout autumn 2018 and spring 2019. Notably, interventions play an important role in study activity. In contrast, activity during the summer holiday period is lower, although not zero. When the official studies end, there is a clear drop in activity. Similarly, when group activities end, learning activity decreases. As the group became smaller and the number of common activities was reduced, activity remained low but peaked a couple of times. The peaks mainly occurred due to workshops. During the extended study period, it is clear that it is primarily interventions that cause the students to take action: activity peaks on workshop days but otherwise remains rather low.
Overall activity as a function of date.
Discussion
In this case study we examined the online study activity of PTE students. We analyzed the PTE students’ online activity using Moodle’s event log. By doing so, we were able to identify the active days and time slots, most and least used materials, impact of interventions of educators, and stumbling blocks to student learning of the study unit. Our results help educators to improve the quality of PTE student learning.
As teaching and learning move increasingly online, there is a need to understand when adult students have time for their learning activities. This will enable educators to design more effective study schedules. Our results reveal that the most active study days were Monday and Tuesday, and early to mid afternoon were the busiest learning times. These times therefore play an important role in PTE student learning. Teachers’ shared expertise could be a way forward in the busiest learning times (Salonen & Savander-Ranne, 2015).
The PTE students studied mainly during weekdays. More specifically, they preferred traditional working hours, from 8 am to 4 pm. This is in clear contrast to degree students, who have been found to study most at evenings and weekends (Suhonen, 2016a; Suhonen & Tiili, 2015). Some explanatory factors of these differences may be the length of study unit and student life situation. The degree students in the studies by Sami Suhonen and colleagues had assignment deadlines every Sunday and study units lasted for 8 weeks. The PTE students under study, in contrast, studied for the whole year and only had a few large assignment deadlines. The difference in life situation is also apparent—unlike degree students, many PTE students have a family and work in addition to their teacher education studies.
The highest peaks in student activity occur after interventions, such as contact days, webinars, and messages from educators. In previous studies, deadlines were observed to cause a peak in activity a day or two before the deadline (Suhonen, 2016b). In the teacher education under study, the study modules and learning tasks were rather extensive and there were no weekly deadlines. Therefore, deadline dates do not emerge as peaks in the activity graph.
In our data, the most frequent openings reveal that basic information was read several times: general information and news were most clicked. Various interaction forums are important for PTE students. This information fits well with the idea that to build a new identity as a qualified teacher, peers and teacher educators are needed. It is also interesting to note that the third exercise of the basic studies in education (“classics of education”) was opened more often (642 clicks) than the first (“learning perception table”, 78 clicks) or second (“human life and development assignment”, 541 clicks) exercises. This is contrary to our logical pre-assumption that the first exercise would have been the most popular. In the case under study, the students could complete the basic studies in education according to their own timetable and in any order. It seems, therefore, that the first exercise was scheduled too early on in their studies at a time when the students had many other things to cope with, whereas by the time of the third exercise the students had become more settled in their studies. Based on this observation, the basic information of the study unit should be made easily accessible and the structure of the study unit should be coherent. Accordingly, when refining the educational content for the next group of PTE students, educators should focus on the most used materials and assignments.
In blended learning environments, self-regulation is needed as there are only a few contact days and students are responsible for their learning outcomes (Paris & Byrnes, 1989; Zimmerman & Schunk, 2011). As PTE students are adults, the assumption is that they would feel that they are the subject of the studies and their experience of the studies and motivation would be therefore be more likely to be positive (Kukkonen & Marttila, 2017). However, according to our results, student activity increased only immediately after each intervention by the educator. It is demanding to study, work full-time as a teacher, and have a family life. Real life is unpredictable. Unexpected work or family situations can force students to put their studies on hold. The consequent challenge, then, is how to resume studying when the work or family load normalizes. Here, Zimmerman’s (1990) statement comes into play: self-regulated action that is systematically oriented toward achievement is an increasingly important predictor of positive learning outcomes. In addition, as Locke et al. (1968) demonstrated appropriately challenging, and jointly designed goals predict good learning outcomes. These both require the availability and support of an educator.
According to our data, it seems that group activities help maintain regular study patterns. Previous research has shown that it is important for students to feel that they are a member of a learning community (Stenlund et al., 2012). This is in line with the idea of positive learning experiences (Kukkonen & Marttila, 2017) and the importance of being noticed (Chiesa & Hobbs, 2008). In contrast, a sense of being left alone decreases motivation and engagement, especially if the majority of learning activities are located at the PTE student’s work place. Thus, in blended learning, contact days seem to help students form a sense of community and pace their studies. In our data this is manifested by the increased online activity immediately after the contact days. Also, according to Chiesa and Hobbs (2008), attention paid to individual efforts raises the level of performance.
Group activities play an essential role in keeping students’ activity alive. Despite possible teamwork and collaboration among teachers (Salonen & Savander-Ranne, 2015), teaching is fairly solitary work. The PTE students that participated in this research already had experience of working as a teacher. This offered opportunities for self-reflection with peers and for reflection on their own capabilities as a teacher. During this peer learning process, prior concepts of knowing, being and acting are challenged and the students develop, in part, a new teacher identity (Barnett & Coate, 2005) with the support of teacher educators and peers. This is closely connected with the idea of self-efficacy: a student’s positive perception of their own ability predicts good motivation (Bandura, 1997).
Our results reveal that the end of group activities is a stumbling block to learning: when most students from the group graduate, those who are still studying feel left alone. As our data shows, activity during the extension period was very low and only peaked when assistance was provided to the students via workshops. The challenge, then, is to help students who work full-time during their studies to maintain good study motivation and effective time management. The easiest way could be to provide progress information. By doing so, students could see what they have done and what they still need to do to graduate. Contacting students also seems to be a relatively straightforward and beneficial approach (Herodotou et al., 2020).
Even if human behavior is a complex phenomenon we identified a power law (Clauset et al., 2009). PTE students’ behavior in a distance learning environment in the context of blended learning follows Zipf’s law, which models the occurrence of distinct objects in particular sorts of collections (Zipf, 1949). This could open topics for future study. Future researchers could also focus more specifically on motivational strategies (e.g., email reminders, role of social media, systematic positive feedback, demonstrating the benefits of studies) in order to determine which motivational strategies are most effective in digital learning environments. In future studies, it would also be beneficial to examine individual learning paths, for example by studying individual students and their activity and comparing this with their graduation time. Small group activities and their relation to graduation could also be an interesting aspect for further study.
Conclusions and Limitations
Learning analytics provides educators with a means of being informed about the learning process. This helps educators to enhance their teaching practices, assess the impact of teacher interventions, and improve the structure of study units. There is a clear need for contact learning in professional teacher education, even when operating in digital learning environments. There is also a need for joint study, such as studying in groups and peer learning, in a blended learning context. Our data also supports the conclusion that educator availability is an essential factor for good PTE student motivation and engagement in digital learning environments. Interaction forums are most famous online arenas for PTE students. Educators should be there available for students to ensure an effective learning process. More specifically, Monday and Tuesday afternoons are the most effective times for educators to be available for students.
Based on our data, it could be beneficial to plan a post-process for PTE students who do not graduate as planned, including regular group meetings for supporting studies, receiving guidance and meeting peers. This could help students to find new peers and give them a sense that they are not alone in not succeeding to advance in their studies.
There are some significant limitations of this research. First, we lack a means of determining all of the factors that may have had an impact on student activity in the digital learning environment. We only analyzed the learner data collected in log files by the learning management system. Activities such as reading articles online outside Moodle and discussing face-to-face were not tracked. Secondly, we only gathered quantitative data. Qualitative data, such as student interviews would help understand how students reason their everyday choices in their learning activities. In short, we could identify what the students clicked in the digital learning platform, but we could not determine why they clicked. The results of this case study cannot not be generalized. Rather, they are a starting place for understanding of the use of LA for adult learners in a hybrid learning environment.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The data of the research was collected as part of the project OPEKE—Reform of vocational teacher education funded by the Ministry of Education and Culture, Finland.
Open Data and Ethics
The dataset generated and analyzed during the current research is available from the corresponding author on reasonable request. The participating students took part voluntarily and with informed consent. Statistical analysis was performed using non-identifiable data.
