Sage Journals: Discover world-class research

Abstract

Background

The promotion of cycling for children is beneficial from a health and environmental perspective, however road safety and awareness amongst this age group remain a considerable issue. As children are developing their cognitive and physical abilities, they are considered a high-risk group for injuries and fatalities on the roads. Virtual learning environments have demonstrated promising ways to engage children in learning about road risks and teach children about safe cycling.

Intervention

A web-based Virtual Learning Environment (VLE) and Virtual Reality (VR) environment was designed to offer a platform for schoolchildren to learn about safe cycling and to develop skills for them to better detect hazards in traffic. Both learning environments were used by 455 school children and 20 class teachers.

Methods

Fun toolkit methods, specifically designed for child participants were used to assess the design of the technology, and for teachers a qualitative survey to provide evidence around the use of both learning environments.

Results

Results suggest both learning environments were appealing for lower school grade participants. For grades five and six a need to review the relevance and user design of the learning content was evident. The VR environment was highly attractive across all school grades, suggesting that VR could be a feasible way to teach road safety and safe cycling for school children.

Discussion

Although VLE and VR environment were attractive and fun to use, some users of VR experienced motion sickness. This would need to be considered and further examined before engaging a young learner in a VR learning environment. Several considerations are provided for teachers, practitioners, researchers, and designers seeking means to promote safe cycling for children.

Keywords

virtual learning environment virtual reality educational games cycling education traffic safety

Background

Cycling as a mode of transportation has benefits over other modes of transportation that are based on combustion. Cycling is beneficial for mitigating climate change and has a positive impact for public health (Rabl & de Nazelle 2012; Macmillan, Connor, Witten et al., 2014). Moreover, cycling is directly linked to 11 out of 17 goals of the United Nations Global Goals. For cities that seek efficient land-use, green human-centered public spaces, and sustainable growth, promoting cycling offers a socially just and efficient transportation mode reducing traffic congestion, environmental pollution, and noise. Although the health benefits outweigh the negative outcomes of possible road accidents associated with cycling, fatalities and injuries remain a serious concern. (Götschi, Garrard & Giles-Corti, 2016; Oja, Titze, Bauman et al., 2011; de Hartog, Boogaard, Nijland & Hoek 2010; Gotschi, 2011; IRTAD, 2014; Bo Andersen, Schnohr, Schroll & Hein, 2000). Children are considered a significantly high-risk group within a traffic environment. Despite the gradual decrease in cycling related road incidents in recent years, cycling still accounts for most child deaths in road traffic accidents to date (Ishii, Hitosugi, Kandori et al., 2022).

In Finland, the majority of children aged 7 to 12 who experience road related injuries and fatalities are cyclists. (Liikenneturva, 2019 This is aligned with international comparisons (OECD/ITF, 2013). Child cyclists are out of the line of sight for many drivers. They are vulnerable to traffic as they develop their cycling skills, they are in the process of developing their cognitive judgement of danger and their fine motor skills.

Compared to adult cyclists, children detect and focus less on other road users, they can sometimes miss warning signals or cues and react to them slower, and in general have lesser situational awareness than adults (Melin, Peltomaa, Schildt & Lehtonen, 2018).

Safety related bicycle instruction has shown promise in improving schoolchildren’s knowledge about bicycle safety (Hooshmand, Hotz, Neilson & Chandler, 2014; Riaz, Cuenen, Janssens et al., 2019) although the effect may not be long-term (Van Schagen & Brookhuis, 1994). Cycling skills intervention programmes have been shown to improve children's cycling-related knowledge, and self-perceived confidence. Notably, these programmes did not affect children's willingness to cycle to school (Mandic, Flaherty, et al., 2018). In a systematic review of children and youth bicycle skills training intervention programmes, educational and skills training may increase knowledge of cycling safety but does not appear to translate into less injuries or fatalities recorded in road safety statistics (Richmond, Zhang, Stover, Howard & Macarthur, 2014).

In formal education systems, some centers of Early Childhood Education and Care (ECEC), preschools, and elementary schools, typically provide basic introductions to road safety navigation and cycling education to the extent that is stated by national curriculums. National guidelines and curriculums vary extensively. Prominent cycling countries such as the Netherlands, Denmark and Germany have a mandatory cycling education for all schoolchildren (See Pucher, Dill & Handy, 2010). The purpose of cycling education is to teach children how to ride their bicycles safely and to minimize potential injuries in traffic (Ellis, 2014). There are many purposes for cycling education and cycling per se that reach beyond a focused cycle road safety initiative such as sustainable urban development, promotion of healthy lifestyle and active commuting, and mitigation of climate change. For consistency, this article will focus on the promotion of road safety amongst child cyclists within education settings.

In the same light as vehicle driver education, formal cycling education usually consists of practical on-bike training and off-bike training. This can take place outdoors, on site or in a classroom setting. Relatively short practical on-bike training improves a child's cycling skills, (Ducheyne, De Bourdeaudhuij, Lenoir & Cardon, 2013), and encourages children to use bicycles (Hatfield, Boufous & Eveston, 2019). Off-bike training such as classroom instruction can teach children knowledge about traffic legislation, signs, and safety measurements. In this study, the focus is on the off-bike training taking place in the formal education context.

Online driver education and traffic simulators have been an inseparable part of driver education in past decades. Recently, research has reported on how child pedestrians and cyclists could be informed and educated with the assistance of computers and simulators. Creating effective traffic simulators for children is complex, where the causal connections to real-life naturalistic change of behavior in traffic environment is hypothetical. Confirmation in the real traffic environment imposes ethical considerations of child´s safety and may be difficult to measure, although not completely impossible (See Kircher & Ahlström, 2023). As causation may be complex to confirm, there is still evidence that simulators may be one of the tools to teach children road traffic safety knowledge and expose them to traffic based scenarios within a safe virtual environment within the classroom.

Several pilot studies involving computers and simulators have trialed with child participants, youth, and adults (Table 1). Child pedestrians might take voluntary risks when crossing a street when in a hurry (Charron, Feston & Guégen, 2012). Compared to children, adult pedestrians were more precise in detecting hazard road crossing situations than children in a mixed reality environment (Meir, Parmet & Oron-Gilad, 2013). According to Morrongiello, Corbett, Beer, and Koutsoulianos (2018), training within a virtual pedestrian environment can improve children’s conceptual understanding and crossing behaviors.

Table 1.

Pilots, prototypes, and evidence regarding virtual bicycle education.

Apparatus	Interaction	Stimuli & footage	Learning target	Sample	Gamification elements	Evidence	Context	Reference
23" Acer T232HL + HP EliteBook 850 G2 + built traffic software prototype	Touch screen	Video (GoPro Hero HD1,∼90 cm (cyclists view), 170° FOV)	Situation awareness	n=36, aged 9 - 10*	Scores, hit rate, feedback	SA can be trained using a learning game	Lab	Lehtonen et al. (2017b)
23" Acer T232HL + Dell Latitude E6400 + built traffic software prototype	Touch screen	Video (GoPro Hero HD1,∼90 cm cyclists view, 170° FOV)	Situation awareness	n=49, aged 8 - 9*	Hit rate	Not sure if SA can be trained by using a learning game	Lab	Lehtonen et al. (2017a)
Moodle	Mouse	Photos & videos (view from handlebar)	Traffic safety knowledge	n=44, aged 9 - 13	Relatable character, badges, time limit, graphs	E-learning platform can be viable option to train traffic knowledge	School	Riaz et al. (2019)
Desktop computer + Remote Eye Tracking Device (RED) + tracking software	Mouse	Video (GoPro Hero 2, cyclists view, 170° field of view)	Hazard perception	n=78, aged 9 **	Non mentioned	Simulation improved response rate, reaction times and earlier detection of the hazard.	School	Zeuwts et al. (2017)
22-inch lapstop computer screen + eye-tracking experiment designing software	Mouse	Video (GoPro Hero2, full HD, 170°FOV	Hazard perception	n=11, aged 8 - 9*	Non mentioned	Childrens hazard perception can be studied and measured with aforementioned research instruments.	School	Vansteenkiste et al. (2016)

A replication study by Lehtonen, Sahlberg, et al., (2017b) found adults possessed higher-level situational awareness compared to children, however this report did not deliver robust evidence in improving child cyclists’ situational awareness. A follow-up study by Lehtonen, Airaksinen, et al., (2017a) found a learning game could be used to train children (and adult) cyclists’ situational awareness. This is particularly relevant to the child participants who felt the learning game was both educational and fun. Riaz, Cuenen, Janssens, Brijs and Wets (2019) investigated a gamified e-learning platform to inform elementary school children on traffic safety. The repeat measurements of the results demonstrated that the e-learning platform increased student engagement and their performance in scores for risk management modules.

Zeuwts, Vansteenkiste, Deconinck, Cardon and Lenoir (2017) performed a cluster-randomized control trial to train child cyclists hazard perception skills. Compared to the control group, trained participants detected more hazardous situations and reacted faster. It was suggested that those children who received treatment demonstrated improved processing and prediction of potentially hazardous situations as well as a decrease in their decision processing time.

The studies did not report the type of gamification mechanics that were applied, did not refer to literature around game-based learning (e.g., Kiili, 2005; Girard, Ecalle & Magnan, 2013), and whether these simulators or digital games were able to host optimal learning conditions (e.g., Admiraal, Huizenga, Akkerman & Dam, 2011; Hamari, Shernoff, Rowe, Coller, Asbell-Clarke & Edwards, 2016).

Many of the fore-mentioned digital learning environments are research spin-offs that have an important role as research instruments (e.g., examining children´s hazard perception). They contribute to the improvement in our understanding of these learning environments and their effects on learning. But most, if not all, are not available for large scale use for the majority of the public (e.g., schools), and have not yet been widely tested in schools nor with schools’ devices. Examples are non-existent regarding the design, implementation, and use of these environments in their intended context of use. Children as users and schools as the context may challenge the material, technological and digital experiences these learning environments produce. Therefore, more research and studies are needed to review the effectiveness of these learning environments.

Methodology

VLE and VR environment were designed and tested in elementary schools with school children and their class teachers as research participants. A web-based Virtual Learning Environment (VLE) and a Virtual Reality (VR) learning environment was designed for school children to engage them in learning around road and cycling safety. This included skills and attitudes incorporating digitalization and gamification methods. Both learning environments were designed by the Finnish Road Safety Council (Liikenneturva). The study followed a pre-test and post-test designed to measure differences in evaluation scores of elementary school children using VLE VR learning environment designed to engage children in road safety education with a specific focus on safe cycling. Research questions are as follows: (1) how school children and class teachers evaluate VLE and VR environment designed for safe cycling education, and (2) based on results and findings from user evaluation, what design implications can be addressed to inform further development of virtual learning environments focusing on safe cycling education. In a broader context, this study aimed to address and increase the agency of research participants, especially school children, when designing VLEs and VRs for children by facilitating youth voice and enabling co-creation linked to the United Nations Global Goals number 4 quality education.

Intervention

User evaluation was considered important to inform the future design process of both learning environments (VLE and VR) and to support in scaling them for national reach. User evaluation within this study was exploratory in its nature to understand the overall performance of both learning environments, and how young participants engage with them through play. The target group for which these learning environments was designed were comprehensive education grades three and four (approx. age groups 9-10). However, the design was also interested in how older students would perceive the use of learning environments to inspect the age appropriateness. Concluding, the service is a combination of a VLE and VR environment called Filla&Rilla. Illustrations and examples of Filla&Rilla are presented in Figure 1. When introduced during school year 2018, Filla&Rilla had 1500 active users. In the school year 2022, Filla&Rilla had 15 419 active users.

Figure 1.

From the left - children follow the journey of two virtual characters Filla and Rilla who introduce the user to different tasks such as problem-solving, quizzes, puzzles and 360-degree videos.

Filla&Rilla VLE is a web-based environment that can be accessed with any device where an internet browser is available. Filla&Rilla VLE is built on a WordPress Content Management System (CMS) and the individual tasks are built with content collaboration framework H5P. The interaction with Filla&Rilla VLE content can be either with a mouse (clicking) and/or touch based interface. Devices such as laptops, desktop computers, tablets and smartphones are increasingly found in modern schools enabling school children to access Filla&Rilla VLE content in and outside schools.

Filla&Rilla VR requires specific VR equipment that are not typically found in schools. This equipment can be ordered by schools directly from service provider the Finnish Road Safety Council. These VR boxes (Figure 2) contain a VR headset and a smartphone that is linked to the VR headset device and runs VR content within an application library on the VR Headset.

Figure 2.

Setting up Filla&Rilla VR for students in school classroom.

In Filla&Rilla VR the interaction in the environment is done by moving a small circle sight in the field-of-view that acts in the same manner as a traditional handheld pointing device (i.e., mouse). Interaction is enabled when the user aims the circle at the expected target area within the game interface. The logical task in Filla&Rilla VR is to choose a level in a VR lobby. Levels represent cycling in a realistic traffic simulation. After choosing the level, the user may start the task. The levels are 360-degree videos that have been recorded in a real traffic environment (see Table 2). The video starts automatically, and the user is not expected to pedal or maneuver a bicycle, yet the task is to target one's focus on other users in the simulated traffic environment. A user can score points within the game when they tag other vehicles, pedestrians, and other cyclists in the busy traffic environment. By turning their head, the user can see whether or not vehicles are emerging to pedestrian crossing, or for example they can recognize other cyclists within the game.

Table 2.

Filla&Rilla VR technical specifications.

Camera(s)	Rig	Mounting	Height	Framerate	Stitching	Editing and color definition	360 plug-in to Adobe Premiere	Stabilizing	Rendering	Coding
GoPro HERO4 Black x 6	Freedom 360	Helmet	29" mountain bike with a 185 cm tall camera user	60 FPS	Kolor Autopano Giga & Kolor Autopano Pro	Adobe Premiere Pro	Mettle Skybox Studio	SynthEyes	Adobe Premiere Encoder, codec h265	Unity

Both Filla&Rilla VLE and VR were designed to be fun for children to use, and to incorporate gamification elements. In the VLE, these elements are virtual avatars, namely Filla and Rilla, game-based levels, scores, quiz tasks, memory games, timers in tasks, and feedback all based on content specific items relevant to cycling safety. Filla&Rilla VR learning environment also uses scoring, audio feedback and progressive level design. Gamification elements covered sufficiently the known gamification taxonomies (Toda, Klock, et al., 2019). and delivered gamification elements regarding performance (progression, level, point, stats), ecological (choice, chance, time), and personal (sensation, objective, puzzle, novelty), and to some extent fictional (the story of Filla and Rilla avatars).

Designing Sustainable Technology With Children

Research involving children requires careful consideration since children may not understand what they are participating and consenting to (Read, Fitton & Horton, 2014). Therefore, Participatory Design (PD) was considered an asset to guide practice, knowledge creation, values, and theory (See Frauenberger, Good, Fitzpatrick & Iversen, 2015; Barendregt, Torgersson, Eriksson & Börjesson, 2017). The work of Read, Fitton and Horton (2014) about consent and inclusive participation influenced procedures taken in the study. Children were considered as testers delivering their subjective experiences about learning environments under use and evaluation. Teachers were considered both testers and informants since they also reported what they had seen and witnessed in the classroom when children used the learning environments and made explicit and implicit suggestions for the design based on these findings (See Druin, 2002).

Schools are complex social communities involving local dynamics, cultures and social factors that need to be considered in the research design process (Wang & Hannafin, 2005). For instance, the tech savviness of teachers and their attitudes toward digital games as learning tools may influence results gathered from schoolchildren (All, Castellar & Van Looy, 2016). In addition, maneuvering and navigating research in a school environment obligates working with several stakeholders. Typical stakeholders when working in schools may include regional and local education authorities, school leaders, teachers, caregivers and of course the school children themselves. When working with children, ensuring the informed consent to participate in research is of high ethical importance. This research study has been conducted following the ethical requirements established by the national board of ethics.

Children can be experienced users of technology. Adults may not always fully understand children’s needs as regards technology, or how children would like to interact with technology (Druin, 2002; Fails, Guha & Druin, 2013). Since adults may have a limited understanding of children’s technology use, including children in a carefully planned co-creation design process of technology may have benefits when designing for children as intended end-users. First, handing children the role of a participant in the design process has been known to raise the overall acceptability and success of the design outcomes (Druin, 1999; Kujala, 2003; Kiili, De Freitas, Arnab & Lainema, 2012). Secondly, creating conditions for children's participation is valuable as it gives a voice to a marginalized group and ensures evidence-based realization of the digital needs of children. (Fails, Guha & Druin, 2013). Involving children in the participating and design of technology has also wider implications that may go beyond just designing technology with children. For instance, children participating in the development of the school environment has been linked with improvements to learning, motivation and school engagement (Könings, Seidel & van Merriënboer, 2014). Sanoff (1999, p. 10) describes participation as direct public involvement in decision-making processes that may determine participants' quality and direction in life. Therefore, supported co-creation of the design processes of technology-based learning environments is likely beneficial for all stakeholders involved. It should be seen as fostering school democracy and providing possibilities for learning. In conclusion, this study aimed not only to produce research outcomes but to interact with all school stakeholders, especially children, to provide meaningful experiences and possibilities to interact, design, and imagine with technology.

Sample

The research study took place in Finnish elementary schools in Southern Finland. Convenience sampling was used. A sample of schools were recruited based on their availability and willingness to participate in research activities. Following the recruitment phase, a total of 20 elementary school classes, consisting of 455 students and 20 class teachers, participated in the study. These classes were drawn from five elementary schools located in three different municipalities (Table 3).

Table 3.

School children participants based on their gender and school grade.

Grade		3rd	4th	5th	6th	Total
Girls	Count	85	48	43	61	237
Boys	Count	82	50	40	46	218
Total	Count	167	98	83	107	455
	% of Total	36,7%	21,5%	18,2%	23,5%	100,0%

After initial analysis of the data, a total of 29 participants were excluded from the dataset because of missing data and/or a reliability concern. Class teachers were included in the study as they could provide more information about the contextual use and implementation of the two learning environments when the researcher was not present at the school. For instance, teachers provided details regarding the frequency of use, how they saw VR fit as regards pedagogical aims, and they provided other useful qualitative commentary. Out of 20 class teachers, 13 (n=13) answered the survey of teachers (65 %). Teacher participants mean age was 44 years, and gender distribution was 9 women and 4 men.

Preliminary questions from students were asked regarding the use of VR content and hardware. Out of 325 participants, 62 participants (19 %) reported they or someone in their household possessed VR equipment whereas 81 percent (n=263) of the participants reported no VR equipment was present in their household. 39.7 percent of the participants had never experienced VR before, 38.6 percent reported they had tried VR once or twice, 19 percent reported more than once or twice, and 2.6 percent reported they use VR occasionally.

Instruments

This study incorporated The Fun Toolkit (Read & Macfarlane, 2000) often used with children when evaluating interactive products. Instruments used by child participants were (1) Smileyometer, (2) Again-again table, and (3) social variables regarding the use of VR. For teachers, a typical digital survey was used where questions were asked about the use of Filla&Rilla VLE and VR in the school. For children, paper forms were used because of conveniency.

Both Smileyometer and Again-again table have been designed to evaluate interactive products with children. Smileyometer uses smiley’s instead of numerical likert-scales where numbers represent an abstract subjective construct. In the Smileyometer, smileys represent feelings or expressions: (1) Awful, (2) Not very good, (3) Good, (4) Really good, and (5) Brilliant. The Smileyometer has been especially used with children to measure ‘funness’, the quality of something being fun or enjoyable. According to Read, MacFarlane and Casey (2002), children have unique experiences and environments compared to adults and fun is a natural concept for children to understand. The Smileyometer has been validated and used extensively when evaluating interactive products with children (e.g., Read, MacFarlane & Casey, 2002, Zaman, Abeele, & De Grooff, 2013; van der Sluis, van Dijk & Perloy, 2012). Again-again is a simple table where a child participant is being asked whether they would like to do this (activity) again by choosing from options Yes, Maybe or No. According to Read (2008), this supports the idea we most likely want to return to an activity we have liked and can be also thought of as an endurability measurement of an activity and the engagement felt during it.

Research Protocol

Research involved minor participants, and therefore informed consent of children and their caregivers was pivotal. The process of informed consent first began by contacting elementary school teachers who were willing to take part in the research project. Prior to the actual research activities, teachers had the opportunity to interact with both VLE and VR offered to schools free of charge. Teachers also had the opportunity to discuss any concerns and additional information required with research personnel. After teachers were recruited and informed, families were contacted to inform parents/guardians about the research that was about to start in their school. This was done by a paper sheet that included a consent form and detailed description about the research and its objectives. To take part in the research, participants required parental/guardian(s) consent and a formal signature in the consent form that was gathered by the teacher. Children who did not have parental/guardians’ consent still had the same opportunities to use, try-out and play with both virtual environments without taking part in the actual research or data collection activities.

In schools when researcher first entered a new class researcher read a formal transcript introducing children to the goals and objectives of the research. Children then had the opportunity to ask questions regarding the research. In addition, before gathering any data or feedback, the researcher informed that children were free to end their participation at any time and/or use the program and devices without handing any feedback or data regarding their experiences at any time. The researcher gave a short demo tour showing both learning environments for the participants.

Following the pre- and post-test design, the data collection began by asking participants to fill in a paper form before participants used the VLE or VR environment. Pre- and post-test paper forms were identical and asked the same questions. After participants filled in the pre-test form, participants started to use Filla&Rilla VLE and VR. The first facilitated experience lasted 45 min class, after which, participants were asked to complete the post-test form. After data collection was completed by the participants, both VLE and VR remained in the schools and were used by students and teachers throughout the semester for a period of approximately three months. After the piloting period was over, the researcher repeated data collection via paper form identical in format to the first testing period. During the pilot, participants were not actively encouraged to use the environments by the researcher to have a clear picture of the frequency that would not actively be influenced by an external advocate outside of school. Therefore the frequency of use was based on the teacher and their motivation.

Statistical Analysis

Statistical analysis was designed to answer the research questions and to understand the differences between grades, gender, and social variables. IBM SPSS software was used for statistical analysis. Before analysis, preliminary data checks were carried out. Normality of both pre- and post-test were examined using the Kolmogorov–Smirnov test of normality resulting in violations of the assumption of normality. Both pre- and post-test data included outliers and/or missing data from either pre- or post-test data. According to Rousseeuw and Hubert (2011), outliers can be errors, recorded under exceptional circumstances or can belong to another population. Outliers were treated according to the process of understanding the possible presence of outliers (See Aguinis et al., 2013). After deleting 29 cases from the population, normality within the data was revised, and instead of Kolmogorov-Smirnov test, Kurtosis and Skewness were examined based on applicable parameters (< 2). Levene’s test was carried to check the homoscedasticity of the datasets resulting in failure in post-test. Therefore, Welch’s one-way Analysis of Variance (ANOVA) was used over traditional one-way ANOVA. Pre- and post-test datasets were compared and processed to exclude cases that did not have full data sufficient for comparisons. This resulted in 232 users (n=232) with complete scores from both pre- and post-test questionnaires that were used for analysis of variance tests. Since qualitative material was limited and samples were small, qualitative material is used to illustrate the quantitative findings.

Results

Illustrated in Figure 3, pre-test results indicate there is a statistically significant difference between grades determined by Welch´s one-way ANOVA (F(3,237) = 9.748, p=.000). Partial ETA square results show that 17 % of variance is associated with grade level (η2=.166). Following, Tukey post-hoc test revealed third graders evaluated VLE higher than fourth (p=.05), fifth (p=.024) and significantly higher than sixth graders (p=.000). Continuing with how post-test and follow-up studies were evaluated, post-test results follow the same pattern where significant statistical difference was found between different grades (F(3,242) = 5.301, p=0.001), and now only between third and sixth graders according to Tukey post-hoc test (p = 0.001). Since the effect of grade diminished in post-test (η2=.062) an assumption was made and affirmed that indeed 28 % of the post-test variance was associated with pre-test results (η2=.278).

Figure 3.

User evaluation scores before and after using Filla&Rilla VLE with boxplots, confidence intervals (CI95%) and means.

Figure 4.

Would participants want to use Filla&Rilla VLE again.

Figure 5.

User evaluation mean scores of Filla&Rilla VR.

Figure 6.

Did users of Filla&Rilla VR felt sick after using it (n=436)?

The role of gender in evaluation scores was reviewed. According to the Chi-Square Test an association between gender and evaluation was observed in pre-test (χ2(3) = 9.780, p = .021) where boys (M=3,92) evaluated the VLE higher than girls (M=3,72). Surprisingly, as an association occurred in post-test results as well (χ2(4) = 14.205, p = .007) this time girls (M=3,57) evaluated the VLE slightly higher than boys (M=3,53). In addition, the follow-up post pilot phase study revealed (χ2(4) = 13.362, p = .010) statistical difference where girls (M=3,84) again evaluated the VLE higher than boys (M=3,54).

To deliver more evidence, participants were asked if they wanted to use the VLE again after their first try and again in the follow-up study (Figure 4). On average, participants used the VLE 3.4 times (n=233, SD 1.75). With three scale measure (again-again table). Out of 245 participants 54.3 % wanted to try the VLE again whereas 38.8 % answered “Maybe” and 6.9 % “No”. In addition, Welch´s ANOVA and Tukey post-hoc test confirmed significant statistical difference (F(3,241) = 2.758, p=0.001) between groups and more precisely, between third and sixth graders (p=.000). Repetition was done in follow-up study which confirmed the assumption that participants willingness to continue using the VLE turned more negative over time, and that this negative willingness grows from third to upper grades. Unfortunately grade six post-test data was insufficient to be presented here.

Filla&Rilla VR was evaluated remarkably high at five-level scale by all grades shown in Figure 5. Significant statistical differences between grades were found in pre-test (F(3,428) = 20.868, p=.000) and post-test results (F(3,427) = 17.398, p=.000). In pre-test, 3^rd grade results were significantly higher from 4^th grade (p=.006), 5^th grade (p=.000), and 6^th grade (p=.000) according to Tukey HSD. Furthermore, 4^th and 6^th grade (p=.001) and 5^th and 6^th grade (p=.032) showed significant differences.

Post-test results follow a similar pattern to the pre-test. 3^rd grade results differ from 4^th grade (p=.054) and are significantly higher than 5^th grade (p=.000) and six (p=.000). In addition, 4^th and 6^th grade have a statistical difference (p=.001) according to Tukey HSD.

Since VR may expose users to so-called VR sickness that has similar symptoms to that of motion sickness, a specific focus was put on the assessment of the behavior of users. A question was asked immediately after the use of Filla&Rilla VR - “Do you feel sick?” (Figure 6).

Three out of four (78 %) participants did not report any sort of sickness, vertigo, sensation or behavior that would indicate sickness. Every fifth participant (21 %) reported they felt a bit weird, slightly odd, or somehow different than before they started to use Filla&Rilla VR. Only three participants (1%) reported they felt very sick but no severe sickness (e.g. vomiting) was witnessed. All participants were able to carry out their school day uninterrupted. Participants were asked whether they would like to use VR again. Out of 367 participants 72 % answered Yes, 22 % Maybe and 6 % No. Researcher asked participants after using the VR headset, whether they needed help in using the VR. This was considered important by participants. VR headsets and VR environments are still quite novel pedagogical tools in education, and they are not considered as mainstream devices found in schools such as computers. Therefore, it was important to track what level of autonomy the students had with the use of VR headset and if they were able to navigate the VR environment independently. Out of 436 participants, 11 % reported they needed the assistance of teacher or researcher, and 89 % managed to wear the headset and accomplish the tasks in the environment without assistance. For those seeking assistance, a common problem was that the VR headset was not perfectly fitting to their head and users complained that the headset was a bit loose even after all possible adjustments were made.

A survey for teachers revealed further information regarding the use of both Filla&Rilla VLE and VR. Teachers (n=13) were asked how many hours they spent with VLE and VR during the whole pilot study. Teachers were not explicitly encouraged to use these environments, this was left for teachers own motivation and willingness. On average, teachers spent 7.5 hours of their time in school for both VLE and the VR environment. Teachers also reported that on average their students spent 4.3 hours with both environments. Teachers were asked how their class felt using the Filla&Rilla VLE. They responded there were a little too many tasks that required writing, and for small children some of the tasks were confusing, in many cases they needed the help of the teacher to progress. Teachers also responded that the content and material was on-point; it was seen as meaningful and important to teach. Asking the same question regarding VR, most of the teachers reported their students were really engaged, drawn, and excited about getting to use VR headsets and engage in VR content. A teacher reported, “They [students] were really excited, some of them were surprised how real it felt, many would have wanted to try it again immediately”. Teachers also reported using VR could be pedagogically feasible in other subject domains like in natural sciences. Although they reported positive experiences and promising other fields of use for VR, teachers also noted that VR is not suitable for every child because of the potential of motion sickness.

Discussion

Safe cycling education, whether on-bike or off-bike, plays a role in informing children and adolescents of road safety and cycling knowledge. Research related to the effect of safe cycling education interventions and programmes is mixed. It remains in need of high-quality research studies demonstrating the link between programmes and incident statistics (Richmond et al., 2014). However, it is pivotal to seek new measures and methods to address educationalists on how they could make-use of available educational material in a feasible way that would also appear engaging for children to learn about safe cycling and road use. This is of particular relevance to the need for more awareness around traffic legislation, safe attitudes in a traffic environment with the incorporation of meaningful tasks and objectives.

Appropriately designed and informed virtual learning environments may also play an interesting role in the development of a scalable tool for improved delivery of safe cycling education. As the road safety education agenda is rigorous, it is cardinal that VLE and VR environments are properly designed, tested and aimed at educating children, and their effect and contribution to the road safety agenda will be thoroughly examined. It is worth designing such learning environments to be used by children that are openly sought after as pedagogical tools by parent/guardians and teachers alike.

Based on the results, Filla&Rilla VLE was evaluated moderately high by grade three participants, but this effect diminished as higher-grade students evaluated the VLE with lower scores. As the VLE was targeted and designed for grades three and four explicitly, it seems the design met the given objectives. Although there was some critique towards over emphasis on text material in the tasks. Scores and their relation to gender was assessed, and girls scored the VLE higher than boys in post- and follow-up tests. Although this affiliation was interesting, it is a trend that needs further examination.

Participants were asked whether they would be willing to use the VLE again Participants´ willingness to continue using the VLE turned more negative the more they spent time with it. This may indicate that there were high expectations towards the VLE and its learning content, or some expectations were not met during long-term use, and eventually the VLE started to lose its attractiveness. For future design iterations it is necessary to review the gamification elements and tasks. It is important to consider the long-term engagement and if it is enough. Filla&Rilla VLE included gamification elements and presented a thorough set of tasks included in five dimensions of gamification taxonomy (Toda et al., 2019). There is room for improvement regarding the fictional and social dimensions of FillaRilla VLE since at this design stage there was very little to total absence of fictional storytelling or social cooperation. It is also important to assess how each gamification element contributes to users’ subjective engagement, and how specific gamification elements contribute to safe cycling education if any. In addition, the aesthetics and “playfulness” of the Filla&Rilla VLE may need reinvention and further user design consideration for older students to understand why participants from grade four, five and six in this study did not find the Filla&Rilla VLE as appealing as younger participants. Teenagers are known to be drawn to motor vehicles, bikes, and cars over bicycles, and this early association could be important to consider in the future design. In addition, making cycling cool may very well be the first step toward increasing cycling and encouraging children and young people to cycle. (Underwood et al., 2014). Teachers reported Filla&Rilla VLE was both pedagogically feasible and supported the implementation of the national core curriculum, yet some teachers also reported the content was dull. A review of content is recommended based on this feedback from teachers. This would require a revision of the tasks and the addition of more engaging tasks over tasks that require writing based answers. However, it is important to note that although some learning content would be experienced as dull or not fun, it does not indicate it would not be effective in the transfer of knowledge.

Filla&Rilla VR was evaluated with high scores among users in all grades, but statistics also show the level of enjoyment or fun decreased as participants were older. This is aligned with the trend found also in the Filla&Rilla VLE evaluation scores. However, the scores were particularly high indicating that the use of VR for educational purposes is appealing and fun for school children as well as for teachers. The difference between 3^rd and 6^th grade is quite significant and consistent since both pre and post test results have a similar pattern across grades. This is also the case with 4^th and 5^th grade results. There is a trend where 3^rd grade students evaluated Filla&Rilla VR very highly whereas participants got older, their perception of fun captured lower scores. During the study a large proportion (40 %) of the participants were experiencing VR for the first time, and this may have affected the evaluation scores, particularly regarding the pre-test scores where participants may have had high expectations. Motion sickness appears to be an issue for some users of Filla&Rilla VR. This is an issue in majority of the VR applications to some extent and with varying symptoms (Chang, Kim, & Yoo, 2020). Although the study did not witness any major incidents with sickness, and 80 percent of users did not report any kind of sickness, the possibility of getting sick requires supervision of the teacher who is ultimately responsible for student’s wellbeing during school hours. It is also important to keep the users seated since as there is a possibility that without supervision students would stand up potentially causing injury. Furthermore, there are certain limitations of how long it is advisable to use per session.

As VR is remains a relatively new and novel technology inside a classroom, it requires further co-creation, design consideration and an examination on how to best to apply it for pedagogical use in schools.

Limitations and Suggestions for Further Future Research

There are several limitations within this study. Firstly, this study does not make a claim that using either Filla&Rilla VLE or VR environment would contribute to the user’s knowledge regarding road safety and cycling or creating a change in users’ behavior in real life settings. These certainly should be examined in future research with a more robust research design in a laboratory simulation. It is also important to remember that the study focuses only on off-bike training. Secondly, this study did not shed light on what specific elements users felt were appealing in Filla&Rilla learning environments, although this study did gather evidence regarding the aspects that were not enjoyed. It is likely that in future research, there is a need to examine more closely what tasks are more appealing for children, and for what age groups. Therefore, as this study interacted with children as test group, it would be beneficial in the future to co-design virtual learning environments with children and to raise the level of their involvement to ensure youth voice in both design and research of such technology. Regarding the experience with VR environment and motion sickness, it would be beneficial to merge the VR headset with an exercise bicycle in the classroom and compare results with and without the exercise bicycle. This would examine whether a tangible element would lower the probability of motion sickness. Since the VR environment used 360-degree video, it would also be important to evidence, what would be the difference between real-life videos vs. a 100 % animated virtual environment. Finally, large scale randomised controlled trial research studies and longitudinal research studies would be desired to evidence the cognitive and behavioral outcomes that using Filla&Rilla may produce in real life situations.

Conclusion

Safe cycling education program Filla&Rilla was designed through a virtual learning environment and head mounted virtual reality learning environment targeted for use by school children and their teachers in Finland. The web-based virtual learning environment was designed to deliver safe cycling education and a virtual reality learning environment was designed to teach skills to detect hazards in traffic. A total of 455 students and 20 class teachers participated in the research studying what expectations, experiences, and perceptions both students and teachers had when they used learning environments designed to engage children in safe cycling education. A child centered toolkit was used to capture child participants subjective experiences through a pre- and post-test on how fun it was to use learning environments in school. A survey was done for teacher participants to obtain more information about the content of this pilot study in the classroom.

A VLE was evaluated relatively positively among 3^rd and 4^th grade participants in pre- and post-test questionnaires indicating that the environment and the learning content was appealing for intended target groups. However, evaluation scores decreased among older child participants. For 6^th grade students there is a need to assess whether the learning content is appealing and relevant. The VR learning environment was evaluated exceptionally high in all grade levels yet also showed a similar pattern where evaluation scores decreased among older participants. For 40 percent of the participants, this study exposed them for the first time to virtual reality. Motion sickness was witnessed with every fifth student reporting slight sickness, and only a percent of students reported they felt sick. Almost 80 percent did not report any feelings of sickness. Teachers reported that they used both learning environments on average 4.3 hours during the piloting phase lasting three months. Teachers reported their students felt particularly drawn to VR content, whereas the VLE had too many tasks that some of the students felt were un-engaging or that there were too many tasks that involved writing.

This study did not provide evidence as to whether students effectively learned safe cycling skills, nor did it review students' behavior altered real-life traffic environments. There is, however, growing evidence that off-bike simulators may be a feasible way to deliver both road and safe cycling education for children. It is important to design and assess what is scalable, feasible and a working solution to provide these learning environments within mainstream formal education settings. This study demonstrated that VLE and VR environments designed for safe cycling education are appealing for children to use particularly for 3^rd and 4^th grade students, which creates possibilities for further national cycling education programs and supports the role of teachers to engage children in cycling through the curriculum. VR can be an effective part of the spectrum of pedagogical tools and methods that supports teachers in the delivery of safe cycling education. However, this would require supervision within the classroom, as well as further development of these environments in their design.

Footnotes

Acknowledgements

The research was supported by the Finnish Road Safety Council and people involved in the creation and implementation of Filla&Rilla. Sincere thanks to Ida Maasalo and Jen Hesnan for discussions and help with the study. Sincere thanks to all schools, headmasters, teachers, parents/guardians, and all children and youth involved for their valuable contribution to this research.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Finnish Cultural Foundation´s Pirkanmaa Regional Fund [grant number 50192006].

ORCID iD

Jaakko Vuorio

Author Biography

Jaakko Vuorio is a Doctoral Researcher in Humans and Technologies at Tampere University.

References

Admiraal

Huizenga

Akkerman

Dam

G. ten.

(2011). The concept of flow in collaborative game-based learning. Computers in Human Behavior, 27(3), 1185–1194. https://doi.org/10.1016/j.chb.2010.12.013

Aguinis

Gottfredson

R.K.

Joo

(2013). Best-practice recommendations for defining, identifying, and handling outliers. Organ. Res. Methods. Grubbs Hunter & Schmidt Rousseeuw & Leroy. https://doi.org/10.1177/1094428112470848

Barendregt

Torgersson

Eriksson

Börjesson

(2017). Intermediate-Level Knowledge in Child-Computer Interaction: A Call for Action. In Proceedings of the 2017 Conference on Interaction Design and Children (IDC '17). Association for Computing Machinery, New York, NY, USA, 2017, 7–16. https://doi.org/10.1145/3078072.3079719

Bo Andersen

Schnohr

Schroll

Ole Hein

(2000). All-cause mortality associated with physical activity during leisure time, work, sports, and cycling to work. Archives of Internal Medicine, 160(11), 1621–1628.

All

Castellar

E. P. N.

Van Looy

(2016). Assessing the effectiveness of digital game-based learning: Best practices. Computers & Education, 92, 90-103. https://doi.org/10.1016/j.compedu.2015.10.007

Chang

Kim

H. T.

Yoo

(2020). Virtual reality sickness: a review of causes and measurements. International Journal of Human–Computer Interaction, 36(17), 1658-1682. https://doi.org/10.1080/10447318.2020.1778351

Charron

Festoc

Guéguen

(2012). Do child pedestrians deliberately take risks when they are in a hurry? An experimental study on a simulator. Transportation Research Part F: Traffic Psychology and Behaviour, 15(6), 635–643. https://doi.org/10.1016/j.trf.2012.07.001

de Hartog

J. J.

Boogaard

Nijland

Hoek

(2010). Do the health benefits of cycling outweigh the risks? Environmental Health Perspectives, 118(8), 1109–1116. https://doi.org/10.1289/ehp.0901747

Druin

(1999). Cooperative inquiry: developing new technologies for children with children. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '99). Association for Computing Machinery, New York, NY, USA, 1999, 592–599. https://doi.org/10.1145/302979.303166

10.

Druin

(2002). The role of children in the design of new technology. Behaviour and information technology, 21(1), 1-25.

11.

Ducheyne

De Bourdeaudhuij

Lenoir

Cardon

(2013). Does a cycle training course improve cycling skills in children? Accident Analysis and Prevention, 59, 38–45. https://doi.org/10.1016/j.aap.2013.05.018

12.

Ellis

(2014, January). Bicycle safety education for children from a developmental and learning perspective (Report No. DOT HS 811 880). Washington, DC: National Highway Traffic Safety Administration.

13.

Fails

J. A.

Guha

M. L.

Druin

(2013). Methods and techniques for involving children in the design of new technology for children. Foundations and Trends® in Human–Computer Interaction, 6(2), 85-166.

14.

Frauenberger

Good

Fitzpatrick

Iversen

O. S.

(2015). In pursuit of rigour and accountability in participatory design. International journal of human-computer studies, 74, 93-106.

15.

Girard

Ecalle

Magnan

(2013). Serious games as new educational tools: How effective are they? A meta-analysis of recent studies. Journal of Computer Assisted Learning, 29(3), 207–219. https://doi.org/10.1111/j.1365-2729.2012.00489.x

16.

Gotschi

(2011). Costs and benefits of bicycling investments in Portland, Oregon. Journal of Physical Activity & Health, 8 Suppl 1(Suppl 1), 49–58.

17.

Götschi

Garrard

Giles-Corti

(2016). Cycling as a part of daily life: A review of health perspectives. Transport Reviews, 36(1), 45–71. https://doi.org/10.1080/01441647.2015.1057877

18.

Hamari

Shernoff

D. J.

Rowe

Coller

Asbell-Clarke

Edwards

(2016). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54, 170–179. https://doi.org/10.1016/j.chb.2015.07.045

19.

Hatfield

Boufous

Eveston

(2019). An evaluation of the effects of an innovative school-based cycling education program on safety and participation. Accident Analysis and Prevention, 127, 52–60. https://doi.org/10.1016/j.aap.2019.02.021

20.

Hooshmand

Hotz

Neilson

Chandler

(2014). BikeSafe: Evaluating a bicycle safety program for middle school aged children. Accident Analysis and Prevention, 66, 182–186. https://doi.org/10.1016/j.aap.2014.01.011

21.

IRTAD , 2014. Road Safety Annual Report 2014. In International Traffic Safety Data and Analysis Group (IRTAD). OECD, Paris.

22.

Ishii

Hitosugi

Kandori

et al. Physiological status and anatomical severity factors associated with child versus adult bicyclist fatalities based on a national trauma dataset. Sci Rep 12, 18354 (2022). https://doi.org/10.1038/s41598-022-21949-z

23.

Kiili

(2005). Digital game-based learning: Towards an experiential gaming model. Internet and Higher Education, 8(1), 13–24. https://doi.org/10.1016/j.iheduc.2004.12.001

24.

Kiili

De Freitas

Arnab

Lainema

(2012). The design principles for flow experience in educational games. In Procedia Computer Science (Vol. 15, pp. 78–91). https://doi.org/10.1016/j.procs.2012.10.060

25.

Kircher

Ahlström

(2023). Children and youngster's gaze behaviour when cycling in familiar environments. Journal of Cycling and Micromobility Research, 1, 100006. https://doi.org/10.1016/j.jcmr.2023.100006

26.

Kujala

(2003). User involvement: A review of the benefits and challenges. Behaviour & Information Technology, 22(1), 1–16. https://doi.org/10.1080/01449290301782

27.

Lehtonen

Airaksinen

Kanerva

Rissanen

Ränninranta

Åberg

(2017a). Game-based situation awareness training for child and adult cyclists. Royal Society Open Science, 4(3). https://doi.org/10.1098/rsos.160823

28.

Lehtonen

Sahlberg

Rovamo

Summala

(2017b). Learning game for training child bicyclists’ situation awareness. Accident Analysis and Prevention, 105, 72–83. https://doi.org/10.1016/j.aap.2016.07.036

29.

Liikenneturva

2019. Finnish Road Safety Council. Alakouluikäisten lasten henkilövahingot liikenteessä. Tilastokatsaus. Injuries and Fatalities in Traffic Involving Children Aged 7–12. Statistical Review. https://www.liikenneturva.fi/sites/default/files/materiaalit/Tutkittua/Tilastot/tilastokatsaukset/tilastokatsaus_alakouluikaiset.pdf

30.

Macmillan

Connor

Witten

Kearns

Rees

Woodward

(2014). The societal costs and benefits of commuter bicycling: Simulating the effects of specific policies using system dynamics modeling. Environmental Health Perspectives, 122(4), 335–344. https://doi.org/10.1289/ehp.1307250

31.

Mandic

Flaherty

Pocock

Kek

C. C.

McArthur

Ergler

Bengoechea

E. G.

(2018). Effects of cycle skills training on children's cycling-related knowledge, confidence and behaviours. Journal of Transport & Health, 8, 271-282. https://doi.org/10.1016/j.jth.2017.12.010

32.

Melin

M. C.

Peltomaa

Schildt

Lehtonen

(2018). Where do people direct their attention while cycling? A comparison of adults and children. Transportation research part F: traffic psychology and behaviour, 58, 292-301. https://doi.org/10.1016/j.trf.2018.06.017

33.

Meir

Parmet

Oron-Gilad

(2013). Towards understanding child-pedestrians’ hazard perception abilities in a mixed reality dynamic environment. Transportation Research Part F: Traffic Psychology and Behaviour, 20, 90–107. https://doi.org/10.1016/j.trf.2013.05.004

34.

Könings

K.D.

Seidel

van Merriënboer

J.J.G.

(2014). Participatory design of learning environments: integrating perspectives of students, teachers, and designers. Instr Sci 42, 1–9, 2014. https://doi.org/10.1007/s11251-013-9305-2

35.

Morrongiello

B. A.

Corbett

Beer

Koutsoulianos

(2018). A pilot randomized controlled trial testing the effectiveness of a pedestrian training program that teaches children where and how to cross the street safely. Journal of pediatric psychology, 43(10), 1147-1159.

36.

OECD/International Transport Forum (2013). Cycling, Health and Safety, OECD Publishing/ITF. https://doi.org/10.1787/9789282105955-en

37.

Oja

Titze

Bauman

de Geus

Krenn

Reger-Nash

Kohlberger

(2011, August 1). Health benefits of cycling: A systematic review. Scandinavian Journal of Medicine and Science in Sports. John Wiley & Sons, Ltd (10.1111). https://doi.org/10.1111/j.1600-0838.2011.01299.x

38.

Pucher

Dill

Handy

(2010). Infrastructure, programs, and policies to increase bicycling: An international review. Preventive Medicine, 50(SUPPL.), S106–S125. https://doi.org/10.1016/j.ypmed.2009.07.028

39.

Rabl

de Nazelle

(2012). Benefits of shift from car to active transport. Transport Policy, 19(1), 121–131. https://doi.org/10.1016/j.tranpol.2011.09.008

40.

Read

J. C.

(2008). Validating the Fun Toolkit: an instrument for measuring children’s opinions of technology. Cognition, Technology & Work, 10(2), 119–128. https://doi.org/10.1007/s10111-007-0069-9

41.

Read

J. C.

Fitton

Horton

(2014). Giving ideas an equal chance: inclusion and representation in participatory design with children. In Proceedings of the 2014 conference on Interaction design and children 2014, (pp. 105-114).

42.

Read

J. C.

MacFarlane

S. J.

(2000). Measuring fun—usability testing for children. Computers and Fun 3, BCS HCI Group, York, England

43.

Read

J. C.

MacFarlane

Casey

(2002). Endurability, engagement and expectations: Measuring children’s fun. In Interaction design and children (Vol. 2, No. 2002, pp. 1-23). Eindhoven: Shaker Publishing.

44.

Riaz

M. S.

Cuenen

Janssens

Brijs

Wets

(2019). Evaluation of a gamified e-learning platform to improve traffic safety among elementary school pupils in Belgium. Personal and Ubiquitous Computing. https://doi.org/10.1007/s00779-019-01221-4

45.

Richmond

S. A.

Zhang

Y. J.

Stover

Howard

Macarthur

(2014). Prevention of bicycle-related injuries in children and youth: a systematic review of bicycle skills training interventions. Injury prevention, 20(3), 191-195.

46.

Rousseeuw

P.J.

Hubert

(2011). Robust statistics for outlier detection. Wiley Interdiscip. Rev. Data Min. Knowl. Discov 1(1), 73–79 (2011). https://doi.org/10.1002/widm.2

47.

Sanoff

(1999). Community participation methods in design and planning. John Wiley & Sons.

48.

Toda

A.M.

Klock

A.C.T.

Oliveira

et al. Analysing gamification elements in educational environments using an existing Gamification taxonomy. Smart Learn. Environ. 6, 16 (2019). https://doi.org/10.1186/s40561-019-0106-1

49.

Underwood

S. K.

Handy

S. L.

Paterniti

D. A.

Lee

A. E.

(2014). Why do teens abandon bicycling? A retrospective look at attitudes and behaviors, Journal of Transport & Health, Volume 1, Issue 1, 2014, Pages 17-24, ISSN 2214-1405. https://doi.org/10.1016/j.jth.2013.12.002

50.

van der Sluis

van Dijk

Perloy

(2012). Measuring fun and enjoyment of children in a museum: Evaluating the Smileyometer. In Spink

A.J.

Grieco

Krips

O.E.

Loijens

L.W.S.

Noldus

L.P.J.J.

Zimmerman

P.H.

(Eds), Proceedings of Measuring Behavior 2012. Utrecht, The Netherlands, August 28-31, 2012, 86-89, Available at SSRN: https://ssrn.com/abstract=2956298

51.

van Schagen

I.N.L.G.

Brookhuis

K.A.

(1994). Training young cyclists to cope with dynamic traffic situations, Accident Analysis & Prevention, Volume 26, Issue 2, 1994, Pages 223-230, ISSN 0001-4575. https://doi.org/10.1016/0001-4575(94)90092-2

52.

Vansteenkiste

Zeuwts

Cardon

Lenoir

(2016). A hazard-perception test for cycling children: An exploratory study. Transportation Research Part F: Traffic Psychology and Behaviour, Volume 41, Part B, 2016, Pages 182-194, ISSN 1369-8478. https://doi.org/10.1016/j.trf.2016.05.001

53.

Wang

Hannafin

M.J.

(2005). Design-based research and technology-enhanced learning environments. ETR&D 53, 5–23. https://doi.org/10.1007/BF02504682

54.

Zaman

vanden Abeele

Dirk de Grooff

(2013). Measuring product liking in preschool children: An evaluation of the Smileyometer and This or That methods. International Journal of Child-Computer Interaction 1(2), 61–70. https://doi.org/10.1016/j.ijcci.2012.12.001

55.

Zeuwts

L. H.

Vansteenkiste

Deconinck

F. J.

Cardon

Lenoir

(2017). Hazard perception training in young bicyclists improves early detection of risk: A cluster-randomized controlled trial. Accident Analysis & Prevention, 108, 112-121. https://doi.org/10.1016/j.aap.2017.08.024

Studying the Use of Virtual Reality Learning Environments to Engage School Children in Safe Cycling Education

Abstract

Background

Intervention

Methods

Results

Discussion

Keywords

Background

Methodology

Intervention

Designing Sustainable Technology With Children

Sample

Instruments

Research Protocol

Statistical Analysis

Results

Discussion

Limitations and Suggestions for Further Future Research

Conclusion

Footnotes

Acknowledgements

Declaration of Conflicting Interests

Funding

ORCID iD

Author Biography

References