Sage Journals: Discover world-class research

Abstract

The education community has gradually recognized the value of augmented reality (AR) technology and its positive impacts on student learning performance. Still, few studies have proposed specific solutions and results to confirm this phenomenon for first graders in English vocabulary learning. Hence, this research explores the effects of AR apps on first graders’ English learning motivation and performance in addition to analyzing the moderating effects of learning styles. For this purpose, the class was divided into the experimental group that used an AR app in learning and a control group that used traditional learning methods. A series of stochastics analyses were carried out. Based on the results, this study finds that incorporating AR apps into English vocabulary learning can significantly improve first graders’ motivation and English vocabulary learning performance. The study’s findings will be beneficial to educational practitioners and researchers in developing instructional theories, strategies, and materials and improving elementary students’ learning motivation and performance.

Keywords

augmented reality elementary education E-learning motivation

Introduction

In the context of ESL/EFL (English as a second language/English as a foreign language), vocabulary learning is considered to be the key to mastering English (Yaacob et al., 2019). Strong performance of vocabulary learning will help develop language skills (Rahimi & Allahyari, 2019), including listening, speaking, reading, and writing. Akbulut and Cardak (2012) believe that successful foreign language courses have several common features, including early language learning and creative use of technology. Mahadzir and Phung (2013) indicate that primary schools focusing on English learning lack creative use of technology and students’ active participation. AR can interact with objects in the virtual or real world and learn through experiments, participation, and interaction, increasing learners’ motivation and attention (Singhal et al., 2012). Santos et al. (2016) note that AR technology can lead to better retention of new vocabulary. With the rapid development of digital technology, several state-of-the-art technologies are applied to foreign language learning to improve language learning effectiveness (Zhang & Zou, 2020). AR, deepening learning interactions by imposing digital information on natural physical settings (Chen et al., 2020), is a promising way for improving vocabulary learning performance.

However, low motivation for vocabulary learning is one of the main problems (Legault et al., 2006). Motivation includes one’s efforts and desire to achieve goals and is closely related to the process of language learning and can increase learning performance (Liu et al., 2021). It is a critical element for educators in helping students achieve better learning outcomes (Paas et al., 2005). Therefore, improving students’ learning motivation is a goal that education units strive for. Studies have found that using AR for learning can reduce student anxiety, increase motivation, improve learning effectiveness, and increase satisfaction (Bursali & Yilmaz, 2019; Chen et al., 2020; Di Serio et al., 2013; Hsu, 2017). Furthermore, Hsu (2017) demonstrates that learning styles will also significantly affect anxiety in the context of AR in English learning. Therefore, this study sets the research object as first graders and explores the literature on AR technology’s impact on English learning, in order to understand the impacts of integrating AR apps into English vocabulary learning on the motivation and performance of elementary school students. Further, this study takes learning styles as a moderating variable to explore whether different learning styles have an impact on student learning motivation and performance.

The research questions are: (1) Will learning using AR apps have a more substantial effect on first graders’ motivation and learning performance than traditional learning? (2) Will learning style moderate the impact of learning method (i.e., AR apps learning vs. traditional learning) on first graders’ motivation and learning performance? This research will propose in-depth suggestions for teachers for how to use AR apps in learning based on the research results. It may also be used as a basis for researchers to develop educational strategies and theories.

Literature Review

Learning Motivation Theory

Pintrich (1999) argued that learning motivation is an essential factor in improving learning performance. Early scholars have proposed several arguments about motivation. McDougall’s (1908) instinct theory of motivation considers that individual behavior is driven by instinct. Hull’s (1943) driven reduction theory advocates that individuals have an internal drive. Motivation is the core of learning and is necessary in education (Maehr & Meyer, 1997). Hulleman et al. (2008) define motivation as a driving force and believe that motivation can drive individuals to participate in activities. Motivation can also be defined as why a person wants to repeat a behavior (Elliot & Covington, 2001). Motivation provides learners with the motivation to work hard for learning and persist in learning (Rost, 2006). Activity Theory (AT) posits that human activities are motived by biologically or culturally constructed needs. A need becomes a motive once directed at the activity’s focus, giving the direction to the activity (Engeström, 1999). The importance of activity theory in language learning is well recognized (Ellis, 1997; Yu & Lee, 2015). Schinke-Llano (1995) contends that teachers should not regard the language itself as the goal for learning in the process of language learning but should make students use language as a tool through activities and dialogs. Yu and Lee (2015) show that student motives could directly influence students’ participation in group activities from an activity theory perspective. Moreover, students’ engagement in activities is affected by motivation (Dincer et al., 2019).

Gardner et al. (2004) separate the two concepts of motivation and orientation. Motivation is the degree of investment (motivational intensity), desire to learn (willingness to learn), and attitudes toward learning the language (attitudes toward learning the language). However, orientation represents the reasons for learning a language, like integrative orientation and instrumental orientation. Instrumental orientation is related to external regulation. Integrative orientation is related to more self-determined extrinsic motives and intrinsic motives (Noels, 2001a, 2001b). The dichotomy makes it easy for people to ignore other factors that affect language learning motivation. Therefore, later scholars argued that situational factors of motivation should be integrated into research and learning (Guay et al., 2000). The exploration of learning motivation and learning situations allows the theory and practice to complement each other. Keller (1983) integrated motivational-related theories and proposed the ARCS motivation model to facilitate students’ learning motivation. The ARCS motivation model summarizes learning motivation’s main factors into Attention, Relevance, Confidence, and Satisfaction. Keller believes that if the learning design and materials do not arouse students’ interest or attention, learning outcomes and performance will be affected. The ARCS model can provide effective and systematic guidance, which is very important in the design of AR teaching materials, especially in improving students’ motivation (Mahadzir & Phung, 2013).

Augmented Reality and English Learning

The concept of AR is derived from the reality-virtuality continuum of Milgram and Kishino (1994). They regard the natural environment and the virtual environment as the two ends of a continuous system. The middle of the system is defined as mixed reality. In mixed reality, the end close to the virtual environment is called augmented virtuality. Conversely, the end closer to the natural environment is called AR (Walker et al., 2017). AR takes the actual environment as the background, adds virtual elements to the natural environment, and presents virtual reality in 3D in a real-time interactive way (Azuma et al., 2001). AR was first used for pilot training (Caudell & Mizell, 1992). Because of the development of digital technology and the popularization of mobile devices, AR has been widely used in many fields, including architecture (Lin & Hsu, 2017), maintenance (Schwald & De Laval, 2003), entertainment (Ozbek et al., 2004), education (Akçayır & Akçayır, 2017; Bacca et al., 2014; Nincarean et al., 2013; Radu, 2012, 2014), medicine (De Buck et al., 2005), and psychotherapy (Chicchi Giglioli et al., 2015; Juan et al., 2005; Wrzesien et al., 2013).

In the research of technology-enhanced learning (TEL), technology plays a vital role in the learning and teaching process. It can enhance learning effectiveness and efficiency (Goodyear & Retalis, 2010). Ahmadi and Reza (2018) believe that technology can support interactions between teachers and students and can help students develop thinking skills and make learning more efficient. AR technology has many advantages in the learning environment. It can help students explore in the real world (Dede, 2009), and by adding virtual elements to the real world also helps students observe tiny things (Wu et al., 2013). Thus, the application of emerging technologies such as AR into learning is a topic of concern to many educators and scholars (Akçayır & Akçayır, 2017; Bower, 2008; Dalgarno & Lee, 2010; Dunleavy et al., 2009; Kye & Kim, 2008; Walker et al., 2017). The immersive and interactive features of AR can enhance students’ satisfaction, help students understand the learning content more comprehensively, and promote interaction and collaboration between students (Dalgarno & Lee, 2010; Dunleavy et al., 2009). These technological features of AR are believed to improve kinesthetic learning tasks, help memorize cognitive processes (Chien et al., 2010; Dunleavy et al., 2009), and enhance students’ learning motivation (Di Serio et al., 2013). AR provides students with novel learning situations to enable them to become more engaged in the learning process (Lee et al., 2009). It can be seen that augmented reality can effectively enhance learning effectiveness since it can enhance spatial concepts through direct interaction with the virtual image of stereo projection (Liarokapis et al., 2004) and improve the interactivity of learning (Kaufmann & Schmalstieg, 2003).

Barreira et al. (2012) adopted the AR game-based system-MOW (Matching Objects and Words) for teaching experiments to children. They found that students who received AR learning were better than those who received traditional methods. Juan et al. (2010) apply AR games to vocabulary learning for children. Mahadzir et al. (2013) conducted experiments on primary school students, finding that AR can make students show curiosity about the course and stay focused throughout the learning process. Through AR technology, the interaction and cooperation between students can be enhanced. Santos et al. (2016) indicate that using AR applications can provide good system usability and leads to better retention of new words in the vocabulary. Chen and Chan (2019) demonstrated that AR technology can improve vocabulary learning in kindergartens and expand children’s vocabulary. Rozi et al. (2021) state that AR will make learning English vocabulary more attractive and argue that it can help students remember and learn vocabulary.

The contextual learning theory emphasizes that knowledge is related to the learning context through interaction and effective participation in the learning process, and the true meaning of knowledge is understood (Lave & Wenger, 1991). Creating a contextual learning environment can affect students’ understanding and form a strong interaction between teachers and students (Sahin, 2019). Self-determination theory (SDT) posits that learning needs to be driven by motivation (Deci & Ryan, 2000). People will adjust their actions according to the degree of satisfaction of their needs (Deci & Ryan, 1985, 2000). AR and related applications are believed to enhance students’ learning motivation (Ogawa, 2016; Walker et al., 2017). Further, previous studies have explored the use of AR to teach from the Flow Theory and find that students will be more focused when engaging in meaningful activities, increasing student motivation (Ibáñez et al., 2014; Liao, 2006). Based on the foregoing discussion, this research contends that applying AR apps to primary school students’ English vocabulary learning can enhance motivation and learning effectiveness. Hence, we hypothesize (Figure 1):

H1. Learning via AR apps has a stronger effect on motivation than traditional learning.

H2. Learning via AR apps has a stronger effect on learning performance than that of traditional learning.

Figure 1.

Research framework.

Learning Style

Gregorc (1979) defines learning style as a person’s preference for adapting to the environment. It is a behavior formed through the influence of personal internal subjective factors and external environmental factors. Keefe (1982) defined personal learning style as the interaction between perception and learning environment. He believed that learning style is a stable indicator of learners’ response to the learning environment. Students’ learning performance is mainly affected by their ability and learning style, and students usually choose their learning activities based on their preferred learning style (Derakhshan & Shakki, 2018; Sternberg & Grigorenko, 1997). Generally speaking, learning styles are the personal characteristics of learners through learning behaviors, ways of receiving education, and ways of interacting with the learning environment (Chang et al., 2009; Reiff, 1992; Tseng et al., 2008). The learning styles are not only the individual differences between students but also affect the learning performance of students. There is a voluminous literature on learning styles (Bandler & Grinder, 1975; Derakhshan & Shakki, 2018; Kolb, 1976, 2007; Ocepek et al., 2013). Among them, Kolb’s (1976) learning style theory and Visual, Auditory, and Kinesthetic (VAK) learning style theory (Bandler & Grinder, 1975) provides a comprehensive overview of how learners process and learn new information. They are widely used in the field of adaptive learning (Akbulut & Cardak, 2012).

Many scholars have discussed individual differences in learning styles (Derakhshan & Shakki, 2018; Dunn & Dunn, 1993; Reid, 1995). After reviewing the literature on learning styles, this research argues that the VAK learning style theory proposed by Bandler and Grinder (1975) is the most suitable for elementary students. VAK learning style theory divides learning styles into visual, auditory, and kinesthetic types through sensory differences (Fleming & Mills, 1992). Visual learners preferred to learn visually, meaning that they prefer to read and need visual stimulation when learning. In the classroom, diagrams and text instructions can help visual students to learn. Auditory students tend to learn verbally and like to have group discussions, communication, and teamwork. These students are usually suitable for verbal instructions in learning (Oxford & Anderson, 1995). Kinesthetic students prefer experiential learning and interacting physically with the learning environment, and enjoy outdoor learning, theater performances, and interviews (Kinsella, 1995). Hence, given the above arguments, we hypothesize:

H3. Learning style moderates the relationship between learning method and motivation.

H4. Learning style moderates the relationship between learning method and learning performance.

Methodology

Participants and Measurement Instruments

In this study, the independent variable is the learning method, and the dependent variables are learning motivation and learning performance. Four questionnaires are commonly-used to assess learning motivation (Gopalan et al., 2020): the Instructional Materials Motivation Survey (IMMS) (Keller, 1987), the Motivational Strategies for Learning Questionnaire (MSLQ) (Pintrich et al., 1993; Yang et al., 2020), the Student Motivation toward Science Learning Questionnaire (SMTSL) (Tuan et al., 2005), and the Self-Regulation of Academic Motivation (SRAM) (Sonia Gonzalez et al., 2006). The IMMS is used to determine students’ motivation in learning since the IMMS survey constructs are derived from the ARCS model of motivation (Huang et al., 2006). The ARCS scale is also used by previous studies to measure learning motivation (Bolliger et al., 2010; Green & Sulbaran, 2006; Rodgers & Withrow-Thorton, 2005) and is aligned with the objectives of this research. Hence, this study adopts the IMMS (Keller, 1983, 1987) to develop the 25 items with a 5-point Likert scale as an index for evaluating first graders’ motivation. Further, the learning style is treated as a moderating variable. The VAK Learning Style Scale, which classifies respondents into visual, auditory, and kinesthetic learners is used to measure students’ learning styles (Appendix 1), with 13 items adapted from previous research (Bandler & Grinder, 1975; Beatrice, 1995), This research is conducted in a public elementary school in northern Taiwan. The participants are first graders from two classes, one as the experimental group and one as the control group. The experimental group used an AR app, while the control group used traditional learning methods for English vocabulary learning.

Experiment Procedure

After selecting the participants and the developing instruments, the formal experiment is conducted by this research (Figure 2). First, to establish the reliability of the researcher’s self-translated and revised survey questionnaires of these instruments, a pretest with 25 first graders is conducted before the formal treatment.

Figure 2.

Experiment procedure.

This research explores the impacts of learning via AR apps on the motivation and learning performance of first graders with different learning styles. Before the treatment, the scores on the ARCS, VAK, and English vocabulary proficiency of the two groups are measured to confirm that there is no difference between the experimental and control groups. Next, the subjects are asked to fill out the VAK scale. First-graders do not understand the questionnaire or cannot answer the questionnaire. They will be assisted by the class instructor/tutor, on both the ARCS scale and the VAK scale. Students are classified into a learning style based on which one of the three learning styles of the VAK scale they score the highest on. For example, 7V3A4K is classified as V. If a student had the same score for two or more styles, the class teacher classified the participants as a single learning style. The ARCS Learning Motivation Scale is then conducted to understand the first graders’ motivation before the experiment. Finally, the vocabulary ability test is conducted to confirm whether the two groups of subjects had the same level of English vocabulary before the experiment.

The formal experiment is as follows. The experimental group used vocabulary cards with the Aurasma AR app installed on tablets, while the control group used traditional learning and vocabulary cards in learning (Figure 3). The learning aids used by the students in the experimental group are vocabulary cards and the AR app. The teacher taught using vocabulary cards and then guided the students in operating the AR app. By contrast, students in the control group use only vocabulary cards. In this study, the experimental group and control group are taught by the same teacher during the teaching experiment to avoid teacher effects from interfering with the experiment. After the experiment, the learning motivation of ARCS, learning styles of VAK, and the learning performance were measured. The purpose is to compare whether the subjects’ learning motivation and learning outcomes were affected by the treatment.

Figure 3.

AR app and vocabulary cards.

Analyses and Results

The data for all dependent and categorical variables are submitted to the SPSS program. For analyzing before-treatment scores, this study first uses MANOVA (multivariate analysis of variance) to confirm that there is no difference in ARCS between the two groups. It then uses ANOVA (analysis of variance) to analyze whether the individual A, R, C, and S motivation scores are different. The critical assumption for using MANOVA is that dependent variables must exhibit normal distributions with the same variance. Nevertheless, using standardized 5-point scales to measure dependent variables, this assumption is roughly correct (Laosirihongthong et al., 2003). Since Hair et al. (1995) indicate that the MANOVA is robust to the assumption, minor violations would have little impact.

Previous scholars have also examined the normality of univariate variables from skewness and kurtosis and observe that values for skewness and kurtosis between −2 and +2 are considered acceptable in showing a normal univariate distribution (George & Mallery, 2010). The study uses the 5-point Likert scale to measure ARCS. The four motivation subscale scores of skewness and kurtosis are almost all between −2 and +2 (Table 1), with only one violation among the 16 comparisons. Another assumption of homogeneity of the covariance matrices was tested with Box’s M test. The value of Box’s M is 10.82 (p = .45), a non-significant result for the two groups at the p = .05 level. Therefore, MANOVA is suitable for analyzing the data.

Table 1.

Descriptive of Motivation Sores Before Treatment.

	Experimental group n₁ = 23				Control group n₂ = 24				Total N = 47
	A	R	C	S	A	R	C	S	A	R	C	S
Mean	4.58	4.50	4.19	4.51	4.36	4.27	4.29	4.53	4.67	4.38	4.22	4.52
Std.	0.13	0.12	0.16	0.12	0.12	0.12	0.16	0.12	0.60	0.57	0.76	0.57
Skew.	−1.11	−0.91	−0.48	−1.04	−1.27	−1.03	−1.29	−1.28	−1.32	−0.86	−0.93	−1.15
Kurt.	0.16	−0.25	−1.24	0.53	1.82	2.29	0.60	0.95	1.85	0.71	−0.23	0.64

Note. See Appendix for abbreviations in Tables 1 to 5.

Table 2.

Before-Treatment Analysis for Mean Difference of ARCS and Vocabulary Proficiency.

Group effects	One-way MANOVA	One-way ANOVA
Group effects	ARCS	A	R	C	S	VP
F-value	0.926 (Wilks’ λ = .009)	1.56	1.92	0.08	0.03	0.47
p-Value	.46	.22	.17	.77	.87	.50

Note. VP = vocabulary proficiency.

Table 3.

Two-way MANCOVA of ARCS.

Effects	Statistics and significance of MANCOVA
Covariance	Wilks’ λ	F-value	p-Value
A	0.77	2.50	.06
R	0.79	2.22	.09
C	0.87	1.29	.29
S	0.59	5.93	.000***
Learning method (AR vs. non-AR)	0.70	3.66	.013*
Learning style (V, A, K)	0.71	1.60	.14
Interaction	0.72	1.51	.17

p < .05. **p < .01. ***p < .001.

Table 4.

After-Treatment Analysis of Main Effects of Learning Method on Sub-scores of ARCS.

	A		R		C		S
	AR	nAR	AR	nAR	AR	nAR	AR	nAR
Mean	4.69	4.52	4.70	4.38	4.67	4.28	4.80	4.68
F-value	1.16		6.95		4.34		1.00
p-Value	.29		.01**		.04*		.32

p < .05. **p < .01. ***p < .001.

AR = AR app learning; nAR = non-AR app learning.

Table 5.

Two-Way ANCOVA of Sub-Scores of ARCS.

Effects	Significance of ANCOVA (F-Value)
Effects	A	R	C	S
Covariance	12.88***	14.91***	15.11***	40.12***
Learning method (AR vs. non-AR)	0.28	4.4*	9.55***	2.84*
Learning style (VAK)	0.10	0.87	4.10**	1.28
Interaction	0.06	0.43	2.68*	0.65

p < .1. **p < .05. ***p < .01.

MANCOVA and ANOVA are then used to analyze differences in learning motivation and English vocabulary proficiency between the control and experimental groups. The results show no significant difference between the two groups of students’ learning motivation and English vocabulary proficiency (Table 2). Therefore, after-treatment analysis may be performed.

Two-way MANCOVA is used to determine the learning method’s main effects and moderating effects of learning style on motivation (dependent variable). We use before-treatment scores of ACRS as the covariance, the learning method (AR app learning vs. non-AR app learning) and learning style (V, A, K) as independent variables, and the after-treatment scores on the ARCS as the dependent variables. As shown in Table 3, the learning method does significantly increase student motivation.

After using one-way ANOVA for analysis of the A, R, C, and S scores, it is found that the motivation of the group that learned via AR app is higher than that of the non-AR app learning group across all four sub-scales of A, R, C, and S (Table 4). Hence, H1 is supported.

Although, the interaction effects of the two-way MANCOVA are not significantly different (Table 3), after using two-way ANCOVAs to analyze A, R, C, and S individually, it is found that C exhibits a significant interaction effect (Table 5). Hence, H3 is partially supported.

This research also examines whether there is a difference in learning performance after the treatment. The results show that after the experimental group students received AR app learning, their average performance scores (M = 76.52) are significantly different (p = .001) from the average performance scores before the treatment (M = 51.30) (Table 6). The after-treatment average performance scores (M = 80.00) are also significantly different from the before-treatment average performance scores (M = 58.75, p = .028) in the control group.

Table 6.

Learning Performance Comparisons Before and After the Treatment.

	Experimental group			Control group
	Before	After	∆_E	Before	After	∆_C
Mean	51.30	76.52	25.22	58.75	80.00	21.25
Std.	37.08	26.73		37.33	31.20
t-Value	–3.648***			–2.347***

Note. ∆ = After-before.

p < .1. **p < .05. ***p < .01.

We then used the differences-in-differences test to check whether there is a difference in the degree of performance improvement between the two groups. Results (Table 6) show that the extent of performance improvement in the experimental group (mean improvement scores of the experimental group, i.e., $\bar{Δ_{E}}$ = 25.22) is greater than that of the control group (mean improvement scores of the control group, that is, $\bar{Δ_{C}}$ , = 21.25). However, the difference between mean improvement scores of the experimental group) and the mean improvement scores of the control group, that is, $\bar{Δ_{E}} - \bar{Δ_{C}}$ , was not significant (t-value = 0.35, p-value = .73). Hence, H2 is partially supported.

This study further explores the differences between the two learning methods in motivation and performance. The results show that learning methods significantly impact students’ motivation, indicating that student motivation improved significantly, with a higher slope, after learning via AR app. Similarly, student learning performance improved significantly, with a steeper slope, after learning via AR app (Figures 4 and 5). Hence, H4 is supported.

Figure 4.

The motivation score before and after learning.

Figure 5.

The performance score before and after learning.

Discussion and Implications

This study shows that applying AR apps to language learning can improve student learning motivation. After the experiment, the experimental group students exhibited a significant improvement in motivation. By contrast, the control group students showed no significant difference in motivation after the experiment, confirming previous findings (Di Serio et al., 2013). In the app group, overall learning motivation improved. These results are consistent with previous studies on applying AR technology in learning, which show that it can effectively increase student motivation (Di Serio et al., 2013; Mahadzir & Phung, 2013; Singhal et al., 2012). The main reason for the improvement of students’ motivation appears to be the technological characteristics of AR, including the technological novelty and the immersive interactive experience. Di Serio et al. (2013) find that students show significant improvement in the motivational elements of attention and satisfaction after applying AR learning. However, our results indicate that students have a significant improvement in the two motivational elements of attention and relevance. This inconsistency is likely because Di Serio et al. (2013) explored the application of AR to Art courses. However, this study focuses on English vocabulary learning for first graders. The difference in the teaching context may lead to the partial inconsistency.

Ogawa (2016) believes that AR’s technological novelty can attract students’ attention and make them more involved in the learning process. This research also posited that AR apps can increase students’ motivation to learn. AR’s tracking technology can superimpose 3D stereoscopic images in appropriate positions, bringing students a different degree of immersive experience (Di Serio et al., 2013). Compared with traditional digital learning, this method can make students feel more immersed in the learning content (Yuen et al., 2011), enhancing the motivational element of relevance. Further, the experimental subjects of this study are first graders. According to the literature, AR’s technological features are particularly suitable for junior elementary school students since text comprehension is more difficult for students in lower grades. The use of audio-visual content related to real situations can create a better learning environment for students and enhance their learning motivation (Lee et al., 2009).

This research also shows that learning performance improved significantly after the experiment, which means that both learning methods can improve students’ learning performance. However, compared with the traditional learning method, integrating AR into learning did not lead to a significant difference in student learning performance. The reason may be due to experiment time limitations. In this study, experiments are conducted in classroom units, meaning that students’ learning performance cannot achieve significant differences in such a short period. Future studies can extend the experiment period to a semester or a school year, to observe students’ learning performance changes after receiving AR app learning.

Learning style has a moderating effect but is not significant. The reason may be that both learning methods stimulate the students’ three senses of vision, hearing, and kinesthesia in different ways. The traditional learning method stimulates the students’ auditory and visual senses through chanting and teaching vocabulary cards. It encourages the students’ kinesthetic senses through classroom interaction. In using AR apps to assist learning, students’ auditory and visual senses are stimulated through the 3D images, and the kinesthetic senses are encouraged in the process of operating the tablet and interacting with classmates. Therefore, students with different learning styles will receive appropriate stimulation using either learning method. VAK analysis enables understanding of the individual’s learning style, helping the student become more focused and improving learning (Gilakjani, 2011). Therefore, this study contends that when applying AR app learning, teachers should understand students’ learning styles and then design appropriate AR teaching materials that can improve learning motivation under the sub-standards of ARCS to stimulate student motivation to learn more and achieve expected learning performance.

Concluding Remarks and Limitations

AR apps can effectively improve students’ English learning motivation, especially for lower grade students in elementary schools. Therefore, when applying AR to learning, teachers can use the novelty of the technology and the presentation of 3D stereoscopic images to design teaching content that attracts students to enhance their attention. Further, teachers can use the immersive and interactive environment provided by AR to design curriculum-related activities to enable students to associate learning content with their own experience through interactive learning and immersive experience, enhancing the motivation of relevance.

This research is limited by time and space and points worthy of further discussion and improvement remain. First, since the research subject is limited to first graders, it is recommended that future researchers expand the research participants to make the research sample more diversified and extend the scope of inference of the research results. In addition, this research only explores English vocabulary learning in elementary schools. Future research can also focus on different fields/courses to increase its generalizability. Finally, this research is conducted in the classroom as a unit, and future research is recommended to longitudinally observe the learning performance of the whole grade or the whole school across a more extended period of time.

Footnotes

Appendix 1

Appendix 2 Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jung-Yu Lai

References

Ahmadi

Reza

(2018). The use of technology in English language learning: A literature review. International Journal of Research in English Education, 3(2), 115–125. https://doi.org/10.29252/ijree.3.2.115

Akbulut

Cardak

C. S.

(2012). Adaptive educational hypermedia accommodating learning styles: A content analysis of publications from 2000 to 2011. Computers & Education, 58(2), 835–842. https://doi.org/10.1016/j.compedu.2011.10.008

Akçayır

(2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Review of Educational Research, 20, 1–11. https://doi.org/10.1016/j.edurev.2016.11.002

Azuma

Baillot

Behringer

Feiner

Julier

MacIntyre

(2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21(6), 34–47. https://doi.org/10.1109/38.963459

Bacca

Baldiris

Fabregat

Graf

Kinshuk . (2014). Augmented reality trends in education: A systematic review of research and applications. Educational Technology & Society, 17(4), 133–149.

Bandler

Grinder

(1975). The structure of magic (vol. 1): Science and behavior. Books Palo.

Barreira

J. B.

Maximino

Pereira

L. C.

Telmo

Emanuel

Magalhães

(2012, June 20–23). MOW: Augmented reality game to learn words in different languages: Case study: learning English names of animals in elementary school [Conference session]. CISTI 2012: 7th Iberian conference on information systems and technologies, Madrid, Spain. https://ieeexplore.ieee.org/document/6263236/keywords#keywords

Beatrice

J. A.

(1995). Learning to study through critical thinking. Irwin Professional Publishing.

Bolliger

D. U.

Supanakorn

Boggs

(2010). Impact of podcasting on student motivation in the online learning environment. Computers & Education, 55(2), 714–722. https://doi.org/10.1016/j.compedu.2010.03.004

10.

Bower

(2008). Affordance analysis – Matching learning tasks with learning technologies. Educational Media International, 45(1), 3–15. https://doi.org/10.1080/09523980701847115

11.

Bursali

Yilmaz

R. M.

(2019). Effect of augmented reality applications on secondary school students’ reading comprehension and learning permanency. Computers in Human Behavior, 95(June), 126–135. https://doi.org/10.1016/j.chb.2019.01.035

12.

Caudell

Mizell

. (1992, January 7–10). Augmented reality: An application of heads-up display technology to manual manufacturing processes [Conference session]. Hawaii international conference on system sciences, Kauai, HI.

13.

Chang

Y.-C.

Kao

W.-Y.

Chu

C.-P.

Chiu

C.-H.

(2009). A learning style classification mechanism for E-learning. Computers & Education, 53(2), 273–285. https://doi.org/10.1016/j.compedu.2009.02.008

14.

Chen

M.-P.

Wang

L.-C.

Zou

Lin

S.-Y.

Xie

Tsai

C.-C.

(2020). Effects of captions and English proficiency on learning effectiveness, motivation and attitude in augmented-reality-enhanced theme-based contextualized EFL learning. Computer Assisted Language Learning, Advance online publication. https://doi.org/10.1080/09588221.2019.1704787

15.

Chen

R. W.

Chan

K. K.

(2019). Using augmented reality flashcards to learn vocabulary in early childhood education. Journal of Educational Computing Research, 57(7), 1812–1831. https://doi.org/10.1177/0735633119854028

16.

Chicchi Giglioli

I. A.

Pallavicini

Pedroli

Serino

Riva

. (2015). Augmented reality: A Brand new challenge for the assessment and treatment of psychological disorders. Computational and Mathematical Methods in Medicine, 2015, 862942. https://doi.org/10.1155/2015/862942

17.

Chien

C.-H.

Chen

C.-H.

Jeng

T.-S.

(2010, March 17–19). An interactive augmented reality system for learning anatomy structure. Proceedings of the international multiconference of engineers and computer scientists, Hong Kong, China.

18.

Dalgarno

Lee

M. J. W.

(2010). What are the learning affordances of 3-D virtual environments? British Journal of Educational Technology, 41(1), 10–32. https://doi.org/10.1111/j.1467-8535.2009.01038.x

19.

De Buck

Maes

Ector

Bogaert

Dymarkowski

Heidbüchel

Suetens

. (2005). An augmented reality system for patient-specific guidance of cardiac catheter ablation procedures. IEEE Transactions on Medical Imaging, 24(11), 1512–1524. https://doi.org/10.1109/TMI.2005.857661

20.

Deci

E. L.

Ryan

R. M.

(1985). The general causality orientations scale: self-determination in personality. Journal of Research in Personality, 19(2), 109–134. https://doi.org/10.1016/0092-6566(85)90023-6

21.

Deci

E. L.

Ryan

R. M.

(2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268. https://doi.org/10.1207/s15327965pli1104_01

22.

Dede

(2009). Immersive interfaces for engagement and learning. Science, 323(5910), 66–69. https://doi.org/10.1126/science.1167311

23.

Derakhshan

Shakki

(2018). An investigation into the relationship between Iranian EFL high- and low-proficient learners and their learning styles. Sage Open, 8(4), 1–10. https://doi.org/10.1177/2158244018809408

24.

Dincer

Yeşilyurt

Noels

K. A.

Vargas Lascano

D. I.

(2019). Self-determination and classroom engagement of EFL learners: A mixed-methods study of the self-system model of motivational development. Sage Open, 9(2), 1–15. https://doi.org/10.1177/2158244019853913

25.

Di Serio

Á.

Ibáñez

M. B.

Kloos

C. D

. (2013). Impact of an augmented reality system on students’ motivation for a visual art course. Computers & Education, 68(October), 586–596. https://doi.org/10.1016/j.compedu.2012.03.002

26.

Dunleavy

Dede

Mitchell

(2009). Affordances and limitations of immersive participatory augmented reality simulations for teaching and learning. Journal of Science Education and Technology, 18(1), 7–22. https://doi.org/10.1007/s10956-008-9119-1

27.

Dunn

R. S.

Dunn

K. J.

(1993). Teaching secondary students through their individual learning styles: Practical approaches for grades 7-12. Prentice Hall.

28.

Elliot

A. J.

Covington

M. V.

(2001). Approach and avoidance motivation. Educational Psychology Review, 13(2), 73–92. https://doi.org/10.1023/a:1009009018235

29.

Ellis

(1997). SLA research and language teaching. ERIC. https://elt.oup.com/catalogue/items/global/linguistics/oxford_applied_linguistics/9780194372152?cc=tw&selLanguage=zh&mode=hub

30.

Engeström

(1999). Activity theory and individual and social transformation. Perspectives on activity theory, 19(38), 19–30. https://doi.org/10.1017/CBO9780511812774.003

31.

Fleming

N. D.

Mills

(1992). Not another inventory, rather a catalyst for reflection. To Improve the Academy, 11(1), 137–155. https://doi.org/10.1002/j.2334-4822.1992.tb00213.x

32.

Gardner

R. C.

Masgoret

A. M.

Tennant

Mihic

(2004). Integrative motivation: Changes during a Year-long intermediate-level language course. Language Learning, 54(1), 1–34. https://doi.org/10.1111/j.1467-9922.2004.00247.x

33.

George

Mallery

(2010). SPSS for Windows step by step: A simple guide and reference 17.0 update (10th ed.). Allyn & Bacon. https://lib.ugent.be/catalog/rug01:001424067

34.

Gilakjani

A. P.

(2011). Visual, auditory, kinaesthetic learning styles and their impacts on English language teaching. Journal of Studies in Education, 2(1), 104–113. https://doi.org/10.5296/jse.v2i1.1007

35.

Goodyear

Retalis

(2010). Technology-enhanced learning. Sense Publishers. https://brill.com/view/title/37631

36.

Gopalan

Bakar

J. A. A.

Zulkifli

A. N.

(2020). A review of motivation theories, models and instruments in learning environment. Journal of Critical Reviews, 7(6), 554–559. https://doi.org/10.31838/jcr.07.06.100

37.

Green

Sulbaran

(2006). Motivation assessment instrument for virtual reality scheduling simulator [Conference session]. E-Learn: World conference on e-learning in corporate, government, healthcare, and higher education, San Diego, CA.

38.

Gregorc

A. F.

(1979). Learning/teaching styles: Potent forces behind them. Educational Leadership, 5, 234–237. https://www.scirp.org/(S(i43dyn45teexjx455qlt3d2q))/reference/ReferencesPapers.aspx?ReferenceID=954114

39.

Guay

Vallerand

R. J.

Blanchard

(2000). On the assessment of situational intrinsic and extrinsic motivation: The situational motivation scale (SIMS). Motivation and Emotion, 24(3), 175–213. https://doi.org/10.1023/a:1005614228250

40.

Hair

J. F.

Anderson

R. E.

Tatham

R. L.

Black

W. C.

(1995). Multivariate data analysis with readings (4th ed.). Prentice-Hall.

41.

Hsu

T.-C.

(2017, March). Learning English with augmented reality: Do learning styles matter? Computers & Education, 106, 137–149. https://doi.org/10.1016/j.compedu.2016.12.007

42.

Huang

Diefes-Dux

Imbrie

P. K.

(2006). A preliminary validation of attention, relevance, confidence and satisfaction model-based instructional material motivational survey in a computer-based tutorial setting. British Journal of Educational Technology, 37(2), 243–259. https://doi.org/10.1111/j.1467-8535.2005.00582.x

43.

Hull

C. L

. (1943). Principles of behavior: An introduction to behavior theory. Appleton-Century.

44.

Hulleman

C. S.

Durik

A. M.

Schweigert

S. A.

Harackiewicz

J. M.

(2008). Task values, achievement goals, and interest: An integrative analysis. Journal of Education & Psychology, 100(2), 398–416. https://doi.org/10.1037/0022-0663.100.2.398

45.

Ibáñez

M. B.

Di Serio

Á.

Villarán

Delgado Kloos

(2014). Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness. Computers & Education, 71, 1–13. https://doi.org/10.1016/j.compedu.2013.09.004

46.

Juan

C. M.

Llop

Abad

Lluch

(2010, July 5–7). Learning words using augmented reality [Conference session]. 10th IEEE international conference on advanced learning technologies, Sousse, Tunisia.

47.

Juan

M. C.

Alcañiz

Monserrat

Botella

Baños

R. M.

Guerrero

(2005). Using augmented reality to treat phobias. IEEE Computer Graphics and Applications, 25(6), 31–37. https://doi.org/10.1109/MCG.2005.143

48.

Kaufmann

Schmalstieg

(2003). Mathematics and geometry education with collaborative augmented reality. Computer Graphics, 27(3), 339–345. https://doi.org/10.1016/s0097-8493(03)00028-1

49.

Keefe

. (1982). Assessing student learning styles. In Keefe

J. W.

(Ed.), Student learning styles and brain behavior (pp. 1–18). National Association of Secondary School Principals.

50.

Keller

J. M

. (1983). Motivational design of instruction. In Reigeluth

C. M.

(Ed.), Instructional design theories and models: An overview of their current status (pp. 383–434). Lawrence Erlbaum.

51.

Keller

J. M.

(1987). Development and use of the ARCS model of instructional design. Journal of Instructional Development, 10(3), 2–10. https://doi.org/10.1007/bf02905780

52.

Kinsella

(1995). Understanding and empowering diverse learners in ESL classrooms. In Reid

J. M.

(Ed.), Learning styles in the ESL/EFL classroom (pp. 170–194). Heinle & Heinle.

53.

Kolb

D. A.

(1976). Learning style inventory. McBer and Company.

54.

Kolb

D. A.

(2007). The Kolb learning style inventory. Hay Resources Direct.

55.

Kye

Kim

(2008). Investigation on the relationships among media characteristics, presence, flow, and learning effects in augmented reality based learning. International Journal for Education Media and Technology, 24(4), 193–223. https://doi.org/10.17232/kset.24.4.193

56.

Laosirihongthong

Paul

Speece

M. W.

(2003). Evaluation of new manufacturing technology implementation: An empirical study in the Thai automotive industry. Technovation, 23, 321–331. https://doi.org/10.1016/s0166-4972(01)00115-8

57.

Lave

Wenger

(1991). Situated learning: Legitimate peripheral participation. Cambridge University Press.

58.

Lee

Choi

Park

J. I.

(2009). Interactive E-learning system using pattern recognition and augmented reality. IEEE Transactions on Consumer Electronics, 55(2), 883–890. https://doi.org/10.1109/tce.2009.5174470

59.

Legault

Green-Demers

Pelletier

(2006). Why do high school students lack motivation in the classroom? Toward an understanding of academic amotivation and the role of social support. Journal of Education & Psychology, 98(3), 567–582. https://doi.org/10.1037/0022-0663.98.3.567

60.

Liao

(2006). A flow theory perspective on learner motivation and behavior in distance education. Distance Education, 27(1), 45–62. https://doi.org/10.1080/01587910600653215

61.

Liarokapis

Mourkoussis

White

Darcy

Sifniotis

Petridis

Basu

Lister

P. F.

(2004). Web3D and augmented reality to support engineering education. World Transactions on Engineering and Technology Education, 3(1), 11–14. http://www.wiete.com.au/journals/WTE&TE/Pages/Vol.3,%20No.1%20(2004)/03_Liarokapis29.pdf

62.

Lin

C.-H.

Hsu

P.-H.

(2017). Integrating procedural modelling process and immersive VR environment for architectural design education. MATEC Web of Conferences, 104, 6.

63.

Liu

Y. C.

Wang

W.-T.

Lee

T.-L.

(2021). An integrated view of information feedback, game quality, and autonomous motivation for evaluating game-based learning effectiveness. Journal of Educational Computing Research, 59, 3–40. https://doi.org/10.1177/0735633120952044

64.

Maehr

M. L.

Meyer

H. A.

(1997). Understanding motivation and schooling: Where we’ve been, where we are, and where we need to go. Educational Psychology Review, 9(4), 371–409. https://doi.org/10.1023/A:1024750807365

65.

Mahadzir

N. N. N.

Phung

L. F.

(2013). The use of augmented reality pop-up book to increase motivation in English language learning for national primary school. IOSR Journal of Research & Method in Education (IOSR-JRME), 1(1), 26–38. https://doi.org/10.9790/7388-0112638

66.

McDougall

. (1908). An introduction to social psychology. Methuen.

67.

Milgram

Kishino

. (1994). A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12), 1321–1329.

68.

Nincarean

Alia

M. B.

Halim

N. D. A.

Rahman

M. H. A.

(2013). Mobile augmented reality: The potential for education. Procedia – Social and Behavioral Sciences, 103, 657–664. https://doi.org/10.1016/j.sbspro.2013.10.385

69.

Noels

K. A.

(2001a). Learning Spanish as a second language: Learners’ orientations and perceptions of their teachers’ communication style. Language Learning, 51(1), 107–144. https://doi.org/10.1111/0023-8333.00149

70.

Noels

K. A.

(2001b). New orientations in language learning motivation: Towards a model of intrinsic, extrinsic, and integrative orientations and motivation. Motivation and Second Language Acquisition, 23, 43–68. https://doi.org/10.3138/cmlr.57.3.424

71.

Ocepek

Bosnić

Nančovska Šerbec

Rugelj

(2013). Exploring the relation between learning style models and preferred multimedia types. Computers & Education, 69, 343–355. https://doi.org/10.1016/j.compedu.2013.07.029

72.

Ogawa

T. A.

(2016). Vocabul-AR-Y: Action research project of Aurasma to support vocabulary. Retrieved May 5, 2021, from https://scholarspace.manoa.hawaii.edu/bitstream/10125/40192/Ogawa_Scholarspace_4-22-16.pdf

73.

Oxford

R. L.

Anderson

N. J.

(1995). A crosscultural view of learning styles. Language Teaching, 28(4), 201–215. https://doi.org/10.1017/s0261444800000446

74.

Ozbek

C. S.

Giesler

Dillmann

(2004). Jedi training: Playful evaluation of head-mounted augmented reality display systems. Proceedings Volume 5291, Stereoscopic Displays and Virtual Reality Systems XI, (2004).

75.

Paas

Tuovinen

J. E.

van Merriënboer

J. J. G.

Aubteen Darabi

(2005). A motivational perspective on the relation between mental effort and performance: Optimizing learner involvement in instruction. Educational Technology Research and Development, 53(3), 25–34. https://doi.org/10.1007/bf02504795

76.

Pintrich

P. R.

(1999). The role of motivation in promoting and sustaining self-regulated learning. International Journal of Educational Research, 31(6), 459–470. https://doi.org/10.1016/s0883-0355(99)00015-4

77.

Pintrich

P. R.

Smith

D. A. F.

Garcia

Mckeachie

W. J.

(1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53(3), 801–813. https://doi.org/10.1177/0013164493053003024

78.

Radu

(2012, November 5–8). Why should my students use AR? A comparative review of the educational impacts of augmented-reality [Symposium]. 2012 IEEE International symposium on mixed and augmented reality (ISMAR), Atlanta, GA.

79.

Radu

(2014). Augmented reality in education: A meta-review and cross-media analysis. Personal and Ubiquitous Computing, 18(6), 1533–1543. https://doi.org/10.1007/s00779-013-0747-y

80.

Rahimi

Allahyari

(2019). Effects of multimedia learning combined with strategy-based instruction on vocabulary learning and strategy use. Sage Open, 9(2), 1–14. https://doi.org/10.1177/2158244019844081

81.

Reid

J. M.

(1995). Learning styles in the ESL/EFL classroom. ERIC.

82.

Reiff

J. C.

(1992). Learning styles. What research says to the teacher series. National Education Association.

83.

Rodgers

D. L.

Withrow-Thorton

B. J.

(2005). The effect of instructional media on learner motivation. International Journal of Instructional Media, 32(4), 333.

84.

Rost

(2006). Generating student motivation. World View, 1–4.

85.

Rozi

Larasati

Lestari

(2021). Developing vocabulary card base on augmented reality (AR) for learning English [Conference Session]. IOP Conference Series: Materials Science and Engineering.

86.

Sahin

(2019). Contextual learning strategies in the early stages of architecture education. New Trends Issues Proceedings on Humanities Social Sciences, 6(1), 313–320. https://doi.org/10.18844/prosoc.v6i1.4183

87.

Santos

M. E. C.

Lübke

A. I. W.

Taketomi

Yamamoto

Rodrigo

M. M. T.

Sandor

Kato

(2016). Augmented reality as multimedia: The case for situated vocabulary learning. Research and Practice in Technology Enhanced Learning, 11(1), 4. https://doi.org/10.1186/s41039-016-0028-2

88.

Schinke-Llano

(1995). Reenvisioning the second language classroom: A Vygotskian approach. In Eckman

F. R.

Mileham

Weber

R. R.

Highland

Lee

P. W.

(Eds.), Second language acquisition theory and pedagogy (pp. 21–28). Routledge.

89.

Schwald

De Laval

(2003). An augmented reality system for training and assistance to maintenance in the industrial context. Journal of WSCG, 11(1–3), 425–432. http://wscg.zcu.cz/wscg2003/Papers_2003/I23.pdf

90.

Singhal

Bagga

Goyal

Saxena

(2012). Augmented chemistry: Interactive education system. International Journal of Computer Applications, 49(15), 1–5. https://doi.org/10.5120/7700-1041

91.

Sonia Gonzalez

M. D.

Brickman

McInerney

D. M

. (2006). Self-regulation of academic motivation: Advances in structure and measurement. University of Western Sydney, Australia. http://www.aare.edu.au/05pap/abs05.htm

92.

Sternberg

R. J.

Grigorenko

E. L.

(1997). Are cognitive styles still in style? American Psychologist, 52(7), 700–712. https://doi.org/10.1037/0003-066x.52.7.700

93.

Tseng

J. C. R.

Chu

H.-C.

Hwang

G.-J.

Tsai

C.-C.

(2008). Development of an adaptive learning system with two sources of personalization information. Computers & Education, 51(2), 776–786. https://doi.org/10.1016/j.compedu.2007.08.002

94.

Tuan

Chin

Shieh

(2005). The development of a questionnaire to measure students’ motivation towards science learning. International Journal of Science Education, 27(6), 639–654. https://doi.org/10.1080/0950069042000323737

95.

Walker

McMahon

D. D.

Rosenblatt

Arner

(2017). Beyond Pokémon: Augmented reality is a universal design for learning tool. Sage Open, 7(4), 1–8. https://doi.org/10.1177/2158244017737815

96.

Wrzesien

Bretón-López

Botella

Burkhardt

J.-M.

Alcañiz

Pérez-Ara

M. Á.

Del Amo

A. R.

(2013). How technology influences the therapeutic process: Evaluation of the patient-therapist relationship in augmented reality exposure therapy and in vivo exposure therapy. Behavioural and Cognitive Psychotherapy, 41(4), 505–509. https://doi.org/10.1017/S1352465813000088

97.

H.-K.

Lee

S. W.-Y.

Chang

H.-Y.

Liang

J.-C.

(2013). Current status, opportunities and challenges of augmented reality in education. Computers & Education, 62, 41–49. https://doi.org/10.1016/j.compedu.2012.10.024

98.

Yaacob

Shapii

Saad Alobaisy

Al-Rahmi

W. M.

Al-Dheleai

Y. M.

Yahaya

Alamri

M. M.

(2019). Vocabulary learning strategies through secondary students at Saudi school in Malaysia. Sage Open, 9(1), 1–12. https://doi.org/10.1177/2158244019835935

99.

Yang

Q.-F.

Chang

S.-C.

Hwang

G.-J.

Zou

(2020, April). Balancing cognitive complexity and gaming level: Effects of a cognitive complexity-based competition game on EFL students’ English vocabulary learning performance, anxiety and behaviors. Computers & Education, 148, 103808. https://doi.org/10.1016/j.compedu.2020.103808

100.

Yuen

S. C. Y.

Yaoyuneyong

Johnson

(2011). Augmented reality: An overview and five directions for AR in education. Journal of Educational Technology Development and Exchange, 4(1), 119–140. https://doi.org/10.18785/jetde.0401.10

101.

Lee

(2015). Understanding EFL students’ participation in group peer feedback of L2 writing: A case study from an activity theory perspective. Language Teaching Research, 19, 572–593. https://doi.org/10.1177/1362168814541714

102.

Zhang

Zou

(2020). Types, purposes, and effectiveness of state-of-the-art technologies for second and foreign language learning. Computer Assisted Language Learning. Advance online publication. https://doi.org/10.1080/09588221.2020.1744666

Impacts of Augmented Reality Apps on First Graders’ Motivation and Performance in English Vocabulary Learning

Abstract

Keywords

Introduction

Literature Review

Learning Motivation Theory

Augmented Reality and English Learning

Learning Style

Methodology

Participants and Measurement Instruments

Experiment Procedure

Analyses and Results

Discussion and Implications

Concluding Remarks and Limitations

Footnotes

Appendix 1

Appendix 2

Declaration of Conflicting Interests

Funding

ORCID iD

References