Abstract
New teaching and learning practices related to gamified tools in online learning environments have become even more important during the pandemic and will continue to reshape educational processes, including higher education institutions, in the post-pandemic period. In line with this, the present study aimed to investigate the effects of gamification with Web 2.0 tools including Kahoot!, Socrative, Quizizz, and Mentimeter on EFL learners’ motivation and academic achievement levels in online learning environments through a quasi-experimental research method and was based on a mixed-method sequential explanatory research design. The participants of the study are 60 freshman learners studying at a state university in Türkiye. The data were obtained through a scale, an Achievement Test, and a semi-structured interview. The quantitative data were analyzed via statistical measures, and the qualitative data were analyzed through content analysis. The findings indicated that the treatment via gamified Web 2.0 tools had a statistically significant difference in favor of the experimental group in terms of the participants’ motivation/course interest levels and the experimental group scored higher on the Achievement Test compared to the control group.
Plain language summary
The study aims to investigate the effects of gamification with Web 2.0 tools including Kahoot!, Socrative, Quizizz, and Mentimeter on EFL learners’ motivation and academic achievement levels in online learning environments. The participants of the study are 60 freshman learners studying at a state university in Türkiye. The data were obtained through two scales, an achievement test, and a semi-structured interview. The findings indicated that the treatment via gamification tools has a statistically significant difference in favor of the experimental group in terms of participants’ motivation / course interest levels. The experimental group had higher scores in the achievement test when compared to the control group as well. As a limitation, the findings may not be generalizable to other students of different contexts and different settings as the data were collected from only 60 learners. Despite the limitations, the current study has several implications in today’s digital world. Each level of educational institution needs to provide its learners with interactive learning settings which arouse interest in the course and motivate them for more dynamic participation / engagement process. The current study is believed to reveal an alternative way in that the instruction with the help of Web 2.0 tools in online learning environments contributes to increase learner course interest/motivation and academic achievement levels that possibly decrease course dropouts in those settings.
Introduction
The spread of coronavirus (Covid-19) has affected people’s lives and education in many ways. According to UNESCO’s report, on April 1, 2020, educational institutions, including schools and universities, were closed in 185 countries, affecting 1,542,412,000 students, 89.4% of total enrollment (Marinoni et al., 2020). Within this regard, educational practices have undergone digital transformation, especially during the pandemic crisis, when online learning was considered a panacea (Dhawan, 2020). As the traditional classroom teaching and learning process has been replaced by online learning environments, educational institutions are recommended to conduct more flexible teaching and learning practices through online platforms. Similarly, since the digital generation provides easy access to information and technology, students have demands for the use of technology, and therefore, instructors should prioritize activities and learning environments in which students can take part more actively in order to achieve permanent learning while creating instructional design (Gündüz & Akkoyunlu, 2020). Although online learning dates to the 1980s and the number of learners engaged in online learning platforms is constantly increasing, some challenges with student motivation arise in these settings, leading to high levels of dropout (Park & Choi, 2009). The difficulties or challenges are mostly related to technological issues such as download/installation errors, login issues, and problems with audio and video files, etc., leading students to find online teaching and learning both boring and uninteresting. Furthermore, it is indicated that distance and personalized instructional processes refer to the biggest challenges of online teaching as students desire two-way interaction, but occasionally online course content is purely theoretical, which can be a barrier for learners to practice and learn in an effective way (Dhawan, 2020). Similarly, it is revealed that the lack of opportunity to interact and compete with other students, as in classrooms, makes students feel isolated in distance learning or online learning platforms, leading to a decrease in motivation levels (Balaban-Sali, 2008; Kamal et al., 2021; X. Zhang & Cui, 2010). Therefore, motivation is regarded to be essential for online learning (Akhasbi et al., 2021; Hashemian & Soureshjani, 2011) and it is stated that gamification is highly associated with motivation especially since it attracts more and more attention every day due to its potential to motivate students (Dichev & Dicheva, 2017) and with reference to language learning, it is argued that sharing information with others through Web 2.0 technologies also increases learners’ motivation to learn a language (Cephe & Balçıkanlı, 2012), as creating Web-based gamified environments encourages learners to acquire an online identity and interact with their peers in a foreign language (Stevenson & Liu, 2010).
Objectives of the Study
Regarding the importance of motivation and students’ achievement levels, it is noted that poor motivation is likely to result in less satisfaction and less achievement in online learning environments, indicating a need for consideration by both researchers and teachers to improve students’ motivation levels for online learning (Berestova et al., 2022; Yu, 2022). Therefore, various practices regarding teaching and learning strategies, cooperation between teacher and learner, computer applications, and gamification could perform as facilitators for promoting students’ online learning achievements (Yu, 2022). However, gamification is not adequately presented in ELT research, particularly in the context of English as a foreign language (EFL) (Cárdenas-Moncada et al., 2020; Phuong, 2020) and it is also a nascent field of research regarding empirical evidence in Second Language Acquisition (SLA) (Boudadi & Gutiérrez-Colón, 2020). Furthermore, it is indicated that only a few of the previous research studies addressed gamification in online learning environments (Huang et al., 2019). While it is recognized that motivation is an important aspect of the teaching and learning process, contemporary studies investigating its impact on online learning environments are limited and address the need to reassess motivational factors in these platforms (Hartnett, 2016). Within this respect, the purpose of the present study is to investigate the effects of gamification with Web 2.0 tools including Kahoot!, Socrative, Quizizz, and Mentimeter on EFL learners’ motivation, and academic achievement levels in online learning environments through a quasi-experimental research method and mixed method sequential explanatory research design as it is indicated that most of the research studies within EFL world focus on the investigation of the effects of several specific games or game platforms based on quantitative data, but qualitative research is also needed and is believed to add valuable insights into the field (Phuong, 2020). Besides, this study is considered timely as it aims to reveal various online practices for instructors, teaching practitioners, or any stakeholders to utilize in online learning environments both during and after the time of the global pandemic as it is stated that the practices, methods, and techniques used during the pandemic period and the widespread use of digital technologies may lead to the evolution of live online education and training to the new normal in the coming period (Tang et al., 2021).
It is historically recognized that gender differences influence how and in what ways students learn and progress in classroom learning, but gender-specific discussions in online learning are somewhat contradictory, making it necessary to seek a clearer explanation of whether or how students’ perceptions vary by gender in online learning environments (Harvey et al., 2017). Thus, in this study, gender was considered an important variable to search for whether it has an impact on participants’ perceptions of gamification with Web 2.0 tools, their academic achievement, and their motivation in online learning environments.
Defining Gamification and Game Elements
Gamification is defined as “a new technology that incorporates elements of gameplay in nongame situations” (Prince, 2013, p. 162) and has been widely adopted in educational settings over the years since it has become a trend (Toda et al., 2019). The interest in gamification in recent years appeals to the idea that it also has an impact on behavior (Buckley & Doyle, 2016).
To design a game-based teaching and learning process with a positive impact, it should have the necessary elements namely components, mechanics, and dynamics and all of them should be combined to promote a needs-oriented procedure of learning (Bicen & Kocakoyun, 2018). The aforementioned game elements within the hierarchical order could be illustrated in Figure 1 (Costa et al., 2017, p. 6):
Game elements hierarchy.
As shown in Figure 1, among the game elements, game components relate to achievements, avatars, badges (visual representations of achievements), boss fights (hard challenges), collections, combat (short-lived battle), content unlocking, gifting, leaderboards, levels, points, quests, social graphs, virtual goods (game assets), and teams; game mechanics include challenges (puzzles), chance, competition (win and lose), cooperation (working together for a shared goal), feedback, resource acquisition (getting useful items), rewards (benefits), transactions (trading), turns, win states, and game dynamics come at the top and refer to constraints (limitations), emotions, narrative (the storyline), progression (participant’s development), and relationships (interactions) (Werbach & Hunter, 2012, pp. 78–80). These categories that make up the hierarchy are defined and classified according to their level of abstractness. Each component is linked to one or more game mechanics, and each mechanic can be linked to one or more game dynamics. In other words, every element has a connection with its higher levels, except for the dynamic, which represents the top level in the hierarchical order (Werbach & Hunter, 2012).
Since this study aims to investigate the effects of gamification through Web 2.0 tools in online learning environments, the game elements included in the selected Web 2.0 applications and discussed in the literature can be seen in Table 1.
Game Elements in Selected Web 2.0 Tools.
As shown in Table 1, four Web 2.0 applications were selected for use in this study as there are various game elements in each application with some similar and differentiating components.
Theories for Gamification
When the literature is examined, it is seen that there are various theories associated with gamification. These include Motivation Theory, Self-Determination Theory, Achievement Goal Theory, Social Learning Theory, and Situated Learning Theory (S. Kim et al., 2018). This study focuses on Motivation Theory, which assumes that motivation is one of the most important factors that can influence the success of gamification along with gamification mechanics and elements (Bovermann & Bastiaens, 2020). Regarding the theory, it is argued that gamification fosters a combination of two types of motivation, intrinsic and extrinsic motivation, to increase learner motivation and engagement (Muntean, 2011). In this regard, game elements are highly adaptable to the L2 teaching and learning process, since an intrinsically motivated person performs a task or an activity for pleasure and challenge, which can be demonstrated by issues such as achievement, cooperation, and sense of belonging, while an extrinsically motivated one acts based on some extrinsic goals such as badges, points, rewards, and levels (Flores, 2015).
With reference to motivational design models, one of the widely used is Keller’s attention, relevance, confidence, and satisfaction (ARCS) model (K. Li & Keller, 2018) which assumes that students will be motivated if they feel they can succeed and that their learning is valuable (Hodges, 2004). The model has been used in many disciplines as ARCS makes it easy for course designers and teaching practitioners to follow by providing very detailed recommendations for key teaching components/content in enhancing student motivation (Song & Kao, 2023).
Gamification and Motivation
To have a better insight into the concept of motivation, it needs to be defined first in that if someone is characterized as motivated for anything, then s/he is enthusiastic and moved to perform it while one with no impulse is regarded to be unmotivated. In line with this perspective, those people who play and work with others are associated with the issue of motivation (Ryan & Deci, 2000). Therefore, gamification and motivation are interrelated, as gamification refers not only to a technological application but also to a methodology that many organizations use to increase motivation (Dichev & Dicheva, 2017), as gamification is seen as an innovative approach to increase motivation (Sailer et al., 2014). Furthermore, from a pedagogical perspective, gamification is considered an evolving approach to have more motivated and engaged learners with the help of using various game elements in educational environments (Dichev & Dicheva, 2017) that include points, leaderboards, badges, achievements, and levels (Nicholson, 2013). Besides, it is indicated that gamification is differentiated from traditional approaches in that it utilizes competition explicitly as a kind of motivational tool (Buckley & Doyle, 2016; Sailer et al., 2014).
Gamification and Student Achievement
It is known that online learning is not a new trend, and with the number of online learners growing every day, it is becoming increasingly important to identify and explore the learning strategies or practices they need to achieve academic success in online learning environments (Peechapol et al., 2018). Within this regard, it is suggested that gamification arouses students’ interest in the academic subject matter (Alsadoon et al., 2022) and both motivation and learning achievement are important factors supporting optimal gamification learning (Q. Zhang et al., 2021).
The relationship between gamification and achievement was discussed in the literature. In a study with 3.202 participants, Bai et al. (2020) investigated the effects of gamification on learners’ academic performance in several educational settings and the data resulted in an overall statistically significant medium effect size in favor of the treatment group. Turan et al. (2016) examined the effects of gamification strategies on learners’ achievements, cognitive load levels, and perceptions with a mixed-method, and based on the results a significant difference emerged between the two groups with higher achievement in favor of the experimental group and the experimental group also outperformed the control group in terms of the cognitive load levels. Besides, it was revealed that learners had positive perceptions regarding gamification. Zhan et al. (2022) aimed to construct a systematic framework and explore the effects of gamification on learners’ academic achievement levels, motivation, cognitive load, and thinking skills in programming education through a meta-analysis, and based on the results, it was revealed that gamification has the greatest effect on learners’ motivation, followed by their academic achievement levels.
Motivation and Academic Achievement
Student motivation plays a role as one of the key factors in both participation in the learning process and a high level of academic performance (Çakıroğlu et al., 2017). When the literature is examined, it is seen that there are some motivation components related to the relationship between motivation and academic achievement, and some of these components, such as extrinsic goal orientation, intrinsic goal orientation, and the value of the subject, are directly related to the academic achievement of individuals (Orhan Özen, 2017). Similarly, it has been observed that students’ motivation towards digital learning is affected by both internal and external aspects, as the global pandemic has led to a rapid transition from traditional practices to online learning platforms (Gustiani, 2020). The relationship between motivation and academic achievement has been investigated in the literature. In a study, Ayub (2010) explored the relationship between students’ intrinsic and extrinsic motivation on their academic performance, and based on the findings they concluded that there was a significant correlation. Similarly, Sikhwari (2014) investigated the relationship between motivation, self-concept, and academic achievement levels of learners studying at a university in South Africa via quantitative cross-sectional survey design and concluded that there were significant correlations between the variables. Furthermore, the findings confirmed the importance of self-concept and motivation in learners’ academic achievement.
Previous Studies on Gamification
Gamification has been the focus of research in different disciplines since it became a trend in various fields. In this direction, various studies have been conducted for educational purposes. A study aiming to analyze whether game-based vocabulary learning affects Chinese EFL learners’ self-confidence, motivation, and vocabulary achievement found that the treatment significantly benefited the experimental group in terms of motivation to learn vocabulary and self-confidence but did not affect their vocabulary learning achievement (R. Li, 2021). In another research study Hassan et al. (2021) aimed to investigate the challenges and benefits of using Web 2.0 tools among ESL (English as a Second Language Learners) learners studying at the Eastern Mediterranean University in North Cyprus at the time of the global Covid-19 pandemic and based on the results they concluded that via Web 2.0 tools, it is possible to help learners promote collaborative, flexible and independent learning as well as increase competency for technology to fulfill learning aims. Furthermore, Gündüz and Akkoyunlu (2020) aimed to explore the effects of gamification in a flipped online learning environment and whether it increases participation, interaction data, and achievement through a mixed-method sequential explanatory research design, and based on the findings, it was indicated that the experimental group had higher scores compared to the control group regarding all the three variables.
Abusa’aleek and Baniabdelrahman (2020) aimed to analyze the effects of gamification on sixth-grade EFL students’ reading comprehension levels in a public school in Jordan with experimental and control groups and based on the results, they concluded that there was a statistically significant difference in favor of the post-test scores of the experimental group in three levels of reading comprehension including literal, critical and inferential. In another study, Arunsirot (2021) investigated whether integrating the gamification approach in English language classroom was effective on English-majored students’ syntactic knowledge, and based on the results, it was revealed that the gamification approach has a significant impact and could be used to improve students’ English syntactic knowledge.
Various research studies have been conducted with selected Web 2.0 tools up to now. Since Kahoot! is utilized in many areas, it has been a focus of research for years in several contexts. Chiang (2020) aimed to understand Chinese EFL learners’ attitudes toward Kahoot! in reading class and indicated that the participants had positive attitudes towards using it but some negative feelings in terms of its use as a testing tool. Since Socrative has features of immediate feedback and real-time teaching and learning procedures, its effects have been investigated in the literature. Concerning ESL classrooms, El Shaban (2017) designed a qualitative study using Socrative to explore learners’ perceptions and through the results, it was revealed that the use of Socrative along with active learning activities increased learners’ engagement and collaboration and enhanced their critical thinking abilities. There have been various studies regarding the use of the Quizizz platform for educational purposes. Zuhriyah and Pratolo (2020) investigated EFL learners’ views on the use of Quizizz as an assessment tool and the results showed that students thought it was an interesting tool and contributed to their motivation, confidence, and reading skills. Through another research study, Dewi et al. (2020) aimed to explore the effects of mobile-assisted language learning (MALL) using Quizizz on higher education learners’ grammar mastery through a quasi-experimental research design and it was revealed that Quizizz is a useful tool to teach grammar specifically in higher education level. Last but not least; Gokbulut (2020) investigated the effects of Kahoot! and Mentimeter word cloud activity on prospective teachers of the Department of Primary School Education at a state university in Türkiye, and the findings indicated that both Kahoot! and Mentimeter applications are beneficial for e-learning environments.
Present Study
Despite its use in different settings and contexts along with revealed positive effects on the learning process and motivation for years, gamification does not seem to be handled much from an empirical aspect which leads to the need for a theoretical basis concerning its motivational effects (Sailer et al., 2014). In line with this, Dichev and Dicheva (2017) suggest that although online learning needs stronger motivation and provides a more promising area to apply gamification, there is a lack of studies regarding gamified online learning. Unlike the previous studies that are mostly descriptive or make a comparison between traditional classrooms and online learning settings (Paul & Jefferson, 2019; Singh et al., 2020), the current study which is based on a quasi-experimental research design and whose control and experimental group participants are all online learners is regarded to give insights into the literature regarding the effects of gamified Web 2.0 tools including Kahoot!, Quizizz, Socrative and Mentimeter on EFL learners’ motivation, and academic achievement levels in online learning environments.
Gamification with Web 2.0 Tools
Web 2.0 tools are indicated to be good practices of gamification activities for educational purposes as gamified Web 2.0 tools refer to Web-based applications along which instructors prepare various types of interactive questions about the course content and present them to their students (Öden et al., 2021). However, Bilgin (2022) emphasizes that benefiting from Web 2.0 technologies in both foreign and second-language learning environments seems to be an unexplored field of study, although it has great potential and importance. Since gamification provides learners with several opportunities through the games played with Web 2.0 tools, it is seen that students seem to be more engaged and motivated when they are given the chance to challenge each other during the games and obtain rewards at the end (Çınar et al., 2022). Furthermore, Web 2.0 tools provide both learners and instructors with the opportunity to have more interactive and user-developed content and may create “real-world” situations for learners to be able to enhance their listening and speaking skills which are not provided by Web 1.0 tools (Stevenson & Liu, 2010). Similarly, it is indicated that Web 2.0 applications including Kahoot!, Quizizz, Duolingo, or several classroom games could be used in the foreign language teaching and learning process (Çınar et al., 2022). Cephe and Balçıkanlı (2012), based on their research findings on pre-service teachers, argue that the use of Web 2.0 tools in the process of language teaching and learning enables students to develop an awareness of digital literacy and computer technologies in today’s digital world, as they are given the opportunity to experience learner autonomy for their own learning at the programming, monitoring, and finally evaluation stages in terms of online activities.
Selected Web 2. 0 Tools in the Study
Kahoot! is an online game-based platform consisting of questionnaires and quizzes and is considered one of the best online applications for educational purposes due to its potential to provide learners with a problem-solving process, critical thinking skills, and a meaningful and fun learning environment (Dellos, 2015) and encourages learners to engage in the learning process and improve their language skills (Muhridza et al., 2018).
Among student response programs, Socrative is used for both formative and summative procedures and is considered a great tool for L2 learners, with several features that allow participants to answer questions through trial and error, helping them to reduce anxiety levels through gamified strategies such as live results and immediate feedback (Flores, 2015).
Quizizz is an online student response system that provides fun multiplayer activities that participants can engage in at their own pace. Along with other fields, Quizizz is acknowledged to be effective and plays a crucial role in the English language teaching and learning process (Degirmenci, 2021).
Mentimeter is an online program that enables instructors to create interactive and real-time presentations (Pichardo et al., 2021). In the word cloud feature of Mentimeter, repetitive words by the users are situated larger in the center while the other ones are settled smaller at the edges.
Gamification and EFL Learning
Gamification is considered to be useful in teaching contexts and is especially popular in EFL/ESL settings today (S. Zhang & Hasim, 2022). Redjeki and Muhajir (2021) state that gamification can be used by teaching practitioners to improve students’ English language skills as it offers students an innovative and captivating learning experience. Since games have been part of foreign language pedagogy for many years, the use of gamification practices seems to be more relevant in language teaching than in other educational fields (Phuong, 2020).
Regarding gamification, it is suggested that creating web-based gamified environments encourages learners to assume an online identity and interact with their peers in a foreign language (Stevenson & Liu, 2010; Wang &Vásquez, 2012). Besides, Web 2.0 technologies also increase their motivation to learn languages because most of today’s learners spend most of their time doing online activities (Cephe & Balçıkanlı, 2012). Furthermore, gamification fits well into a globalized and multicultural English language classroom mainly in higher education, and a more culturally diverse and technologically oriented learning environment (Redjeki & Muhajir, 2021).
Like other fields, gamification is utilized by many in L2 teaching and learning to improve linguistic skills consisting of reading, speaking, listening, and writing and to foster both an interactive and collaborative atmosphere along with gamified instruments that also enable instructors to provide their learners with meaningful experiences (Flores, 2015). Furthermore, Ahmed et al. (2022) indicate that gamification could also be used to develop and improve EFL learners’ idiomatic knowledge.
Research Questions
In the light of the reviewed literature, the research questions of the study are as follows:
1. Does gamification with Web 2.0 tools have an impact on EFL learners’ motivation levels in online learning environments?
1.1. What are EFL learners’ views of the effects of gamification with Web 2.0 tools on their motivation levels in online learning environments?
1.2. Do learners’ perception changes differ by their gender?
2. Does gamification with Web 2.0 tools have an impact on EFL learners’ academic achievement levels in online learning environments?
2.1. What are EFL learners’ views of the effects of gamification with Web 2.0 tools on their academic achievement levels in online learning environments?
2.1. Do learners’ perception changes differ by their gender?
Methodology
Design of the Study
It is acknowledged that the mixed-method approach is getting more interest over time regarding gamification and its effects (Alsawaier, 2019) across online platforms (Alhalafawy & Zaki, 2022). In line with this, the current study was conducted through a mixed-method sequential explanatory research design that regards collecting and analyzing the quantitative, and then qualitative data in consecutive order. From the qualitative perspective, a semi-structured interview was held with participants to explain the quantitative results broadly (Ivankova et al., 2006).
Participants
The participants of the study who were intact groups included 60 freshman EFL learners studying at a state university in Türkiye in different departments including Turkish Language and Literature, Biology, Physics, Chemistry, Geography, History, and Mathematics. The course which formed the focus of the study is Compulsory Foreign Language acknowledged by the Council of Higher Education (CoHE) in Türkiye. A Demographic Information Form was created by the researchers including participants’ gender, departments, periods of learning English, previous experiences with online learning environments and Web 2.0 tools, mobile device preferences, perceived computer, and motivation levels that are presented in Table 2.
Demographic Information of Participants (in Number).
As shown in Table 2 different parameters were included in the form in order to ensure that both experimental (N = 30) and (N = 30) control group participants share similar features or backgrounds. Furthermore, 10 volunteer participants took part in the semi-structured interview process.
Instruments
Two instruments were used in order to collect quantitative data for the purposes of the study. Firstly, the Course Interest Survey (Keller, 2010) is a situation-specific and self-report scale that can be used to assess students’ motivational attitudes in the context of both traditional face-to-face classroom teaching and instructor-facilitated synchronous/asynchronous online applications. Keller’s (1983, as cited in K. J. Kim & Frick, 2011) so-called ARCS (Attention, Relevance, Confidence, and Satisfaction) model has been applied in many educational institutions and also emphasizes that instruction becomes more motivating when it (a) provides higher learner attention (task engagement), (b) consists of activities and course content that students find more relevant, (c) expands learner confidence (self-efficacy), and (d) creates more satisfaction among students with what they have learned.
It is formed of 34 items in total and has a suggested level of Cronbach Alpha values for each subscale namely attention (.84), relevance (.84), confidence (.81), satisfaction (.88), and .95 for the total scale (Keller, 2010, p. 281). Regarding the current study, the Cronbach alpha values were also calculated, and based on the results of the Course Interest Survey (CIS), it was calculated as .90 for the pre-test and .91 for the post-test scores.
Another instrument that was used for the quantitative data is the Achievement Test which was developed by the researchers and included 25 multiple-choice questions. In terms of the development of the test, a literature review was held, and an item pool was generated. Then, a pilot test form was created and sent to the English Language instructors who have been working at a university for over 10 years. Based on the feedback on the Expert Evaluation Form, a question and some of the answers were revised. Then, a final version of the test was formed on the Google Forms platform and was sent to the instructors at the School of Foreign Languages in order to apply it to their learners on a voluntary basis. One hundred and two volunteer freshman students took part in the pilot study and a Test Analysis Program (TAP, version 19.1.4) was conducted to ensure the reliability and validity of the test, and the KR-20 reliability coefficient scores of 25 items were calculated as 0.87. Kuder Richardson formula ranges between 0 or 1 and the closer the value is to 1, the stronger the correlation, for example, if a KR-20 test result indicates between .70 and .89, there is a strong correlation (Schober et al., 2018). Since the KR-20 value was calculated as .87 it refers to a strong correlation in the test. As a result of the analyses, item difficulty indices were found to be between 0.41 and 0.82, and item distinctiveness indices were calculated as between 0.34 and 0.75. Furthermore, the mean discrimination index was valued at 0.52 and the mean item difficulty was revealed as 0.64 in the test. With reference to item difficulty, it is suggested that p values below .20 refer to difficult items and need to be reviewed for any confusing language or content, and p values above .90 indicate very easy items. Besides, the item discrimination value varies between 0.0 and 1.00 and the higher the value, the more distinctive the item is (Boopathiraj & Chellamani, 2013). Thus, both the item difficulty values, and item distinctiveness indices seem to be between the referenced values in the test.
A semi-structured interview which relates to a verbal interchange between the interviewer and the interviewee through asking questions in order to overcome the limitations of the pre-determined questions in the scales (Longhurst, 2003) was conducted to collect qualitative data and it was analyzed via content analysis. Field expert opinions were held, and a pilot test was run with two volunteer students to find out any problematic or unambiguous questions. As the participants were not sure about the meaning of the term, academic achievement, it was first explained and then interviewed. After making sure that there were no problems, the interview participants were selected voluntarily and they were informed about the purpose of the interview, how it would be conducted, and how long it would take. The interview process was held on the Microsoft Teams platform and analyzed via content analysis. There were eight questions in the interview and the questions asked the participants whether, in which direction, and in what ways web 2.0-supported gamification activities affected their motivation and academic achievement in online learning.
As in other studies involving human subjects, ethical considerations issues were considered. After obtaining official permission from the original developer of the scale, and the ethical approval report approved by the ethics committee of their university, the researchers informed the participants about the aims and procedures of the study and assured them that participation was voluntary, and their identities would be kept confidential.
Validity and Trustworthiness of the Study
It is well acknowledged that validity which refers to the meaningfulness of research elements (Drost, 2011), and reliability which relates to the consistency of research measurements (Knapp & Mueller, 2010) “lie at the heart of competent and effective study” (Thanasegaran, 2009, p. 35). Therefore, they need to be ensured before conducting research. Within this regard, threats to internal validity suggested by Fraenkel et al. (2011) in terms of the quantitative data and the trustworthiness criteria regarding the qualitative data suggested by Lynch (1996) and how they were overcome for the study are discussed as follows:
Threats to Internal Validity for Quantitative Data
Several measurements were taken to ensure the validity and reliability of the study. To overcome the threat regarding subject characteristics, a Demographic Information Form was created by the researchers, and it included information on various variables such as participants’ gender, departments, age, previous experiences with online learning environments, and Web 2.0 tools to ensure that the groups have similar backgrounds. Concerning the participants, the threat regarding the loss of subjects/mortality is believed to be handled as the current study is run on online platforms and requires voluntary participation. Furthermore, participants were informed that the intervention was not an experiment but a regular part of the course training, as participants’ opinions about a research study can affect internal validity, known as the Hawthorne effect. Similarly, since the participants of the current study are graduate learners and it ensures the required time, the maturation threat is thought to be avoided. Furthermore, as they do not have extremely high/low levels of pre-intervention performance and there is a control group, the regression threat is also ensured.
Some specific locations such as better-equipped classrooms or conditions may lead to obtaining alternative explanations for research results (Fraenkel et al., 2011). Since the current study was conducted on online platforms, there is no location threat, and since all data tools were created on Google Forms, an online platform, it is believed that the instrumentation threat is met. To overcome the testing threat, the study was designed on a quasi-experimental research method with a comparison group, and similar instruments were administered to the groups as pre-test and post-test to ensure that any improvement was due to the treatment. Otherwise, a significant difference could be expected for the control group participants in the study. Since the treatment was carried out on online platforms and based on the principles acknowledged in the New Normalization Guide by CoHE in Türkiye at the time of the Covid-19 pandemic all over the world, there did not occur any unplanned/unexpected cases during the data collection procedure and because the researchers were the only ones who conducted the implementation/treatment for both groups in online learning environments, the implementation threat is believed to be overcome accordingly.
Trustworthiness Principle for Qualitative Data
Various techniques to ensure the trustworthiness criteria (Guba & Lincoln, 1989 as cited in Lynch, 1996) were also applied. As one of the researchers was also a lecturer in the research setting and was the one conducting the intervention, the criterion of prolonged engagement and, as a natural consequence, persistent observation was established to ensure credibility and a deeper understanding of the context of the study. Furthermore, it needs to be indicated that throughout the research study, an ongoing process was run via getting the opinions of unbiased peers to ensure peer debriefing, and the whole process was checked by field professionals reiteratively in line with the member checks criterion. With reference to the negative case analysis, research hypotheses were reformulated based on the research process, and “why” questions were added to the yes/no questions to get further explanation of the semi-structured interview. Similarly, since progressive subjectivity suggests a continuing recording or archiving process for changing or emerging constructions and making comparisons among them, participants’ opinions were taken throughout the research study accordingly. To ensure transferability; an in-depth description of qualitative data was revealed comprehensively to provide a thick description and with reference to the dependability, each step in terms of the evaluations was documented by the researchers in detail. Since the qualitative data gathering was recorded on the online platform, it is possible to evaluate the process by an outside reviewer in the following period. Therefore, the criterion for confirmability was also ensured in this way.
Data Collection Procedures
Considering the pre-test scores showing that the experimental and control group participants had similar characteristics in terms of motivation and academic achievement, the treatment including gamification activities through Web 2.0 tools was started to be carried out with the experimental group, while the control group was simultaneously taught the course topics on the Microsoft Teams platform through traditional PowerPoint presentations.
Since a pilot study was run before the main application, several arrangements were made accordingly. Thus, two different activity links, one for virtual classrooms and one for self-paced learning for those who followed the online course on their mobile phones and needed another device as clickers were shared with the participants for Kahoot! activity and two different leaderboards were shown via the researchers’ screen sharing.
As there were no problems during the pilot study of the Quizizz activity, the experimental group participants were informed about the general framework, including the rules, the various power-ups, and the redemption question where participants could have a second chance for three questions they had previously answered incorrectly. Then, a live test was initiated by the researchers and participants answered at their own pace. The whole process and leaderboard were presented as a real-time event through the researchers’ screen sharing. Unlike the previous activity, participants did not need another device to take part in the Quizizz platform.
The participants were informed about the gamification activity on the Socrative platform as there were no problems during the pilot study. Unlike a kind of game code as in other applications, a room name was created by the researchers and the participants joined the activity by logging in, and a live quiz was started. The whole process and instant feedback were shared with the participants through the researchers’ screen sharing.
The following week, the participants were given brief information on another online learning program, Mentimeter, and how to take part in a word cloud activity. Since the previous course topic was on “verbs used in the classroom,” they were asked what kind of verbs they remembered. Then, a game code was shared with them, and they were asked to go to menti.com to type their answers. Each participant had a chance to type three entries and submit for multiple times.
Data Analysis
Several statistical measurements and analyses were run in the study. Before applying a parametric or non-parametric test, a Test of Normality was conducted first as it is suggested that the normality assumption needs to be ensured (Ghasemi & Zahediasl, 2012) because it is regarded to be a prerequisite (Mishra et al., 2019; N. M. Razali & Wah, 2011). Furthermore, an independent samples t-test was utilized in order to analyze any statistically significant difference regarding two types of data that are not influenced by one another (Nunan & Bailey, 2009) and a Mann-Whitney U test, which also functions as the non-parametric alternative to the independent samples t-test, was run when the normality assumption is violated and asymmetrical (Nachar, 2008). Besides, an analysis of covariance (ANCOVA) which aims to assess the effect of a treatment on some post-test scores while adjusting for baseline pretest scores (Oakes & Feldman, 2001) was also used in the study for the purpose of controlling potential confounding variables.
Findings
Findings Regarding the First Research Question
A Test of Normality was run for the Course Interest Survey and the results for the experimental and control groups indicated that the pre-test scores were not distributed normally (p = .008) which suggested running a non-parametric test. Therefore, a Mann-Whitney U test was conducted, and the results were indicated in Table 3.
Mann Whitney U Test Results Regarding Participants’ Pre-Test Scores in CIS.
Based on the results shown in Table 3, it was revealed that there is not a statistically significant difference between experimental and control group participants’ pre-test scores in CIS (U = 434.0, p > .05). Besides, taking the mean values into account, it is possible to indicate that the groups are similar to each other before the treatment. The Cohen’s d was calculated as 0.10 and within the framework of these results, ANCOVA analysis was performed to determine the significant difference between the post-test scores of the experimental and control groups corrected according to the pre-test scores in terms of post-tests within the framework of What Works Clearinghouse (WWC) standards and the findings obtained are presented in Table 4.
ANCOVA Results of Groups’ Post-Test Scores in CIS Adjusted for Pre-Test Scores.
The gamification with Web 2.0 tools was the independent variable and the dependent variable was the scores on the CIS test; students’ scores on the pretest of the CIS test served as the covariate in this analysis. Before ANCOVA analysis, it was checked that the conditions for normality, linearity, homogeneity of variances, homogeneity of regression slopes, and reliable measurement of covariates were met. According to Table 4, as a result of ANCOVA analysis of post-test scores corrected according to pre-tests, it was concluded that there was a statistically significant difference between the post-test scores of the experimental and control groups, F(1,57) = 19.48, p = .000, η2 = 0.25. Therefore, it could be indicated that the treatment with Web 2.0 tools has a positive impact on participants’ course interest and motivation levels.
Since the survey was created to investigate learners’ course interests; namely, motivation levels (Keller, 2010), participants’ views of the effects of gamification via Web 2.0 tools on their motivation were also obtained through semi-structured interview results, and the qualitative findings regarding the first sub-question were indicated as follows: Of course. To be honest, I was more motivated to see if the activity would be repeated every week and I thought that if there was an activity, I would attend the class to participate in it. (Interviewee 6) It encourages learning as our names are ranked 1st and 2nd on the leaderboard. It provides competition to answer earlier. It is nice to have our names on the list and this encourages us. In terms of motivation, I would like these gamification activities to continue. I think Kahoot! is the most useful app among them because I felt more comfortable as we all answered the same question at the same time. (Interviewee 5).
The findings obtained from both quantitative and qualitative data indicate that the treatment positively affects learners’ course interest or motivation levels.
With reference to the second sub-question on gender differences, a Kolmogorov-Smirnov test was run to analyze the homogeneity level of experimental group participants’ pre-test and post-test scores in CIS. Based on the results, it was revealed that there is a normal distribution in their pre-test and post-test scores which addressed running a parametric test, and the results were shown in Table 5.
Independent Samples t-Test Results of the Experimental Group Participants’ Scores Concerning Gender in CIS.
As shown in Table 5, it was revealed that there is not a statistically significant difference between pre-test scores of male and female participants (Mfemale = 125.60, SD = 14.05; Mmale = 126.40 SD = 15.45, d = 0.05), t(28) = −0.142, p = .602. Similarly, the p-value of the findings indicated that there is not a statistically significant difference between their post-test scores (Mfemale = 134.20, SD = 14.58; Mmale = 140.20, SD = 8.43), t(28) = −1.198, p = .241 as well.
The findings are also in line with qualitative research data that are indicated as follows: Of course. When there is a subject that I do not know, I go to research it directly and this increases my motivation. (Interviewee 10—Male) From my point of view, I think gamification activities definitely increase motivation. (Interviewee 8—Female) When you associate something with your own need, you learn it more quickly. If what we associate with ourselves while learning a language is passing the exam, even if it has a quantitative effect, it is short-lived, but if it is approached within the framework of a general vital purpose such as I need to live this language in my life, I think it also increases motivation. It motivates me because I like to study English. (Interviewee 7—Male)
Findings Regarding the Second Research Question
A Test of Normality was also run for each analysis of the pre-test and post-test scores of the groups in the Achievement Test and the homogeneity level indicated that the scores were not distributed normally, leading to the implementation of a non-parametric test and the findings were revealed in Table 6.
Mann Whitney U Test Results of Participants’ Pre-Test and Post-Test Scores in the Achievement Test.
The findings revealed that there was no statistically significant difference between the pre-test scores of both groups in the Achievement Test (U = 404.5, p > .05). In addition, the Cohen’s d effect size value for Achievement pretest was calculated as 0.16. Concerning participants’ post-test scores in the test, a Mann-Whitney U test was also conducted, and the findings indicated that there was no statistically significant difference between the two groups’ post-test scores (U = 348.00, p > .05). However, the mean scores showed that the experimental group participants had higher scores compared to the control group.
The quantitative findings were also supported by qualitative data findings with reference to the first sub-question obtained through the semi-structured interview process and the results were indicated as follows: I find gamification activities beneficial for success because games, entertainment, and competition are parts of life. I think follow-up activities have a reinforcing effect. To give an example, there are people I know who don’t know English at all but only learn English by playing games. I believe that the most important thing while learning a language is to keep it alive in daily life. I think it is effective because games are also related to current life. (Interviewee 7) Gamification activities enable the subject and words to take more place in our memory and reinforce the subjects and words. I think it helps a lot in this respect. (Interviewee 8) … Gamification activities help us remember words and spellings, make them memorable, and make the lesson more efficient. (Interviewee 5) When I saw my name at the bottom of the list on the leaderboard, there was a need to look at my mistakes. I saw that it was beneficial in that way. (Interviewee 10)
Participants’ responses to semi-structured interview questions also supported the idea that the selected Web 2.0 tools could be beneficial for students’ academic achievement levels.
Tests of Normality were also run to analyze the homogeneity level of experimental group participants’ scores concerning gender in the Achievement Test and the findings suggested that there was a normal distribution in their pre-test scores. Therefore, an independent samples t-test was run, and the findings were revealed in Table 7.
Independent Samples t-Test Results of the Experimental Group Participants’ Pre-Test Scores Concerning Gender in the Achievement Test.
The results indicated that there was no statistically significant difference between the pre-test scores of females and males (Mfemale = 76.30, SD = 17.51; Mmale = 82.00, SD = 16.46, d = 0.16) in the Achievement Test t(28) = −0.857, p = .595. Nevertheless, taking the p-value scores in the Kolmogorov-Smirnov test into account, a non-parametric test was run for their post-test scores whose results were illustrated in Table 8.
Mann Whitney U Test Results of the Experimental Group Participants’ Post-Test Scores Concerning Gender in the Achievement Test.
As shown in Table 8, it was revealed that there was no statistically significant difference between females and males in terms of their post-test scores in the Achievement Test (U = 72.50, p > .05). Similar findings were also revealed in the qualitative data that were indicated as follows: When I saw my name at the bottom of the list on the leaderboard, there was a need to look at my mistakes. I saw that it was beneficial in that way. (Interviewee 10—Male) It creates a competitive environment. We could see that our friends whom we had never heard of were participating in the activities and I think it’s nice. If one can do it, I feel like I can too. One day, people can try harder so that their names will appear on that leaderboard, and in this way, they can start to show more interest in the lesson. (Interviewee 8—Female).
Discussion
The findings of the study regarding the first research question indicated that the treatment through selected Web 2.0 tools including Kahoot!, Socrative, Quizizz, and Mentimeter has a positive impact on participants’ post-test scores of course interest or motivation levels in favor of the experimental group and the quantitative findings were supported through the qualitative findings on the semi-structured interview. Furthermore, the results of the study are in line with several other research studies cited in the literature. In a research study that investigated whether the implementation of Kahoot! has an impact on the improvement of learner engagement through a mixed-method design along with 96 undergraduates, it was indicated that learners’ engagement and course performance enhanced with the help of Kahoot! activities (Bawa, 2019). Through another research study which was run at a research-intensive higher education institution in New Zealand, it was revealed that the incorporation of Kahoot! promoted a better engagement process and contributed to enhancing participants’ learning experiences, motivation, and classroom dynamics (Licorish et al., 2017).
It is discussed in several research studies that gender variable seems to be controversial with reference to online learning outcomes (Yu, 2021), and the present study also aimed at investigating learners’ perceptions regarding gender differences in their course interest or motivation levels on the second sub-question. As a result of the findings, it was concluded that there is not a statistically significant difference between male and female participants of the experimental group regarding their pre-test and post-test scores in CIS. In a similar vein, Kaya and Balta (2016) aimed to investigate university prep school students’ attitudes towards Socrative as a response system and the findings indicated that it is a beneficial tool that increases learners’ engagement without differences regarding gender and it promotes an interactive atmosphere in English language classes. However, Lim and Kim (2003) concluded that in terms of learner characteristics, gender had a statistically significant impact in favor of female learners compared to male participants in online learning. Furthermore, Tsay et al. (2018) investigated the effects of learner background and revealed that females participate in activities in online learning environments significantly more than males.
Concerning the findings on the second research question regarding the effects of Web 2.0 tools on learners’ academic achievement in online learning environments, it was revealed that there is no statistically significant difference between the experimental and control group participants. However, the findings indicated higher mean scores in favor of the experimental group which addressed the idea that the treatment has a positive impact on learners’ academic achievement in terms of their post-test scores. Furthermore, the qualitative findings on the first sub-question revealed that the participants had positive perceptions of the treatment with Web 2.0 tools during their studies in online learning environments. Similar findings were also suggested in the literature. In a research study by Baig (2011), the effectiveness of online learning on 10th-grade level learners was investigated. Based on the results, it was indicated that the experimental group had higher scores than the control group in the achievement test. Denny et al. (2018) examined whether student activity involving game elements mediated the relationship between students’ test scores and gamification, and the results showed that the experimental group had higher scores. On the contrary, it turned out that many of the control group students had lower test scores. Similarly, Yildirim (2017) designed a true experimental research study along with 97 university students in Türkiye during the 2014-2015 academic year. Based on the results through participants’ pre-test and post-test scores, it was indicated that the treatment via gamified teaching practices has a positive impact on student achievement and their attitudes towards the course. Maesaroh et al. (2020) aimed to explore the effects of Kahoot! and Socrative activities on grammar achievement of learners with high and low-interest levels based on an experimental research study with a 2x2 factorial design and the results showed that both applications are effective on grammar mastery of learners with different interests and Kahoot! got higher mean score than Socrative. Alsadoon et al. (2022) conducted a research study during the time of Covid-19 pandemic and investigated whether a gamified e-learning environment had an impact on the improvement of eighth-grade learners’ achievement, satisfaction, and motivation along with a quasi-experimental method and the results indicated that the treatment increased learners’ motivation and satisfaction but not a statistically significant impact on their achievement levels. However, Domínguez et al. (2013) conducted a research study with higher education learners and revealed that although learners who completed their tasks with gamified experience had higher scores in practical assignments and in their overall scores, they performed lower scores in written assignments. It is thought that this differentiation may be due to differences in study design and content related to the gamification process (Turan et al., 2016). Groening and Binnewies (2019) also argue that digital achievements, which are the cornerstones of gamification, increase the motivation and performance of learners only if they are designed correctly.
The current study also aimed to analyze whether participants’ gender differences affect their perceptions of academic achievement levels through the second sub-question and based on the findings, it was revealed that there is not a statistically significant difference between experimental group participants’ pre-test and post-test scores in the Achievement Test in terms of gender. Similarly, in a study, Nistor (2013) found that gender differences did not have a significant impact on participants’ learning outcomes, with women outperforming men in engagement, while men were more stable in terms of attitudes. Furthermore, Yu (2021) aimed to investigate the effects of education levels, personality traits, and gender on students’ online learning outcomes with a mixed design and found no statistically significant difference regarding participants’ gender.
Conclusion and Implications
Unlike previous research studies which mainly focus on a comparison of the effects of instruction between traditional and online learning platforms, the present study aimed to investigate the effects of gamification with web 2.0 tools on EFL learners’ motivation and academic achievements in online learning environments in two groups as experimental and control who are both online learners that were instructed through online applications following the period of the sudden outbreak of the global Covid-19 pandemic. Since it is acknowledged that the rapid transformation from traditional classroom teaching to online learning platforms during the pandemic has affected learners’ motivation toward digital learning both intrinsically and extrinsically (Gustiani, 2020), it has become more important to provide learners with new approaches or applications to overcome the threat of course drop-outs. Since it is suggested that resources that may support both teaching and learning practices are critically important now, they will continue to reshape the educational processes even after the Covid-19 pandemic (Adams, 2020; Pichardo et al., 2021). From this perspective, Web 2.0 tools are seen to increase online collaborative learning and generally support student learning (Chou & Chen, 2008).
The study has some limitations in that the data were collected from 60 freshman university students studying at a state university in Türkiye. Therefore, the findings may not be generalizable to other students of different contexts and different settings and a large-scale online learning experience via Web 2.0 tools could be created. Since the present study investigated the effects of gamification in online learning environments via Web 2.0 tools along with gender differences, other variables that were included in the demographic information form such as participants’ previous experience with online learning, mobile device preferences, perceived computer self-efficacy levels may be the other sources of future research studies. Due to the development of technology day by day and the fact that this study is based on the effects of gamification with selected Web 2.0 tools in online learning environments and was conducted in a certain period of the pandemic, different findings may emerge at other times or in different geographies. Therefore, further studies can be conducted in this direction. The data were collected through selected Web 2.0 tools such as Kahoot!, Quizizz, Socrative, and Mentimeter. Therefore, other Web 2.0 tools can be conducted in other studies with different participants in different settings. Lastly, since the current study was based on a mixed method sequential explanatory research design, other types of research methods or designs may also be used in other research studies.
Despite the limitations, the current study has several implications in today’s digital world where information can be reached from everywhere and individuals are required to be accustomed to the changing and emerging constructions every day. Therefore, all levels of the educational institution should provide interactive learning environments that arouse interest in their students and motivate them for a more dynamic participation or engagement process, and it is believed that the current study will reveal an alternative path. The use of Web 2.0 tools in online learning environments contributes to the increase of learners’ interest, motivation, and academic achievement levels, while at the same time reducing drop-outs in these environments.
It is believed that the practices, applications, or methods used in online learning environments during the pandemic will continue to reshape educational practices in the post-pandemic period (Robson et al., 2022; Tartavulea et al., 2020). In line with this, based on the decision by the CoHE in Türkiye entitled “Procedures and Principles Regarding Distance Education in Higher Education Institutions,” a maximum of 40% of the courses in the programs and the courses specified in Article 5-i including Compulsory Foreign Language at Turkish universities can be delivered through both formal and online instruction. The present study was conducted in online learning environments to give insights and reveal online practices as compulsory courses including Foreign Language have been decided to be instructed through online platforms in a synchronous way in many higher education institutions in Türkiye and will continue to be instructed in online learning environments in the following years.
As the rapid transition from traditional applications to online platforms, mostly during the time of global pandemic, has led to major changes in our daily lives and accordingly our educational practices, many instructors have turned to using direct transfer for their online lessons without making significant changes in their teaching due to limited knowledge in online learning environments (Başal & Eryılmaz, 2021). Nevertheless, Köksal (2004) emphasizes that it is very important for language teachers to be aware of the latest and greatest equipment and what is appropriate and applicable in each teaching situation. In this regard, Cesur (2021) states that if the technological tools used in online foreign language teaching are easier, more meaningful, and fun, they will have a positive effect on the process. Therefore, elective courses regarding different methods and applications to use in online learning environments can be provided for prospective teachers within English Language Teacher Education Programs. Furthermore, various seminars and in-service training programs can be organized for faculty members, EFL instructors, teachers, educators, and other stakeholders in the field of education to increase participants’ motivation and course interest levels in online learning environments and technology teaching courses can also be designed for educators of various teaching levels. Besides, in relation to the CIS questionnaire used in the study and based on the sub-headings of attention, confidence, relevance, and satisfaction, Izmirli and Izmirli (2015) state that if the relevant dimensions are taken into consideration when designing courses in online learning environments, learners’ motivation and performance will increase.
Moreover, because of the advances in today’s technological world, Web 2.0 tools can also be used in the assessment and evaluation process in foreign language teaching and learning environments and necessary up-to-date changes can be made in the curricular contents in terms of the use of technology and decision makers can consider best practices to implement the tools for necessary adjustments (Orava & Worrall, 2011). Furthermore, Tasir and Al-Dheleai (2019) suggest that since participants feel disconnected from others in online learning environments, their social presence can be fostered through more interactive and collaborative learning opportunities supported by Web 2.0 tools in online learning environments.
Footnotes
Author’s Note
This paper was produced from the doctoral thesis written by the first author under the supervision of the second author.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
Data Availability Statement
The dataset supporting the conclusions of this article is included within the article and for more details can also be kindly requested from the corresponding author whenever necessary.