Sage Journals: Discover world-class research

Abstract

Background

The fields of gamification and game-based learning are growing and changing quickly, with researchers investigating the use of gamification in education using various methods and in numerous case studies, however, no research investigates the significance pre-perception of higher education students has in gamification, game-based learning, and serious games studies.

Objective

We perform a systematic mapping to identify the gaps in our understanding on the effects of pre-perception in gamification, focusing on the under-researched area of gamification and game-based learning applications for higher education.

Results

We present three main findings: (1) Current research on the effects of higher education’s students' pre-perception of gamification and game-based learning are limited. (2) The limited available research indicates perception towards gamification and game-based learning in participants may impact on the effectiveness of game-based solutions, but more research is required to better understand the relationship. (3) A significant body of new research work is required to examine how student perception can affect the effectiveness of gamification and game-based learning solutions in higher education.

Conclusion

Our understanding of the effect pre-perception may have on the results reported by research on gamification applications for higher education is limited. We encourage more, non-empirical research, to be conducted around pre-perception, and more research that considers gamification and game-based learning applications for higher education.

Keywords

gamification game-based learning higher education student perception systematic mapping

Background and Motivation

Gamification, defined as the use of game elements in non-game contexts (Deterding et al., 2011), has been widely reported to provide positive changes in academic performance and intrinsic motivation to students across multiple levels of study, including higher education (Bai et al., 2020; Buckley & Doyle, 2016; Chapman & Rich, 2018). Similarly, game-based learning, defined as games used towards learning and education, is also considered as an effective method for making learning more engaging across several age groups (Tang et al., 2009). The academic field of gamification in education interacts with multiple related academic domains, including edutainment (Egenfeldt-Nielsen, 2011), serious games (Susi et al., 2007) and more. The wide adoption and availability of sensors and actuators have impacted the field of game-based solutions for education. Research considers the relationship between serious games, gamification, and the internet of things (Henry et al., 2021).

Research into the impact perception has on adoption or successful intervention is limited. Research into the relationship between perception of autonomous vehicles and their adoption or acceptance (Zhu et al., 2020) discovered that self-perception and perception of autonomous vehicles had an indirect influence on adoption. Considering their findings, we question what effects pre and post perceptions of gamification and game-based learning interventions have on effectiveness metrics, such as adoption rate, in higher education. Other existing research has designed and validated a scale to score user’s gamification preferences based on personality traits (Tondello et al., 2016). Their scale is effective for designing gamification solutions but does not consider the pre-perception or confirmation bias a student may have towards gamification or game-based learning.

Focusing on the prior perceptions of gamification and game-based learning, no existing mapping study explores the effects of pre-perception against these contexts and the outcomes generated from exposure. Significantly, limited research explores existing bias in perception prior to exposure of a gamification or game-based application. Different tools are used to measure the pre-perception and perception of a student post exposure to a game-based solution for education. Commonly, perception is measured through questionnaires (Roodt and Saunders, 2017, Tuparova et al., 2020), though there are studies that use a combination of interview groups, one-on-one interviews and questionnaires (Hosseini et al., 2019, Berns et al., 2016) to enrich their understanding of participant perception. Considering the size of the academic field of gamification, there has been less focus and emphasis on student’s prior perception on gamification in existing studies most studies focus on the pre and post gamification concerning learning outcomes and user experience. Though post experience and perception on gamification have been explored (Khuzzan et al., 2021), there is a lack of understanding in the significance and role of prior perception in the success of gamification and game-based solutions in education. Existing research has mapped out gamification studies applied in education (Dicheva et al., 2015), considering the impression of students as part of evaluation. Most relevant systematic reviews or systematic mappings focus on the effectiveness of serious games and gamification in a particular area, such as healthcare (Wang et al., 2016), education (Boyle et al., 2016) or manufacturing (Menandro and Arnab, 2020) Other systematic reviews in the field, consider how empirical studies may inform behaviour (Hammady and Arnab, 2022) and learning (Lameras et al., 2017).

One systematic mapping to our knowledge exists that considers how perceived skills in science, technology, engineering, and mathematics (STEM) related subjects may explain an over-representation of studies involving gamification in education (Dicheva et al., 2015). Their study does begin to consider how perceptions of any nature may inform the results we note in gamification studies. We identify the need for a systematic mapping that examines how the academic field considers the effect of prior perceptions of gamification and game-based learning in their results, so we can begin considering how to design gamification systems that engage a wider demographic.

Since, no existing research maps pre-perception or perception across multiple gamification studies. Pre-perception towards gamification and game-based learning can be considered as confirmation bias. Research into how confirmation bias may alter the receptiveness of personalised recommendations (Schwind and Buder, 2012), identified a model for reducing the effectiveness of confirmation bias and acknowledged the impact towards receptiveness. No existing research considers the impact of confirmation bias has on higher education students’ likelihood to adopt gamification intervention and perform well in them, when considering gamification for the improvement of academic performance.

A systematic mapping by Alhammad and Moreno (Alhammad and Moreno, 2018) into gamification for enriching software engineering learning experiences uncovered a lack of conclusive data around the perception of students and noted a lack of understanding in the field towards the underlying causes toward successful gamification. A recent systematic mapping study (Alhammad and Moreno, 2018) focused on the negative effects of gamification in education and learning. Their mapping discovered negative effects around motivation, outcomes and considered best practises of gamification applications.

Our paper explores the pre-perceptions and perceptions of participants on gamification and game-based learning, mapping existing studies that investigate the pre-perception and perception of participants against the education application. We map out the relationship between student pre-perception and perception and the effectiveness of the education intervention and propose how new research may investigate student pre-perception and the impact on effectiveness for gamification in higher education. Effectiveness in this context considers either adoption rate or academic performance. The ambiguity of effectiveness is fostered by the diversity of research and the context of application of gamification in education by the literature in our corpus.

Furthermore, we hypothesise that our results can be generalised across multiple gamification and game-based applications if consistency between perception collection tools and methods for analysing the data is achieved. Our paper promotes new research in the field of gamification, presenting key considerations around effective gamification applications.

Methodology of systematic mapping

Our study builds upon the systematic mapping guidelines updated by Petersen et al (Petersen et al., 2015) and considers the following research questions:

• RQ1: What studies explore the pre-perception and perception of gamification and game-based learning in higher education students since 2015?

• RQ2: Does pre-perception of gamification and game-based learning impact the effectiveness of the intervention?

• RQ3: What tools are used to measure gamification and game-based learning perception and pre-perception?

• RQ4: What future research is required to better understand the relationship between pre-perception, game-based learning and gamification?

After going through the five suggested steps of systematic mapping planning, we have developed a two-stage search strategy. Stage 1 involves database search followed by manual sifting. Stage 2 compliments Stage 1 with a manual search to ensure sufficient representation of our database. Initial database search was conducted in August 2021. To ensure currency of the corpus, we repeated the database search in August 2022 to include new publications since the first search. These two database searches resulted in Database 1, which was then complimented by the manual search that formed Database 2. In this section, we present our methodology from the aspects of source selection, search criteria, and eligibility assessment for both stages followed by an explanation of the data coding and analysis process.

Source Selection

To construct our mapping database, in Stage 1, we performed searches on four widely used databases: IEEE Xplore, Scopus, ACM, and Web of Science. We chose multiple databases based on the experience of Petersen et al (Petersen et al., 2015) which, according to Shibuya et al. (Shibuya et al., 2022), helps reduce the selection bias compared to single database search. Stage 1 produced Database 1.

In Stage 2, we sifted out the five most cited publications in Database 1 by August 2022 and conducted a manual search on their reference lists. We select relevant items based on reading through their titles, abstracts, and author keywords. This stage produced Database 2. We then combined Database 1 and 2 together and read through the full text to identify items to be included in our corpus for this systematic mapping.

Search Criteria

To add articles to our corpus, in Stage 1, we used key words: “gamification”, “perception”, and “higher education” which were grouped as a set. The synonym for gamification used was: “gamified”. To consider game-based learning, we include the term “game-based”. Therefore, the search string used on each database is (“gamification” OR “gamified” OR “game-based”) AND (“perception”) AND (“higher education”), with minor variance due to the different search functions of each platform. The search starting year is 2015 when the previous gamification mapping paper was published. Table 1 summarises the number of items found in each database.

Table 1.

Database search outcome.

Search Fields	IEEE Xplore	Scopus	ACM	Web of Science
Full Text and Metadata or All Fields or Anywhere	395	7240	168	80
Document Title or Title or Article Title	0	2	0	3
Abstract	4	71	1	30
Author Keywords or Keyword	1	10	1	1

With the above criteria, in Stage 1, our database search turned out 395 articles in IEEE Xplore, 7240 articles in Scopus, 168 articles in ACM, and 80 articles in Web of Science based on applying the search string in all fields (i.e., full text and metadata or all fields or anywhere), totalling 7883 items. Our initial results raised concerns of database search accuracy due to such large number of results as a quick review of the items found a large number of irrelevant publications. We therefore further refined the search to title, abstract, and author keywords, which turned out a total number of 71 items after removing duplication (Database 1).

In Stage 2, our manual search through the five most cited publications’ reference lists using the same key words above have turned out 74 articles by title, narrowed down to 34 after removing articles published prior to 2015. We then read through the abstracts and excluded papers that did not state the study was related to gamification, game-based approaches or the pre-perception or perception of gamification or game-based perception which further narrowed down the list to 11 (Database 2).

Eligibility Assessment for the Final Analysis

To evaluate the eligibility of the retrieved articles, the first and second author manually checked the inclusion criteria listed below by reading the publications’ titles and abstracts. When it was impossible to make a clear decision, other parts of the article, especially the methodology and data collection statements, were examined during consensus meetings between the first and second author.

We adopted consensus meetings to verify our data collection during the screening stages. More specifically, when the initial database search results are exported, both the first and second author conducted the duplication removal process. The first author did it by applying a set of python codes and the second author did it manually. Once the full list of database search results is confirmed during a consensus meeting, the work of reading through the title, year, abstract, and full paper is split evenly where possible between the first and second author. After each round of screening, a consensus meeting is held to discuss items that either party is not certain about before moving on to the next round. The items in question are investigated and confirmed by both the first and second author during the consensus meeting. The final stage is to determine whether an item should be included in the final mapping corpus. After completing this process for the initial database search in 2021, the same approach is applied for the manual search as well as the updated database search in 2022.

We only incorporated an article into the corpus when all four inclusion criteria were met:

• Topic relevance: an article’s topic is relevant to perceptions of gamification in a higher education context. We manually examined whether the aim, goal, or research questions of a study is related to understanding how gamification (or gamified and game-based) intervention is perceived in higher education, including how elements such as students’ attitude, motivation, learning experience, satisfaction, and performance are affected with or without gamification intervention.

• Peer-reviewed: an article is a peer-reviewed publication either in conference proceedings or peer-reviewed journals.

• Language: an article is written in English.

• Year: an article that is published since 2015.

• We excluded an article if it meets any of the following criteria:

• No-match topic: an article’s topic that is irrelevant to true gamification intervention was excluded. For example, an article that used sports games in PE teaching class did not fit this criterion because the nature of the class makes gamification less of an intervention.

• No empirical data: an article that is not a completed study, or without empirical data, or actual conclusions and conclusive findings was excluded.

• Non-English articles: we excluded article’s whose main bodies are written in non-English language even if their titles and abstracts are in English. This is because we could not conduct meaningful mapping analysis in a non-English language.

• Lack of method and data descriptions: we excluded articles that are lack of explanations in methods or data. For example, when the sample size is unclear or there is a lack of methodological explanations. We found this happen in many conference abstracts where the full texts were also not available.

The data collection and screening process is shown in Figure 1. After reading through the full text of 82 items in Database 1 and 2, our corpus is comprised of 46 eligible publications for mapping, as illustrated in Figure 1. More detailed account is provided on the extraction and classification process in the next section.

Figure 1.

Flow chart of corpus collection for systematic mapping. The stages of the mapping are noted on the left and the process from start to finish is illustrated from top to bottom.

Data Coding and Analysis

Following the data screening process, we start the extraction and classification process by studying the 46 articles in greater depth. As stated in the next section, we firstly present a descriptive account of the corpus in terms of source and venue of publication, referred to as the topic-independent classification by Petersen et al (Petersen et al., 2015). We then start the topic-specific classification by investigating the aspects below.

Process for gathering student pre-perception and perception on gamification and game-based solutions

As all articles explore perception towards gamification or games-bases solutions with higher education students, we ask how perception has been gathered. Firstly, we look at whether a study gathers perception towards a specific gamification solution or gamification as a generic concept. Secondly, we investigate at which point has perception been gathered throughout students’ exposure to gamification or a game-based solution.

Relationship Between Perception and Effectiveness

In this aspect, we categorise the corpus based on the reported beliefs and preconceptions of gamification by higher education students or the perception of a gamification or game-based solution after exposure discovered in those studies (i.e., positive, negative, neutral, and N/A) and the type of effectiveness revealed by them (i.e., positive, negative, neutral, and N/A). We gather the perception values reported in the studies and map them against the effectiveness results noted in the same studies. For example, if a study noted overall a positive perception of their solution, we mapped that against their overall noted level of effectiveness. This would facilitate our understanding on the relationship between perception towards gamification and the impact gamification has on students.

We hypothesise we will see similar results from student perceptions to similar research in the field around the acceptance of game-based monitored through teacher responses (Bourgonjon et al., 2013). We could not gain an insight into pre-conception and beliefs of students of gamification and game-based solutions in higher education due to a lack of literature in the field. Our full results are discussed in 4.2.

Methods for Measuring Gamification and Games-Based Perception

Additionally, we look at the methods adopted by studies in our corpus to measure perception of gamification in existing literature to obtain an overview of the current research status in this field.

Lastly, we analysed the future research sections and identified keywords to form a word cloud, which provides a text data visualisation based on the crucial academic areas for future research contained in the selected articles. In the word clouds, the font sizes represent the frequency of topic appearances within the corpus’ author keywords (the maximum font size is set to 50 pt and decreases analogous to the decreasing frequency of the word per 1 pt). The world cloud classifies the most commonly stated areas of future works by the literature in the corpus, helping address RQ4.

Results

Our paper categorises the results from the systematic mapping into four sections. We characterise the corpus in Subsection 4.1. Following this, Subsection 4.2 details the relationship between perception and effective gamification. Lastly, Subsection 4.3 outlines the tools used for measuring student perception and Subsection 4.4 amalgamates the future works identified by literature from the corpus. To promote the replicability of our results, we have uploaded our data to GitHub (https://github.com/doctorhenry/Gamification-Perception-Mapping-Data, Accessed 04/04/2023).

Corpus Description

Our collection of literature derives from 10 publishers and 41 publication venues. Analysis on the metadata from the articles in our corpus present Springer as the most popular publisher for research that considers student perception of gamification in higher education. Elsevier was the second most common publisher. A full breakdown of the analysis totalling publishers is presented in Figure 2. 15 articles were classed as “conferences or others” as not all peer-reviewed publications had a publisher related to them in their citation data. Each of the publication venues were accessed to identify missing publisher information.

Figure 2.

Distribution of publishers in the corpus. Others describes conference publications and publishers with less than two papers.

An in-depth analysis of the corpus followed, detailing the publishing venues. The focus of the analysis was to highlight common publishing venues in the research area of gamification, game-based solutions and student perception in higher education. Publishing venues that only contained a single article were ignored for this phase of analysis.

Figure 3 presents the most common publishing venues from the articles in our corpus. Computers in Human Behaviour (Elsevier) and European Conference of Games Based Learning totalled three publications each, with International Conference on Education and New Learning Technologies (EDULEARN), Sustainability (MDPI), Education Sciences (MDPI) and Education and Information Technologies (Springer) presenting the second most common publishing venues, with two articles each.

Figure 3.

Distribution of most common publishing venues in the corpus. Venues with less than two publications counts have been excluded.

All included articles in the corpus measure student perception towards gamification and game-based solutions. Differences emerge in the methods used to gather students’ perception on gamification and game-based solutions, the methodology and the rationale. This subsection explores the articles in the corpus to identify:

1. If the data on perception considers a specific solution or gamification or a game-based solution in general

2. If perception was gathered before exposure to gamification or game-based solutions, afterwards, or both

As seen in Figure 4, out of the 46 papers in the corpus, 37 related to research specific to a particular gamification or game-based solution. The remaining formed conceptual studies. All 37 studies that investigated the effectiveness of a gamification or game-based solution obtained data on student perception after exposure. This finding highlights a gap in existing literature that identifies pre-perception or confirmation bias, towards gamification and game-based approaches prior to exposure. Such research would further our understanding on demographic and personality traits that can contribute towards the effectiveness of a gamification or game-based solution.

Figure 4.

The distribution of studies from the corpus that relate to a specific gamification or game-based solution.

The Relationship Between Perception and Effective Gamification Intervention

Focusing on RQ2 of the study, as described in the Section 1, the analysis here investigates if the literature from the corpus can identify any data connection between student perception of gamification and gamification effectiveness. From the 32 articles focused on gamification, 18 included results on student perception post exposure and effectiveness, considering factors such as academic performance. As seen in Figure 5, 16out of the 18 studies reported both positive perception and positive effectiveness. 1 of the 18 papers reported positive results on the effectiveness of the gamification solution and neutral results towards student perception. Lastly, 1 out of the 18 papers reported neutral results on the effectiveness of gamification and positive results on student perception. The results of our mapping highlight that the corpus of literature is too small to draw conclusions between the relationship of pre-perception and the effectiveness of a gamification intervention.

Figure 5.

The count of all studies reporting on effectiveness of their gamification intervention and the student perception. Effectiveness is represented by the letter E and Student Perception is represented by the letter P.

Including the studies that only reported on either effectiveness of gamification or perception towards it, and the studies that did not present results on either, alters our results but maintains a positive outlook towards gamification effectiveness and perception.

Figure 5 also illustrates the total of studies that did not include either any data or partial data towards the effectiveness of gamification and student perception of gamification. In most cases, the nature of the studies or the context prevented the authors from completing the required data collection and analysis. 10 out of 33 studies related to gamification reported positive results towards the perception of gamification but did not report findings on the effectiveness of gamification. Most studies in this category are either preliminary or focus solely on perception.

The Relationship Between Perception and Effective Games-Based Intervention

We note similar comparable results to gamification when focusing on game-based interventions, though the sample size is reduced. Out of the 46 papers in our corpus, 14 focused on game-based interventions. Figure 6 presents the distribution of literature against the same criteria used to map gamification interventions in the previous section. Out of the 14 related papers, 5 reported positive results on student perception and effectiveness.

Figure 6.

The count of all studies reporting on effectiveness of their game-based intervention and the student perception. Effectiveness is represented by the letter E and Student Perception is represented by the letter P.

The second most common result was positive student perceptions, but no data reported on effectiveness. Again, the studies in this category that did not note results on effectiveness were early in nature, commonly testing the perception of prototypes before developing the intervention itself. 2 out of 14 papers reported neutral student perception results and positive results on effectiveness and 2 out of 14 papers also reported no data on student perception or the effectiveness of an intervention. Finally, 1 out of 14 papers reported positive results on student perception and no results on the effectiveness of the intervention.

Methods for Measuring Gamification Perception

This subsection explores the different tools the articles in our corpus used to measure student perception aligning with RQ2, focusing on literature related to gamification. Most studies used questionnaires alone to measure perception (n=18). Questionnaires varied from a collection of question focusing on quantitative, qualitative data or both. The second most common method for measuring student perception of gamification was a combination of assessments and questionnaires (n=6). We consider assessments for studies that used academic performance to validate the effectiveness of gamification and consider the perception of students during the same experiment.

Figure 7 presents the complete set of results. We annotate the rows of data as follows:

• A: Assessment

• O: Observation

• F: Focus Group

• Q: Questionnaire

• I: Interview

Figure 7.

The distribution of tools used to measure perception of gamification across the literature in the corpus focusing on gamification. The letters in the left-hand column describe the following: A: Assessment, O: Observation, F: Focus group, Q: Questionnaires, I: Interview.

Our analysis highlights the small number of studies that use multiple methods of determining student perception, compared with the use of questionnaires alone. A combination of methods can aid in understanding the impact of student perception on the effectiveness of gamification and should be considered for future nonempirical studies that aim to gain a more longitudinal insight into the effectiveness of gamification in higher education and what drives effective gamification intervention.

Methods for Measuring Games-Based Perception

Our results focusing on game-based solutions find similar patterns in the tools used for feedback. Most studies used questionnaires alone (n=6), with the second most popular method for gathering feedback being interviews alone (n=2) and questionnaires and assessments combined (n=2). Figure 8

Figure 8.

Finally, we note that the remaining literature related to game-based approaches divide evenly (n=1) between questionnaires and interviews, no tools used, focus group alone and questionnaires and focus groups. While there are no real differences between gamification and game-based solutions, we highlight the difference in sample size retrieved from our criteria, which could produce a statistical imbalance. We consider this point in our limitations section of this paper.

Crucial Academic Areas for Future Research

Our analysis around RQ4 drew the most common keywords from the future works sections of the articles in our corpus. Not all papers stated future works. We highlight the most common areas of future work through a word cloud as seen in Figure 9. As part of the analysis, we removed the phrases “students, future works and future research” to reduce common words with little relevance to the future work being discussed.

Figure 9.

Common keywords identified from future works sections of the literature in the corpus for gamification and game-based interventions.

The most common word in future works was effectiveness. The word refers to authors from the articles that noted future research is required to better determine the effectiveness of their work. The trend of research in our corpus highlights that more early work or preliminary work is available on studies that measure student perception of gamification since 2015 than complete studies. This finding matches the results from subsection 4.3, where studies that only used questionnaires to measure student perception made up 50% of the corpus.

Other notable keywords from future works include intrinsic motivation, gamification intervention, gamified course, and game-based intervention. Studies that mentioned intrinsic motivation in their future works express a desire to gain a deeper insight into their results. The key phrases of gamification intervention and gamified course and based intervention do not directly consider future works in most instances but are descriptors that contextualise the author’s research.

Finally, we highlight their key phrases from the word cloud, period of data, dimensions of data and range of data. All of three key phases relate to authors identifying a lack of extensive data in the studies from our corpus, describing more elaborate experiments with larger sample sizes, over a longer period and with a greater range of data.

Discussion

Our findings revealed a gap in existing literature showing that there is insufficient understanding on gamification and game-based perception prior to exposure which could provide insight into studies that struggled to determine their effectiveness as an education solution, albeit the possible correlation between positive perception towards gamification and its effectiveness in education. Therefore, there is a need for extensive studies to gather perceptions prior to and after exposure to a gamification solution, while investigating the effectiveness of the proposed solutions. More comprehensive methods to measure gamification perception and effectiveness are also plausible, particularly with larger samples and over longer periods of time. We further elaborate our findings below.

Perceptions of Gamification and Games-Based Solutions

As stated in the findings section, our corpus (n=46) reveals several approaches in gathering perceptions towards gamification and game-based solutions in existing literature while no studies reported on pre-perceptions. Our corpus includes different perspectives to approach gamification have been adopted. For instance, some look at gamification itself and study students’ general perception or acceptance towards it (Ofosu-Ampong et al., 2020, Tuparova et al., 2020, Thi Van Pham and Duy Nguyen, 2021), or look at game-based solutions using the same criteria (Subhash and Cudney, 2018) some through the angle of learners’ satisfaction, motivation, and engagement (Bahji et al., 2015). Others focus on more specific aspects of gamification such as the effects of groups (Ahmad et al., 2021), fun in the game-based learning process (Whitton and Langan, 2018), and gamification as a competitive process (Clements et al., 2017). However, this group of studies, due to their generic nature in enquiry, have not revealed detailed insights on the effectiveness of gamification as an education solution or intervention.

We do note that a study by Menando and Arnab (Denden et al., 2021) has revealed that some effects of gender and personality may cause differences on students’ perception of gamification in education. While our systematic mapping found no correlation between such perceptions and their impact on the effectiveness of the gamification intervention, we do recommend that further studies do investigate the relationship between personality traits and pre-perceived perceptions around gamification. For example, future research may wish to consider how learning styles have an impact on pre-perception of gamification.

For the studies in our corpus that have investigated specific gamification solution (n=25) perceptions are measured for students that have used or experienced gamification. The same is true for studies on game-based solutions with a specific solution (n=12). This, as previously indicated, represents a significant gap in which resides the potential bias on both perceptions and effectiveness of the gamification or game-based solution in question. Nevertheless, these studies have presented a seemingly positive picture in that most of them (33 out of 37) discovered positive perceptions towards gamification (n=) or game-based solutions (n=10). For example, (Manzano-Le´on et al., 2021) focus on developing and validating a tool to measure perception towards a game-based solution rather than looking at perception itself. (Belova and Zowada, 2020) use gamification as a tool to facilitate teaching and highlight student misconceptions. Whereas works of (Durso et al., 2017) and (De-Marcos et al., 2017) both suggest a neutral perception towards a game-based solution yet reveal positive results in its effectiveness in education. More specifically, (Durso et al., 2017) find that students acknowledge the various skills being developed by the game-based software but express critics about the user experience of it, whereas (De-Marcos et al., 2017)’s study indicates positive learning but no positive perceptions towards the game-based solution. While positive perceptions are a promising trend in the research area, we still emphasise the need for a clearer understanding on how these perceptions are formed any what impact previous experiences may have on the perceptions that are noted after exposure to gamification and game-based solutions.

Amongst those 38 studies that uncovered positive perceptions towards gamification or game-based solutions, only 5 game-based projects and 16 gamification projects have also found that positive perceptions are associated with positive effectiveness in education. The rest 16 (gamification n=11, game-based n=5) have either found neutral results in effectiveness or no results at all. For instance, Alawadhi and AbuAyyash (2021) find that although there is a general positive response towards gamification (Kahoot!) there is no significant effect on academic performance. This is echoed by Shernoff et al. (2020) who discover a positive learning experience in a game-based solution amongst students but no significance in enhancing their conceptual understanding. Studies that do not reveal the effectiveness of gamification solutions have found a wide range of students’ positive perceptions towards gamification which are associated with key words such as ‘fun and excitement’ (Mloza-Banda and Scholtz, 2017), ‘entertaining’ (Emblen-Perry, 2018), ‘engaging’ (Pettit et al., 2015, Ruiz, 2021, Kuo and Chuang, 2016), ‘motivating’ (Rajˇsp et al., 2017,Ruiz, 2021, ,Leaning, 2015), ‘positive learning environment’ (Ruiz, 2021), ‘confidence in competence’ (Fernandez-Antolin et al., 2021), and ‘positive experience’ (Costa et al., 2020). Similarly, for game-based solutions, studies with matching criteria also note keywords such as “entertaining” (Fernandez-Antolin et al., 2021) and “confidence in competence” (Martin-Hernandez et al., 2019). Although these studies do not directly point out whether gamification or game-based solutions can be an effective solution in improving students’ academic performance, it is evident that gamification and game-based solutions play an important role in improving students’ learning experience and enriching teachers’ teaching practice, hence facilitating pedagogical enhancement. We suggest that more research considers publishing their early data around the effectiveness of their solution to help strengthen the insight around the topic, where the research is focused on a particular application rather than a concept.

Towards a clearer understanding of student perception and gamification and game-based effectiveness

As stated in the results section, our study discovered 21 studies that note positive perception and effectiveness with gamification (n=16) and game-based (n=5) solutions. The related work in our corpus that developed game-based solutions, investigated the perception towards game-based solutions post exposure but only 8 out of the 14 related studies consider the effectiveness of their solutions in the field. We note a similar distribution of research that do not report on the effectiveness of their work, relating to gamification, with only 14 out of 32 publications presenting results on the effectiveness of their work. Considering the effectiveness of game-based and gamification solutions at trial phase can help gain a better understanding between how and why we would use games approaches for a variety of applications, and as such, future research should consider testing the effectiveness of their solution as a priority in their research methodology.

We also note that effectiveness is currently considered in context of various goals, in both gamification and game-based research. For example, some studies considered knowledge and competencies development as an indicator of effective learning through a games -based solution (Cheung and Ng, 2021, Mattera et al., 2016, Hamzeh et al., 2017, De-Marcos et al., 2017) or a gamification solution (Ahmad et al., 2021, Lorenzo-Alvarez et al., 2020, Campillo-Ferrer et al., 2020, Medina et al., 2019, Mas-Machuca et al., 2019, Nicolaidou, 2018, Tao and Zou, 2021). Other studies focused on student engagement with a gamification solution (Cardenas-Moncada et al., 2020, Ruiz, 2021, Kuo and Chuang, 2016, Thi Van Pham et al., 2021) or a game-based solution (Shernoff et al., 2020) as an indicator of effectiveness. Certain studies considered the post-perception students had on a game-based solution (Emblen-Perry, 2018, Barr, 2018, Fernandez-Antolin et al., 2021, Durso et al., 2017, Berns et al., 2016 ) or gamification (Alt and Raichel, 2020, Hosseini et al., 2019, Mohamad et al., 2020, Jimenez-Rodríguez et al., 2020, Pettit et al., 2015, Alawadhi and Abu-Ayyash, 2021,Costa et al., 2020, Yildirim, 2017, Stansbury and Earnest, 2017, Leaning, 2015, La Cruz et al., 2022) and its appropriateness as a tool for learning as the indicator to effectiveness. The diversity on what defines an effective gamification and game-based solutions is fostered through the variety in the variety of applications and hypotheses included in the corpus. While we do not highlight this diversity as a limitation in the gamification field, it is important we note the diversity of effectiveness in context of specific areas of application and how it can be interpreted in the field.

Based on the included research, we find that skills development and learning can be improved through gamification and game-based interventions and perception towards gamification and game-based interventions is largely positive. Unfortunately, all cited research investigated perception of their gamification or game-based intervention and did not investigate what perception students had towards gamification prior to exposure. We find a substantial gap in existing literature that does consider pre-existing perceptions towards game-based solutions and gamification in higher education and the impact such perceptions may have on the effectiveness of their solutions.

Limitations and Suggestions for Future Research

Our study is affected by the following limitations. First, we explored papers that were written in English or had an English version available. Our study does not map literature that has been written in other languages which can result in appropriate research not being included into this paper. Secondly, the sourced literature derived from the databases and index websites stated in 4.1. The databases and index websites provide wide access to peer-reviewed literature, but it is possible that further relevant research was subsequently excluded should they not have been available. We accompanied our database search with a manual search to reduce the risk of missing relevant research due to the difference in search terms between each database. The difference in search terminology and method could have reduced the appropriate research results made available, which could have been a factor in the difference we note in sample size between games-based applications and gamification literature in our corpus. Lastly, as a growing field, publications in this area remain limited, with a large portion coming from conference papers. However, with the trend of moving to playful and experiential learning in education in a post pandemic era, gamification bears significant influence in shaping the future of education.

Conclusion

Our paper explores the existing research that measures student pre-perception and perception of gamification and game-based solutions in higher education compares to the reported effectiveness in the related studies. In doing so, we investigate what available research considers the effect pre-existing perception may have towards the effectiveness of gamification and game-based solutions in higher-education learning. We reviewed 145 papers as part of a systematic mapping, considering papers published since the last relevant mapping in the field was conducted (Dicheva et al., 2015). After excluding literature that did not meet the expected quality, did not consider gamification or a game-based solution, or did not relate to student perception, we arrived at a corpus of 46 papers. Our corpus did not identify any literature that had considered pre-perceptions of gamification or game-based solutions in higher education learning. We identify this as an area for future research and invite researchers that work around gamification in higher education to consider how pre-perceptions may influence the effectiveness of their work.

From the research that does measure student perception, the majority (n=21) reported positive student perception post exposure and a positive result in the effectiveness of gamification (n=16) and game-based solutions (n=5). Our mapping identifies a positive trend for using gamification and game-based solutions in higher education, meaning more research in the field should be considered for this age demographic. Our research also revealed that questionnaires alone are the most common method (n=24) for measuring student perception in gamification (n=16) and game-based solutions (n=6). Alongside this finding, we note that the majority of the work in gamification and game-based solutions was preliminary or early work. While questionnaires are viable instruments for measuring student perception, we note that future research could benefit from a mixed approach that coincides with more longitudinal work.

In summary, we highlight the need for new research that considers pre-existing perception of gamification and game-based solutions before exposure, helping identify how it may effect on effectiveness. We also note that more complete research with large sample sizes is required to more conclusive findings around gamification and game-based solutions in higher education.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

John Henry

Sylvester Arnab

Author Biographies

John Henry received the B.Sc. degree in Computer Animation and Visualisation from Liverpool John Moores University, Liverpool, United Kingdom, in 2014 and the PhD degree in combining Serious Games with the Internet of Things from Liverpool John Moores University, Liverpool, United Kingdom, in 2018. He is currently a Senior Lecturer at the Department of Computing and Mathematics in Manchester Metropolitan University where he also leads the Games Science Lab, which focuses on human centred computing advancements in game-based approaches. He also leads a cross-faculty theme on Serious Games for Health, fostering cross-institution collaboration on the topic. His research interests lie in gamification and serious games with sensor integration for education and healthcare.

Fujia Li received the BA degree in English and translation from Hunan Agricultural University, China in 2010, the MSc degree in international management from the University of Exeter, UK in 2012, and a PhD in management from the University of Exeter, UK in 2019. He is currently a Lecturer in Entrepreneurship at University of Exeter Business School and the Teaching and Learning Lead of the British Academy of Management (BAM) Entrepreneurship Special Interest Group (ENT SIG). Cliff is experienced in conducting qualitative empirical research with SMEs in an international context. His recent research interests span across several interdisciplinary areas such as responsible innovation and entrepreneurship in developing economies, the dynamics of entrepreneurial ecosystems and resilience in times of crisis, as well as perception and impact of gamification in an entrepreneurship education setting.

Sylvester Arnab received the B.Sc. degree in computer science from the University in Manchester, UK in 1998, M.Sc. degree in distributed multimedia systems from University of Leeds, UK in 2001, and a PhD in engineering from University of Warwick, UK in 2009. He is currently a Professor in Game Science (Applied Games) at Coventry University, where he is leading the Ludic Design research group within the Centre for Post digital Cultures. Sylvester cofounded the multi-award winning GameChangers initiative in 2015 that focuses on empathic play and game design that has been adapted in various countries, including Malaysia, Indonesia, and Vietnam. He also published a book on Game Science in Hybrid Learning Spaces in 2020 based on his 10-year research in serious games and game-based learning.

References

Ahmad

Zeeshan

Marriam

Samreen

Ahmed

(2021). Does one size fit all? investigating the effect of group size and gamification on learners’ behaviors in higher education. Journal of Computing in Higher Education, 33(2):296–327.

Alawadhi

Abu-Ayyash

E. A.

(2021). Students’ perceptions of kahoot!: An exploratory mixed-method study in efl undergraduate classrooms in the uae. Education and Information Technologies, 26(4):3629–3658.

Alhammad

M. M.

Moreno

A. M.

(2018). Gamification in software engineering education: A systematic mapping. Journal of Systems and Software, 141:131–150.

Alt

Raichel

(2020). Enhancing perceived digital literacy skills and creative self-concept through gamified learning environments: Insights from a longitudinal study. International Journal of Educational Research, 101:101561.

Bahji

S. E.

El Alami

Lefdaoui

(2015). Learners’ attitudes towards extended-blended learning experience based on the s2p learning model. International Journal of Advanced Computer Science & Ap- plications, 1(6):70–78.

Bai

Hew

K. F.

Huang

(2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30:100322.

Barr

(2018). Student attitudes to games-based skills development: Learning from video games in higher education. Computers in human behavior, 80:283–294.

Belova

Zowada

(2020). Innovating higher education via game-based learning on misconceptions. Education Sciences, 10(9):221.

Berns

Isla-Montes

J.-L.

Palomo-Duarte

Dodero

J.-M.

(2016). Motivation, students’ needs and learning outcomes: A hybrid game-based app for enhanced language learning. SpringerPlus, 5(1):1– 23.

10.

Bourgonjon

De Grove

De Smet

Van Looy

Soetaert

Valcke

(2013). Acceptance of game-based learning by secondary school teachers. Computers & Education, 67:21–35.

11.

Boyle

E. A.

Hainey

Connolly

T. M.

Gray

Earp

Ott

Lim

Ninaus

Ribeiro

Pereira

(2016). An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education, 94:178–192.

12.

Buckley

Doyle

(2016). Gamification and student motivation. Interactive Learning Environments, 24(6):1162–1175.

13.

Campillo-Ferrer

J.-M.

Miralles-Martınez

S´anchez-Ib´an˜ez

(2020). Gamification in higher education: Impact on student motivation and the acquisition of social and civic key competencies. Sustainability, 12(12):4822.

14.

C´ardenas-Moncada

V´eliz-Campos

V´eliz

(2020). Game-based student response systems: The impact of kahoot in a chilean vocational higher education efl classroom. Computer- Assisted Language Learning Electronic Journal (CALL-EJ), 21(1):64–78.

15.

Chapman

J. R.

Rich

P. J.

(2018). Does educational gamification improve students’ motivation? if so, which game elements work best? Journal of Education for Business, 93(7):315–322.

16.

Cheung

S. Y.

K. Y.

(2021). Application of the educational game to enhance student learning. In Frontiers in Education, volume 6, page 623793. Frontiers Media SA.

17.

Clements

A. J.

Ahmed

Henderson

(2017). Student experience of gamified learning: a qualitative approach. In European Conference on Games Based Learning, pages 88–94. Academic Conferences International Limited.

18.

Costa

F. A.

Viana

Raleiras

(2020). What do higher education students have to say about gamification? In International Conference in Methodologies and intelligent Systems for Technology Enhanced Learning, pages 56–65. Springer.

19.

De-Marcos

Garcia-Cabot

Garcia-Lopez

(2017). Towards the social gamification of e-learning: A practical experiment. International journal of engineering education, 33(1):66–73.

20.

Denden

Tlili

Essalmi

Jemni

Chen

N.- S.

Burgos

(2021). Effects of gender and personality differences on students’ perception of game design elements in educational gamification. International Journal of Human-Computer Studies, 154:102674.

21.

Deterding

Dixon

Khaled

Nacke

(2011). From game design elements to gamefulness: Defining gamification. Proceedings of the 15th International Academic MindTrek Conference on Envisioning Future Media Environments - MindTrek ’11, pages 9–11.

22.

Dicheva

Dichev

Agre

Angelova

(2015). Gamification in education: A systematic mapping study. Journal of educational technology & society, 18(3):75–88.

23.

Durso

Reginato

Cornacchione

(2017). Using a simulation enterprise game to develop important abilities in accounting students of a brazilian higher education institution. In INTED2017 Proceedings, pages 7317–7326. IATED.

24.

Egenfeldt-Nielsen

(2011). Beyond edutainment: Exploring the educational potential of computer games. Lulu.com.

25.

Emblen-Perry

(2018). Promoting education for sustainability through game-based learning: Using the sustainable strategies game to improve students’ knowledge and skills of sustainable business practices. In Handbook of Sustainability Science and Research, pages 849–866. Springer.

26.

Fernandez-Antolin

M.-M.

del R´ıo

J. M.

Gonzalez-Lezcano

R.-A.

(2021). The use of gamification in higher technical education: perception of university students on innovative teaching materials. International Journal of Technology and Design Education, 31(5):1019–1038.

27.

Hammady

Arnab

(2022). Serious gaming for behaviour change: a systematic review. Information, 13(3):142.

28.

Hamzeh

Theokaris

Rouhana

Abbas

(2017). Application of hands-on simulation games to improve classroom experience. European Journal of Engineering Education, 42(5):471–481.

29.

Henry

Tang

Mukhopadhyay

Yap

M. H.

(2021). A randomised control trial for measuring student engagement through the internet of things and serious games. Internet of Things, 13:100332.

30.

Hosseini

Hartt

Mostafapour

(2019). Learning is child’s play: Game-based learning in computer science education. ACM Transactions on Computing Education (TOCE), 19(3):1–18.

31.

Jimenez-Rodrıguez

Garcia

T. B.

Luque

V. A.

(2020). Perception of nursing students about the implementation of greens© methodology in nursing studies. Nurse Education Today, 92:104495.

32.

Khuzzan

S. M. S.

Harun

N. Z.

Hanid

Habib

S. N. H. A.

(2021). Students’ perceptions towards the use of gamification. Central Asia and the Caucasus, 22(5):509–523.

33.

Kuo

M.-S.

Chuang

T.-Y.

(2016). How gamification motivates visits and engagement for online academic dissemination–an empirical study. Computers in Human Behavior, 55:16–27.

34.

La Cruz

Laura

K. M.

Gebera

O. W. T.

Copaja

S. J. N.

(2022). Application of gamification in higher education in the teaching of english as a foreign language. In Perspectives and Trends in Education and Technology, pages 323–341. Springer.

35.

Lameras

Arnab

Dunwell

Stewart

Clarke

Petridis

(2017). Essential features of serious games design in higher education: Linking learning attributes to game mechanics. British journal of educational technology, 48(4):972–994.

36.

Leaning

(2015). A study of the use of games and gamification to enhance student engagement, experience and achievement on a theory-based course of an undergraduate media degree. Journal of Media Practice, 16(2):155–170.

37.

Lorenzo-Alvarez

Rudolphi-Solero

Ruiz- Gomez

M. J.

Sendra-Portero

(2020). Game-based learning in virtual worlds: a multiuser online game for medical undergraduate radiology education within second life. Anatomical sciences education, 13(5):602–617.

38.

Manzano-Le´on

Camacho-Lazarraga

Guerrero-Puerta

M. A.

Guerrero-Puerta

Alias

Aguilar-Parra

J. M.

Trigueros

(2021). Development and validation of a questionnaire on motivation for cooperative playful learning strategies. International Journal of Environmental Research and Public Health, 18(3):960.

39.

Martın-Hern´andez

Azkue-Beteta

Gil- Lacruz

Gil-Lacruz

(2019). Working effectively with others through game-based learning. Proceedings of EDULEARN19, pages 3074– 3079.

40.

Mas-Machuca

Jordan

Tresserres

(2019). Implementation of game-based learning in higher education: an example in hr management. In HEAD’19. 5th International Conference on Higher Education Advances, pages 1043–1050. Editorial Universitat Politecnica de Valencia.

41.

Mattera

Baena

Pinto

(2016). Games hit the books: How physical games can improve higher education students’ class- engagement and learning outcomes. In ICERI2016 Proceedings, pages 6486– 6494. IATED.

42.

Medina

Jarauta

Calduch

Hervas

(2019). Impact on the students learning of a teaching innovation program at the university of barcelona. In 11th International Conference on Education and New Learning Technologies (EDULEARN) Conference Location Palma, SPAIN, pages 7284–7288. IATED-INT ASSOC TECHNOLOGY EDUCATION & DEVELOPMENT Location VALENICA.

43.

Menandro

Arnab

(2020). Game-based mechanical engineering teaching and learning. Smart and Sustainable Manufacturing Systems, 5(2).

44.

Mloza-Banda

Scholtz

(2017). A gamified system for learning enterprise resource planning systems: Investigating the user experience. In 2017 1st International Conference on Next Generation Computing Applications (NextComp), pages 209–214. IEEE.

45.

Mohamad

Arif

F. K. M.

Alias

B. S.

Yunus

M. M.

(2020). Online game-based formative assessment: Distant learners post graduate students’ challenges towards quizizz. International Journal of Scientific and Technology Research, 9(4):994–1000.

46.

Nicolaidou

(2018). Turn your classroom into a gameshow with a game-based student response system. In European conference on games-based learning, pages 487–XXII. Academic Conferences International Limited.

47.

Ofosu-Ampong

Boateng

Anning-Dorson

Kolog

E. A.

(2020). Are we ready for gamification? an exploratory analysis in a developing country. Education and Information Technologies, 25(3):1723–1742.

48.

Petersen

Vakkalanka

Kuzniarz

(2015). Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology, 64:1–18.

49.

Pettit

R. K.

McCoy

Kinney

Schwartz

F. N.

(2015). Student perceptions of gamified audience response system interactions in large group lectures and via lecture capture technology. BMC medical education, 15(1):1–15.

50.

Rajˇsp

Beranic

Hericko

Horng-Jyh

P. W.

(2017). Students’ perception of gamification in higher education courses. In Central European Conference on Information and Intelligent Systems, pages 69–75. Faculty of Organization and Informatics Varazdin.

51.

Roodt

Saunders

(2017). Digital game- based learning for the net generation: perceptions of university students. In Proceedings of EDULEARN17 Conference, pages 3–4.

52.

Ruiz

C. G.

(2021). The effect of integrating kahoot! and peer instruction in the spanish flipped classroom: the student perspective. Journal of Spanish Language Teaching, 8(1):63–78.

53.

Schwind

Buder

(2012). Reducing confirmation bias and evaluation bias: When are preference-inconsistent recommendations effective–and when not? Computers in Human Behavior, 28(6):2280–2290.

54.

Shernoff

D. J.

Ryu

J.-C.

Ruzek

Coller

Prantil

(2020). The transportability of a game-based learning approach to undergraduate mechanical engineering education: effects on student conceptual understanding, engagement, and experience. Sustainability, 12(17):6986.

55.

Shibuya

Hamm

Pargman

T. C.

(2022). Mapping hci research methods for studying social media interaction: A systematic literature review. Computers in Human Behavior, 129:107131.

56.

Stansbury

J. A.

Earnest

D. R.

(2017). Meaningful gamification in an industrial/organizational psychology course. Teaching of Psychology, 44(1):38–45.

57.

Subhash

Cudney

E. A.

(2018). Gamified learning in higher education: A systematic review of the literature. Computers in human behavior, 87:192–206.

58.

Susi

Johannesson

Backlund

(2007). Serious games: An overview.

59.

Tang

Hanneghan

El Rhalibi

(2009). Introduction to games-based learning. In Games-based learning advancements for multi-sensory human computer interfaces: Techniques and effective practices (pp. 1-17). IGI Global.

60.

Tao

Zou

(2021). Students’ perceptions of the use of kahoot! in english as a foreign language classroom learning context. Computer Assisted Language Learning, pages 1–20.

61.

Thi Van Pham

Duy Nguyen

(2021). Students’ perceptions of gamification: A case study in business courses at an institution of higher education. In 2021 3rd International Conference on Modern Educational Technology, pages 120–124.

62.

Thi Van Pham

Thi Thao Ho

Duy Nguyen

(2021). An investigation into students’ motivation and learning effectiveness in gamified learning experiences via kahoot! at a higher education institution in vietnam. In 2021 5th International Conference on Education and Multimedia Technology, pages 5–10.

63.

Tondello

G. F.

Wehbe

R. R.

Diamond

Busch

Marczewski

Nacke

L. E.

(2016). The gamification user types hexad scale. In Proceedings of the 2016 annual symposium on computer-human interaction in play, pages 229–243.

64.

Tuparova

Tuparov

Orozova

(2020). Educational computer games and gamification at the higher education–students’ points of view. In 2020 43rd International Convention on Information, Communication and Electronic Technology (MIPRO), pages 1579–1584. IEEE.

65.

Wang

DeMaria Jr

Goldberg

Katz

(2016). A systematic review of serious games in training health care professionals. Simulation in Healthcare, 11(1):41–51.

66.

Whitton

Langan

(2018). Fun and games in higher education: an analysis of uk student perspectives. Teaching in Higher Education.

67.

Yildirim

(2017). The effects of gamification-based teaching practices on student achievement and students’ attitudes toward lessons. The Internet and Higher Education, 33:86–92.

68.

Zhu

Chen

Zheng

(2020). Modelling the acceptance of fully autonomous vehicles: a media-based perception and adoption model. Transportation research part F: traffic psychology and behaviour, 73:80–91.

On the Pre-Perception of Gamification and Game-Based Learning in Higher Education Students: A Systematic Mapping Study

Abstract

Background

Objective

Results

Conclusion

Keywords

Background and Motivation

Methodology of systematic mapping

Source Selection

Search Criteria

Eligibility Assessment for the Final Analysis

Data Coding and Analysis

Process for gathering student pre-perception and perception on gamification and game-based solutions

Relationship Between Perception and Effectiveness

Methods for Measuring Gamification and Games-Based Perception

Results

Corpus Description

The Relationship Between Perception and Effective Gamification Intervention

The Relationship Between Perception and Effective Games-Based Intervention

Methods for Measuring Gamification Perception

Methods for Measuring Games-Based Perception

Crucial Academic Areas for Future Research

Discussion

Perceptions of Gamification and Games-Based Solutions

Towards a clearer understanding of student perception and gamification and game-based effectiveness

Limitations and Suggestions for Future Research

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iDs

Author Biographies

References