Sage Journals: Discover world-class research

Abstract

Entertainment digital games (EDGs) can be used to promote real-world-relevant learning, even if they have not been specifically designed for educational purposes. This is especially true for EDGs that engage players in contexts that mimic real-world problem scenarios and support the development of knowledge and skills that can be transferred to out-of-game contexts. In these cases, gameplay learning can be a very effective means of promoting those cognitive capabilities that are central to addressing real-world challenges requiring complex problem-solving (CPS), but at the same time are particularly difficult to foster through formal educational environments and approaches. However, what features of EDGs can simultaneously promote player enjoyment and CPS processes, which specific CPS skills can be fostered, and by which mechanisms, are questions that remain largely unanswered in the current literature. This paper presents a protocol for a systematic review that aims to address this gap by examining relevant analysis and design frameworks for entertainment games. Selected frameworks will be reviewed by combining a game-centric and a player- centric perspective to identify structural elements of gameplay environments and tasks that may affect psychological processes relevant to the promotion of CPS capabilities. To this end, each framework will be subjected to a formal content analysis in which data will be extracted, coded, and analyzed based on the Work System Theory (WST) and the Cognitive Work Analysis (CWS) frameworks. The main outcome of the proposed systematic review will be a knowledge base that can help researchers, developers and practitioners to select the most appropriate methodological frameworks for the analysis and design of entertainment games capable of promoting CPS skills through gameplay learning. In addition, this protocol can also inform and guide further reviews of methods for analyzing and designing educational games.

Keywords

entertainment game analysis entertainment game design methodological frameworks gameplay features gameplay learning cognitive complex problem-solving capabilities

Introduction

Digital games can be powerful learning environments, capable of promoting learning processes and outcomes that are difficult to achieve in ordinary educational contexts (Boyle et al., 2016; Fabricatore & Lopez, 2019; Gee, 2003; Prensky, 2003; Whitton, 2014; Wouters et al., 2013). For this reason, research on game-based learning has investigated for over three decades how to develop and use games specifically designed to promote desirable learning (Abt, 1987; Avila-Pesantez et al., 2017; De Freitas, 2018; Fabricatore et al., 2020; Rieber, 1996; Squire, 2005). In parallel with this attention to educational games, there is also growing interest in entertainment games (i.e., commercial digital entertainment products designed for recreational purposes) and how they can be used to promote meaningful learning (Clark et al., 2016; Jayakanthan, 2002; Martinez et al., 2022; McClarty et al., 2012; Paraskeva et al., 2010).

Game research has amply recognized that good entertain- ment games have the ability to engage players in highly motivating experiences that elicit significant gameplay learn- ing, i.e., the integrative development of knowledge, attitudes, and cognitive, social, and practical skills required to pur- sue game goals (Boyle et al., 2016; Connolly et al., 2012; Fabricatore, 2018; Gee, 2003; Vlachopoulos & Makri, 2017). Furthermore, depending on game requirements and contextualization (e.g., objectives, settings, and narratives that drive and underpin gameplay activities), what players learn through gameplay may be relevant outside of the game and transferable to real-world scenarios and activities (Boyle et al., 2016; Clark et al., 2016; Fabricatore & Lopez, 2019; Granic et al., 2014; Hobbs et al., 2006; McClarty et al., 2012). Finally, players learn through entertainment games because they choose to do so. Gameplay learning is fully intrinsically motivated, i.e., it is motivated solely by the goals, contextu- alizations, and mechanics that games provide (Egenfeldt-Nielsen, 2006; Fabricatore, 2000; Gee, 2003; Mitchell & Savill-Smith, 2004; Prensky, 2002) and, thus, it represents an optimal learning experience (Habgood et al., 2005; Guay et al., 2008; Malone & Lepper, 1987; Shernoff & Csikszentmihalyi, 2009). The combination of these factors (i.e., transferability to out-of-game contexts, intrinsic moti- vation, and integrative development of knowledge, attitudes, and skills) make gameplay learning a particularly relevant form of learning to investigate, emphasizing the importance of understanding which features can make entertainment games enjoyable and at the same time suitable for promoting learning processes and outcomes relevant to contexts outside the game (Boyle et al., 2016; Clark et al., 2016; Connolly et al., 2012; Granic et al., 2014; Martinez et al., 2022; Prensky, 2002).

The heterogeneity of meaningful learning processes involved in entertainment games can lead to different learning outcomes of varying relevance (Biggs & Collis, 2014; Fabricatore & Lopez, 2019; Ivanitskaya et al., 2002; Tynja¨la¨, 1997). One such outcome that is critical for addressing real-world challenges is the development of cognitive capabilities appropriate to engage in complex problem-solving (CPS) (e.g., situational assessment and decision-making under uncertain conditions) (Do¨rner & Funke, 2017; Fabricatore et al., 2020; Fischer et al., 2011; Funke et al., 2018). Complex problem-solving is a self-regulated process aimed at defining and executing courses of actions suitable to achieve and maintain desirable environmental states in dynamically changing environments (Do¨rner & Funke, 2017). Complex problem-solving therefore requires the ability to continuously monitor problem scenarios, and iteratively define and adapt action plans in order to timely respond, based on incomplete and uncertain information, to relevant environmental changes (Do¨rner & Funke, 2017; Do¨rner & Wearing, 1995; Fischer et al., 2011; Funke et al., 2018).

Despite the increasing need to acquire and apply CPS skills, formal learning environments find difficulty in promoting them effectively (Do¨rner & Funke, 2017; Fabricatore & Lopez, 2019; Fabricatore et al., 2020; Jonassen & Rohrer-Murphy, 1999). Entertainment games, however, can address these issues because they have the potential to engage players in enjoyable tasks set in complex scenarios that elicit CPS skills and foster their development (Eseryel et al., 2014; Fabricatore et al., 2020; Gyaurov et al., 2021; Whitton, 2014). It is therefore important to understand these opportunities and analyze how gameplay features designed to promote player engagement and enjoyment can also support CPS capabilities. Game research provides a variety of methodological frameworks for developing engaging entertainment games and facilitates the analysis of gameplay features that can promote or hinder player enjoyment (e.g., Avedon, 1981; De Byl, 2015; Hunicke et al., 2004; Kiili et al., 2014; Raybourn, 1997; Sweetser & Wyeth, 2005). To our knowledge, however, only a few of these frameworks have considered the impact of such gameplay features on learning (e.g., Deterding, 2015; Fabricatore, 2018; Fabricatore et al., 2019), and none of them have specifically addressed CPS.

In light of these observations, this paper presents a protocol for a systematic review of existing methodological frameworks for the analysis and design of entertainment games. This review will contribute to the state of the art by investigating: (a) which gameplay features can simultaneously promote player enjoyment and CPS capabilities, and (b) which CPS capabilities can be promoted by these gameplay features and by which mechanisms.

The systematic review will adopt an integrative approach to analyze the gameplay features proposed by the selected methodological frameworks, since these tend to be either game-centric or player-centric (Fabricatore, 2018; Fabricatore et al., 2019). Game-centric frameworks emphasize structural elements of a gameplay environment that affect player engagement by defining gameplay objectives, tasks, challenges, and means provided to the player, and the mechanics by which these elements interact with each other (Fabricatore, 2018; Fabricatore et al., 2019; Larsen & Schoenau-Fog, 2016; Sicart, 2008). Player- centric frameworks focus on the psychological processes that drive player engagement, and the aspects of player activities that make them comprehensible, meaningful, motivating, and therefore engaging (Bernhaupt, 2010; Fabricatore, 2018; Fabricatore et al., 2019; Ryan et al., 2006). In any problem scenario, the structural elements that define a problem environment and the psychological processes involved in a problem-solving activity interact, and are equally important to define demands, dynamics, and outcomes of a problem- solving process (D’zurilla & Goldfried, 1971; Newell &Simon, 1972). Because of this, both game-centric and player-centric features should be considered when investigating what may elicit or foster CPS in a gameplay activity.

Therefore, the integrative approach of the proposed review will analyze: (a) structural elements of gameplay processes and environments involved in gameplay tasks (e.g., goals, workflows, artifacts, and actors), (b) their interplay (e.g., interaction mechanics), and (c) the effects these may have on players’ cognitive processes and capabilities that are relevant to CPS (e.g., motivation to learn, elicitation and support of decision-making processes, stimulation of abductive reasoning, etc.). Specifically, the review will conceptualize gameplay as a goal-directed activity (Fabricatore, 2018), and consequently identify, describe, and analyze gameplay features by relying on two frameworks, namely the Work System Theory (WST) (Alter, 2013) and the Cognitive Work Analysis (CWS) (Roth and Bisantz, 2013). The rationale for this choice is the following.

Work System Theory models goal-directed activities as sociotechnical task processes that: (a) are performed by actors (human and non-human) who use tools, knowledge, and collaboration to achieve their intended goals, and (b) are defined by key environmental conditions under which the tasks are performed (Alter, 2013). Work System Theory provides formal approaches to analyzing the key elements of the domain in which the work is performed and their potential function in facilitating or hampering practical task performance (Alter, 2013). Work System Theory will be therefore used to analyze each methodological framework selected for full review and to identify and classify the structural elements of a gameplay environment that the framework considers and that may affect gameplay tasks.

The CWS framework provides formal methods for identifying cognitive processes and capabilities involved in work activities and analyzing how they might be stimulated, facilitated, or hampered by a variety of interacting elements in a work environment (Roth and Bisantz, 2013). The identification of relevant cognitive processes and capabilities involved in gameplay, and how the gameplay elements affect them, will be carried out by analyzing each of the selected methodological frameworks using CWS methods. The analysis will be based on selected taxonomies relevant to classifying and describing the cognitive processes and capabilities involved in CPS, such as the Cattel-Horn-Carroll theory of cognitive abilities (Schneider and McGrew, 2018), the constructivist framework for problem-based learning (Savery and Duffy, 1995), and the computer-simulated theory of CPS (Do¨rner and Wearing, 1995).

The proposed review will be guided by the following research questions:

• Which gameplay features can simultaneously promote player enjoyment and CPS capabilities?

• Which specific CPS capabilities can be promoted by gameplay features?

• By which mechanisms can these CPS capabilities be promoted by gameplay features?

In addition, the following research question will help examine the methodological soundness of the analyzed frameworks:

• What are the theoretical or conceptual underpinnings of different frameworks?

The relevance of this systematic review protocol is emphasized by (a) the recognized contribution of protocols to reducing bias in research (Kitchenham and Charters, 2007; Moher et al., 2015), (b) the specific need for protocols to guide games research (Caldero´n & Ruiz, 2015; Saucedo- Araujo et al., 2020), and (c) the potential of this protocol to inform future reviews of methods for analyzing and designing educational games. In addition, we highlight the novelty of the proposed systematic review. To our knowledge, this is the first work of its kind to approach the field of game studies with the intention of examining how available methodological frameworks cover gameplay features that may promote both player enjoyment and cognitive CPS capabilities through gameplay learning (Neil, 2012; Neves & Zagalo, 2021). The proposed systematic review will therefore complement and expand the existing knowledge body provided by previous reviews of game analysis and design frameworks, which have been focused on either general gameplay features (Almeida & Silva, 2013; Caroux et al., 2015), isolated factors of the player experience (Avila-Pesantez et al., 2017; Boyle et al., 2016), or game-based learning in specific domains such as history courses (Ghulamani et al., 2020) and science education (Tsai & Tsai, 2020). It is therefore expected that the proposed systematic review will be useful to facilitate the comparison and selection of appropriate methodological frameworks for analyzing and designing entertainment games that may foster the development of cognitive CPS capabilities through gameplay learning, which is of great importance in research and practice (Caroux et al., 2015; Fabricatore et al., 2019; Natucci & Borges, 2021; Neil, 2012; Neves & Zagalo, 2021; Tahir & Wang, 2018).

Methods

The protocol for this systematic review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA-P) statement (Moher et al., 2015). The PRISMA-P checklist for the protocol is available in the Supplemental material section (S1 File).

Eligibility Criteria

Eligibility criteria are based on the research questions proposed for this systematic review in the Introduction section. Studies will be included in the review if they:

1. Present a methodological framework for the analysis and design of entertainment games;

2. Have been written in English;

3. Have been published in peer-reviewed journals, book chapters, or conference papers (thus, excluding editorials, abstracts, posters and panel discussions);

4. Have been published between January 1, 2000 and December 31, 2021;

5. Have demonstrable impact, e.g., are papers that either (a) advance empirical studies (e.g., a methodology or instrument applied – in its original form or with modifications – to conduct a case study), or (b) inform the formulation of other methodological papers (e.g., an extension of a theory or an adaptation of an instrument).

Information Sources

Eligible studies will be identified through a comprehensive search of the following four databases: ACM Digital Library, IEEE Xplore, Scopus, and Web of Science. In addition, the reference lists of the identified articles will be hand-searched for other relevant studies (Horsley et al., 2011; Moher et al., 2015).

Search Strategy

The strategy for identifying studies eligible for inclusion in the review will follow the iterative approach proposed in (Lockwood et al., 2017). This process will start from an already completed initial search, which consisted of (a) formulating a pool of key search terms, (b) generating generic query strings based on the identified key search terms, and (c) translating the generic query strings into the syntax required for each database. The initial set of key search terms in step (a) was obtained by analyzing titles, abstracts and keywords of articles in Scopus that describe methodological frameworks for analyzing and designing entertainment games (e.g., ; Carvalho et al., 2015; De Byl, 2015; Hunicke et al., 2004; Sweetser & Wyeth, 2005). These terms were then conceptually organized based on key terms from the research questions (e.g., gameplay, features, framework, analysis, design, etc.) and expanded to include synonyms, variants, and related terms (e.g., videogames and playability, models and approaches, elements and mechanics, development and evaluation, etc.). Subsequently, generic query strings were formulated based on the expanded search terms (step (b)), considering the inclusion criteria as additional filters. The initial list of identified search terms (S2 File) and the preliminary search strategy for all four databases (S3 File) are included in the Supplemental material section. As mentioned earlier, these initial search terms and query strings will be revised iteratively as new data emerge during the development of the systematic review and analysis of selected articles.

Study Selection

The selection of items for review will be done by two reviewers, following the steps suggested in the PRISMA checklist (Page et al., 2021). First, duplicates will be removed and the results will be consolidated in a Mendeley database. Then, the remaining studies will be independently screened by two reviewers based on their title, abstract, and keywords, and marked for inclusion or exclusion based on the eligibility criteria 1–4. A coefficient of inter- reviewer agreement (e.g., Cohen’s Kappa (Cohen, 1960)) will be calculated to confirm that a sufficient level of inter-reviewer agreement has been achieved (Park & Kim, 2015). Remaining disagreements will be resolved through discussion. If consensus is not reached, the disputed articles will be examined by a third reviewer to complete the evaluation (Barbour, 2001; Berends & Johnston, 2005). Subsequently, the list of references of the selected studies will be analyzed to identify additional relevant articles according to the aforementioned screening process (Horsley et al., 2011; Moher et al., 2015).

Finally, each candidate paper will be read completely by both reviewers, who will also collaboratively assess its impact, applying eligibility criterion 5 (demonstrable impact), by determining whether it has been cited by further research, and what type of research referenced it. The details of the study selection process will be documented using a PRISMA flowchart (Page et al., 2021).

Data Extraction and Synthesis

The studies selected for full review will be examined through an inductive qualitative content analysis approach (Hsieh & Shannon, 2005), using a template analysis technique (King, 1998, 2012). Template analysis is a form of thematic analysis of textual data that focuses on the iterative and flexible development of a coding template that can contain both descriptive and interpretive thematic categories (King, 1998, 2012).

Following the template analysis method and based on the WST, CWS, and CPS-relevant taxonomies of cognitive processes and capabilities, the reviewers will formulate an initial analysis schema to code text in order to identify, in each of the reviewed studies: (a) specific gameplay features that are highlighted as relevant to promoting game enjoyment; (b) how these gameplay features should be configured to promote player engagement; (c) what types of CPS-relevant cognitive capabilities may be elicited or supported by the gameplay features; and (d) the mechanisms by which these capabilities may be elicited or supported. Consistent with the template analysis approach (King, 1998), the analysis schema will be iteratively updated by adding and revising categories based on data emerging from the review. On the one hand, gameplay features that promote player enjoyment and their recommended configurations will be coded as descriptive categories in the analysis schema.

On the other hand, the types of CPS capabilities that may be promoted by the identified gameplay features and their associated elicitation or support mechanisms will be derived through a theory-based inductive analysis (Thomas, 2006) based on the relevant literature on Human Factors Engineering, CPS, and human-computer interaction and coded as interpretive categories.

To improve data coding reliability (Cornish et al., 2013; Patton, 2014; Perreault Jr. & Leigh, 1989; Richards & Hemphill, 2018; Shenton, 2004), two reviewers will collaborate using the analysis schema to extract, code, and derive data from each eligible study. Initial codes and analysis schema have already been developed and are available in the Supplemental material section (S4 File).

Discussion

Entertainment games have the potential to engage players in intrinsically motivating gameplay learning processes (Boyle et al., 2016; Connolly et al., 2012; Fabricatore, 2018; Gee, 2003; Vlachopoulos & Makri, 2017), which can promote the development of cognitive CPS capabilities that are particularly relevant for addressing real-world challenges (Eseryel et al., 2014; Fabricatore et al., 2020; Gyaurov et al., 2021; Whitton, 2014). A necessary step to fully exploit this potential is to review existing methodological frameworks for the analysis and design of entertainment games, as even features intended to promote player enjoyment can be very helpful in fostering gameplay learning and, in particular, CPS capabilities. Therefore, the systematic review based on this protocol aims to examine methodological frameworks for the analysis and design of entertainment games to explore which gameplay features can simultaneously promote player enjoyment and gameplay learning, and what mechanisms allow such features to specifically promote CPS.

As previous research has shown, review protocols can inform and guide the development of sound research methods both generally (Kitchenham & Charters, 2007; Moher et al., 2015) and specifically in the field of game studies (Caldero´n & Ruiz, 2015; Saucedo-Araujo et al., 2020). In addition, systematic reviews (e.g., (Bisantz & Roth, 2007; Sun et al., 2019; Unertl et al., 2010; Wei & Salvendy, 2004) represent an appropriate approach to analyze methodologies for analyzing and designing human activities and human-system interactions and, consequently, identifying which aspects of a system can promote desirable effects on the actors involved and how. To our knowledge, in the domain of games and game-based learning studies, this use of systematic reviews has never been leveraged to investigate potential impacts of entertainment game features on gameplay learning, and in particular, on CPS skills. Hence, a strength of the proposed systematic review is that it will be the first of its kind to contribute knowledge relevant to analyze and design gameplay features that promote both gameplay learning and player enjoyment (Neil, 2012; Neves & Zagalo, 2021). In addition, the study selection criteria and the methodological approaches planned to analyze existing game analysis and design methodological frameworks will promote the relevance and reliability of the results of the proposed review. By including only studies that have some demonstrable impact, the review will guarantee that the identified gameplay features are effectively applicable in practical game analysis and design contexts, or to inform novel theoretical work. By combining WST and CWA for the analysis of the selected frameworks, the review will ensure that gameplay features are analyzed based on formal methods, accounting for both structural elements of a game system, and their effects on key psychological processes that may contribute to the development of CPS skills.

The scope of the proposed review will be deliberately limited to game analysis and design frameworks specifically focused on entertainment products. This is because of the educational potentialities that these entertainment games offer, and the lack of knowledge relevant to harness such potentialities. Despite this limitation, the proposed review will serve as an important basis for further research, by applying the same review methodology to examine analysis and design frameworks focused on educational games, and comparing and contrasting them with frameworks conceived for entertainment games. Another important limitation of the proposed review concerns the interpretive nature of qualitative content analyses, which inevitably introduces an element of subjectivity in the review processes. However, in order to mitigate reliability risks, a rigorous data extraction and synthesis process has been planned, as described in the Methods section of this protocol. Further limitations will be addressed in the full systematic review.

In conclusion, the authors expect that the results of the proposed review will facilitate the identification and understanding of characteristics of entertainment games suitable to both engage players and promote CPS. Thus, the review will support game analysts and designers interested in creating new entertainment games or leveraging existing ones to foster CPS gameplay learning. Furthermore, the review will provide a knowledge base helpful to compare and contrast different methodological frameworks for entertainment game analysis and design, and consequently select and combine the most appropriate ones for specific research and practice needs.

Supplemental Material

Supplemental Material - Features of Entertainment Digital Games for Learning and Developing Complex Problem-Solving Skills: A Protocol for a Systemic Review

Supplemental Material for Features of Entertainment Digital Games for Learning and Developing Complex Problem-Solving Skills: A Protocol for a Systemic Review by Dimitar Gyaurov, Carlo Fabricatore, and Andrea Bottino in International Journal of Qualitative Methods

Supplemental Material

Supplemental Material - Features of Entertainment Digital Games for Learning and Developing Complex Problem-Solving Skills: A Protocol for a Systemic Review

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Dimitar Gyaurov

Supplemental Material

Supplemental material for this article is available online.

References

Abt

C. C.

(1987). Serious games. University press of America.

Almeida

M. S. O.

Silva

F. S. C. d.

(2013). A systematic review of game design methods and tools. In International conference on entertainment computing (pp. 17–29). Springer. https://doi.org/10.1007/978-3-642-41106-9_3

Alter

(2013). Work system theory: overview of core concepts, extensions, and challenges for the future. Journal of the Association for Information Systems, 14(2), 72–121. https://doi.org/10.17705/1jais.00323

Avedon

(1981). The structural elements of games. The psychology of social situations (pp. 11–17). Selected readings. https://doi.org/10.1016/b978-0-08-023719-0.50009-7

Avila-Pesantez

Rivera

L. A.

Alban

M. S.

(2017). Approaches for serious game design: A systematic literature review. The ASEE Computers in Education (CoED) Journal, 8(3).

Barbour

R. S.

(2001). Checklists for improving rigour in qualitative research: A case of the tail wagging the dog? British Medical Journal, 322(7294), 1115–1117. https://doi.org/10.1136/bmj.322.7294.1115

Berends

Johnston

(2005). Using multiple coders to enhance qualitative analysis: The case of interviews with consumers of drug treatment. Addiction Research and Theory, 13(4), 373–381. https://doi.org/10.1080/16066350500102237

Bernhaupt

(2010). User experience evaluation in entertainment. In Evaluating user experience in games (pp. 3–7). Springer. https://doi.org/10.1007/978-1-84882-963-3_1

Biggs

J. B.

Collis

K. F.

(2014). Evaluating the quality of learning: The SOLO taxonomy (structure of the observed learning outcome). Academic Press.

10.

Bisantz

Roth

(2007). Analysis of cognitive work. Reviews of Human Factors and Ergonomics, 3(1), 1–43. https://doi.org/10.1518/155723408x299825

11.

Boyle

E. A.

Hainey

Connolly

T. M.

Gray

Earp

Ott

Lim

Ninaus

Ribeiro

Pereira

(2016). An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games. Computers & Education, 94, 178–192. https://doi.org/10.1016/j.compedu.2015.11.003

12.

Caldero´n

Ruiz

(2015). A systematic literature review on serious games evaluation: An application to software project management. Computers & Education, 87, 396–422. https://doi.org/10.1016/j.compedu.2015.07.011

13.

Caroux

Isbister

Le Bigot

Vibert

(2015). Player–video game interaction: A systematic review of current concepts. Computers in Human Behavior, 48, 366–381. https://doi.org/10.1016/j.chb.2015.01.066

14.

Carvalho

M. B.

Bellotti

Berta

De Gloria

Sedano

C. I.

Hauge

J. B.

Rauterberg

(2015). An activity theory-based model for serious games analysis and conceptual design. Computers & Education, 87, 166–181. https://doi.org/10.1016/j.compedu.2015.03.023

15.

Clark

D. B.

Tanner-Smith

E. E.

Killingsworth

S. S.

(2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122. https://doi.org/10.3102/0034654315582065

16.

Cohen

(1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46. https://doi.org/10.1177/001316446002000104

17.

Connolly

T. M.

Boyle

E. A.

MacArthur

Hainey

Boyle

J. M.

(2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661–686. https://doi.org/10.1016/j.compedu.2012.03.004

18.

Cornish

Gillespie

Zittoun

(2013). Collaborative analysis of qualitative data. The SAGE Handbook of Qualitative Data Analysis, 79, 93. https://doi.org/10.4135/9781446282243.n6

19.

De Byl

(2015). A conceptual affective design framework for the use of emotions in computer game design. Cyberpsychology: Journal of Psychosocial Research on Cyberspace, 9(3), Article 4. https://doi.org/10.5817/cp2015-3-4

20.

De Freitas

(2018). Are games effective learning tools? a review of educational games. Journal of Educational Technology & Society, 21(2), 74–84.

21.

Deterding

(2015). The lens of intrinsic skill atoms: A method for gameful design. Human–Computer Interaction, 30(3–4), 294–335. https://doi.org/10.1080/07370024.2014.993471

22.

Do¨rner

Funke

(2017). Complex problem solving: What it is and what it is not. Frontiers in Psychology, 8, 1153. https://doi.org/10.3389/fpsyg.2017.01153

23.

Do¨rner

Wearing

A. J.

(1995). Complex problem solving: Toward a (computersimulated) theory. In Complex problem solving (pp. 65–99). The European perspective.

24.

D’zurilla

T. J.

Goldfried

M. R.

(1971). Problem solving and behavior modification. Journal of Abnormal Psychology, 78(1), 107. https://doi.org/10.1037/h0031360

25.

Egenfeldt-Nielsen

(2006). Overview of research on the educational use of video games. Nordic Journal of Digital Literacy, 1(3), 184–214. https://doi.org/10.18261/issn1891-943x-2006-03-03

26.

Eseryel

Law

Ifenthaler

Miller

(2014). An investigation of the interrelationships between motivation, engagement, and complex problem solving in game-based learning. Journal of Educational Technology & Society, 17(1), 42–53.

27.

Fabricatore

(2000). Learning and Videogames: an Unex- ploited Synergy. In Visser

(Chair) (Ed.), In search of the meaning of learning. Workshop conducted at the annual convention of the association for educational communica- tions and technology (AECT).

28.

Fabricatore

(2018). Underneath and beyond mechanics: An activity-theoretical perspective on meaning-making in gameplay. Columbia University Press.

29.

Fabricatore

Gyaurov

Lopez

(2019). An exploratory study of the relationship between meaning-making and quality in games. Multimedia Tools and Applications, 78(10), 13539–13564. https://doi.org/10.1007/s11042-019-7232-1

30.

Fabricatore

Gyaurov

Lopez

(2020). Rethinking serious games design in the age of covid-19: Setting the focus on wicked problems. In Joint international conference on serious games (pp. 243–259). Springer. https://doi.org/10.1007/978-3-030-61814-8_19

31.

Fabricatore

Lopez

M. X.

(2019). Game-based learning as a meaning making-driven activity process: a human factors perspective. In ECGBL 2019 13th European conference on game-based learning (p. 218). Academic Conferences and publishing limited.

32.

Fischer

Greiff

Funke

(2011). The process of solving complex problems. Journal of Problem Solving, 4(1), 19–42. https://doi.org/10.7771/1932-6246.1118

33.

Funke

Fischer

Holt

D. V.

(2018). Competencies for complexity: problem solving in the twenty-first century. In Assessment and teaching of 21st century skills (pp. 41–53). Springer. https://doi.org/10.1007/978-3-319-65368-6_3

34.

Gee

J. P.

(2003). What video games have to teach us about learning and literacy. Computers in Entertainment (CIE), 1(1), 20–20. https://doi.org/10.1145/950566.950595

35.

Ghulamani

Shah

Khowaja

(2020). Review of components from frameworks/models for dgbl for history based courses. University of Sindh Journal of Information and Communication Technology, 4(1), 9–16.

36.

Granic

Lobel

Engels

R. C.

(2014). The benefits of playing video games. American Psychologist, 69(1), 66, 78. https://doi.org/10.1037/a0034857

37.

Guay

Ratelle

C. F.

Chanal

(2008). Optimal learning in optimal contexts: The role of self-determination in education. volume 49.

38.

Gyaurov

Fabricatore

Bottino

(2021). Development of an instrument to analyse gameplay features promoting complex problem-solving conditions. In European conference on games based learning (p. 296). Academic Conferences International Limited.

39.

Habgood

M. P.

Ainsworth

S. E.

Benford

(2005). Endogenous fantasy and learning in digital games. Simulation and Gaming, 36(4), 483. https://doi.org/10.1177/1046878105282276

40.

Hobbs

Brown

Gordon

(2006). Using a virtual world for transferable skills in gaming education. Innovation in Teaching and Learning in Information and Computer Sciences, 5(3), 1–13. https://doi.org/10.11120/ital.2006.05030006

41.

Horsley

Dingwall

Sampson

(2011). Checking reference lists to find additional studies for systematic reviews. Cochrane Database of Systematic Reviews, 2011(8), MR000026. https://doi.org/10.1002/14651858.MR000026.pub2

42.

Hsieh

H. F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687

43.

Hunicke

LeBlanc

Zubek

(2004). Mda: A formal approach to game design and game research. In Proceedings of the AAAI workshop on challenges in game AI, volume 4 (p. 1722).

44.

Ivanitskaya

Clark

Montgomery

Primeau

(2002). Interdisciplinary learning: Process and outcomes. Innovative Higher Education, 27(2), 95–111. https://doi.org/10.1023/a:1021105309984

45.

Jayakanthan

(2002). Application of computer games in the field of education. The electronic library.

46.

Jonassen

D. H.

Rohrer-Murphy

(1999). Activity theory as a framework for designing constructivist learning environments. Educational Technology Research and Development, 47(1), 61–79. https://doi.org/10.1007/bf02299477

47.

Kiili

Lainema

de Freitas

Arnab

(2014). Flow framework for analyzing the quality of educational games. Entertainment Computing, 5(4), 367–377. https://doi.org/10.1016/j.entcom.2014.08.002

48.

King

(1998). Template analysis.

49.

King

(2012). Doing template analysis. Qualitative Organizational Research: Core Methods and Current Challenges, 426, 77–101. https://doi.org/10.4135/9781526435620.n24

50.

Kitchenham

Charters

(2007). Guidelines for performing systematic literature reviews in software engineering.

51.

Larsen

B. A.

Schoenau-Fog

(2016). The narrative quality of game mechanics. In International conference on interactive digital storytelling (pp. 61–72). Springer. https://doi.org/10.1007/978-3-319-48279-8_6

52.

Lockwood

Porrit

Munn

Rittenmeyer

Salmond

Bjerrum

Loveday

Carrier

Stannard

(2017). Systematic reviews of qualitative evidence. In Joanna Briggs Institute reviewer’s manual. [Internet]. The Joanna Briggs Institute.

53.

Malone

T. W.

Lepper

M. R.

(1987). Making learning fun: A taxonomy of intrinsic motivations for learning.

54.

Martinez

Gimenes

Lambert

(2022). Entertainment video games for academic learning: A systematic review (pp. 1–27).https://doi.org/10.1177/07356331211053848

55.

McClarty

K. L.

Orr

Frey

P. M.

Dolan

R. P.

Vassileva

McVay

(2012). A literature review of gaming in education. Gaming in Education, 1–35.

56.

Mitchell

Savill-Smith

(2004). The use of computer and video games for learning. A review of the literature.

57.

Moher

Shamseer

Clarke

Ghersi

Liberati

Petticrew

Shekelle

Stewart

L. A.

(2015). Preferred reporting items for systematic review and meta-analysis protocols (prisma-p) 2015 statement. Systematic Reviews, 4(1), 1–9. https://doi.org/10.1186/2046-4053-4-1

58.

Natucci

G. C.

Borges

M. A.

(2021). The experience, dynamics and artifacts framework: Towards a holistic model for designing serious and entertainment games. In 2021 IEEE conference on games (CoG) (pp. 1–8). IEEE. https://doi.org/10.1109/cog52621.2021.9619144

59.

Neil

(2012). Game design tools: Time to evaluate. Proceedings of 2012 DiGRA Nordic.

60.

Neves

P. P.

Zagalo

(2021). Game design as an autonomous research subject. Information, 12(9), 367. https://doi.org/10.3390/info12090367

61.

Newell

Simon

H. A.

(1972). Human problem solving, volume 104. Prentice-Hall.

62.

Page

M. J.

McKenzie

J. E.

Bossuyt

P. M.

Boutron

Hoffmann

T. C.

Mulrow

C. D.

Shamseer

Tetzlaff

J. M.

Akl

E. A.

Brennan

S. E.

Chou

Glanville

Grimshaw

J. M.

Hróbjartsson

Lalu

M. M.

Loder

E. W.

Mayo-Wilson

McDonald

Moher

(2021). The prisma 2020 statement: an updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 105906. https://doi.org/10.1016/j.ijsu.2021.105906

63.

Paraskeva

Mysirlaki

Papagianni

(2010). Multiplayer online games as educational tools: Facing new challenges in learning. Computers & Education, 54(2), 498–505. https://doi.org/10.1016/j.compedu.2009.09.001

64.

Park

C. U.

Kim

H. J.

(2015). Measurement of inter-rater reliability in systematic review. Hanyang Medical Reviews, 35(1), 44–49. https://doi.org/10.7599/hmr.2015.35.1.44

65.

Patton

M. Q.

(2014). Qualitative research & evaluation methods: Integrating theory and practice. Sage publications.

66.

Perreault

W. D.

Jr. Leigh

L. E.

(1989). Reliability of nominal data based on qualitative judgments. Journal of Marketing Research, 26(2), 135–148. https://doi.org/10.1177/002224378902600201

67.

Prensky

(2002). The motivation of gameplay: The real twenty- first century learning revolution. On the Horizon, 10(1), 5–11. https://doi.org/10.1108/10748120210431349

68.

Prensky

(2003). Digital game-based learning. Computers in Entertainment (CIE), 1(1), 21–21. https://doi.org/10.1145/950566.950596

69.

Raybourn

E. M.

(1997). Computer game design: New directions for intercultural simulation game designers. In Developments in business simulation and experiential learning: Proceedings of the annual ABSEL conference, volume 24.

70.

Richards

K. A. R.

Hemphill

M. A.

(2018). A practical guide to collaborative qualitative data analysis. Journal of Teaching in Physical Education, 37(2), 225–231. https://doi.org/10.1123/jtpe.2017-0084

71.

Rieber

L. P.

(1996). Seriously considering play: Designing interactive learning environments based on the blending of microworlds, simulations, and games. Educational Technology Research and Development, 44(2), 43–58. https://doi.org/10.1007/bf02300540

72.

Roth

Bisantz

(2013). Cognitive work analysis. In Lee

J. D.

Kirlik

(Eds.), The Oxford handbook of cognitive engineering (pp. 240–260). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199757183.013.0015

73.

Ryan

R. M.

Rigby

C. S.

Przybylski

(2006). The motivational pull of video games: A self-determination theory approach. Motivation and Emotion, 30(4), 344–360. https://doi.org/10.1007/s11031-006-9051-8

74.

Saucedo-Araujo

R. G.

Chillón

Pérez-López

I. J.

Barranco-Ruiz

(2020). School-based interventions for promoting physical activity using games and gamification: A systematic review protocol. International Journal of Environmental Research and Public Health, 17(14), 5186. https://doi.org/10.3390/ijerph17145186

75.

Savery

J. R.

Duffy

T. M.

(1995). Problem based learning: An instructional model and its constructivist framework. Educational Technology, 35(5), 31–38.

76.

Schneider

W. J.

McGrew

K. S.

(2018). The Cat- tell–Horn–Carroll theory of cognitive abilities. In Flanagan

D. P.

McDonough

E. M.

(Eds.), Contemporary intel- lectual assessment: Theories, tests, and issues (4th ed., pp. 73–163). Guilford Press.

77.

Shenton

A. K.

(2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22(2), 63–75. https://doi.org/10.3233/efi-2004-22201

78.

Shernoff

D. J.

Csikszentmihalyi

(2009). Flow in schools: Cultivating engaged learners and optimal learning environments.

79.

Sicart

(2008). Defining game mechanics. Game Studies, 8(2), 1–14.

80.

Squire

(2005). Game-based learning: Present and future state of the field. Masie center e-learning consortium

81.

Sun

Houssin

Renaud

Gardoni

(2019). A review of methodologies for integrating human factors and ergonomics in engineering design. International Journal of Production Research, 57(15-16), 4961–4976. https://doi.org/10.1080/00207543.2018.1492161

82.

Sweetser

Wyeth

(2005). Gameflow: a model for evaluating player enjoyment in games. Computers in Entertainment (CIE), 3(3), 3–3. https://doi.org/10.1145/1077246.1077253

83.

Tahir

Wang

A. I.

(2018). Insights into design of educational games: Comparative analysis of design models. In Proceedings of the future technologies conference (pp. 1041–1061). Springer. https://doi.org/10.1007/978-3-030-02686-8_78

84.

Thomas

D. R.

(2006). A general inductive approach for analyzing qualitative evaluation data. American Journal of Evaluation, 27(2), 237–246. https://doi.org/10.1177/1098214005283748

85.

Tsai

Y.-L.

Tsai

C.-C.

(2020). A meta-analysis of research on digital game-based science learning. Journal of Computer Assisted Learning, 36(3), 280–294. https://doi.org/10.1111/jcal.12430

86.

Tynjala

(1997). Developing education students’ conceptions of the learning process in different learning environments. Learning and Instruction, 7(3), 277–292.

87.

Unertl

K. M.

Novak

L. L.

Johnson

K. B.

Lorenzi

N. M.

(2010, May-Jun). Traversing the many paths of workflow research: developing a conceptual framework of workflow terminology through a systematic literature review. Journal of the American Medical Informatics Association, 17(3), 265–273. https://doi.org/10.1136/jamia.2010.004333

88.

Vlachopoulos

Makri

(2017). The effect of games and simulations on higher education: a systematic literature review. International Journal of Educational Technology in Higher Education, 14(1), 1–33. https://doi.org/10.1186/s41239-017-0062-1

89.

Wei

Salvendy

(2004). The cognitive task analysis methods for job and task design: review and reappraisal. Behaviour & Information Technology, 23(4), 273–299. https://doi.org/10.1080/01449290410001673036

90.

Whitton

(2014). Digital games and learning: Research and theory. Routledge.

91.

Wouters

van Nimwegen

van Oostendorp

van der Spek

E. D.

(2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. https://doi.org/10.1037/a0031311

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.23 MB

0.20 MB