Sage Journals: Discover world-class research

Abstract

Background.

Reframing could be considered to represent individual and collective twofold values in social learning. When reframing is promoted, it benefits to generate learning processes involving collective inquiry occurring in inter and outer formal education. Digital games, popular as an application for the young generation, are considered a specific community in which social learning occurs by playing games. Serious games (SG) are a type of digital game involving educational purposes. Hence SG is selected as a suitable educational strategy to promote reframing. The previous research investigated features of a gameplay experience associated with reframing. This study develops further based on these game features to examine if/how they improved reframing capabilities. From an educational perspective, specific guidelines were proposed as the research findings to assist serious games in designing and applying game features to enhance the gameplay experience of reframing.

Methods.

An expert evaluation was used to assess the guidelines. The seven experts with experience in game design and teaching were chosen to review the designed guidelines. These experts were required to rate the guidelines using a 5-point Likert scale. All experts offered positive feedback on the finding.

Results.

Research data were analysed and validated via the Content Validity Index. The results showed that a total of 5 guidelines was remained and updated.

Conclusion.

These guidelines can be applied to assist students, game designers, related educators, and researchers who use games to promote reframing or require evidence to link game features with social learning.

Keywords

serious games reframing guideline game features game design

Background

Due to the nature and importance of reframing, this study is motivated to focus on it. For the aim of the research, reframing is conceptualised as people’s ability to redefine their and other people’s aims and values, reflect strategies and methods, and reshape the new criteria of success, which is triggered via observation of and communication with other participants and the environment (Brown, 2010; Conklin, 2005; Mezirow, 1990; Wenger et al., 2011).

Reframing is regarded as a learning outcome/value of social learning (Muro & Jeffrey, 2008; Wenger et al., 2011). Social learning explains that learning is an interpersonal process driven by communication among cognition, behaviour, and environmental influences (Bandura, 1977). During this process, people improve together through observing and shaping their emotional attitudes and behaviours and participating in collaborative knowledge-building, decision-making, and problem-solving (Bandura, 1977; Muro & Jeffrey, 2008). Today social learning is seen as a core paradigm for promoting social development and change (Muro & Jeffrey, 2008), as we live in an era defined by complex, multi-stakeholder issues that can only be addressed through collective learning, reflection, and action (Fabricatore et al., 2020).

Still, a big challenge is to foster people’s reframing capability using educational strategies. Reframing represents an educational priority, being central to addressing real-world, collective problem-solving processes involving actors driven by diverse and possibly conflicting viewpoints (Conklin, 2005; Fabricatore et al., 2020; Wenger et al., 2011). Serious games (SG) are considered one of the educational strategies that can be effective in improving students’ critical thinking and problem-solving (Hwang & Chen, 2017), promoting the development of transferable knowledge, skills and attitudes by engaging players in activities driven by meaning-making, and by setting scenarios that mirror real-world situations (Fabricatore & Lopez, 2019). Games thus represent ideal platforms to promote social learning (Bakhanova et al., 2020; Den Haan & Van der Voort, 2018; Fabricatore et al., 2020). However, the literature has highlighted the need for further research, in particular, to investigate the relationship between specific game features and social learning outcomes that these may promote (Den Haan & Van der Voort, 2018; Voulgari & Komis, 2011). The next challenge is to ensure that the educational strategy is adopted, which means the methods and outcomes of SGs’ usage. Previous studies highlight that SGs are designed in order to improve related learning outcomes. For example, Bressler and Tutwiler (2021) propose that mobile augmented reality games provide benefits to support serious science learning; SGs have been shown to successfully promote the acquisition of teachers’ domain-specific knowledge and skills (Johnson & Kim, 2021, Alyaz & Genc, 2016, Lameras et al., 2014). However, little explanation is shown for the methods of game-making relevant to enhancing learning outcomes in these studies. In order to address these challenges, this paper presents a guideline and its verification to help: i) students with critical reflection on their learning; ii) teachers to promote social learning by exploring games features and what we can learn from serious games; iii) game designers to clarify decisions on designing and developing new computer games and highlight the choices to be made when using games in educational contexts. The guideline describes the application and matching of game elements that can be selected in the SG for promoting reframing. These game features have been identified according to content analysis of specific games.

This paper includes six sections. In Section 2, related research background is involved, and the research question is presented. For example, the nature of reframing is discussed and explains the reasons that serious games are utilised as an effective platform to analyse social learning. After this, our previous work is described, involving a model of game features promoting reframing. Section 3 shows the research methodology, including conceptualisation and validation of these guidelines from the experts' panel. The results of this study are shown in Section 4. The way to use related guidelines is discussed, and in the last Section 6, this paper is concluded and states future works.

The nature and importance of reframing

The concept of reframing is derived from the framework of value creation which was first developed by Wenger et al. (2011). Wenger et al. (2011) examined the value creation of social learning based on virtual communities (VC), which comprise more or less distributed collectives. Among these collectives, individuals could share some interests, communicate and are connected with related stimulation, knowledge (e.g., information flows) and joint action. In this context, Wenger et al. (2011) highlighted the importance of fostering reframing, given that its nature supports the transition of community members' views and actions from individualism to collectivism and consequently creates valuable outcomes to promote the operation of the entire community. According to this, reframing could be considered to represent a twofold value. From the individual's perspective, it means an enhanced potential, including critical reflection, questioning personal worldviews, interacting with others, and acquiring capacity that individuals alone could not obtain (Bandura, 1977; Mezirow, 1990). Literature (e.g., Brookfield, 2015; Howard, 2003; Mamede et al., 2008; Saric & Steh, 2017) has also highlighted that reframing is a core element of critical thinking representing a key mediator of other types of learning (e.g., problem-solving) when non-trivial scenarios need to be critically examined. From the collective perspective, reframing represents the important development of the whole community and signifies the development of new, shared ways of interaction (Bandura, 1977; Conklin, 2005; Muro & Jeffrey, 2008). The twofold value of reframing benefits all learning processes involving collective inquiry occurring in inter and outer formal education (Conklin, 2005; Pedaste et al., 2015).

Serious games

Whilst there is no absolute agreement on a definition of serious games (SG), most scholars agree on a core meaning when the primary purpose of SGs is within education (Bontchev et al., 2021). In particular, when applications from relevant areas such as game-based learning (Deterding et al., 2011; Plass et al., 2015) are included rather than simply entertaining (Landers, 2014; Susi et al., 2007; Michael & Chen, 2005). Game-based learning products are, therefore, widely considered a subcategory of SGs (Connolly et al., 2012; Hainey et al., 2016). This paper similarly considers SGs through the conceptualisation of digital games/ game-based learning design to consider how they promote behaviour change and/or the development of knowledge, skills and attitudes (Hainey et al., 2016; Connolly et al., 2012; De Lope & Medina-Medina, 2017). In addition, SGs can be utilised as a specific community where social learning can be generated (Bolter & Grusin, 1999) because video games offer a very rich ground for studies on VC (Tardini & Cantoni, 2005). According to Gee (2007), players have different identities when playing games (e.g., the real player and the virtual character). Tardini and Cantoni (2005) indicate that, based on the relationship between the real player’s and the virtual character’s identities, two different VCs can be constructed, namely the playing community and the player community. In the playing community, for example, the virtual characters interact in the game’s world with other characters or non-playing characters to complete tasks such as killing monsters. In the players’ community, for instance, the interaction is generated between the real players who are interested in the same game (Newman, 2012). In this context, the study adopts SGs as a suitable platform/community to provide guidance for the design of SGs. This can assist students, game designers, related educators and researchers who use games to promote reframing or require evidence to link game features with social learning.

Our previous work on the study and research question

According to these descriptions mentioned before, there exists, therefore, a relationship between games and learning values, with reframing generated in social learning and the use of games supporting social learning (Bakhanova et al., 2020; Medema et al., 2016; Den Haan and Van der Voort, 2018), as shown in Figure 1. Based on the relationship, the previous work presented a model of ten game features promoting four functions of reframing by investigating and analysing 5,940 players’ comments on the TOP 5 role-playing (RPG) games. However, this model focused on examining features of a gameplay experience associated with reframing. Further research is needed to investigate if/how the features improved reframing capabilities. So, from an educational perspective, this article proposes the research question:

Figure 1.

The background to the research.

How to effectively utilise the model to design and apply game features to enhance SGs gameplay experience of reframing?

Firstly, however, we need further clarification on the model and related terms being used.

According to Wenger et al. (2011), reframing is split into four functions, including promoting reflection: learning contents and values trigger rethinking; the change of evaluation: changing the processes and patterns of evaluation; generating new assessments for other people: reframing the assessment standard (e.g., views, relationship) for other people; suggestions for community development: reshaping the new expectation for whole community development (e.g., institutional change and new frameworks).

The model of using role-playing games promoting reframing used a specially created form that described the relationship between game features and the four evaluation criteria of reframing. See Table 1. Table 2 reverses the column order of Table 1. In the first column, it lists the criteria for reframing and in the second, game features. This enables Tables 1 and 2 to be used either from a game feature or a reframing perspective.

Table 1.

The model of the game features promoting reframing.

Game features	Reframing functions
Multiple endings of the game (According to players' selections, the game provides different endings, but these endings cannot appear concurrently.)	Promoting reflectionBased on multiple- endings of the game, players express their various reflections in deciding or discussing the ending of the game with other people. For example, players think about the advantages and disadvantages of the game-ending.
Tasks with emotional experience (Players can gain emotional experience in missions such as friendship, love and adventure.)	Promoting reflectionEmotional experiences enable players to immerse themselves in the game, including triggering relevant emotions (e.g., love) and generating different thoughts or evaluations of emotional value.
A real no-play character (NPC) image(The game system controls the character, whose background story, dialogues and personality are designed to be close to real-life roles.)	Promoting reflectionPlayers are engaged by NPCs whose personality/performance was designed closer to a real living person.
The game design of a particular style (The game is the art synthesis involving sound, theme, music and design details.)	Promoting reflectionWhen players are interested in themes, design details or music, they will be stimulated in their exploration or further thinking based on these elements.
	Suggestions for community developmentWhen players are interested in themes, design details or music, they will be stimulated in their exploration to make some suggestions for improving the development of the game. For instance, if a player does not like the game's background music, he/she is likely to suggest revising the music.
Game stories (The game story is utilised as a key strategy to achieve engagement because it involves multiple contents such as whether or not it has rich and colourful plots, reasonable role arrangement, humorous narration and making players feel a sense of participation.)	Promoting reflectionPlayers have various thoughts based on the game story, such as analysing the game plot, guessing the relationship between game characters, digging out the hidden philosophy of life, and putting forward suggestions for improving the game story.
	Suggestions for community developmentPlayers have various thoughts based on the game story, such as analysing the game plot, guessing the relationship between game characters, digging out the hidden philosophy of life, and putting forward suggestions for improving the game story.
Game avatars (Players control the characters with original skills and personalities.)	Suggestions for community developmentPlayers playing the game avatar (e.g., matching skills) would generate certain ideas to achieve related goals, such as increasing the winning rate. These ideas may include advising developers to change/modify the avatar's personality or proposing new skill patterns or strategies.
	Generating new assessments for other peoplePlayers control the game avatar to display their emotions or behaviours, which can cause them to reflect on the avatar or motivate other players who are seen as enemies or teammates to evaluate the avatar's behaviour and come up with new criteria for evaluating others.
Tasks for problem-solving (Specific difficulties such as quizzes and traps were allocated in the missions, which need to be solved by players.)	Suggestions for community developmentThe player made some suggestions to the developer, including modifying/changing the game (e.g., game time).
	The change of evaluationThe player's multiple methods/strategies are utilised to solve problems/difficulties.
The rank system of the game (A distinction of the rank system of the game, such as game points, generates a culture of collaboration and competition.)	Suggestions for community developmentPlayers put forward suggestions for optimising the game through multiple tests of the rank system. For example, players pay attention to the fairness of the rank system and offer matching rules.
Game tools (Some items are used by players in order to assist them in finishing tasks.)	Promoting reflectionPlayers utilise game tools to associate relevant or similar knowledge.
	Suggestions for community developmentGame tools are reflected by players and include factors such as increasing win rates. They advise developers on modifying or changing game tools.
	The change of evaluationPlayers rethink the functionality of game tools to generate various strategies. For instance, the same tools are used in different environments to solve problems, or different tools are selected to assist game characters.
Game mode (Game mode means the game rule or function.)	Suggestions for community developmentPlayers focus on game rules or functions, making suggestions for game modes (e.g., adding a checking function to review members’ details).

Table 2.

Promoting reframing through the model of the game features (reversing Table 1).

Reframing functions	Game features
Promoting reflection	A real no-play character (NPC) image
	Game stories
	Multiple endings of the game
	The game design of a particular style
	Tasks with emotional experience
	Game tools
Generating new assessments for other people	Game avatars
Suggestions for community development	Game stories
	Game avatars
	Game mode
	The game design of a particular style
	Tasks for problem-solving
	Game tools
	The rank system of the game
The change of evaluation	The game design of a particular style
The change of evaluation	Tasks for problem-solving

Research methodology

The research method is composed of conceptualisation and validation, including identification of the guidelines for SGs design based on research aims and content validation of the guidelines.

Conceptualisation

The construction of guidelines is examined to cover the situations in which guidelines are possibly used, from the perspective of students, teachers and game designers’ applications, respectively. For example, game designers were likely to design a game that is required to cover all reframing functions, so the guideline would provide a relevant strategy. According to this, the guidelines were generated. See Figure 2.

Figure 2.

The methodology for conceptualisation.

Content validation

For the second part of the methodology, expert evaluation was adopted to investigate the content validation of guidelines. In light of some studies, an expert evaluation rating scale for domain content was five to seven experts (e.g., Dumas & Grajo, 2022; Razali et al., 2022). Hence, this research selected seven experts from their focus areas of teaching (n =3) or game design (n =4). In this study, all seven experts have more than five years of experience in relevant fields, especially three experts with over ten years of experience. It is important to select experts who focus on their fields (De Jans et al., 2017).

The research method that expert evaluation was used to identify issues and support experts offering advice was adopted (Khowaja & Salim, 2019; Nielsen & Molich, 1990). The data for this part were collected using a questionnaire to obtain an evaluation from the experts who were given the guidelines and two weeks to finish the survey. The questionnaire's content includes the description of guidelines with two more added columns. For one of the additional columns, a 5-point Likert scale (1 Strongly Disagree, 5 Strongly Agree) was utilised to require the experts to indicate the level of agreement. In another column, the experts' extended comments were listed.

In order to measure the relevancy and clarity of the guidelines, the Content Validity Index (CVI) (Zamanzadeh et al., 2015) was utilised. As Zamanzadeh et al. (2015) indicate that prerequisites of assessment (e.g., if there is a potential bias happening) could be completed by carrying out this method that calculates both item levels (I-CVIs) and the scale level (S-CVI). This investigation obtained the I-CVI by dividing the number of experts associated with the assessment element by the total number of experts (Razali et al., 2022); S-CVI is calculated using the total number of elements to divide the sum of the obtained I-CVIs (Razali et al., 2022; Zamanzadeh et al., 2015).

The consensus analysis (i.e., measuring consensus and evaluating experts’ opinions) (Heiko, 2012) was also used in the second part of the methodology in order to enhance the verification of the guidelines. According to Heiko (2012), the Interquartile Range (IQR) was frequently utilised in the consensus among the answers as a measurement of variability in terms of the distribution of quartiles of the data. The data is divided into three equal parts, including the first, second and third quartiles and IQR (the first quartile subtracted from the third quartile) (Razali et al., 2022). The elements with an IQR score should remain when the score is one or less. Otherwise, the elements were eliminated.

Results

The results are presented in the following, including the description of the guidelines and the validity findings.

The description of guidelines

Q1 was eliminated, according to the results of the validity of guidelines. (The old version: each game feature can be utilised independently. Some SGs only cover one game feature, which also means that there is the potential to promote reframing for these games.)

Q2 Game features will be matched to assist game design and selection of the existing games, according to different game features that support reflection of different dimensions (i.e., promoting reflection, the change of evaluation, generating new assessments for other people and suggestions for community development). For example, such tasks for problem-solving (promoting: the change of evaluation; suggestions and for community development) and game stories (promoting: promoting reflection; suggestions for community development) can be selected synchronously to create value in three dimensions (i.e., promoting reflection; the change of evaluation; suggestions and for community development).

Q3 was revised. The new version is as follows: some serious games, which cover related one or more game features (e.g., multiple endings of the game; Tasks with emotional experience), can be considered to have the potential of promoting reframing. These serious games can be utilised to create reflective environments (inside or outside classrooms). When using the existing serious games, Table 3 can be completed to help understand the nature of the games for promoting reframing.

Table 3.

An example of using a promoting-reframing-value game.

Serious game	Covers game features	Potentials of promoting reframing
e.g. WU KONG SHI ZI https://www.gongfubb.com/	e.g. Game story	e.g. Promoting reflection;Suggestions for community development
…	…	…

Q4 This model can provide a reference/standard for identifying the quality of serious games. For example, good serious games will be recommended to students, teachers, schools, or parents. From the perspective of promoting reframing, if the game covers related game features, this means that the game can be confirmed as having the potential to promote reflection.

Q5 To promote a specific criterion of reframing, some related game features will be considered in advance. The research found that some game features showed positive effects on promoting reframing. For example, two game features (the game design of a particular style and game stories) supported promoting reflection.

Q6 The following matching effects of game features need to be considered. Some identified game features are consistently oriented at supporting the same criterion of reframing. For example, tasks with emotional experience, game avatars and game stories together impact promoting reflection. Secondly, there is wide variability in how the game features promote criteria of reframing. Some criteria included one or three game features that supported two or three criteria, while others used mixed game features (e.g., some supported two or three criteria, others supported one).

The validity of guidelines

The overall guidelines were evaluated with high content validity (Shrotryia & Dhanda, 2019). The item content validity index (I-CVI) ranged from 0.72 to 0.86. Zamanzadeh et al.(2015) highlight that if i) I-CVI is greater than 0.79, the item is appropriate; ii) I-CVI is between 0.70 and 0.79, the item needs to be revised; iii) I-CVI is less than 0.70, the item should be eliminated. Q1 of the Items, which was a 0.72 I-CVI, indicated the need for revising it. The scale content validity index (S-CVI/Ave) was 0.84, which was acceptable. Because for reflecting content validity, some studies recommend that S-CVI should be 0.8 or higher is acceptable (Grant & Davis, 1997; Polit & Beck, 2004). The research showed the I-CVI's relevancy using the modified Kappa statistic (Shrotryia & Dhanda, 2019; Zamanzadeh et al., 2015), as the Kappa provided the degree of agreement beyond chance (Wynd et al., 2003; Li, 2021). The formula is as follows: K = (I-CVI – Pc)/ (1- Pc), Pc = [N!/A!( N-A)!] * 0.5N (Zamanzadeh et al., 2015), where Pc = the probability of chance agreement, N = number of experts, A = number of experts who agree the item is related (Li, 2021; Rodrigues et al., 2017; Zamanzadeh et al., 2015; Razali et al., 2022). The Kappa value is considered excellent when it is greater than 0.74; if the value is between 0.60 and 0.74, this means good; the value between 0.40 and 0.59 is considered fair (Rodrigues et al., 2017). The results of the five items were excellent because the values were 0.86 (>0.74) and Q1 was fair with Kappa values of 0.40. See table 4.

Table 4.

The validity of guidelines (E1-E7 means expert 1-expert 7).

Items	E1	E2	E3	E4	E5	E6	E7	Experts in agreement	I-CVI	Pc	Kappa (k)
Q1	0	0	1	1	1	1	1	5	0.72	0.16	0.40
Q2	1	1	0	1	1	1	1	6	0.86	0.05	0.75
Q3	1	0	1	1	1	1	1	6	0.86	0.05	0.75
Q4	1	1	0	1	1	1	1	6	0.86	0.05	0.75
Q5	1	1	1	1	0	1	1	6	0.86	0.05	0.75
Q6	1	0	1	1	1	1	1	6	0.86	0.05	0.75
								S-CVI/Ave	0.84

The IQR of the five recommended guidelines is less than or equal to one, indicating that it is significantly relevant. Heiko (2012) states that the factor with an IQR score is one or less (< = 1) should remain and eliminate the factor with a score greater than one (>1). Another guideline with a 1.5 IQR score would be removed (Heiko, 2012) (Table 5)

Table 5.

The IQR of the guidelines (E1-E7 means expert 1-expert 7).

Items	E1	E2	E3	E4	E5	E6	E7	The first quartile	The third quartile	IQR
Q1	2	3	5	4	4	4	5	3	4.5	1.5
Q2	3	4	4	4	5	5	5	4	5	1
Q3	2	4	4	5	5	5	5	4	5	1
Q4	3	4	4	4	4	5	5	4	4.5	0.5
Q5	3	4	4	4	4	5	5	4	4.5	0.5
Q6	2	4	4	4	4	5	5	4	5	1

Discussion

Revision for guidelines

The seven experts did not provide extended comments to revise these guidelines. According to the findings, Q1 was eliminated. Nevertheless, Q1 involves the content that each game feature can be utilised independently, which is suggested to integrate into Q3, and the new Q3 is described in Section 4.1.

How to use these guidelines

The users (i.e., students, teachers and game designers) who utilise this paper can reference the following flow processes to use related guidelines.

(1) The research background is provided to students, teachers and game designers, enabling them to understand why the model has been proposed.

(2) The related terms and concepts, such as reframing, are mentioned in the research background. This is to ensure that students, teachers and game designers know the aim of the model, which is to use specific game features to promote the four functions of reframing. Meanwhile, the related descriptions of the model are offered in Tables 1 and 2, which explain the relationship between the four functions of reframing and game features. The information should be read in advance of using the model.

(3) The guidelines are used based on the model, which will get more benefits. The model is described using Tables 1 and 2, which include two columns (‘Game features’ and ‘Functions of reframing’). In Table 1, the first column lists game features, and the second column lists related criteria of reframing that have been promoted by the game feature, together with how the game feature promotes these specific criteria. For example, game stories can be found in the first column, and the reader can then check the second column of game stories to find that game stories support two criteria of reframing (i.e., promoting reflection and suggestions for community development). Table 1 can therefore be used to identify game features present within a game and what value they may help to reframe, for example, developing a community. Table 2 reverses the column order, so in the first column, it lists the criteria of reframing and in the second game features. It can therefore be used when particular reframing is desired, for example, promoting reflection, and particular game features can then be identified that may enable this to occur.

Conclusion

This research provides specific guidelines to apply, clarifying how the game features promote reframing from the perspectives of students, teachers and game designers. The validation of experts with rich experience and skills in the relevant areas was used to verify the guidelines. This can give weight to the outcomes, but if a prototype of the reframing game can be investigated, that would be better. In future, a simple prototype will be planned, and the same experts assess the prototype will be adopted as another verification cycle.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Statement

The research does not involve human participants and/or animals, so the 1964 Helsinki Declaration and its later amendments or comparable ethical standards are not applicable to this paper.

Consent Statement

All testers agree with their testing data that can be published.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

ORCID iD

Lanlan Gao

Author Biographies

Dr Lanlan Gao is an associate professor of Computer Science in China and assistant lecturer of Computer Science at Maynooth University (Maynooth International Engineering College).

Dr Rupert Ward is a Professor of Learning Innovation and Associate Dean (International) within the School of Computing and Engineering at the University of Huddersfield.

Dr Carlo Fabricatore is a human factor scholar and computer scientist at E.H.E.University.

References

Alyaz

Genc

Z. S.

(2016). Digital game-based language learning in foreign language teacher education. Turkish Online Journal of Distance Education, 17(4). doi:https://doi.org/10.17718/tojde.44375

Bakhanova

Garcia

J. A.

Raffe

W. L.

Voinov

(2020). Targeting social learning and engagement: What serious games and gamification can offer to participatory modelling. Environmental Modelling & Software, 134, 104846. doi:https://doi.org/10.1016/j.envsoft.2020.104846

Bandura

(1977). Social learning theory (Vol. 1). Prentice Hall: Englewood Cliffs. https://www.decisionskills.com/uploads/5/1/6/0/5160560/bandura_sociallearningtheory.pdf

Bolter

J. D.

Grusin

(1999). Remediation: Understanding new media. MIT Press. doi:https://doi.org/10.1108/ccij.1999.4.4.208.1

Bontchev

B. P.

Terzieva

Paunova-Hubenova

(2021). Personalisation of serious games for learning. Interactive Technology and Smart Education. doi:https://doi.org/10.1108/ITSE-05-2020-0069

Bressler

D. M.

Tutwiler

(2021). ‘Play Is Serious Learning': Using Mobile Augmented Reality Gaming to Support Science Learning. In Handbook of Research on Innovations in Non-Traditional Educational Practices (pp. 79-106). IGI Global. doi:https://doi.org/10.4018/978-1-7998-4360-3.ch005

Brookfield

S. D.

(2015). Critical reflection as doctoral education. New Directions for Adult and Continuing Education, 2015(147), 15–23. doi:https://doi.org/10.1002/ace.20138

Brown

V. A.

(2010). Collective inquiry and its wicked problems. In Tackling wicked problems (pp. 61-83). Routledge. http://hdl.handle.net/1885/61616

Conklin

(2005). Dialogue mapping: Building shared understanding of wicked problems. John Wiley & Sons, Inc. https://dl.acm.org/doi/abs/10.5555/1237968

10.

Connolly

T. M.

Boyle

E. A.

MacArthur

Hainey

Boyle

J. M.

(2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661–686. doi:https://doi.org/10.1016/j.compedu.2012.03.004

11.

De Jans

Van Geit

Cauberghe

Hudders

De Veirman

(2017). Using games to raise awareness: How to co-design serious mini-games? Computers and education, 110, 77–87. doi:https://doi.org/10.1016/j.compedu.2017.03.009

12.

De Lope

R. P.

Medina-Medina

(2017). A comprehensive taxonomy for serious games. Journal of Educational Computing Research, 55(5), 629-672. doi:https://doi.org/10.1177/07356331166813

13.

Den Haan

R. J.

Van der Voort

M. C.

(2018). On evaluating social learning outcomes of serious games to collaboratively address sustainability problems: A literature review. Sustainability, 10(12), 4529. doi:https://doi.org/10.3390/su10124529

14.

Deterding

Khaled

Nacke

L. E.

Dixon

(2011). Gamification: Toward a definition. In CHI 2011 gamification workshop proceedings (Vol. 12, pp. 1-79). Vancouver BC, Canada. http://gamification-research.org/wp- content/uploads/2011/04/CHI11_Workshop_Gamification.pdf

15.

Dumas

C. M.

Grajo

L. C.

(2022). The Content Validity and Inter-Rater Reliability of the Occupational Therapy Pediatric Inventory of Cognitive Skills (OT-PICS): An Assessment Tool of Functional Cognition in Children. Occupational therapy in health care, 36(1), 84-100. doi:https://doi.org/10.1080/07380577.2021.1972381

16.

Fabricatore

Gyaurov

Lopez

(2020). Rethinking serious games design in the age of COVID-19: Setting the focus on wicked problems. Paper presented at the Joint International Conference on Serious Games. https://www.springerprofessional.de/en/rethinking-serious-games-design-in-the-age-of-covid-19-setting-t/18527038

17.

Fabricatore

Lopez

M. X.

(2019). Game-based learning as a meaning making-driven activity process: a human factors perspective. Paper presented at the ECGBL 2019 13th European Conference on Game-Based Learning. doi: https://doi.org/10.34190/GBL.19.177

18.

Gee

J. P.

(2007). What video games have to teach us about learning and literacy. New York, NY: St. Martin’s Griffin. https://blog.ufes.br/kyriafinardi/files/2017/10/What-Video-Games-Have-to-Teach-us-About-Learning-and-Literacy-2003.-ilovepdf-compressed.pdf

19.

Grant

J. S.

Davis

L. L.

(1997). Selection and use of content experts for instrument development. Research in nursing & health, 20(3), 269-274. doi:https://doi.org/10.1002/(SICI)1098-240X(199706)20:3<269::AID-NUR9>3.0.CO;2-G

20.

Hainey

Connolly

T. M.

Boyle

E. A.

Wilson

Razak

(2016). A systematic literature review of games-based learning empirical evidence in primary education. Computers & Education, 102, 202-223. doi:https://doi.org/10.1016/j.compedu.2016.09.001

21.

Heiko

A. V. D. G.

(2012). Consensus measurement in Delphi studies: review and implications for future quality assurance. Technological forecasting and social change, 79(8), 1525-1536. doi:https://doi.org/10.1016/j.techfore.2012.04.013

22.

Howard

T. C.

(2003). Culturally relevant pedagogy: Ingredients for critical teacher reflection. Theory into practice, 42(3), 195-202. doi:https://doi.org/10.1207/s15430421tip4203_5

23.

Hwang

G. J.

Chen

C. H.

(2017). Influences of an inquiry‐based ubiquitous gaming design on students’ learning achievements, motivation, behavioural patterns, and tendency towards critical thinking and problem-solving. British Journal of Educational Technology, 48(4), 950-971. doi:https://doi.org/10.1111/bjet.12464

24.

Johnson

L. L.

Kim

G. M.

(2021). Experimenting with game-based learning in preservice teacher education. English Teaching: Practice & Critique. doi:https://doi.org/10.1108/ETPC-10-2019-0125

25.

Khowaja

Salim

S. S.

(2019). Serious game for children with autism to learn vocabulary: an experimental evaluation. International journal of human-computer interaction, 35(1), 1–26. doi:https://doi.org/10.1080/10447318.2017.1420006

26.

Lameras

Petridis

Torrens

Dunwell

Hendrix

Arnab

(2014). Training science teachers to design inquiry-based lesson plans through a serious game. In Proceedings of the Sixth International Conference on Mobile, Hybrid and Online Learning (pp. 86-91). https://core.ac.uk/download/pdf/228142696.pdf

27.

Landers

R. N.

(2014). Developing a theory of gamified learning: Linking serious games and gamification of learning. Simulation & Gaming, 45(6), 752-768. doi:https://doi.org/10.1177/10468781145636

28.

(2021). Inclusive Leadership Questionnaire: The Design and Validation of a Theory-based Instrument. Columbia University. https://www.proquest.com/openview/b01ed9e04ad15a8545b57962df143517/1?pq-origsite=gscholar&cbl=18750&diss=y

29.

Mamede

Schmidt

H. G.

Penaforte

J. C.

(2008). Effects of reflective practice on the accuracy of medical diagnoses. Medical education, 42(5), 468-475. doi:https://doi.org/10.1111/j.1365-2923.2008.03030.x

30.

Medema

Furber

Adamowski

Zhou

Mayer

(2016). Exploring the potential impact of serious games on social learning and stakeholder collaborations for transboundary watershed management of the St. Lawrence River Basin. Water, 8(5), 175. doi:https://doi.org/10.3390/w8050175

31.

Mezirow

(1990). Fostering critical reflection in adulthood (pp. 1-20). San Francisco: Jossey-Bass Publishers. https://participativelearning.org/pluginfile.php/561/mod_resource/content/4/How_Critical_Reflection_triggers_Transformative_Learning_-_Mezirow.pdf

32.

Michael

D. R.

Chen

S. L.

(2005). Serious games: Games that educate, train, and inform. Muska & Lipman/Premier-Trade. https://www.researchgate.net/publication/234812017_Serious_Games_Games_That_Educate_Train_and_Inform

33.

Muro

Jeffrey

(2008). A critical review of the theory and application of social learning in participatory natural resource management processes. Journal of environmental planning and management, 51(3), 325-344. doi:https://doi.org/10.1080/09640560801977190

34.

Newman

(2012). Videogames. Routledge. doi:https://doi.org/10.4324/9780203143421

35.

Nielsen

Molich

(1990, March). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 249-256). doi:https://doi.org/10.1145/97243.97281

36.

Pedaste

Mäeots

Siiman

L. A.

De Jong

Van Riesen

S. A.

Kamp

E. T.

Tsourlidaki

(2015). Phases of inquiry-based learning: Definitions and the inquiry cycle. Educational research review, 14, 47-61. doi:https://doi.org/10.1016/j.edurev.2015.02.003

37.

Plass

J. L.

Homer

B. D.

Kinzer

C. K.

(2015). Foundations of game-based learning. Educational psychologist, 50(4), 258-283. doi:https://doi.org/10.1080/00461520.2015.1122533

38.

Polit

D. F.

Beck

C. T.

(2004). Nursing research: Principles and methods. Lippincott Williams & Wilkins. https://otg-dv.ru/sites/default/files/webform/nursing-research-by-denise-polit-85d16e4.pdf

39.

Razali

N. E. M.

Ramli

R. Z.

Mohamed

Zin

N. A. M.

Rosdi

Diah

N. M.

(2022). Identifying and validating game design elements in serious game guideline for climate change. Heliyon, 8(1), e08773. doi:https://doi.org/10.1016/j.heliyon.2022.e08773

40.

Rodrigues

I. B.

Adachi

J. D.

Beattie

K. A.

MacDermid

J. C.

(2017). Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskeletal Disorders, 18(1), 1-9. doi:https://doi.org/10.1186/s12891-017-1914-5

41.

Saric

Steh

(2017). Critical reflection in the professional development of teachers: Challenges and possibilities. CEPS journal, 7(3), 67-85. doi:https://doi.org/10.25656/01:14908

42.

Shrotryia

V. K.

Dhanda

(2019). Content validity of assessment instrument for employee engagement. Sage Open, 9(1), 2158244018821751. doi:https://doi.org/10.1177/215824401882175

43.

Susi

Johannesson

Backlund

(2007). Serious games: An overview. https://www.diva-portal.org/smash/record.jsf?pid=diva2%3A2416&dswid=-1917

44.

Tardini

Cantoni

(2005). A semiotic approach to online communities: Belonging, interest and identity in websites’ and videogames’ communities. In Proceedings of the IADIS International Conference e-Society (pp. 371-378). https://www.researchgate.net/profile/Lorenzo_Cantoni2/publication/266218884_A_SEMIOTIC_APPROACH_TO_ONLINE_COMMUNITIES_BELONGING_INTEREST_AND_IDENTITY_IN_WEBSITES'_AND_VIDEOGAMES'_COMMUNITIES/links/5510112b0cf224726ac4f2a6/A-SEMIOTIC-APPROACH-TO-ONLINE-COMMUNITIES-BELONGING-INTEREST-AND-IDENTITY-IN-WEBSITES-AND-VIDEOGAMES-COMMUNITIES.pdf

45.

Voulgari

Komis

(2011). Collaborative learning in massively multiplayer online games: A review of social, cognitive and motivational perspectives. Handbook of research on improving learning and motivation through educational games: Multidisciplinary approaches, 370-394. doi:https://doi.org/10.4018/978-1-60960-495-0.ch018

46.

Wenger

Trayner

De Laat

(2011). Promoting and assessing value creation in communities and networks: A conceptual framework. https://www.researchgate.net/profile/Maarten-Laat/publication/220040553_Promoting_and_Assessing_Value_Creation_in_Communities_and_Networks_A_Conceptual_Framework/links/0046353536fa177004000000/Promoting-and-Assessing-Value-Creation-in-Communities-and-Networks-A-Conceptual-Framework.pdf

47.

Wynd

C. A.

Schmidt

Schaefer

M. A.

(2003). Two quantitative approaches for estimating content validity. Western journal of nursing research, 25(5), 508–518. doi:https://doi.org/10.1177/019394590325299

48.

Zamanzadeh

Ghahramanian

Rassouli

Abbaszadeh

Alavi-Majd

Nikanfar

A. R.

(2015). Design and implementation content validity study: development of an instrument for measuring patient-centred communication. Journal of caring sciences, 4(2), 165–178. doi:https://doi.org/10.15171/jcs.2015.017

Items	E1	E2	E3	E4	E5	E6	E7	The first quartile	The third quartile	IQR
Q1	2	3	5	4	4	4	5	3	4.5	1.5
Q2	3	4	4	4	5	5	5	4	5	1
Q3	2	4	4	5	5	5	5	4	5	1
Q4	3	4	4	4	4	5	5	4	4.5	0.5
Q5	3	4	4	4	4	5	5	4	4.5	0.5
Q6	2	4	4	4	4	5	5	4	5	1

Items	E1	E2	E3	E4	E5	E6	E7	The first quartile	The third quartile	IQR
Q1	2	3	5	4	4	4	5	3	4.5	1.5
Q2	3	4	4	4	5	5	5	4	5	1
Q3	2	4	4	5	5	5	5	4	5	1
Q4	3	4	4	4	4	5	5	4	4.5	0.5
Q5	3	4	4	4	4	5	5	4	4.5	0.5
Q6	2	4	4	4	4	5	5	4	5	1

Guidelines of Serious Game Design for Promoting Reframing

Abstract

Background.

Methods.

Results.

Conclusion.

Keywords

Background

The nature and importance of reframing

Serious games

Our previous work on the study and research question

Research methodology

Conceptualisation

Content validation

Results

The description of guidelines

The validity of guidelines

Discussion

Revision for guidelines

How to use these guidelines

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

Ethical Statement

Consent Statement

Data Availability Statement

ORCID iD

Author Biographies

References

Items	E1	E2	E3	E4	E5	E6	E7	The first quartile	The third quartile	IQR
Q1	2	3	5	4	4	4	5	3	4.5	1.5
Q2	3	4	4	4	5	5	5	4	5	1
Q3	2	4	4	5	5	5	5	4	5	1
Q4	3	4	4	4	4	5	5	4	4.5	0.5
Q5	3	4	4	4	4	5	5	4	4.5	0.5
Q6	2	4	4	4	4	5	5	4	5	1