Evaluating the Effectiveness of the Leading Change for Sustainability in Schools Simulation: A Research and Development Project

Abstract

Background

Although a growing number of simulations have been developed for the purpose of educating for sustainability, published reports consist primarily of prescriptive essays, case descriptions, and commentaries rather than empirical studies. Moreover, only a small number of the empirical studies have used experimental designs to assess their effects on learning. This article addressed the need for validated active learning tools that can be used by educators in educating for sustainability.

Aim

This article presents the design and initial evaluation of the Leading Change for Sustainability in Schools (LCSS) computer simulation. The study examined the effects of the simulation on student engagement, skills in formulating and executing change management strategies, and the application of knowledge to the simulation challenge.

Method

This project employed the research and development method for product design and evaluation. A three-week simulation-based learning intervention was conducted with 32 experienced K-12 school teachers and administrators studying in a Master degree program in Vietnam. The research employed a quasi-experimental, time series design to assessed change in learners’ knowledge and skills following participation in the Leading Change for Sustainability in Schools intervention. Repeated measures ANOVA was used to evaluate week-by-week changes in learning outcomes.

Results

The study found that the simulation-based learning module organized around the Leading Change for Sustainability in Schools (Vietnamese version) simulation was highly engaging for students. Students played the simulation an average of 24 times outside of class during the three-week module (18 hours per student). Students’ skills in formulating and executing change strategies for sustainability improved significantly over the three-week module; 28 students reached the highest level of success on the final assessment. Students also demonstrated significant improvement in their ability to incorporate change management principles into their strategies, indicating improvement in higher-order thinking skills.

Keywords

education for sustainable development simulation sustainability active learning change management

UNESCO’s Education for Sustainable Development (ESD) 2030 framework challenges educators to employ active, participatory approaches to teaching and learning about global sustainability challenges (Leal-Filho et al., 2015; Mogensen & Schnack, 2010; UNESCO, 2020). Among experiential learning approaches, educators have argued that simulations offer several strengths in the context of educating for sustainability (Littlewood et al., 2013; Prado et al., 2020; Svoboda & Whalen, 2017). Specifically, they provide an active, motivating, and engaging learning environment (Chatpinyakoop et al., 2022; Padgett et al., 2019), while also enabling students to learn how to implement holistic interdisciplinary solutions to complex problems (Doyle & Brown, 2000; Huang et al., 2022; Phillips & Graeff, 2014; Stanitsas et al., 2019).

This potential has led numerous educators to design simulations that can be used in educating for sustainability (De la Torre et al., 2021; Hallinger et al., 2022; Svoboda & Whalen, 2017). Nonetheless, efforts to establish the effectiveness of sustainability simulations and games has lagged the broader research literature in both scope and quality (Stanitsas et al., 2019; Vlachopoulos & Makri, 2017). For example, in a systematic review of 376 sustainability simulations and serious games, the Authors found that, “The knowledge base is overly weighted towards ‘commentaries’ (55%) and lacks a critical mass of empirical studies (33%). Moreover, the empirical knowledge base is dominated by studies that rely on non-experimental research designs and descriptive [statistical] methods” (Hallinger et al., 2020b, p. 120358).

These features of the literature are limiting the development of a sound knowledge base in the domain of educating for sustainability (Hallinger et al., 2020b; Lozano et al., 2017; Stanitsas et al., 2019). This article presents the design and evaluation of the Leading Change for Sustainability in Schools (LCSS) computer simulation. The objectives of this article include.

1. To present design features of a simulation-based learning module that aims to prepare K-12 educators with the knowledge and skills needed to implement a whole-school approach to educating for sustainability;

2. Assess learners’ behavioral engagement during an instructional intervention organized around the simulation-based learning module;

3. Evaluate the effects of simulation-based learning on students’ skills in formulating and executing strategies for integrating sustainability into a K-12 school system;

4. Evaluate the effects of simulation-based learning on students’ ability to apply knowledge of change management principles to a sustainability challenge.

The R&D process used in this project yielded two versions of the LCSS simulation: a generic English language version (LCSS) and a culturally-adapted, Vietnamese language version (LCSS-V). This article presents the results of a field test of the LCSS-V version of the simulation. The field test was conducted with 32 experienced K-12 teachers and administrators studying in a Master degree program in Vietnam. The field test employed a quasi-experimental, time-series research design that assessed change in learners’ knowledge and skills resulting from their participation in the intervention.

The significance of this research lies in two areas. First, the R&D project yielded educational tools that can be used in the preparation and professional development of teachers and school leaders. Second, the empirical research reported in this article provides initial empirical findings on the effectiveness of the LCSS simulation. The latter findings contribute to the broader literature on the effectiveness of simulation-based learning with respect to learner engagement and learning outcomes (Hallinger & Wang, 2020a; Nguyen, 2015; Phillips & Graeff, 2014; Stanitsas et al., 2019; Verkuyl & Hughes, 2019).

The Use of Simulations in Educating for Sustainability

Scholars have noted that the complex nature of sustainability challenges requires educational methods that develop systems thinking, incorporate inter-disciplinary knowledge, enhance collaboration skills, and foster knowledge application (Leal Filho et al., 2015; Mogensen & Schnack, 2010; Salas-Zapata et al., 2018). Thus, this literature features numerous calls for the use of “active learning” methods. These include such as problem-based learning, simulations, serious games, project-based learning, and service learning (Lozano et al., 2017; Prado et al., 2020; Stanitsas et al., 2019).

Simulation-based learning refers to a pedagogical method that provides a problem-solving context which challenges learners to make decisions aimed toward solving a challenge that mirrors a real-world problem. For example, medical educators frequently employ “simulated patients” to teach medical diagnostic and motor skills (McGaghie et al., 2010; So et al., 2019). Alternatively, simulations can take the form of computer simulations that present a digital context for solving problems presented in a text or multi-media format (Dankbaar et al., 2016; Gatti et al., 2019; Hoang et al., 2022; Verkuyl & Hughes, 2019). In simulations, learners make decisions that have visible consequences which are conveyed in feedback that becomes the basis for reflection. This reflection then informs further rounds of results-oriented diagnosis, problem-solving and decision-making.

While simulations and serious games share some common elements, Imlig-Itenand and Petko (2018) highlighted several distinguishing features.

[E]ach serious game relies on an enriched system of rules and game mechanics that constitute the playful interaction between both the serious games and with other players, which is not required in educational simulations. . . Especially digital serious games may include additional aspects such as graphical presentations, game characters, sound-effects and a soundtrack, voice-acting, as well as controls that require a certain degree of dexterity, which would be considered as distracting in educational simulations. (p. 403)

The attraction of simulations lies in their demonstrated ability to engage learners (e.g., Nguyen, T. N., 2015; Padgett et al., 2019; Phillips & Graeff 2014), develop practical skills (Doyle & Brown, 2000; Prado et al., 2020; Steadman et al., 2006), change learner attitudes (Chatpinyakoop et al., 2022), and enhance knowledge application (Chatpinyakoop et al., 2022; Gatti et al., 2019; Huang et al., 2022; Phillips & Graeff, 2014; Salas et al., 2009). Yet, as noted above, our recent review of the literature found a need for empirical studies that validate the efficacy of, “sustainability-related simulations and games in relation to change in learner knowledge, attitudes, and skills related to different sustainability foci” (Hallinger et al., 2020b, p. 13). This conclusion was consistent with findings reported in other systematic reviews of research on simulations and serious games used in the domain of sustainability Katsaliaki & Mustafee, 2015; Stanitsas et al., 2019; Vlachopoulos & Makri, 2017). For example, in their review of 77 sustainability-oriented games, Stanitsas et al. (2019) found that a, “large number of studies of SGs (on SD or not) do not properly define their research method and lack quantitative results” (p. 933).

Moreover, reviews of research on simulations in general (Hallinger & Wang, 2020a), and sustainability in particular (Hallinger et al., 2020b), have concluded that the knowledge base is heavily weighted toward studies from economically developed, Anglo-European societies. For example, the authors' recent review concluded as follows:

These data highlight a notable geographical imbalance in this literature. This finding implies a need for expanding their design and use in emerging regions of the world. While some existing simulations may ‘transfer’ to these emerging regions, others may require ‘contextualization’ (see Hallinger & Wang, 2020a). In many cases, simulations that are grounded in the ‘local context’ may offer the greatest potential for changing beliefs and behaviors (Hallinger et al., 2020b, p. 12)

These needs framed the current study which was conducted in the context of the Vietnamese educational system. Our review of relevant literature found that research and practice in educating for sustainability are only beginning to emerge in Vietnam (Kieu et al., 2016; Nguyen, A.N. et al., 2022a). For example, in a recent Scopus search, the authors could only locate a few empirical studies of educating for sustainability conducted in K-12 schools in Vietnam (Hoang & Kato, 2016; Nguyen, A.N. et al., 2022a; Nguyen, L. et al., 2022b). The limited attention to sustainability in Vietnam’s education community was captured by Nguyen (2019) whose article was aptly titled, Searching for education for sustainable development in Vietnam.

Similarly, research on the use of simulations is still in its nascent stage in Vietnam where didactic instruction predominates (Hoang et al., 2022; Nguyen, T.N., 2015). These findings supported the dual foci on sustainability and simulation-based learning adopted in this research. Moreover, the results from this study in Vietnam hold relevance for educators in other societies where educators have not widely adopted a focus on sustainability or simulation-based learning. Indeed, research that validates the effectiveness of active learning methods outside of Anglo-European contexts has been identified as necessary to the development of a global knowledge base (Choon‐Eng Gwee, 2008; Frambach et al., 2014; Hallinger & Wang, 2020a; Lu et al., 2014; National Academies of Sciences, Engineering, and Medicine (NASEM), 2018).

The Intervention

The intervention evaluated in this study consisted of a three-week, graduate education course module organized around the LCSS-V simulation. We classify LCSS-V as a simulation due to multi-faceted features that seek to mirror a real-world problem context. For example, at the outset, the simulation presents learners with an organizational challenge that resembles the abbreviated problem description that might be used in a teaching case. Similarly, the learner(s) is placed in a contextualized organizational role (i.e., a project management team) that works with stakeholders whose roles mirror those of company staff. Moreover, unlike in a serious game, the instructional approach used with the simulation did not rely on inter-group competition. In this section of the article, we describe the instructional process used in the intervention (Nguyen & Hallinger, 2022).

The LCSS-V Simulation

LCSS is a web-based simulation that challenges learners to bring about comprehensive change toward sustainability in a K-12 school system. The LCSS simulation incorporates a “whole-school approach to educating for sustainability” (Henderson & Tilbury, 2004; Tilbury & Cooke, 2005; UNESCO, 2020). Whole-school approaches to educating for sustainability seek to reorient basic features of the school (e.g., curriculum, teaching, management, community relations, operations) around sustainability goals, values, and practices (Dharmapiya & Saratun, 2015; Henderson & Tilbury, 2004; UNESCO, 2020). This contrasts with earlier approaches to educating for sustainability that were often limited to the implementation of a curriculum unit on the environment (Dharmapiya & Saratun, 2015). This whole school approach aims to develop environmentally and socially responsible citizens who are equipped to address current and future sustainability challenges.

LCSS-V is a culturally adapted, Vietnamese language version of the English language LCSS simulation (Nguyen & Hallinger, 2022). Thus, the following description of the LCSS-V simulation's content, design and instructional use also applies to the LCSS simulation.

The Simulation Challenge

In the LCSS-V simulation, the learner assumes the role of a member of a project team appointed by the Director of Education who has initiated the “One Future” sustainability initiative. The project team is tasked with guiding the transformation of the school system toward the integration of more sustainable values and practices. Pilot implementation of the One Future project takes place over a three-year period with 24 stakeholders in the District Office, a Primary School, and a Secondary School (see Figure 1). Consistent with a multi-stakeholder view of sustainable development (Clifton & Amran, 2011), the stakeholders in the simulation encompass a wide range of organizational roles (e.g., District Director, Board member, Principal, Assistant Principal, teacher, supplier, community member, student). The simulation provides brief descriptions of the various stakeholders who possess varying knowledge and attitudes toward sustainability issues.

Figure 1.

The Leading Change for Sustainability in Schools (LCSS) Simulation Gamescreen (English language version).

Method of Play

The learner(s) must formulate and execute a change transformation strategy for the school system. The project team has at its disposal 18 change activities, each of which has a specific cost (see Figure 1). During the three years of implementation, the project team is provided with an annual budget of Bits (i.e., time and money). The project team will spend its budget on activities aimed at informing, engaging, empowering, and supporting the stakeholders in the transformation process. Some activities will be useful in raising awareness (e.g., Talk to First Time, Survey Stakeholders, Assess Sustainable Practices). Some can be used to generate interest among stakeholders (e.g., Talk to Second Time, School Visit, Sustainability Retreat). Still others can support the use of new sustainable practices (e.g., Use Sustainable Practices, School Support Group, Share Sustainability Success). However, these purposes are implicit. The learners will only realize how and when to use the different activities through trial‐and‐error learning. Thus, while there are better (and worse) sequences for implementing these activities successfully, the learners will discover these through the simulated experience.

Stages of Change

The simulation screen visualizes five stages of the change process: Awareness, Interest, Preparation, Practice, and Sustainability (see Figure 1). This stage-based framework is grounded in research on organizational change (e.g., Fullan, 2015; Grant, 2010; Hall & Hord, 2006; Kotter, 2012). If the project team's change strategy (i.e., decision-making sequence) meets the needs and concerns of individuals and the organization, stakeholders will gradually move through these stages of change. However, because resistance to change is a typical feature of organizations, the player(s) will encounter numerous obstacles during the change process. An overview of the three-year process of playing the simulation is shown in Figure 2.

Figure 2.

Three-year Process of Playing the Leading Change for Sustainability in Schools Simulation.

Goals of the One Future Project Team

When playing the LCSS-V simulation, the One Future project team (i.e., the learner or team of learners) is tasked with achieving two goals. The first is to move as many of the 24 stakeholders as possible into the Sustainability Stage by the end of the three years of simulated implementation. If a large number of stakeholders reach the Sustainability stage, it implies that the project team has been successful in changing stakeholder practices and the school culture. Notably, research finds that behavioral change in educational organizations tends to be quite slow (Fullan, 2015; Hall & Hord, 2006; Kotter, 2012; Lozano & Garcia, 2020). Thus, the designers made it very difficult to move all 24 stakeholders into the Sustainability Stage by the end of the three-year simulation.

The second goal is to accumulate as many ‘Bennies’ (i.e., productivity benefits) as possible over the course of the three-year simulation. Successful transformation should be visible in the school system’s impact on the triple bottom line of educational outcomes. These include results on economic (e.g., employability of graduates, budget usage, community development initiatives), social (e.g., inclusive values, health, safety), and environmental (e.g., waste reduction, reduction of carbon emissions) indicators (Elkington, 2013; MacDonald, 2009). Bennies accrue when the project team is successful at implementing activities that impact the triple bottom line. For example, if a team successfully Uses Sustainable Practices, the participating stakeholders will each move one space and the project team will gain 150 Bennies (e.g., by incorporating climate science into the curriculum, by using suppliers who meet social inclusion or environmental impact criteria, by reducing plastic usage in the school, by developing a project-based learning project on flood prevention with the community).

Formulating and Executing a Change Strategy

The project team’s ability to successfully engage stakeholders in the change and gain Bennies depends upon development of an effective change strategy. Over 750 hidden decision rules are embedded in the simulation. These decision rules determine whether stakeholders “move” (i.e., change) and Bennies are gained in response to an activity. These decision rules are based on change management theories that propose how, why, and when people adopt new attitudes and behaviors in response to organizational change initiatives (Hallinger et al., 2022; Nguyen & Hallinger, 2022). The LCSS simulations incorporate change principles derived from the concerns-based adoption model (Hall & Hord, 2006), diffusion of innovations (Crandall et al., 1986; Rogers, 2010), organizational change for sustainability (Benn et al., 2014; Doppelt & McDonough, 2017; Lozano & Garcia, 2020; Sroufe, 2017), systems thinking (Senge et al., 2015), and strategic organizational change (Fullan, 2015; Kotter, 2012).

The simulation also incorporates interdisciplinary knowledge extracted from sustainability-related disciplines (e.g., science, engineering, management, education). For example, the simulation employs the triple bottom line concept of sustainability outcomes (Elkington, 2013), whole-school models of educating for sustainability (Tilbury & Cooke, 2005), sustainability leadership and management (Lozano & Garcia, 2020), social sustainability (Dempsey et al., 2011), circular economy (Geissdoerfer et al., 2017), and stakeholder theory (Clifton & Amran, 2011). Illustrative examples of specific sustainability practices are incorporated into the 185 dialogue boxes that provide feedback on “what happened” each time the project team implements an activity. For example, a dialogue box will appear on the screen if the team successfully implements Share Sustainability Success in the primary school.

The stakeholder team shows how a project-based learning unit on flood prevention and control has contributed innovative solutions to a persisting problem in the community. Stakeholders in the Preparation Stage or beyond move 1 space. Gain 50 Bennies for each person in the Practice stage and 100 Bennies for each person in the Sustainability Stage.

In this way, learners gradually learn about a wide range of sustainability concepts and practices in the context of solving the challenge of implementing change in the simulation. This approach to learning conceptual knowledge in a problem-solving context is also consistent with the principles of problem-based learning (Barrows & Tamblyn, 1980).

The Learning Module

Design Features

The intervention encompassed three weekly 2.5-hour classes that took place in March-April, 2022 at a university in Vietnam. The design of the LCSS module was guided by Kolb’s experiential learning model. Kolb (1984) proposed that all learning begins with experience. However, people only learn from experience if they reflect upon it. This reflection can take many forms including process observation, recording the results of what happened, structured sharing of observations with others, input from a colleague or coach, and collective problem-solving. Reflection can lead to a reconceptualization of the problem or task, and the formulation of new approaches or strategies. This phase is followed by active experimentation resulting in new experiences Kolb (1984).

Figure 3 illustrates how Kolb’s (1984) model was incorporated into the design of the simulation-based learning module. Notably, the effectiveness of the experiential learning cycle depends on learner engagement. For example, only by playing the simulation multiple times do students gain a sufficient body of experience that allows them to notice the patterns that combine to create an effective change strategy. Pattern recognition has been identified as a key component in the development of expertise across various professions (Franklin, 2013; Sternberg, 1998).

Figure 3.

Experiential Learning Cycle Enacted in the Instructional Design.

The Learning Activities

In the first class, the instructor (i.e., the lead researcher) introduced the goals of the course, defined key sustainability concepts, presented the LCSS simulation challenge, and explained the method of play. Next, three-member student teams read through the information provided, formulated initial change strategies, and played the simulation for one hour. Due to time constraints, none of the teams could play beyond the first year. This period of simulation play was followed by a whole class debriefing led by the instructor. No change theories were introduced in this class. Instead, in line with the tenets of problem-based learning (Barrows & Tamblyn, 1980), students were challenged to think about how to address the problem “before” being provided with theoretical content on change management. By the end of class, the students were familiar with the simulation challenge, the characteristics and attitudes of the 24 stakeholders, the 18 change activities, and the rules of play.

The class ended with the instructor encouraging the students to play the full simulation individually “at least three times” at home during the week. Students were also expected to participate in an Online Discussion Forum hosted on Moodle.

The Online Discussion Forum was an integral means of structured, collective reflection. Weekly reflection questions posted by the instructor were formulated to help students process what they were learning, and prompt them to help peers solve problems encountered while playing the simulation. For example, the instructor prompts in the Online Discussion Forum in the first week were the following.

1. What was the most difficult problem you have encountered in implementing change for sustainability in the school district? Were you able to solve it? If so, how?

2. What is the most important piece of advice you would like to share with others?

3. What is the most significant problem on which you would like advice from others?

The instructional sequence in the second and third weeks followed a similar pattern. The instructor began these classes by responding to issues raised in the Online Discussion Forum. This was followed by a mini-lecture that introduced specific sustainability concepts and change theories. Students were prompted to consider how the change theories could be applied to the challenge in the simulation (Nguyen & Hallinger, 2022).

Next, newly formed student teams were given an hour to play the simulation. Team members rotated weekly to promote knowledge sharing and encourage students to restate their assumptions and strategies. During the second week, most teams were able to complete two years of the simulation in class. In the third week, most teams completed the full three-year simulation within the one hour allotted to the team learning activity. Finally, the instructor concluded each class by engaging the students in a debriefing session that involved playing the simulation as a whole class.

As the simulation session unfolded, the instructor initiated a class discussion with questions about what happened and why. Students were asked to share their strategies and identify the change management principles that supported their strategies. This collective reflection aided students in checking their assumptions, reformulating their strategies, and applying theoretical concepts to the simulation challenge. Students also continued to play the simulation, read assigned materials, and participate in the Online Discussion Forum from home during these weeks.

The module concluded with an individual simulation exam at the start of the fourth class. Each student was given 30 minutes to play a single simulation session to completion. Individual student scores on the simulation exam were recorded and applied toward the course grade.

Method

Research Design

This study employed a quasi-experimental, repeated-measures times-series research design (Campbell & Stanley, 2015). Quantitative data were collected at multiple points to track changes in learner knowledge and skills during the three-week intervention. This research design overcomes weaknesses associated with pre/post-test, quasi-experimental designs by enabling the researchers to link observed changes to the process of the intervention through multiple observations (Campbell & Stanley, 2015).

This design is illustrated in Figure 4, where X represents the simulation intervention executed during the three weeks of the course, and O represents data collection points. Specifically, O₁ refers to the baseline, the final results of the first full simulation session played by a student at home after class in week one. O₂ refers to the student’s result from the last session played by a student in week one prior to the second class. Similarly, O₃ and O₄ refer to the results of the last sessions played by students at the end of the second and third weeks. Finally, O₅ refers to the results collected in the simulation exam conducted during class in the fourth class (i.e., after three weeks of learning). It should be noted that in most cases O₄ and O₅ were only separated by a single day.

Figure 4.

Repeated Measures Time Series Quasi-Experimental Research Design. O represents a data collection point, X represents execution of the intervention.

Sampling and Participants

It is often challenging to employ random sampling when testing educational interventions. This is especially the case when seeking to test the viability of an intervention that has yet to be trialed. Thus, we employed convenience sampling in this study.

The target group for the LCSS-V simulation consisted of K-12 school teachers and administrators. The participants in this study were 32 second-year graduate students taking a course on ‘Effective Management of Schools’ in the Faculty of Education at a university in Vietnam. There were 11 male and 21 female graduate students. Their work experience in the education sector was varied (M = 6.63 years, SD = 5.116, Min = 1 year, Max = 20 years). This sample was deemed suitable for an initial study of the simulation’s viability and effectiveness. As experienced educators, the participants could provide useful feedback on the simulation’s authenticity and utility, as well as research data for the study (Barrows & Tamblyn, 1980; Rystedt & Sjöblom, 2012).

Data Collection

Salas and colleagues (2009) proposed a set of best practices for measuring learning outcomes when using simulations. In addition to measuring traditional learning through survey perceptions and test results, they suggested capturing the final performance on a simulation to measure effects on skill performance (Salas et al., 2009). Although the researchers also collected multiple data types on learning engagement, knowledge, and attitudes, this article focuses on the performance metric-oriented approach recommended by Salas et al. (2009).

Procedures

Data collection was continuous throughout the three-week module. Data collection capabilities were programmed into the simulation software hosted on a web server. Data concerning the learning process and outcomes were saved to a file on the server every time a student completed a simulation session. Each student had a unique username, thereby making it possible to track individual student results throughout the module. All of the students signed participant consent forms agreeing to the collection and use of data for the research study. The research was approved by the researchers' Institutional Review Board of the researchers’ university.

Learner Engagement

We operationalized the “behavioral engagement” of learners as the number of simulation sessions played to completion by students during the module. A corollary measure was the amount of time that students played the simulation outside of class. These measures offered insight into the extent to which students were motivated to play the simulation.

Typically, the first few times a student played the simulation, it could take up to 90 minutes to finish. However, by the fourth or fifth simulation session, students could complete the simulation in as little as 20 minutes. This practice of playing the simulation in multiple iterations was consistent with Pringle et al.’s (2010) finding that playing a simulation more than once yielded value-added results to student learning. Both the frequency and duration of play were tracked and saved unobtrusively by the LCSS-V software for each student.

Skill Performance

One advantage of simulations lies in their ability to develop skills that students can apply beyond the classroom (Doyle & Brown, 2000; Salas-Zapata et al., 2018; Steadman et al., 2006). Skills in formulating and executing change strategies for sustainability were tracked by monitoring several performance outcomes on the simulation (Salas et al., 2009). The simulation offers three metrics by which the effectiveness of a player’s change strategy can be evaluated at the conclusion of a simulation session: the number of Bennies gained (0-12,000), number of stakeholders reaching the Sustainability Stage (0-24), Change Management Level achieved (six levels: Apprentice, Novice, Manager, Leader, Expert, Change Master).

For this article, we selected the Change Management Level achieved by students in each completed simulation session as the skill metric. At the conclusion of each simulation session, the software assigned one of six Change Management Levels to the student’s result as a summative measure of skill in executing a successful change strategy. The Change Management Level is a discrete measure based on the number of Bennies gained through the player’s strategy during the simulation session. However, players can only achieve a high number of Bennies if they can move a critical mass (e.g., 18) of the stakeholders into the Sustainability Stage during the three-year simulation. Thus, for the sake of parsimony, we decided to use Change Management Level as the measure of skill in formulating and executing an effective change strategy for sustainability.

Knowledge Application

We were similarly interested in the extent to which the simulation module enhanced the ability of students to integrate change management principles into their change strategies. We identified seven change management principles that, if fulfilled, would contribute to a successful change strategy (see Table 1). These seven principles, drawn from Kotter’s (2012) model of organizational change, were operationalized in terms of activity sequences that could be enacted in the simulation (see Table 1). The more of these change principles a player incorporates into a change strategy, the greater the likelihood of succcess. We programmed the software to track the enactment of these change principles, display these on the simulation dashboard, and save these data at the end of each session played by a student.

Table 1.

Conceptual and Operational Definition of Kotter’s Stages Embedded in the LCSS-V Simulation.

Principle	Concept	Condition
Create a Sense of Urgency	There must be continuous top-down, bottom-up, and multi-channel communication about the change's purposes, benefits, and impact on stakeholders and the organization.	Talk to 2X at least 9 people AND Gather Social Information AND Survey Stakeholders OR Assess Sustainable Practices.
Create a Guiding Team	Successful change requires a team of stakeholders with sufficient power, influence, motivation, and expertise to assume ownership and lead the change.	Talk to 3 social influencers AND talk to Mr.An, Ms.Én, and Mr.Oánh 2 times AND School Visit in Year 1.
Create and Communicate a Shared Vision	An organizational change must be guided by a shared vision of the organization that stakeholders aim to create in the future.	Year 1: Successful Create Shared Vision AND Communicate Vision AND Sustainability Retreat; AND Communicate Vision in Year 2 OR Year 3.
Enable People to Act	Successful change involves developing new attitudes and capacities among the people responsible for implementation in the daily operation of their jobs.	School Visit in Year 1 AND Sustainability retreat in Year 1 or Year 2 AND Sustainability Workshop at least one time in each school.
Create Quick Wins	During the change process, quick wins are needed to show that 'the change can work here' and motivate skeptics to join the change effort.	Get at least 2 people into Sustainability Stage in Year 1OR at least 5 people in Practice stage or beyond in Year 1.
Change the School Culture	Over time, actions must be taken to reenergize stakeholders and embed the change into the culture.	Use Sustainable Practice at least two times AND One Future Celebration AND Share Sustainable Success OR School Support Group.
Make the Change Stick	Achieving sustainable change requires revisions to institutional policies that support the change in practice.	Open Class Demonstration OR Sustainability Report AND Policy Revision.

Thus, at any point, while playing the simulation, a student could consult the LCSS-V dashboard and see how many change principles had been enacted. These seven change principles, taught to students in the third class, became a heuristic used by students for enhancing the effectiveness of their change strategies. Thus, the number of change principles (i.e., from 1-7) enacted in a student’s change strategy was employed as a measure of knowledge application (Salas et al., 2009).

Data Analysis

Learner engagement was analyzed through descriptive statistics that tracked the frequency and duration of student simulation sessions played at home during the three weeks. Both skill performance and knowledge application were evaluated using a similar analytical strategy. One-way repeated measures ANOVA was used to analyze week-by-week change in the Change Management Levels achieved by students. ANOVA produced pairwise comparisons of the student’s results on this metric at five data points. This analysis assumed that statistically significant, week-to-week change in this metric represent evidence of the simulation-based learning module’s effects on change management skills.

Prior to running the ANOVA, Mauchly's test was used to assess the homogeneity of variance of the difference between samples, commonly referred to as sphericity (Hinton et al., 2014). If Mauchly's test yields a p-value less than 0.05, it indicates a violation of the assumption of sphericity. In this case, it would be necessary to apply a correction using the Greenhouse-Geisser test before executing ANOVA. Once the correction was applied, the ANOVA could be used to generate pairwise comparisons of the week-to-week results on Change Management Levels and Change Management Principles. Due to the small sample size used in the study, Shapiro-Wilk and Friedman Rank tests were used as follow-up checks on the robustness of the results.

Results

Learner Engagement

As noted earlier, the experiential learning model used in conjunction with the LCSS-V simulation module assumed that success requires a high level of active student engagement. Two outliers were identified in the sample; one student who played the simulation 195 times during the three weeks, and another who played only four times. After removing the outliers, the average number of simulation sessions played during the three-week module was 24 times per student. During the week following the first class, students played the simulation at home an average of 6 times. The frequency of student play increased to an average of 8 times in the second week, and 10 times in the third week.

A high level of learner engagement was similarly reflected in the amount of time students devoted to the simulation outside class. Students spent an average of 18 hours engaged in self-directed, independent learning with LCSS-V outside class during the three weeks (i.e., an average of 6 hours per week). This time commitment did not include assigned readings or responses to the Online Discussion Form. Since playing the simulation represented a choice by each student, these data suggest that most of the students found the process of learning with the simulation highly motivating and engaging.

Skill Performance: Change Management Level

It would be natural to expect some degree of improvement in the Change Management Level as students played the simulation over time (Pringle et al., 2010; Salas et al., 2009). However, in practice, two conditions are necessary for this to occur. First, students must gain experience by playing the simulation (i.e., engagement). While this might seem obvious, unless students found the simulation motivating, they would not commit the time needed to learn from the simulation. Engagement data presented in the prior subsection suggested that this condition was met.

The second condition concerns the other elements of the experiential learning cycle, namely reflection and reconceptualization. As noted earlier, the module incorporated several reflection activities (e.g., team-based learning during class, structured whole-class debriefing, Online Discussion Forum), as well as readings, and input from the instructor in the theory-oriented mini-lectures. Thus, students engaged in the full experiential learning cycle multiple times during the three weeks.

Descriptive analysis of student results on Change Management Level revealed that students found the simulation quite challenging. None achieved beyond the lowest level (i.e., Apprentice) on their baseline simulation session played at home (see Figure 5). This result meant that the students could not move a significant number of stakeholders beyond the early stages of the change process, and failed to gain even a low number of Bennies. Indeed, a majority of the students failed to achieve beyond the Novice stage (i.e., the 2nd of 6 stages) during the first week. These results reflected a limited ability to conceptualize the change process, execute a successful change strategy, or apply their prior experience and knowledge to the simulation problem.

Figure 5.

Week-by-Week Distribution of Change Management Levels Achieved by Students.

There was, however, a pattern of continuous improvement during the three-week module. By the end of the first week, 14 students had reached the Manager stage or higher (see Figure 5). This improvement was associated with the experience of playing the simulation multiple times. In addition, students appeared to benefit from structured reflection via the Online Discussion Forum, where they actively shared strategies for overcoming problems they were encountering in the simulation.

By the end of the second week, 22 of the 32 students had advanced at least as far as the Leader level (see Figure 5). Achieving the Leader level requires the execution of a change strategy that moves a majority of the stakeholders to the Sustainability Stage and yields a substantial number of Bennies (e.g., >5,000). From a skill perspective, the Leader level represented the proficiency standard desired by the instructor. These results were aligned with students’ continued engagement with the simulation during the second week.

By the end of the third week, 30 of the 32 students had reached the proficiency standard (i.e., Leader Level); 27 students reached the Change Master level. Results on the final exam largely mirrored the third-week results. This was expected since the exam was given only one day after most students had completed their last third-week simulation sessions. Nonetheless, the consistency between the week three result played by students at home and the exam result played with a time limit in class offered useful verification of the reliability of this measure of skill performance.

While these patterns suggested meaningful change in students’ skills, the statistical significance of these observed changes required confirmation through inferential analysis. In the first test, Mauchly's test revealed a violation of the sphericity assumption, with a significant result (Maunchly’s W = 0.135; Chi Square = 58.918; df = 9; p < 0.001. Therefore, we applied the Greenhouse-Geisser correction to adjust the degrees of freedom, mean square, and p-value. The results were significant (F = 128.738, p < 0.001).

Next, one-way repeated measures ANOVA was executed to assess the significance of week-to-week differences in the Change Management Levels achieved by students (see Table 2). All paired comparisons were significantly different, except for the result between week three and the final exam where the measures were only separated by a single day. Moreover, because most students were already achieving at the Expert or Change Master level by the end of the third week, a ceiling effect limited the scope of additional change.

Table 2.

Pairwise Comparisons of Change Management Levels Achieved by the Students Week by Week.

(I) factor1	(J) factor1	Mean Difference (I-J)	Std. Error	Sig.^a	99.9% Confidence Interval for Difference^a
(I) factor1	(J) factor1	Mean Difference (I-J)	Std. Error	Sig.^a	Lower Bound	Upper Bound
Baseline	Week 1	-1.562***	.304	.000	-2.920	-.205
	Week 2	-3.500***	.298	.000	-4.829	-2.171
	Week 3	-4.656***	.166	.000	-5.395	-3.917
	Final test	-4.750***	.119	.000	-5.280	-4.220
Week 1	Baseline	1.563***	.304	.000	.205	2.920
	Week 2	-1.937***	.308	.000	-3.310	-.565
	Week 3	-3.094***	.319	.000	-4.515	-1.672
	Final test	-3.187***	.309	.000	-4.567	-1.808
Week 2	Baseline	3.500***	.298	.000	2.171	4.829
	Week 1	1.938***	.308	.000	.565	3.310
	Week 3	-1.156***	.254	.001	-2.291	-.021
	Final test	-1.250***	.266	.000	-2.435	-.065
Week 3	Baseline	4.656***	.166	.000	3.917	5.395
	Week 1	3.094***	.319	.000	1.672	4.515
	Week 2	1.156***	.254	.001	.021	2.291
	Final test	-.094	.113	1.000	-.599	.411
Final test	Baseline	4.750***	.119	.000	4.220	5.280
	Week 1	3.188***	.309	.000	1.808	4.567
	Week 2	1.250***	.266	.000	.065	2.435
	Week 3	.094	.113	1.000	-.411	.599

Based on estimated marginal means.

***. The mean difference is significant at the .001 level.

^aAdjustment for multiple comparisons: Bonferroni.

Results of the Shapiro-Wilk test indicated that the data were not normally distributed (Final Exam: Calc W = 0.436; p = 0.000). Thus, the researchers conducted a Friedman Rank Test for differences in medians as an additional check of the robustness of the ANOVA results. These results were also significant (Chi-square = 103.410; df = 4; p < 0.001). Thus, we could conclude with confidence that the week-by-week changes in student skills in formulating and executing change strategies for sustainability shown in Figure 5 were statistically significant.

Knowledge Application: Enactment of Change Principles

As noted earlier, seven change principles were tracked for each simulation session played by the students. The enactment of these principles in the students’ change strategies was used as a measure of their ability to apply knowledge of change management principles to the sustainability challenge. During their baseline sessions, the students, on average, failed to enact any of Kotter's (2012) change management principles in their strategies (see Figure 6).

Figure 6.

The Week-by-Week Distribution of Kotter Principles Enacted in Students’ Change Strategies.

By the end of the first week, their performance on this criterion had begun to improve (i.e., M = 2.09 of the 7 principles enacted). Although Kotter’s change principles had not yet been taught in the module, students were beginning to arrive at some of the principles through trial-and-error learning. As the students began to learn from their mistakes, they drew on their prior knowledge to make sense of what worked and what did not. This reflection was enhanced by collective problem-solving among the students and with the instructor in the Online Discussion Forum.

A similar process of gradual improvement continued in the second and third weeks (see Figure 6). Kotter’s principles were taught explicitly in the third-class meeting, where the instructor prompted students to brainstorm how they could apply these principles in their simulation strategies. On the final simulation exam, 27 of the 32 participants (84%) incorporated at least four change principles into their strategy.

Again, inferential analysis was employed to determine whether the week-to-week changes in Figure 6 were statistically significant. Mauchly's Test confirmed that the sphericity assumption had not been violated, indicating that ANOVA could be run without further correction. Results of the ANOVA (see Table 3) affirmed that week-to-week changes in the average number of change management principles enacted by the students were statistically significant (p < 0.001). The only exception was the difference between the week three result and the final exam, again explained by the single-day gap between the two measurements. The Friedman Rank Test yielded a significant result (Chi-square = 96.976; df = 4; p < 0.001). This reaffirmed that students were continuously developing their ability to incorporate change principles into their strategies.

Table 3.

Pairwise Comparisons of the Number of Kotter Change Principles Enacted Week by Week.

(I) factor1	(J) factor1	Mean Difference (I-J)	Std. Error	Sig.^a	99.9% Confidence Interval for Difference ^a
(I) factor1	(J) factor1	Mean Difference (I-J)	Std. Error	Sig.^a	Lower Bound	Upper Bound
Baseline	Week 1	-2.031^***	.293	.000	-3.337	-.726
	Week 2	-3.406^***	.224	.000	-4.405	-2.408
	Week 3	-4.656^***	.236	.000	-5.709	-3.604
	Final test	-4.437^***	.250	.000	-5.552	-3.323
Week 1	Baseline	2.031^***	.293	.000	.726	3.337
	Week 2	-1.375^***	.297	.001	-2.700	-.050
	Week 3	-2.625^***	.310	.000	-4.009	-1.241
	Final test	-2.406^***	.323	.000	-3.849	-.964
Week 2	Baseline	3.406^***	.224	.000	2.408	4.405
	Week 1	1.375^***	.297	.001	.050	2.700
	Week 3	-1.250^***	.215	.000	-2.211	-.289
	Final test	-1.031^**	.282	.009	-2.290	.227
Week 3	Baseline	4.656^***	.236	.000	3.604	5.709
	Week 1	2.625^***	.310	.000	1.241	4.009
	Week 2	1.250^***	.215	.000	.289	2.211
	Final test	.219	.214	1.000	-.736	1.174
Final test	Baseline	4.438^***	.250	.000	3.323	5.552
	Week 1	2.406^***	.323	.000	.964	3.849
	Week 2	1.031^**	.282	.009	-.227	2.290
	Week 3	-.219	.214	1.000	-1.174	.736

Based on estimated marginal means.

**. The mean difference is significant at the .01 level.

***. The mean difference is significant at the .001 level.

^aAdjustment for multiple comparisons: Bonferroni.

These statistical results suggest that students were becoming more sophisticated in their ability to apply change management principles to the sustainability challenge. This was also evident in student posts to the Online Discussion Forum. Their posts emphasized the higher levels of knowledge attainment gained through learning with the simulation (e.g., application, analysis, synthesis).

My own experience is engaging in simulation made me think more systematically i.e., taking notes about characters, and their positions. I examined the activities, organized the strategy, and played many times. After each time, I draw lessons and revised the strategy (for example which activity should be undertaken before another, how to get more bennies). Besides, I joined the Connection Forum, and discussed different approaches and strategies with my classmates and instructor. That gave me more ideas about how to improve my strategy. From that, I was able to develop my own successful strategy. (P3)

In the psychology course, I learned about formal and informal relationships in the organization. But through the simulation, I see more clearly the importance of understanding informal relationships for leaders to influence people and succeed in change. (P21)

Before learning with simulation, I did a green school project following the ‘trend’. I saw the benefits and tried to perfect that project. After learning with simulation, I can see that I missed many opportunities to involve teachers and staff in creating a sustainable culture at school. I acknowledge that sustainable practices must be demonstrated and values communicated effectively to engage and involve others. But in fact I have not done it yet. We have to use sustainable practices and show people their value. I have to increase their awareness before moving to higher level sustainability practices. (P4)

Discussion

This article described the design and pilot evaluation of a simulation-based learning module organized around the Vietnamese language version of the Leading Change for Sustainability in Schools simulation (LCSS-V). In this section, we identify the limitations of this research, provide our interpretation of the findings, and highlight implications for research and practice.

Limitations

First, the sample size in this study was admittedly small (i.e., 32 educators), and the convenience sample did not include all stakeholder groups for whom the simulation might be relevant. This implies a need for additional studies that incorporate larger, more representative samples. It would also be useful to evaluate the results in other training contexts (e.g., professional development workshops).

Second, as with many studies of educational interventions, it was not possible to include a control group in the setting where the researchers were collecting data. This means that it cannot be determined whether a different learning approach would have been as successful or achieved similar results over a shorter time. Nonetheless, the quasi-experimental design used in the study could determine that learning had taken place (Campbell & Stanley, 2015).

Interpretation of the Findings

This study set out to examine the effects of a simulation-based learning module that was organized and delivered in line with the principles of experiential learning. The learning sequence used in the study is shown in Figure 7

Figure 7.

LCSS Learning Process Model.

. Students used the simulation as a means by which they could gain “experience” in leading change for sustainability. Continuous feedback on results was provided in the simulation through multiple means (e.g., dialogue boxes, dashboard display) and metrics (e.g., stakeholder progress through the stages of change, Bennies, Change Management Level, change principles enacted). This “real-time feedback” prompted students to “reflect” on “what happened and why”. Subsequently, they refined their strategies as they continued to play in class and at home.

The instructor played a key role each week. Instructor activities included providing “just-in-time input” on change management and sustainability theories and practices, leading the class in structured debriefings, and responding to learner queries in the weekly discussion forum. Thus, the experiential learning cycle was fulfilled both in class and at home.

The results of the analyses presented suggest that the LCSS-V simulation-based learning module was not only highly engaging, but also produced impressive learning outcomes. More specifically, the simulation-based learning module helped these educators improve their skills in applying theory to practice, and executing effective strategies designed to transform schools for sustainability. This conclusion was supported by the results of the final simulation exam.

Twenty-eight of the 32 participants reached the level of Change Master (i.e., the highest level) on the final simulation exam. The degree of learner growth was evidenced by the fact that none of the participants could achieve beyond the lowest level on either of the measured learning outcomes in their baseline attempts. Both descriptive and inferential analyses affirmed that strategy formulation and knowledge application gains were strongly associated with the treatment.

Notably, these results were also consistent with the results of a pre/post test of knowledge which found significant meaningful improvements in knowledge of change management concepts associated with the intervention (not tabled). Only through a cyclical process, consisting of multiple iterations of independent practice and structured reflection, could the students succeeded in meeting the sustainability challenge presented in the simulation. The data further affirmed that a majority of the students continued to play the simulation even after reaching the Change Master level during the second and third weeks. This suggests that they were motivated, at least in part, by the intrinsic satisfaction of meeting the challenge of formulating a more effective change strategy.

Qualitative data supported these findings. More specifically, students who attained the Change Master Level were still curious to see if they could gain even more Bennies, move more stakeholders into the Sustainability stage, and/or incorporate more of Kotter’s principles in their change strategies. This was the case even though it would not earn them a higher grade.

The findings from this study are consistent with results from a larger study of the effectiveness of the Leading Change for Sustainability–Business simulation conducted in Thailand (Chatpinyakoop, 2023). Comparison of the results from the two studies is viable due to the fact that the structure of the simulations, the instructional design, and the duration of the treatment were closely aligned. The current study found that students played the simulation an average of 24 times during the three-week intervention. The intervention in Chatpinyakoop’s study with 87 students yielded an average of 22 simulation sessions played per student. Both studies found similar patterns with respect to knowledge application and skill development, though the changes reported in Chatpinyakoop’s were even larger than those of the current study. The fact that both studies found significant, meaningful effects of simulation-based learning on learner engagement, skills, and knowledge application strengthens confidence in the results reported in this article.

Finally, while it was not emphasized in this article, we also wish to call attention to the context in which the main field test was conducted. The simulation was used with Vietnamese educators, few of whom had prior knowledge about sustainability and change management, or experience learning with a simulation. Moreover, it was also the first time that the instructor had taught with a simulation. This suggests that future results could be stronger.

Implications of the Findings

First, it should be reiterated that the R&D process reported in this article yielded two versions of the LCSS simulation. While the empirical research in this article focused on the Vietnamese language version of the LCSS simulation (i.e., LCSS-V), similar studies are planned to validate the English language version. This is a priority in that the English language version will have broader applicability in terms of dissemination.

A second implication lies in the R&D process used to develop the LCSS-V simulation for the Vietnamese cultural context. More specifically, the R&D process used to adapt the LCSS simulation for use in Vietnam could inform the localization of other simulations for different national contexts (see Nguyen & Hallinger, 2022). The redesign of a pre-existing simulation (i.e., Leading Change for Sustainability in Business) for use in the K-12 education context was completed in two distinct steps.

In the first step, the authors revised the organizational context from a corporate setting to that of a K-12 school system. Following the development of a generic LCSS simulation in English, the researchers identified unique features of the structure and culture of the Vietnamese education system and build these into the LCSS-V simulation. For example, school principals in Vietnam report not only to superiors in the Ministry of Education and Training, but also to the Communist Party (Hallinger & Truong, 2016). A simulation that did not take this and other unique features of the educational context into account could lead prospective users to dismiss the simulation as lacking authenticity. Since the effectiveness of simulations is grounded to a significant degree on user perceptions of authenticity (Littlewood et al., 2013; Rystedt & Sjöblom, 2012), this contextual adaptation has implications for others' efforts to localize simulations for maximum impact (Nguyen & Hallinger, 2022).

A third implication lies in the data collection capacity of the LCSS simulations as research tools. In this study, a wide variety of useful data on the learning process and outcomes were collected via the simulation software (Showanasai et al., 2013). The growth of online simulations represents a unique opportunity to obtain powerful research data that would be far more difficult to collect during a typical face-to-face classroom intervention. We encourage other educators to explore the development of similar research capabilities in their own simulations.

A fourth implication can be drawn from the integration of several active learning methods in the instructional design of the intervention. Specifically, the instructional intervention intentionally incorporated principles from problem-based learning (Barrows & Tamblyn, 1980), experiential learning (Kolb, 1984), simulation-based learning (Phillips & Graeff, 2014; Salas et al., 2009), and collaborative learning (Hadwin et al., 2017). This is consistent with research which suggests that educators are increasingly employing instructional repertoires that feature the integration of multiple active learning pedagogies (Davidson & Major, 2014). Our findings offer additional support for this trend.

Fifth, the research offers additional empirical support for the potential of using well-designed simulations outside of Anglo-European contexts. Numerous obstacles impede the use of active learning methods in Vietnamese schools (Kieu et al., 2016; Nguyen, 2019). Nonetheless, this study found that these Vietnamese teachers responded enthusiastically to learning via the simulation. This experience as students learning with a simulation “opened their eyes” to the possibilities of active learning. Their reflections on this learning experience further suggest that using simulation-based learning in teacher training could have the unanticipated effect of reducing resistance to non-traditional learning methods in Vietnamese schools. Moreover, it was noted that the instructor teaching the LCSS-V module was himself a novice in teaching with simulations. Thus, the research offers encouragement for the use of simulations by educators in cultural contexts that have traditionally relied on didactic approaches to teaching and learning.

We close with a final research recommendation. Future research on the effects of this simulation should examine the “knock-on effects” beyond the training setting. More specifically, to what extent does learning with the simulation change educators’ attitudes toward the use of active learning methods in their classrooms and schools? Achieving success in this unexamined domain could represent another critical outcome of using simulations in the education of educators. That would also reduce the subsequent challenges of integrating sustainability concepts and practices into the classrooms and schools of Vietnam and other societies (Kieu et al., 2016).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Thailand Sustainable Development Foundation.

ORCID iD

Philip Hallinger

Data Availability Statement

The data from this study can be accessed by interested researchers at: . If used, please contact the authors for permission and additional information that could be needed for coding.

Author Biographies

Vien-Thong Nguyen is Lecturer and Head of Department in the Faculty of Education, University of Scoial Sciences and Humanities, National Vietnam University, Ho Chi Minh City, Vietnam. He received his PhD in Sustainable Leadership from the College of Management, Mahidol University.

Dr. Philip Hallinger is the Global Talent Professor of Management in the College of Management, Mahidol University (Thailand) and Distinguished Visiting Professor in the Department of Educational Leadership and Management, University of Johannesburg (South Africa).

Dr. Parinya Showanasai is an Adjunct Faculty member in the College of Management, Mahidol University (Thailand) and formerly Assistant Professor of Management at the Prince of Songkhla University (Thailand).

References

Barrows

H. S.

Tamblyn

R. M.

(1980). Problem-based learning: An approach to medical education (Vol. 1). Springer Publishing Company.

Benn

Edwards

Williams

(2014). Organizational change for corporate sustainability. Routledge.

Campbell

D. T.

Stanley

J. C.

(2015). Experimental and quasi-experimental designs for research: Ravenio Books.

Chatpinyakoop

(2023). Higher education for sustainable development: Testing the efficacy of online simulation-based learning in management education. Unpublished doctoral dissertation, Mahidol University, Bangkok, Thailand.

Chatpinyakoop

Hallinger

Showanasai

(2022). Developing capacities to lead change for sustainability: A quasi-experimental study of simulation-based learning. Sustainability, 14(17), 10563.

Choon‐Eng Gwee

(2008). Globalization of problem‐based learning (PBL): Cross‐cultural implications. The Kaohsiung Journal of Medical Sciences, 24, S14–S22.

Clifton

Amran

(2011). The stakeholder approach: A sustainability perspective. Journal of Business Ethics, 98, 121–136.

Crandall

D. P.

Eiseman

J. W.

Louis

K. S.

(1986). Strategic planning issues that bear on the success of school improvement efforts. Educational Administration Quarterly, 22(3), 21–53.

Dankbaar

M. E.

Alsma

Jansen

E. E.

van Merrienboer

J. J.

van Saase

J. L.

Schuit

S. C.

(2016). An experimental study on the effects of a simulation game on students’ clinical cognitive skills and motivation. Advances in Health Sciences Education, 21, 505–521.

10.

Davidson

Major

C. H.

(2014). Boundary crossings: Cooperative learning, collaborative learning, and problem-based learning. Journal on Excellence in College Teaching, 25, 11.

11.

De la Torre

Onggo

B. S.

Corlu

C. G.

Nogal

Juan

A. A.

(2021). The role of simulation and serious games in teaching concepts on circular economy and sustainable energy. Energies, 14(4), 1138.

12.

Dempsey

Bramley

Power

Brown

(2011). The social dimension of sustainable development: Defining urban social sustainability. Sustainable Development, 19(5), 289–300.

13.

Dharmapiya

Saratun

(2015). Cultivating a “sufficiency” mindset: Thailand’s educational strategy for a sustainable society. Thailand Sustainable Development Foundation.

14.

Doppelt

McDonough

(2017). Leading change toward sustainability: A change-management guide for business, government and civil society. Routledge.

15.

Doyle

Brown

F. W.

(2000). Using a business simulation to teach applied skills–The benefits and the challenges of using student teams from multiple countries. Journal of European Industrial Training, 24(6), 330–336.

16.

Elkington

(2013). Enter the triple bottom line. In Henriques

Richardson

(Eds.), The triple bottom line: Does it all add up? (pp. 23–38). Routledge.

17.

Frambach

J. M.

Driessen

E. W.

Beh

Van der Vleuten

C. P.

(2014). Quiet or questioning? Students' discussion behaviors in student-centered education across cultures. Studies in Higher Education, 39(6), 1001–1021.

18.

Franklin

C. L.

II (2013). Developing expertise in management decision-making. Academy of Strategic Management Journal, 12(1), 21.

19.

Fullan

(2015). The new meaning of educational change. Teachers College Press.

20.

Gatti

Ulrich

Seele

(2019). Education for sustainable development through business simulation games: An exploratory study of sustainability gamification and its effects on students' learning outcomes. Journal of Cleaner Production 207, 667–678.

21.

Geissdoerfer

Savaget

Bocken

N. M.

Hultink

E. J.

(2017). The circular economy–A new sustainability paradigm? Journal of Cleaner Production, 143, 757–768.

22.

Grant

A. M.

(2010). It takes time: A stages of change perspective on the adoption of workplace coaching skills. Journal of Change Management, 10(1), 61–77.

23.

Hadwin

Järvelä

Miller

(2017). Self-regulation, co-regulation, and shared regulation in collaborative learning environments. In Zimmerman

D. J.

Schunk

(Eds.), Handbook of self-regulation of learning and performance (pp. 83–106). Routledge.

24.

Hall

G. E.

Hord

S. M.

(2006). Implementing change: Patterns, principles, and potholes. Pearson/Allyn and Bacon.

25.

Hallinger

Showanasai

Chatpinyakoop

Nguyen Vien

(2022). Leading Change for Sustainability-Business: Design and Evaluation of a Sustainability Simulation for Management Education. Available at SSRN 4259730.

26.

Hallinger

Truong

(2016). “Above must be above, and below must be below”: Enactment of relational school leadership in Vietnam. Asia Pacific Education Review, 17(4), 677–690.

27.

Hallinger

Wang

(2020a). The evolution of simulation-based learning across the disciplines, 1965–2018: A science map of the literature. Simulation & Gaming, 51(1), 9–32.

28.

Hallinger

Wang

Chatpinyakoop

Nguyen

V. T.

Nguyen

U. P.

(2020b). A bibliometric review of research on simulations and serious games used in educating for sustainability, 1997–2019. Journal of Cleaner Production, 256, 120358.

29.

Henderson

Tilbury

(2004). Whole-school approaches to sustainability: An international review of sustainable school programs. Australian Government.

30.

Hinton

P. R.

Brownlow

McMurray

Cozens

(2014). SPSS explained. Routledge.

31.

Hoang

L. C.

Thi

H. N. V.

Thi

N. Q. N.

Nhat

T. T. B.

Le Nguyen

Y. N.

(2022). The impact of business simulation games on Vietnamese students’ entrepreneurial intention. VNUHCM Journal of Economics, Business and Law, 6(4), 3574–3588.

32.

Hoang

T. T. P.

Kato

(2016). Measuring the effect of environmental education for sustainable development at elementary schools: A case study in Da Nang city, Vietnam. Sustainable Environment Research, 26(6), 274–286.

33.

Huang

Y.-M.

Silitonga

L. M.

Murti

A. T.

T.-T.

(2022). Learner engagement in a business simulation game: Impact on higher-order thinking skills. Journal of Educational Computing Research, 1(1), 11.

34.

Imlig-Iten

Petko

(2018). Comparing serious games and educational simulations: Effects on enjoyment, deep thinking, interest and cognitive learning gains. Simulation & Gaming, 49(4), 401–422.

35.

Katsaliaki

Mustafee

(2015). Edutainment for sustainable development: A survey of games in the field. . Simulation & Gaming, 46(6), 647–672.

36.

Kieu

T. K.

Singer

Gannon

T. J.

(2016). Education for sustainable development in Vietnam: Lessons learned from teacher education. International Journal of Sustainability in Higher Education, 17(6), 853–874.

37.

Kolb

D. A.

(1984). Experiential learning. Prentice Hall.

38.

Kotter

J. P.

(2012). Leading change. Harvard Business Review Press.

39.

Leal Filho

Manolas

Pace

(2015). The future we want: Key issues on sustainable development in higher education after Rio and the UN decade of education for sustainable development. International Journal of Sustainability in Higher Education, 16(1), 112–129.

40.

Littlewood

K. E.

Shilling

A. M.

Stemland

C. J.

Wright

E. B.

Kirk

M. A.

(2013). High-fidelity simulation is superior to case-based discussion in teaching the management of shock. Medical Teacher, 35(3), e1003–e1010.

41.

Lozano

Garcia

(2020). Scrutinizing sustainability change and its institutionalization in organizations. Frontiers in Sustainability, 1, 1.

42.

Lozano

Merrill

M. Y.

Sammalisto

Ceulemans

Lozano

F. J.

(2017). Connecting competencies and pedagogical approaches for sustainable development in higher education: A literature review and framework proposal. Sustainability, 9(10), 1889.

43.

Hallinger

Showanasai

(2014). Simulation-based learning in management education: A longitudinal quasi-experimental evaluation of instructional effectiveness. Journal of Management Development, 33(3), 218–244.

44.

MacDonald

(2009). Balancing priorities and measuring success: A triple bottom line framework for international school leaders. Journal of Research in International Education, 8(1), 81–98.

45.

McGaghie

W. C.

Issenberg

S. B.

Petrusa

E. R.

Scalese

R. J.

(2010). A critical review of simulation‐based medical education research: 2003–2009. Medical Education, 44(1), 50–63.

46.

Mogensen

Schnack

(2010). The action competence approach and the ‘new’ discourses of education for sustainable development, competence and quality criteria. Environmental Education Research, 16(1), 59–74.

47.

National Academies of Sciences, Engineering, and Medicine (NASEM) . (2018). How people learn II: Learners, contexts, and cultures. National Academies Press.

48.

Nguyen

V. T.

Hallinger

(2022). Creating the Leading Change for Sustainability in Schools (Vietnam) simulation: A research and development project. Developments in Business Simulation and Experiential Learning: Proceedings of the Annual ABSEL conference, (Vol. 49).

49.

Nguyen

A. N.

Nguyen

T. P.

Kieu

K. T.

Nguyen

Y. T. H.

Dang

D. T.

Singer

Lambrechts

(2022a). Assessing teacher training programs for the prevalence of sustainability in learning outcomes, learning content and didactic approaches. Journal of Cleaner Production, 365, 132786.

50.

Nguyen

L. H. P.

Bui

N. B. T.

Nguyen

T. N. C.

Huang

C. F.

(2022b). An investigation into the perspectives of elementary pre-service teachers on sustainable development. Sustainability, 14(16), 9943.

51.

Nguyen

T. N.

(2015). Motivational effect of web-based simulation game in teaching operations management. Journal of Education and Training Studies, 3(2), 9–15.

52.

Nguyen

T. P.

(2019). Searching for education for sustainable development in Vietnam. Environmental Education Research, 25(7), 991–1003.

53.

Padgett

Cristancho

Lingard

Cherry

Haji

(2019). Engagement: What is it good for? The role of learner engagement in healthcare simulation contexts. Advances in Health Sciences Education, 24, 811–825.

54.

Phillips

M. E.

Graeff

T. R.

(2014). Using an in-class simulation in the first accounting class: Moving from surface to deep learning. Journal of Education for Business, 89(5), 241–247.

55.

Prado

A. M.

Arce

Lopez

L. E.

García

Pearson

A. A.

(2020). Simulations versus case studies: Effectively teaching the premises of sustainable development in the classroom. Journal of Business Ethics, 161(2), 303–327.

56.

Pringle

J. D.

Nippak

Isaac

(2010). Simulations as an instruction tool to teach healthcare change management: The influence of repeat simulation testing, gender and additional instructions on performance scores? Journal of Health Administration Education, 27(1), 27–43.

57.

Rogers

E. M.

(2010). Diffusion of innovations. Simon and Schuster.

58.

Rystedt

Sjöblom

(2012). Realism, authenticity, and learning in healthcare simulations: Rules of relevance and irrelevance as interactive achievements. Instructional Science, 40(5), 785–798.

59.

Salas

Rosen

M. A.

Held

J. D.

Weissmuller

J. J.

(2009). Performance measurement in simulation-based training: A review and best practices. Simulation & Gaming, 40(3), 328–376.

60.

Salas-Zapata

W. A.

Ríos-Osorio

L. A.

Cardona-Arias

J. A.

(2018). Knowledge, attitudes and practices of sustainability: Systematic review 1990–2016. Journal of Teacher Education for Sustainability, 20(1), 46–63.

61.

Senge

Hamilton

Kania

(2015). The dawn of system leadership. Stanford Social Innovation Review, 13(1), 27–33.

62.

Showanasai

Hallinger

(2013). Developing tools for research on school leadership development: An illustrative case of a computer simulation. Journal of Educational Administration, 51(1), 72–91.

63.

H. Y.

Chen

P. P.

Wong

G. K. C.

Chan

T. T. N.

(2019). Simulation in medical education. Journal of the Royal College of Physicians of Edinburgh, 49(1), 52–57.

64.

Stanitsas

Kirytopoulos

Vareilles

(2019). Facilitating sustainability transition through serious games: A systematic literature review. Journal of Cleaner Production, 208, 924–936.

65.

Steadman

R. H.

Coates

W. C.

Huang

Y. M.

Matevosian

Larmon

B. R.

McCullough

Ariel

(2006). Simulation-based training is superior to problem-based learning for the acquisition of critical assessment and management skills. Critical Care Medicine, 34(1), 151–157.

66.

Sternberg

R. J.

(1998). Metacognition, abilities, and developing expertise: What makes an expert student? Instructional Science, 3, 127–140.

67.

Sroufe

(2017). Integration and organizational change towards sustainability. Journal of Cleaner Production, 162, 315–329.

68.

Svoboda

Whalen

, et al. (2017). Using experiential simulation to teach sustainability. In Galea

(Ed.), Teaching Business Sustainability: Cases, Simulations and Experiential Approaches (2, pp. 171–179). London: Routledge.

69.

Tilbury

Cooke

(2005). A national review of environmental education and its contributions to sustainability in Australia: Frameworks for sustainability. Canberra: Australian Research Institute in Education for Sustainability.

70.

UNESCO . (2020). Education for sustainable development: A roadmap. UNESCO.

71.

Verkuyl

Hughes

(2019). Virtual gaming simulation in nursing education: A mixed-methods study. Clinical Simulation in Nursing, 29, 9–14.

72.

Vlachopoulos

Makri

(2017). The effect of games and simulations on higher education: a systematic literature review. International Journal of Educational Technology in Higher Education, 14(1), 1–33.