Sage Journals: Discover world-class research

Abstract

Background. We describe and provide an initial evaluation of the Climate Action Simulation, a simulation-based role-playing game that enables participants to learn for themselves about the response of the climate-energy system to potential policies and actions. Participants gain an understanding of the scale and urgency of climate action, the impact of different policies and actions, and the dynamics and interactions of different policy choices.

Intervention. The Climate Action Simulation combines an interactive computer model, En-ROADS, with a role-play in which participants make decisions about energy and climate policy. They learn about the dynamics of the climate and energy systems as they discover how En-ROADS responds to their own climate-energy decisions.

Methods. We evaluated learning outcomes from the Climate Action Simulation using pre- and post-simulation surveys as well as a focus group.

Results. Analysis of survey results showed that the Climate Action Simulation increases participants’ knowledge about the scale of emissions reductions and policies and actions needed to address climate change. Their personal and emotional engagement with climate change also grew. Focus group participants were overwhelmingly positive about the Climate Action Simulation, saying it left them feeling empowered to make a positive difference in addressing the climate challenge.

Discussion and Conclusions. Initial evaluation results indicate that the Climate Action Simulation offers an engaging experience that delivers gains in knowledge about the climate and energy systems, while also opening affective and social learning pathways.

Keywords

climate change energy simulation role-play games for change experiential learning

Background

A rapid transition to a low-carbon, climate-resilient society is not only possible, but could also bring many co-benefits for public health, economic wellbeing, and social equity (Intergovernmental Panel on Climate Change [IPCC], 2018). The scientific imperative for such a transition is clear (IPCC, 2018) and has been formally recognized by the international community through the Paris Agreement (Nuttall, 2016), which calls for limiting global warming to ‘well below’ 2 °C above pre-industrial times (UN Framework Convention on Climate Change [UNFCCC], 2015). Yet, nations’ pledges to the Paris Accord fall short (R. B. Jackson et al., 2018; Jiang et al., 2019; Rogelj et al., 2016), current policies are insufficient to achieve pledged cuts in greenhouse gas (GHG) emissions (Clémençon, 2016), and even those policies may not be upheld (Jacquet & Jamieson, 2016). While public concern about climate change is rising, most people do not view climate change as a serious threat to themselves, and the issue has become politically polarized (McCright & Dunlap, 2011; McCright et al., 2014; Saad & Jones, 2016; Stokes et al., 2015). Social science research has revealed many causes of the wide gap between scientific and public understanding of the climate-energy challenge, including: common misconceptions about climate change and complex systems more generally (e.g., Sterman & Sweeney, 2002); deliberate efforts to misinform the public and stall action (Elsasser & Dunlap, 2013; Oreskes & Conway, 2010); human tendency to discount future impacts (Weber, 2017); and social and cultural barriers to learning and acting on climate change (Doherty & Webler, 2016; Kahan et al., 2012). Effective action on climate change would require rapid and widespread changes to our energy system, economy, and decision-making at all levels of society (IPCC, 2018). Yet, key aspects of the energy system are also poorly understood by most people, including the climate impact of different energy sources (Lee, 2016), the climate impact of bioenergy (Sterman et al., 2018a, 2018b), the rate at which new technologies could transform the energy system (Smil, 2010), the importance of energy efficiency and demand reduction, and the policies needed to foster an energy transition (Lee, 2016; Smil, 2010; Steg et al., 2015). Here, we describe the Climate Action Simulation (formerly ‘World Energy’) an interactive simulation-based role-play designed to inform and engage participants about the transition to a low carbon economy needed to address climate change.

Learning and social sciences have pointed to several approaches to climate and energy communication that hold promise for motivating science-informed action (e.g., Eisenack & Reckien, 2013; Reckien & Eisenack, 2013; Stave et al., 2015). For example, interactive computer models can help overcome cognitive challenges posed by the dynamic behavior of complex systems in general and of the climate-energy system in particular (Sterman, 2011, 2012). Computer models offer a means to compress time and space and enable iterative experimentation when it would otherwise be costly, risky, or, as in the case of climate change, impossible (Sterman, 2012).

Simulation-based role-playing games combine interactive computer models with engaging and social role-playing scenarios, offering a promising approach for climate and energy communication (Crookall, 2010). In the Climate Action Simulation, the term ‘simulation’ has a dual meaning, referring to an interactive computer model that simulates the physical climate and energy systems, as well as to a role-play that simulates the social dynamics of decision-making. Simulations of this sort can be powerful educational tools: they foster deep learning about climate change and sustainability that motivates action by simultaneously accessing analytic, affective, and social learning pathways (Eisenack & Reckien, 2013; Kwok, 2019; Meya & Eisenack, 2018; Reckien & Eisenack, 2013; Rooney-Varga et al., 2018).

In the Climate Action Simulation, participants learn about the dynamics of the climate-energy system by simulating the climate and energy outcomes of their own decisions with the interactive computer model En-ROADS. They also engage in the social dynamics of climate and energy decision-making in a role-play simulation. We explain both the computer and role-play simulations, a typical sequence to conduct the simulation, preliminary evaluation of the tool, and future directions.

Intervention: The Climate Action Simulation

The Climate Action Simulation is designed to help people understand the complexity of the interactions between the climate and energy systems, related policies and assumptions, and some of the social dynamics of climate and energy decision-making while fulfilling Mayer’s (2009) requirements for games to support policy-making. Participants take on the roles of leaders representing different sectors of the economy and governments charged with creating a global energy transition that meets the international goal of limiting warming to well below 2 °C above preindustrial levels. The Climate Action Simulation exposes the causes of ongoing climate change; the scale and urgency of mitigation needed to meet climate goals; expected climate impacts such as global temperature rise, sea level rise, and ocean acidification; and the impacts of different mitigation approaches. More specifically, the simulation is designed to explore: the accumulation of GHGs in the atmosphere and the resulting need for early and strong action to reduce emissions; necessary dynamics of energy-related capital and the limitations they impose on energy transition rates; the importance of regulating CO₂ and other GHGs such as N₂O, CH₄, and F gases; the impact of population and GDP on energy consumption; and the relative leverage and interactions of mitigation approaches (e.g., technological breakthroughs and policies that reduce the cost of low-carbon energy sources also generate moderate rebound effects, increasing energy demand and dampening their beneficial effects). The social dynamics of climate and energy decision-making, such as the resistance of policymakers and others to take action commensurate with the challenge, are explored through the role-play simulation.

En-ROADS: An Interactive Computer Model for Energy Policy and Learning

The Climate Action Simulation is framed by the En-ROADS, or Energy Rapid Overview and Decision Support, computer model (Figure 1). En-ROADS enables users to interactively build energy and societal transition scenarios, providing immediate feedback on the effects of policies and decisions on energy supply, GHG emissions, and expected global climate outcomes over the twenty-first century. It builds on elements of the Kaya identity, which states that CO₂ emissions from fossil fuels are the product of population, GDP per capita, energy use per unit GDP, and carbon intensity per unit energy use (Nakićenović et al., 2000). En-ROADS model sectors include global energy demand, energy supply, GHG emissions (from fossil fuels and other anthropogenic sources), the carbon cycle, and the climate system. The carbon cycle and climate system sectors are drawn from an externally reviewed simple climate model (IPCC, 2013), C-ROADS (Sterman et al., 2012, 2013). The economic and energy capital sectors of En-ROADS generate GHG emissions, which force climate outcomes that are determined by carbon cycle and climate system model sectors (Siegel et al., 2018).

Figure 1.

Screenshot of En-ROADS, showing the main control panels and graphs of the energy mix and expected global temperature outcomes through 2100. Note that many other levers and graphs are available to the user (see: https://www.climateinteractive.org/tools/climate-action-simulation/).

En-ROADS enables users to test policies and assumptions about many aspects of energy demand, supply, and carbon intensity. These include the growth rate of population and GDP; the efficiency of energy-using capital; the potential for new, climate-neutral energy technologies; accelerated retirement of coal plants; mandated emissions standards; economic incentives or penalties for using different energy sources (e.g., an economy-wide price on carbon emissions and subsidies or prices for a particular energy source); regulation of non-CO₂ GHGs; and emissions associated with land use change. En-ROADS incorporates data, structure and equations from EIA, Hyde, WEO, BP, EMF22 models, EMF27 models, and the RCP and SRES scenarios (BP, 2014; Krey et al., 2014; Nakićenović et al., 2000; Vuuren et al., 2011; Weyant & Kriegler, 2014). The model incorporates time delays inherent in the energy system, such as the time needed to commercialize, permit, and build power plants or solar and wind power generation capacity (Siegel et al., 2018). Absent policies that either force early retirement or increase the cost of particular types of energy generation, the model assumes that infrastructure will not be retired before its useful life is expended (e.g., it takes an average of six years to build a coal plant and the average coal plant has a useful life of 30 years; Agarwal & Bayus, 2002).

En-ROADS was designed to be accessible to non-specialists and is fully transparent, enabling users to examine model structure, underlying equations, and assumptions. It is freely available and runs on a laptop or via an online server within less than 0.1 sec, making it ideal for a role-play simulation such as the Climate Action Simulation.

The Climate Action Simulation Role-Play

In the Climate Action Simulation, participants take on the roles of leaders from private industry, government, and NGOs focused on energy, land use, and climate policy and are empowered to make decisions within their own sector, while attempting to influence other groups’ decisions and voting on a carbon pricing policy. Together, their goal is to create a technically and scientifically feasible pathway to meet climate goals, for which they need two or more rounds of negotiations. The facilitator guides the role-play by taking on the role of the UN Secretary General or the Executive Secretary of the UN Framework Convention on Climate Change (UNFCCC). The Climate Action Simulation differs from traditional role-plays because it is framed by an interactive computer model so that participants receive immediate feedback about the expected outcomes of their own decisions based on the best available science through En-ROADS.

Typical simulation sequence: The Climate Action Simulation setup

A typical room layout, sequence, and materials used for the Climate Action Simulation are shown in Table 1 and Figures 2–3. A facilitators’ guide, briefing statements, nametags, table cards, proposal forms, and the En-ROADS model are all freely available online (https://www.climateinteractive.org/). Participants are assigned to delegations, each of which receives its own briefing statement, describing each delegation’s position. Participants do not need to prepare for the simulation, although, if possible, providing the briefing statement in advance prompts them to consider their roles, and perhaps even research their sector prior to playing the Climate Action Simulation. Delegations in the simulation include:

(1) Conventional Energy Supply, representing coal, oil, natural gas, and nuclear energy interests. It makes decisions about investing in research and development to reduce the cost of fossil energy sources (such as hydraulic fracturing reducing the cost of natural gas) and they lobby for taxes or subsidies on fossil fuels.

(2) Renewable Energy Supply, representing the wind, solar, hydropower, and bioenergy sectors. It makes decisions about research and development investments in their own sectors and lobby for carbon prices and subsidies on renewable energy sources.

(3) Energy Efficiency, comprised of government officials who regulate energy-intensive sectors of the economy, including construction, transportation, and white goods sectors. This group has the potential to influence the energy efficiency of new capital in both mobile (i.e., transportation) and stationary (i.e., buildings and appliances) sectors.

(4) Land, Agriculture, and Other Greenhouse Gases, representing large agricultural companies and producers, landowners, and organizations representing smaller agricultural producers. It advises governments on global forestry and agriculture.

(5) Climate Hawks, representing the leaders of the climate action movement and climate think tanks around the world. This group’s position is grounded in climate change science and the intergenerational ethics of climate action. They seek to inform other groups about potential future climate impacts and the scale, urgency, and potential co-benefits of climate action.

(6) Carbon Pricing, representing government officials who set a price on carbon for major economies and decides on how carbon price revenue is spent.

Table 1.

Materials needed for the Climate Action Simulation.

Description	Comments	Purpose
Briefing Statements	Each participant receives a Briefing Statement that is specific to their delegation’s role.	Provides background information on the interests and role of each delegation.
Nametags/Credentials for delegates	One per participant, lists their name and delegation	Facilitates interaction within and between delegations. Optional for smaller sessions.
Table cards	One per delegation.	Facilitates interaction between delegations and helps engage participants with their roles.
Proposal forms	One for each round of negotiation for each delegation.	Clarifies and records delegations’ decisions to be entered into En-ROADS.
Candy or snacks	Provided to the Fossil Fuel delegation.	Provides a prop that symbolizes wealth and power and can be used by delegates as a negotiating tool.
Large paper and markers	Provide to the Climate Hawks delegation.	Provides a prop for making signs and staging a protest by climate activists.
Computer with En-ROADS installed or internet access	One for facilitator (with projector) and, if possible, one per delegation.	Access to En-ROADS to test policy decisions.
Blazer, tie, or scarf	One for facilitator.	Helps facilitator model role-play and demarcate its beginning and end by donning or removing this item.
Blackboard, whiteboard, or flipcharts	Enough space to display delegates’ decisions and to explain aspects of the simulation and its underlying dynamics.	It is helpful to create a proposal summary form in a large format, in the front of the room, before the simulation begins. The facilitator will also need to use a board to record and display explanations and insights during the debrief.
Background reading	Optional; one per participant. Alternatively, provide Briefing Statements in advance and encourage participants to research their delegation’s role prior to the simulation.	Provide context and background information and foster in-depth engagement by participants with their roles.

Figure 2.

Typical room layout for the Climate Action Simulation role-play.

Figure 3.

Typical sequence of facilitation and role-play for the Climate Action Simulation.

Depending on the number of participants and whether or not the facilitator considers geopolitical inequities and dynamics of climate action as key learning goals, delegations from developed, emerging, or developing nations’ governments can be added. We are actively developing briefing statements for these delegations, but do not yet have evaluation data from sessions that incorporate them.

Typical simulation sequence: Introductory briefing

Prior to beginning the negotiations, it is important to offer a brief introduction to the Climate Action Simulation role-play, En-ROADS, and the climate-energy challenge. The freely available presentation slide deck draws connections between the simulation and the real world, provides a high-level overview of the En-ROADS computer model, and lays out the basic causes and consequences of climate change. Facilitators can adapt the presentation to their own educational setting or goals by adding slides that demonstrate climate change impacts relevant to their participants’ geographic region or professional interests, or provide additional information about the science, technology, policy, ethics, or other aspects of the climate-energy challenge. Making real-world relevance clear, facilitating the role-play in earnest, providing a few props (see Table 1), and setting up the room to reflect power imbalances (Figure 2) can all help foster participant engagement.

In the convening presentation, the facilitator describes each sector and its corresponding delegation in the simulation. This ensures that all participants are aware of the interests, expertise, and decision-making capabilities of all delegations. Each delegation’s role is further explained within the context of the drivers of GHG emissions using the Kaya identity (Nakićenović et al., 2000) as a framework (Figure 4).

Figure 4.

The Climate Action Simulation delegations (outlined in black) and the policy decisions they can influence in En-ROADS (grey arrows). The Climate Hawks seek to influence others’ decisions, but cannot input decisions directly into En-ROADS.*While Population and Consumption is not a negotiation group, facilitators can test different population growth and consumption scenarios in the debrief and assess their climate impact.

After the delegates have received their briefing statements, the facilitator, in the role of the UN Secretary General or Secretary of the UNFCCC, formally calls the meeting to order and lays out the context and purpose of convening the Climate Action Simulation delegations by providing an overview of the ‘business-as-usual’ (BAU) emissions scenario and its expected climate impacts. The En-ROADS BAU scenario is calibrated to several sources, including RCP 8.5 (Vuuren et al., 2011), SRES (Nakićenović et al., 2000), SSPs (O’Neill et al., 2014), and EMF 27 (Weyant & Kriegler, 2014), and generates expected warming of 4.2 °C by 2100 compared to preindustrial temperatures. The slide deck summarizes key global impacts at this level of warming, such as global sea level rise and effects on human health, agriculture, and ecosystems. Given the catastrophic consequences of expected impacts at this level of warming, the facilitator urges participants to create a better outcome. The facilitator explains that countries have agreed to limit human-caused warming to ‘well below 2 °C’ and to ‘pursue efforts’ to limit warming to 1.5 °C (UNFCCC, 2015), but are relying on their delegations to provide a technically, economically, and politically feasible pathway to do so. The convening briefing session wraps up with a short explanation of the process that the negotiation will follow.

Typical simulation sequence: First round of negotiations

After the initial briefing, the first round of negotiations begins, consisting of two phases. First, each delegation meets for 15-20 minutes to review their briefing statements, consider the decisions they face, begin formulating their positions and strategy, and construct their initial proposal and plenary speech. During this time, the facilitator speaks with each delegation individually to address any questions and encourages them to reach out to other delegations in order to discover their positions and lobby them as they see fit. Second, if time allows, the facilitator allows the groups to negotiate the terms of GHG emissions reduction for another 15 to 20 minutes. Tensions between delegations often emerge, offering an opportunity for discussion and negotiation (Putko & Rooney-Varga, 2016).

After the first round of negotiations, the facilitator calls to order a plenary session and one representative from each delegation is asked to deliver a 2-minute speech addressing all participants, describing and arguing for their delegation’s proposal. During these plenary presentations, either the facilitator or a delegate from each group records the proposals on the pledge board at the front of the room (Figure 5). Delegations may be called to present in order of their perceived power and influence, i.e., Conventional Energy Supply, Energy Efficiency, Renewable Energy Supply, Land and Agriculture, and Climate Hawks. Next, delegations’ policy decisions are entered into En-ROADS.

Figure 5.

The Climate Action Simulation pledge board. Each delegation makes decisions within their column, which they explain to the entire group in plenary sessions between each negotiation round.

Typical simulation sequence: Short debrief after first round of negotiation

Participants rarely achieve the desired climate goals in the first round of negotiations (see Figure 1 for an example of a typical outcome from a first-round negotiation) and often fall far short of meeting international climate goals. The facilitator conveys the gravity of this outcome by explaining the expected climate impacts given the global temperature rise achieved (see online materials). The gap between achieved and desired climate outcomes offers a rich opportunity for discussion about the dynamics of the climate and energy systems and their inherent inertia. Like all complex dynamic systems, the climate-energy system includes many stocks, flows, time delays, and feedback processes. Key examples include the primary driver of human-caused climate change, the accumulation of atmospheric carbon dioxide (a stock) in response to emissions (its inflow) that is roughly double current sinks (its outflows) (Sterman, 2012). Similarly, an energy transition requires that existing stocks of fossil fuel-based infrastructure decline over time, due to outflows such as retirement, and that the services they provide be gradually replaced (via inflows such as production) by stocks of energy-efficient, low-carbon capital. While the climate and energy systems also include many complex interactions and feedback processes, even the simple process of a stock accumulating in response to its flows is poorly understood by most adults (Cronin et al., 2009; Sweeney & Sterman, 2000). People also tend to underestimate the time required for a stock to accumulate or decline in response to its flows (Rahmandad et al., 2009). Applying these misconceptions to the climate-energy system can lead to gravely underestimating the scale and urgency of action needed to meet international climate goals. For example, using a correlation heuristic to relate carbon dioxide emissions to concentrations leads to the false assumption that stabilizing emissions is sufficient for stabilizing concentrations (Cronin et al., 2009). Instead, as long as emissions exceed removals, carbon dioxide concentrations continue to rise. Similarly, people tend to underestimate the time needed for transition to a low-carbon economy, which would require that the existing stock of fossil fuel-based infrastructure be drained through gradual retirement, as new stocks of low-carbon infrastructure are gradually built. This initial debrief is an ideal opportunity to explain the stock-flow dynamics of both energy infrastructure and atmospheric carbon dioxide concentrations. Having failed to achieve the desired climate outcomes, participants are often eager to understand why their decisions were insufficient and to apply what they learn in subsequent rounds.

Typical simulation sequence: Second round of negotiations

The second round of negotiations offers an opportunity for participants to apply what they have learned so far and to attempt to reach the goal of limiting warming to 2 °C. The facilitator encourages the delegations to talk to and lobby each other for another 20 minutes. If possible, participants should be encouraged to access En-ROADS themselves. Two or three members of each delegation can test different scenarios in En-ROADS and provide policy advice to the rest of their delegation, while other members work on negotiating with participants from other delegations.

The facilitator reconvenes a plenary session and asks the delegations once more to present their negotiation results, policy pledges, and rationale. Again, pledges are recorded and entered into En-ROADS. If the pledges again fall short, the facilitator calls for a third round of negotiations or, if time is limited, moderates a group negotiation, asking participants to offer updated pledges from their seats, which are entered into En-ROADS until the desired temperature outcome is reached.

Typical simulation sequence: Debrief session

Whereas the role-play itself is a vehicle for experiential learning about climate change (Kolb, 1984), the debrief session is when participants make sense of their experience and build new knowledge from it (Crookall, 2010). The debrief offers an opportunity to reflect, find new insights, and cement learning (Crookall, 2010; Hoogen et al., 2016). The debrief offers an opportunity for the facilitator to lead participants through a meaningful discussion about their experience (Lederman, 1992). We recommend including several key components in the debrief through which participants: (1) step out of the role-play to gain a broader perspective on the simulation experience; (2) collectively assess what happened during the simulation; and (3) discuss how the simulation relates to the real world, including how their own actions might affect the climate-energy system (Qudrat-Ullah, 2007; van den Hoogen et al., 2016). The facilitator actively uses En-ROADS during the debrief to further explore how the climate-energy system responds to different interventions, including specific scenarios proposed by participants during the role-play.

The overall Climate Action Simulation debrief lasts between 30 and 60 minutes. Further exploration of En-ROADS behavior via an interactive group discussion, guided by the facilitator, can help participants’ revisit insights or questions that emerged during the simulation. In case the climate goal was not met, this segment offers a means for participants discover a set of decisions that can meet climate goals, such as ambitious improvements in energy efficiency, putting a price on carbon, and aggressively reducing emissions of non-CO₂ greenhouse gases. By simulating the effect of one decision at a time, the sensitivity of the climate-energy system to a particular policy decision can be made clear. Similarly, the effect of extreme conditions or changes to model assumptions can be explored interactively. Combined with their experience during the role-play, this guided discussion leads to the conclusion that no single policy is sufficient. Instead, a combination of ambitious policies is needed to put a downward pressure on fossil fuel emissions while simultaneously stimulating the growth of low-carbon energy sources.

If time allows, we encourage facilitators to include a short period for individual reflection in which participants consider gaps between their prior mental models and the simulated outcomes, creating an opportunity to build new learning. We also encourage facilitators to offer time to reflect on and discuss emotional responses to the simulation, since affective engagement supports learning and sustained commitment to address difficult problems, especially around climate change (Pidgeon & Fischhoff, 2011).

Finally, the facilitator should move the discussion away from the simulation and encourage participants to reflect on how any insights gained from the simulation might be transferred to real-world actions. It is helpful to consider actions, decisions, or policies that are familiar or readily accessible to participants and ask them to consider what decisions or actions they might take in their own lives that would represent or influence broader policies. For example, a facilitator might discuss active transport options and dietary choices available to participants, as well as policies or incentives to support broader use of active transport and adoption of plant-based diets. At this stage in the debrief, participants are often eager to discuss and engage with actions that make a real difference in combatting climate change.

Methods: Evaluating the Climate Action Simulation

We designed a set of evaluations to assess the effectiveness of the Climate Action Simulation. We (1) conducted a focus group led by our project evaluator (co-author Rath); (2) administered a pre-test, post-test design to elicit shifts in participant knowledge of and attitudes associated with the simulation; and (3) collected open-ended responses from participants.

Evaluation Through a Focus Group

In the spring of 2015, our project evaluator conducted a focus group (Krueger & Casey, 2014) with six students who had participated in the Climate Action Simulation as part of a course focused on climate change at the University of Massachusetts Lowell. The focus group was held immediately following the simulation, which was the culminating experience of the course. The students met with the project evaluator for about one hour and discussed key aspects of the simulation. The evaluator took written notes summarizing the students’ discussion and comments. Participation was voluntary and had no influence on academic standing.

Comparison of Pre- and Post-Simulation Survey Responses

A pre- and post-simulation survey was used to evaluate learning outcomes from sessions held in educational settings including a high school (Miami FL, USA), undergraduate students at the University of Massachusetts Lowell, MBA students at the Massachusetts Institute of Technology (Cambridge MA, USA), and undergraduate and graduate students at the ESB Business School of Reutlingen University (Germany) (Table 2). These sessions represent a wide range of the educational settings in which the Climate Action Simulation has been used. The Climate Action Simulation is also used in informal education, such as workshops for sustainability professionals or climate action advocates, which could be evaluated in future work. Participation was voluntary. A set of survey items were designed to assess knowledge and beliefs about the reality and causes of climate change (Leiserowitz et al., 2011). Participants’ affective responses to climate change were elicited using semantic differential scales by asking them how they feel about climate change on a scale between emotional poles, such as discouraged to empowered (Osgood et al., 1957). Other items were designed by our team to test learning outcomes specific to the Climate Action Simulation. These questions covered the accumulation of atmospheric CO₂ in response to emissions (Rooney-Varga et al., 2018) and the potential impact of policies and actions on climate change mitigation. Note that surveys were improved over time and, therefore, some items were not included in the surveys used for the first session held in 2015 while others were specific to that session (Table 2).

Table 2.

Contextual Information for Climate Action Simulation Sessions Included in the Current Evaluation.

Session	Session date	Institution	Location	Type of participants	Number of participants in session	Matched surveys	Race/ethnicity	Gender
1	Spring, 2014	Massachusetts Institute of Technology	Cambridge, MA	Graduate MBA students	60	30	N/A	N/A
2	May 2017	Reutlingen University	Reutlingen, Germany	Undergraduates and graduate business students	42	33	6% African/Black 6% Asian 3% Hispanic/Latinx 85% White	52% Female 49% Male
3	October, 2017	Cushman High School	Miami, FL	High school students	40	16	4% African American/Black 31% Hispanic/Latinx 2% Pacific Islander 53% White 9% Other	51% Female 44% Male 4% Other
4	March, 2018	Cushman High School	Miami, FL	High school students	65	4	20% African American/Black 54% Hispanic/Latinx 25% White 2% Other	53% Female 45% Male 2% Other
5	Spring, 2019	University of Massachusetts Lowell	Lowell, MA	Undergraduate engineering students	137	90	11% African American/Black 18% Asian 7% Hispanic/Latinx 58% White 7% Other	10% Female 85% Male 5% Other

Means of pre- and post-simulation responses were compared using t-tests and Cohen’s d effect sizes were determined to assess the magnitude of differences between pre- and post-survey responses using the pooled standard deviation for the pre- and post-survey responses (Cohen, 1992):

d = ({\bar{i}}_{p o s t} - {\bar{i}}_{p r e}) / \sqrt{(s_{P o s t}^{2} + s_{P r e}^{2}) / 2}

where ${\bar{ι}}_{p r e}$ and ${\bar{ι}}_{p o s t}$ are the mean response values for each survey item and s_pre and s_post are the standard deviations, respectively.

Results

Initial evaluation of the Climate Action Simulation indicates that it is an effective tool for engaging participants with the climate-energy challenge and improving understanding of the scale, urgency, policies, and actions that have the potential to meet international climate goals. The simulation was associated with (i) increasing knowledge of climate change causes, dynamics, and impacts (analytic learning); (ii) affective engagement including greater feelings of urgency and hope (affective learning); and (iii) creating an immersive, social learning experience (social learning).

Examples of open-ended responses from participants illustrate the simulation’s potential for transformative learning and affective engagement with climate change and energy transition. Participants’ comments that reflect learning include:

“It opened my eyes to the numerous aspects that play a role in climate change.” – undergraduate engineering student, University of Massachusetts Lowell.

“I had some prior ideas about the impact of CO₂ emissions on climate change, but wasn’t aware that such a drastic reduction is necessary in such a short time.” – MBA participant, Massachusetts Institute of Technology.

Others’ comments illustrate an increased sense of urgency about climate change, a feeling of empowerment to address it, and a desire to take action, often referring to collective rather than individual action. These comments underscore affective and social learning, both of which are thought to be key in influencing climate change beliefs, risk perception, and behavior (Leiserowitz, 2006; Smith & Leiserowitz, 2014). For example:

“This exercise made me understand that the climate change issue is more pressing and needs urgent attention as well as action.” – MBA participant, Massachusetts Institute of Technology.

“I feel surprisingly excited. All of these problems can be solved so I love to think of the post change world that could be. We have a chance to build a new world.” – MBA participant, Massachusetts Institute of Technology.

“[I felt] powerful. Our group had to make final decision very engaged with interesting negotiations.” – graduate business student at ESB Business School, Reutlingen University, Germany.

“We have to work together in order to find solutions. It is not just about government placing some policies we as the member of the society have to take actions as well.” – undergraduate business student at ESB Business School, Reutlingen University, Germany.

Focus Group Responses

The focus group’s response to the Climate Action Simulation was overwhelmingly positive. The overall feeling generated was one of empowerment, supporting affective learning pathways. The participants felt like the problem was reasonably clear-cut and that the solution was obtainable as long as all delegations worked together (social learning). In the end, they were able to successfully reach the goal of limiting global warming to 2 °C.

Despite having substantial knowledge about the climate-energy system prior to the Climate Action Simulation, the students identified several key insights that they had gained, including:

A better understanding of the energy supply sources that our economy relies on;

A sense of the leverage associated with each policy decision or action, in terms of its potential to mitigate climate change;

That, among the different policy levers in the simulation, there was no single one that was sufficient to meet climate goals;

It was necessary for all of the delegations to work together to meet their shared goal of limiting warming to 2 °C, thus meeting the climate goal.

They also commented that simulation’s focus on solutions and coordination among groups led to a productive outcome (social learning), instilling hope that an energy transition that successfully addresses climate change is possible (affective learning).

Analysis of Survey Responses

Participants showed increased analytic learning through the Climate Action Simulation. Some survey questions (i.e., Table 3, items 7-12) were not fully validated and may have been misunderstood by some participants. In some cases, ceiling effects may have prevented our ability to detect meaningful pre-post changes. For example, the vast majority of participants began the simulation believing that climate change was real (99%) and caused mainly by human activities (72%) (Table 3). Thus, participating in the experience had no discernible impact on their awareness of the reality and cause of the problem. Despite these limitations, we found significant increases in participants’ understanding of emissions pathways needed to stabilize atmospheric CO₂ concentrations and meet international climate goals (Table 3, items 5-6). Participants also showed increased understanding of policies and actions that are likely to deliver effective climate change mitigation. For example, they were more likely to support systemic change (e.g., ‘climate policy that affects everyone’, a price on carbon, and regulation of other greenhouse gases such as methane, nitrous oxide, and F-gases) and less likely to believe that individual action or new technologies would be sufficient to solve the problem (Table 3). Their intent to take action that could influence others increased significantly, such as discussing climate change with their family, friends, or peers, or taking some form of political action (Table 3; items 29-31).

Table 3.

Comparison of pre- and post-Climate Action Simulation survey responses.

	Session numbers†	Pre-Mean	Post-Mean	Max*	Pre-SD	Post-SD	N	t	df	p**	ES***
1. Do you think that climate change is happening?	1-5	0.99	0.98	1.00	0.108	0.131	172	1.000	171	0.319	-0.083
2. Do you think climate change is caused mostly by human activities?	1-5	0.72	0.72	1.00	0.451	0.451	170	0.000	169	1.000	0.000
3. How worried are you about climate change?	1-5	3.06	3.27	4.00	0.768	0.736	171	-4.618	170	0.000	0.279
4. How important is the issue of climate change to you personally?	1-5	3.47	3.79	5.00	0.968	0.956	170	-5.068	169	0.000	0.333
5.Truth of statement: If the rate at which fossil fuels are burned stabilizes, the amount of carbon dioxide in the atmosphere will also stabilize.	1	0.00	0.03	1.00	0.000	0.186	29	-1.000	28	0.326	0.263
6. Truth of statement: If the rate at which fossil fuels are burned decreases dramatically, the amount of carbon dioxide in the atmosphere will stabilize.	1	0.73	0.50	1.00	0.450	0.509	30	2.971	29	0.006	-0.486
7. Agree: The cost of cutting emissions is so high that it makes more sense to wait and see how bad climate impacts are before taking action.	1-5	1.95	1.92	5.00	1.078	1.126	165	0.350	164	0.727	-0.027
8. Agree: If everyone does a little to reduce their own emissions, we could address climate change.	1	2.80	2.40	5.00	1.157	1.192	30	2.262	29	0.031	-0.341
9. Agree: Instead of enacting climate policy that affects everyone, individuals who want to reduce their own emissions can do so.	1	2.00	1.57	5.00	1.174	0.971	30	2.213	29	0.035	-0.402
10. Agree: New technologies will be developed that will solve the problems of climate change.	1-5	3.60	3.40	5.00	0.970	1.117	164	2.206	163	0.029	-0.191
11. Agree: Using current technologies, it is physically possible to dramatically reduce emissions.	1-5	3.46	3.68	5.00	1.099	1.074	164	-2.355	163	0.020	0.202
12. Agree: We could effectively address climate change if there were strong political and social will to do so.	1-5	4.13	4.22	5.00	0.956	0.844	164	-1.115	163	0.267	0.100
13. Support: Regulate emissions of methane, nitrous oxide, and other greenhouse gases.	1	3.66	3.83	4.00	0.670	0.468	29	-1.983	28	0.057	0.298
14. Support: Increase the energy efficiency of our infrastructure of cars, trucks, trains, and planes.	1	3.97	3.97	4.00	0.186	0.186	29	0.000	28	1.000	0.000
15. Support: Increase the efficiency of power plants.	1	3.93	3.89	4.00	0.378	0.315	28	0.372	27	0.713	-0.103
16. Support: Fund more research into renewable clean energy sources, such as solar and wind power.	1	3.75	3.79	4.00	0.518	0.499	28	-0.273	27	0.787	0.070
Support: Fund more research into other new, low carbon energy technologies, such as nuclear fusion.	1	3.38	3.58	4.00	0.898	0.809	26	-1.547	25	0.134	0.225
17. Support: Eliminate all government subsidies for the fossil fuel industry (coal, oil, and natural gas).	1	3.21	3.61	4.00	0.630	0.629	28	-3.034	27	0.005	0.624
18. Support: Eliminate all government subsidies for the renewable energy industry (solar, wind, and geothermal).	1	1.22	1.22	4.00	0.422	0.671	23	0.000	22	1.000	0.000
19. Support: Add a ‘revenue-neutral’ carbon price to all greenhouse gas-emitting energy sources.	1	3.38	3.63	4.00	0.619	0.500	16	-2.236	15	0.041	0.444
20. Support: Enact policy in which wealthy nations pay developing nations to protect their forests.	1	2.92	3.04	4.00	0.881	0.908	24	-0.827	23	0.417	0.140
21. Feelings: Hopeless to hopeful	2-5	3.03	3.15	5.00	0.981	0.974	140	-1.540	139	0.126	0.123
22. Feelings: Discouraged to empowered	2-5	2.91	3.22	5.00	0.861	0.874	140	-4.472	139	0.000	0.357
23. Feelings: Indifferent to engaged	2-5	3.19	3.56	5.00	0.974	0.939	140	-4.670	139	0.000	0.387
24. Feelings: Not guilty to guilty	2-5	2.96	3.14	5.00	1.052	1.023	139	-2.282	138	0.024	0.173
25. Feelings: Calm to outraged/angry	2-5	2.69	2.95	5.00	1.093	1.055	140	-3.349	139	0.001	0.242
26. Feelings: Unconcerned to alarmed	2-5	3.36	3.54	5.00	1.067	0.955	140	-2.601	139	0.010	0.178
27. Feelings: Not afraid to very afraid	2-5	3.14	3.24	5.00	1.047	0.957	140	-1.450	139	0.149	0.100
28. Likelihood: Take action to reduce your personal carbon footprint	2-5	3.18	3.23	4.00	0.766	0.738	137	-0.842	136	0.401	0.066
29. Likelihood: Discuss climate change with your family and friends	2-5	2.97	3.16	4.00	0.795	0.688	137	-3.117	136	0.002	0.256
30. Likelihood: Discuss climate change with your peers	2-5	2.91	3.14	4.00	0.836	0.666	137	-3.498	136	0.001	0.304
31. Likelihood: Take some form of political action in support of climate change policy	2-5	2.32	2.55	4.00	0.895	0.870	135	-2.894	134	0.004	0.261

†

Refers to the source of the data by session number listed in Table 2.

Refers to the maximum possible value for each survey item.

Statistically significant p-values < 0.05 are shaded light blue, p-values < 0.01 are shaded darker blue.

***

ES refers to Cohen’s d effect sizes, with small effect sizes (|ES| between 0.2 and 0.4) shaded light blue and moderate effect sizes (|ES| >0.4) shaded darker blue for statistically significant effects.

Importantly, participants showed gains in their affective engagement, reporting an increase in both worry about climate change, perceived importance of the problem, and a greater sense of outrage about it (Table 3, items 3-4, 23, 25), an emotion that is considered a ‘guardian of justice’ because it spurs social action (Hoffman, 2000). Thus, for this group, the Climate Action Simulation experience resulted in a more realistic understanding of effective climate change and energy policies as well as an increased personal connection to climate change. These results align well with open-ended comments above that describe an ‘eye-opening’ experience that increased participants’ sense of sense of empowerment, outrage, and desire to take action to affect systemic change.

Conclusions and Future Directions

The Climate Action Simulation offers an opportunity to take a high-level, holistic view on the climate-energy system, while enabling deeper exploration of particular aspects or perspectives on it. It also offers a potential for broad diffusion. First, the simulation is relevant across diverse disciplines and topics, ranging from the physical basis of the climate system to energy technologies, engineering, policies, and intergenerational ethics and morality. Second, its flexibility enables facilitators to tailor simulation sessions to diverse learning goals, disciplines, and educational settings. It has been used with audiences ranging from high school students to graduate students, scientists, professors, university administrators, energy policy experts, and business leaders. Third, it can be used with as few as six participants to as many as 100. Finally, it can also be adapted to fit different time requirements: while a full simulation and debrief typically take 2-3 hours, their key points can be compressed into a ~30-minute interactive demonstration. Alternatively, the simulation can be held over a period of several days or as a culminating activity after in-depth learning about the climate and energy systems.

The Climate Action Simulation offers an engaging tool that enables people to learn for themselves about climate change and the energy transition needed to mitigate it. The Climate Action Simulation combines simulation of the climate-energy system through En-ROADS with simulation of the social dynamics of human decision-making through role-play. In doing so, it creates an immersive, social learning experience that is not proscriptive, but instead creates a risk- and cost-free environment in which participants learn from iteration, without accumulating the repercussions of failed decisions. It offers the potential for generating innovative, systemic solutions, through experience of systemic change. Initial evaluation results indicate that the Climate Action Simulation delivers gains in knowledge about the climate-energy system, while also increasing affective engagement with climate change and the energy transition needed to address it.

Prior research has shown that climate change knowledge and affect mutually reinforce each other (van der Linden, 2014) and that affect has an important influence on risk perception and support for climate action (Leiserowitz, 2006; Smith & Leiserowitz, 2014). Social forces are also known to have a strong influence on climate change beliefs (e.g., Dunlap et al., 2016; Kahan, 2012; Postmes, 2015) and willingness to take action (Doherty & Webler, 2016). Research on the impact of the related World Climate simulation has shown that learning through simulations increases participants’ knowledge about impacts, which are linked to gains in their sense of urgency about climate change (Rooney-Varga et al., 2018). That greater sense of urgency has been linked to desire to learn and do more (Rooney-Varga et al., 2018). Similarly, social forces are known to influence our willingness to take action on climate change, with motivation to take action being linked to membership in a group that is taking action together (T. Jackson, 2005). By combining a scientifically rigorous computer model with an engaging and richly social role-play, the Climate Action Simulation offers a promising approach for aligning analytic, affective, and social learning pathways. Participants must work together to create a simulated transition to a low-carbon economy. In the real world, experiences of collective efficacy foster sustained collective action (Bandura, 2000). Initial evaluation of the Climate Action Simulation raises an enticing question: can the experience of simulated collective efficacy motivate real-world, science-based action on climate change?

Between May 2015 and October 2019, the Climate Action Simulation was implemented at least 65 times, reaching more than 2,500 participants in 22 countries in educational settings ranging from secondary to graduate schools and professional workshops. Its use in diverse educational settings suggests potential for widespread propagation. Ongoing and future work on the Climate Action Simulation will improve its accessibility to diverse users, foster its diffusion, and research its impact on learning and intent to take action on climate change in the real world.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Juliette N. Rooney-Varga is an expert on climate change and sustainability. She is the Director of the UMass Lowell Climate Change Initiative and associate professor of Environmental Science.

Contact: juliette_rooneyvarga@uml.edu.

Florian Kapmeier is professor of Strategy at ESB Business School at Reutlingen University, Germany. He received his doctorate from the University of Stuttgart. His research links the system dynamics methodology with empirical research on theory development and testing, focusing on organizational aspects of the understanding of complexity, increasingly addressing environmental sustainability issues. He is a member of both the policy council and the Dana Meadows Award Committee of the System Dynamics Society.

Contact: florian.kapmeier@reutlingen-university.de

John D. Sterman is the Jay W. Forrester professor of Management at the MIT Sloan School of Management and a professor in the MIT Institute for Data, Systems, and Society. He is also the Director of the MIT System Dynamics Group and the MIT Sloan Sustainability Initiative.

Contact: jsterman@mit.edu.

Andrew P. Jones is co-founder and co-director of Climate Interactive. An expert on international climate and energy issues, he is a system dynamics modeler, keynote speaker, and designer of simulation-based learning environments.

Contact: apjones@climateinteractive.org.

Michele Putko has served as a teaching professor in the Department of Mechanical Engineering at UMass Lowell, Associate Dean of Education at the National Security Space Institute, Colorado Springs, CO, and assistant professor in the Department of Civil and Mechanical Engineering at the United States Military Academy, West Point, NY.

Contact: michele_putko@uml.edu.

Kenneth Rath has been involved in educational evaluation since 1999. Ken has been involved numerous projects, ranging from broad programs dedicated to helping underrepresented minority students succeed in the sciences to specific interventions designed to improve learning of particular topics. To date, most of his work has been with programs in both the postsecondary and K-12 arenas.

Contact: krathreer@gmail.com.

References

Agarwal

Bayus

B. L.

(2002.). The market evolution and sales takeoff of product innovations. Management Science, 48(8), 1024–1041.

Bandura

(2000). Exercise of human agency through collective efficacy. Current Directions in Psychological Science, 9(3), 75–78. https://doi.org/10.1111/1467- 8721.00064

BP. (2014). Statistical review of world energy. http://www.bp.com/en/global/corporate/energy-economics/statistical-review-of-world-energy.html

Clémençon

(2016). The two sides of the Paris climate agreement: Dismal failure or historic breakthrough? The Journal of Environment & Development, 25(1), 3–24. https://doi.org/10.1177/1070496516631362

Cohen

(1992). A power primer. Psychological Bulletin, 112(1), 155–159.

Cronin

M. A.

Gonzalez

Sterman

J. D.

(2009). Why don’t well-educated adults understand accumulation? A challenge to researchers, educators, and citizens. Organizational Behavior and Human Decision Processes, 108(1), 116–130.

Crookall

(2010). Serious games, debriefing, and simulation/gaming as a discipline. Simulation & Gaming, 41(6), 898–920. https://doi.org/10.1177/1046878110390784

Doherty

K. L.

Webler

T. N.

(2016). Social norms and efficacy beliefs drive the Alarmed segment[rsquor]s public-sphere climate actions. Nature Climate Change, 6(9), 879–884. https://doi.org/10.1038/nclimate3025

Dunlap

R. E.

McCright

A. M.

Yarosh

J. H.

(2016). The political divide on climate change: Partisan polarization widens in the U.S. Environment: Science and Policy for Sustainable Development, 58(5), 4–23. https://doi.org/10.1080/00139157.2016.1208995

10.

Eisenack

Reckien

(2013). Climate change and simulation/gaming. Simulation & Gaming, 44(2–3), 245–252. https://doi.org/10.1177/1046878113490568

11.

Elsasser

S. W.

Dunlap

R. E.

(2013). Leading voices in the denier choir: Conservative columnists’ dismissal of global warming and denigration of climate science. American Behavioral Scientist, 57(6), 754–776.

12.

Hoffman

M. L.

(2000). Empathy and moral development: Implications for caring and justice. Cambridge University Press.

13.

Intergovernmental Panel on Climate Change. (2013). Summary for policymakers. In Stocker

T. F.

Qin

Plattner

G.-K.

Tignor

Allen

S. K.

Boschung

Nauels

Xia

Bex

Midgley

P. M.

(Eds.), Climate change 2013: The physical science basis. Contribution of working group I to the fifth assessment report of the intergovernmental panel on climate change. Cambridge University Press.

14.

Intergovernmental Panel on Climate Change. (2018). Summary for policymakers. In: Masson-Delmotte

Zhai

Pörtner

H. O.

Roberts

Skea

Shukla

P. R.

Pirani

Moufouma-Okia

Péan

Pidcock

Connors

Matthews

J. B. R.

Chen

Zhou

Gomis

M. I.

Lonnoy

Maycock

Tignor

Waterfield

(Eds.), Global warming of 1.5°C. An IPCC Special Report on the impacts of global warming of 1.5°C above pre-industrial levels and related global greenhouse gas emission pathways, in the context of strengthening the global response to the threat of climate change, sustainable development, and efforts to eradicate poverty. World Meteorological Organization. http://report.ipcc.ch/sr15/pdf/sr15_spm_final.pdf

15.

Jackson

R. B.

Le Quéré

Andrew

R. M.

Canadell

J. G.

Korsbakken

J. I.

Liu

Peters

G. P.

Zheng

(2018). Global energy growth is outpacing decarbonization. Environmental Research Letters, 13(12), Article 120401. https://doi.org/10.1088/1748-9326/aaf303

16.

Jackson

(2005). Motivating sustainable consumption: A review of evidence of consumer behaviour and behaviour change. Sustainable Development Research Network.

17.

Jacquet

Jamieson

(2016). Soft but significant power in the Paris agreement. Nature Climate Change, 6(7), 643–646. https://doi.org/10.1038/nclimate3006

18.

Jiang

Peters

G. P.

Green

(2019). Global rules mask the mitigation challenge facing developing countries. Earth’s Future, 7(4), 428–432. https://doi.org/10.1029/2018ef001078

19.

Kahan

D. M.

(2012). Why we are poles apart on climate change. Nature, 488(7411), Article 255. https://doi.org/10.1038/488255a

20.

Kahan

D. M.

Peters

Wittlin

Slovic

Ouellette

L. L.

Braman

Mandel

(2012). The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change, 2(10), 732–735. https://doi.org/10.1038/nclimate1547

21.

Kolb

D. A.

(1984). Experiential learning: Experience as the source of learning and development. Prentice Hall.

22.

Krey

Luderer

Clarke

Kriegler

(2014). Getting from here to there – Energy technology transformation pathways in the EMF27 scenarios. Climatic Change, 123(3), 369–382. https://doi.org/10.1007/s10584-013-0947-5

23.

Krueger

R. A.

Casey

M. A.

(2014). Focus groups: A practical guide for applied research (5th ed.). SAGE.

24.

Kwok

(2019). Science and culture: Can climate change games boost public understanding? Proceedings of the National Academy of Sciences, USA, 116(16), 7602–7604. https://doi.org/10.1073/pnas.1903508116

25.

Lederman

L. C.

(1992). Debriefing: Toward a systematic assessment of theory and practice. Simulation & Gaming, 23(2), 145–160. https://doi.org/10.1177/1046878192232003

26.

Lee

R. P.

(2016). Misconceptions and biases in German students’ perception of multiple energy sources: Implications for science education. International Journal of Science Education, 38(6), 1036–1056. https://doi.org/10.1080/09500693.2016.1176277

27.

Leiserowitz

(2006). Climate change risk perception and policy preferences: The role of affect, imagery, and values. Climatic Change, 77(1), 45–72.

28.

Leiserowitz

Smith

Marlon

J. R.

(2011). American teens’ knowledge of climate change. https://climatecommunication.yale.edu/publications/american-teens-knowledge-of-climate-change/

29.

Mayer

I. S.

(2009). The gaming of policy and the politics of gaming: A review. Simulation & Gaming, 40(6), 825–862.

30.

McCright

A. M.

Dunlap

R. E.

(2011). The politicization of climate change and polarization in the American public’s views of global warming, 2001–2010. The Sociological Quarterly, 52(2), 155–194. https://doi.org/10.1111/j.1533-8525.2011.01198.x

31.

McCright

A. M.

Dunlap

R. E.

Xiao

(2014). Increasing influence of party identification on perceived scientific agreement and support for government action on climate change in the USA, 2006-2012. Weather, Climate, and Society, 6(2), 194–201. https://doi.org/10.1175/WCAS-D-13-00058.1

32.

Meya

J. N.

Eisenack

(2018). Effectiveness of gaming for communicating and teaching climate change. Climatic Change, 149(3–4), 319–333. https://doi.org/10.1007/s10584-018-2254-7

33.

Nakićenović

Alcamo

Davis

de Vries

Fenhann

Gaffin

Gregory

Grübler

Jung

T. Y.

Kram

La Rovere

E. L.

Michaelis

Mori

Morita

Pepper

Pitcher

Price

Riahi

Roehrl

. . . Dadi

(2000). IPCC Special Report on Emissions Scenarios (SRES). Cambridge University Press.

34.

Nuttall

(2016). Landmark climate change agreement to enter into force [Press release]. http://newsroom.unfccc.int/unfccc-newsroom/landmark-climate-change-agreement-to-enter-into-force/

35.

O’Neill

B. C.

Kriegler

Riahi

Ebi

K. L.

Hallegatte

Carter

T. R.

Mathur

van Vuuren

D. P.

(2014). A new scenario framework for climate change research: The concept of shared socioeconomic pathways. Climatic Change, 122(3), 387–400.

36.

Oreskes

Conway

(2010). Merchants of doubt. Bloomsbury Press.

37.

Osgood

Suci

Tannenbaum

(1957). The measurement of meaning. University of Illinois.

38.

Pidgeon

Fischhoff

(2011). The role of social and decision sciences in communicating uncertain climate risks. Nature Climate Change, 1(1), 35–41.

39.

Postmes

(2015). Climate change and group dynamics. Nature Climate Change, 5, 195–196. https://doi.org/10.1038/nclimate2537

40.

Putko

M. M.

Rooney-Varga

J. N.

(2016). World energy in engineering design [Paper presentation]. American Society of Engineering Education (ASEE) Annual Conference and Exposition, New Orleans, LA, United States. https://www.asee.org/public/conferences/64/papers/15020/view

41.

Qudrat-Ullah

(2007). Debriefing can reduce misperceptions of feedback: The case of renewable resource management. Simulation & Gaming, 38(3), 382–397. https://doi.org/10.1177/1046878107300669

42.

Rahmandad

Repenning

Sterman

J. D.

(2009). Effects of feedback delay on learning. System Dynamics Review, 25, 309–338. https://doi.org/10.1002/sdr.427

43.

Reckien

Eisenack

(2013). Climate change gaming on board and screen: A review. Simulation & Gaming, 44(2–3), 253–257. https://doi.org/10.1177/1046878113480867

44.

Rogelj

den Elzen

Hohne

Fransen

Fekete

Winkler

Schaeffer

Sha

Riahi

Meinshausen

(2016). Paris Agreement climate proposals need a boost to keep warming well below 2 degrees C. Nature, 534(7609), 631–639. https://doi.org/10.1038/nature18307

45.

Rooney-Varga

J. N.

Sterman

J. D.

Fracassi

Franck

Kapmeier

Kurker

Johnston

Jones

A. P.

Rath

(2018). Combining role-play with interactive simulation to motivate informed climate action: Evidence from the World Climate simulation. PLOS ONE, 13(8), e0202877. https://doi.org/10.1371/journal.pone.0202877

46.

Saad

Jones

J. M.

(2016). U.S. concern about global warming at eight-year high. Retrieved from http://www.gallup.com/poll/190010/concern-global-warming-eight-year-high.aspx

47.

Siegel

Homer

Fiddaman

McCauley

Franck

Sawin

Jones

A. P.

Sterman

(2018). En-ROADS simulator reference guide. https://img.climateinteractive.org/wp-content/uploads/2018/04/En-ROADS_Reference_Guide_v74.pdf

48.

Smil

(2010). Energy myth and realities: Bringing science to the energy policy debate. American Enterprise Institute.

49.

Smith

Leiserowitz

(2014). The role of emotion in global warming policy support and opposition. Risk Analysis, 34(5), 937–948. https://doi.org/10.1111/risa.12140

50.

Stave

K. A.

Beck

Galvan

(2015). Improving learners’ understanding of environmental accumulations through simulation. Simulation & Gaming, 46(3–4), 270–292. https://doi.org/10.1177/1046878114531764

51.

Steg

Perlaviciute

van der Werff

(2015). Understanding the human dimensions of a sustainable energy transition. Frontiers in Psychology, 6, Article 805. https://doi.org/10.3389/fpsyg.2015.00805

52.

Sterman

J. D.

(2011). Communicating climate change risks in a skeptical world. Climatic Change, 108, 811–826.

53.

Sterman

J. D.

(2012). Sustaining sustainability: Creating a systems science in a fragmented academy and polarized world. In Weinstein

Turner

R. E.

(Eds.), Sustainability science: The Emerging Paradigm and the Urban Environment (pp. 21–58). New York: Springer.

54.

Sterman

J. D.

Sweeney

(2002). Cloudy skies: Assessing public understanding of global warming. System Dynamics Review, 18(2), 207–240.

55.

Sterman

J. D.

Fiddaman

Franck

Jones

McCauley

Rice

Sawin

Siegel

(2012). Climate Interactive: The C-ROADS climate policy model. System Dynamics Review, 28(3), 295–305. https://doi.org/10.1002/sdr.1474

56.

Sterman

J. D.

Fiddaman

Franck

Jones

McCauley

Rice

Sawin

Siegel

(2013). Management flight simulators to support climate negotiations. Environmental Modelling & Software, 44, 122–135. https://doi.org/10.1016/j.envsoft.2012.06.004

57.

Sterman

J. D.

Siegel

Rooney-Varga

J. N.

(2018a). Does replacing coal with wood lower CO2 emissions? Dynamic lifecycle analysis of wood bioenergy. Environmental Research Letters, 13(1), Article 015007. http://stacks.iop.org/1748-9326/13/i=1/a=015007

58.

Sterman

J. D.

Siegel

Rooney-Varga

J. N.

(2018b). Reply to comment on ‘Does replacing coal with wood lower CO2 emissions? Dynamic lifecycle analysis of wood bioenergy’. Environmental Research Letters, 13(12), Article 128003. https://doi.org/10.1088/1748-9326/aaf354

59.

Stokes

Wike

Carle

(2015). Global concern about climate change, broad support for limiting emissions. https://www.pewresearch.org/global/2015/11/05/global-concern-about-climate-change-broad-support-for-limiting-emissions/

60.

Sweeney

L. B.

Sterman

J. D.

(2000). Bathtub dynamics: Initial results of a systems thinking inventory. System Dynamics Review, 16(4), 249–286.

61.

UN Framework Convention on Climate Change. (2015). Adoption of the Paris agreement (FCCC/CP/2015/L.9/Rev.1). http://unfccc.int/resource/docs/2015/cop21/eng/l09r01.pdf

62.

van den Hoogen

Meijer

. (2016). Debriefing research games. Simulation & Gaming, 47(3), 368–388. https://doi.org/10.1177/1046878116651023

63.

van der Linden

. (2014). On the relationship between personal experience, affect and risk perception:The case of climate change. European Journal of Social Psychology, 44(5), 430–440. https://doi.org/10.1002/ejsp.2008

64.

Vuuren

Edmonds

Kainuma

Riahi

Thomson

Hibbard

Hurtt G.

Kram

Krey

Lamarque J.

Masui

Meinshausen

Nakicenovic

Smith

S. J.

Rose

(2011). The representative concentration pathways: An overview. Climatic Change, 109(1–2), 5–31. https://doi.org/10.1007/s10584-011-0148-z

65.

Weber

E. U.

(2017). Breaking cognitive barriers to a sustainable future. Nature Human Behaviour, 1, Article 0013. https://doi.org/10.1038/s41562-016-0013

66.

Weyant

Kriegler

(2014). Preface and introduction to EMF 27. Climatic Change, 123(3), 345–352. https://doi.org/10.1007/s10584-014-1102-7