Sage Journals: Discover world-class research

Abstract

Recent calls to improve the quality of education in schools have drawn attention to the importance of teachers’ preparation for work in classroom settings. Although the practicum has long been the traditional means for pre-service teachers to learn and practice classroom teaching, it does not always offer student teachers the time, safe practice experiences, repetition, or extensive feedback needed for them to gain adequate knowledge, skills, and confidence. Well-designed simulations can augment the practicum and address these gaps. This study evaluated the design of simSchool (v.1), an online simulation for pre-service teachers, using student teachers’ ratings of selected factors, including realism, appropriateness of content and curriculum, appropriateness for target users, and user interaction. Based on these ratings, the study identified strengths and weaknesses, and suggested improvements for the software. Participant ratings varied considerably but indicated that certain aspects of the simulation, such as its educational value, classroom challenges, and simulated student characteristics, were moderately well received. However, user interface navigation and the range and realism of simulated teacher–student interactions should be improved.

Keywords

simulation simSchool design evaluation pre-service teacher education

Teacher education programs must now train teachers to work in environments that will demand increasingly complex skills and knowledge, along with greater accountability and demonstrated teaching effectiveness. Practice teaching is a long-respected education component of this training, allowing student teachers to build their classroom knowledge, skills, and confidence before taking full responsibility for classroom teaching (Arnett & Freeburg, 2008). However, providing an effective practicum experience is often an administrative and logistical challenge, demanding strong university–school partnerships, the integration of theory and practice, and, ideally, a broad range of teaching situations in the face of limited time and resources (Allen & Wright, 2013; Bradley & Kendall, 2014).

Simulation techniques have been used as training and feedback tools for many years in occupations such as medicine, aviation, military training, and large-scale investment where real-world practice is dangerous, costly, or difficult to organize (for example, see Drews & Backdash, 2013). In pre-service teacher education, classroom simulations can help pre-service teachers to translate their theoretical knowledge into action through repeated trials without harming vulnerable students, and they can provide more practice time and diversity than limited live practicum sessions (Carrington, Kervin, & Ferry, 2011; Hixon & So, 2009).

One such simulation is simSchool (www.simschool.org), designed to provide teaching skills practice in a simulated classroom with a variety of students, each with an individual personality and learning needs. simSchool has been shown in several studies to have potential as a practice and learning tool for pre-service teachers (Badiee & Kaufman, 2014; Christensen, Knezek, Tyler-Wood, & Gibson, 2011; Gibson, 2007). Although simSchool has been under development for more than 10 years (Gibson & Halverson, 2004), very little published research has addressed its design as an instructional tool. To address this gap, the current study evaluated the design of simSchool (v.1) from the perspective of its target users, pre-service teachers, providing both quantitative and qualitative evidence of its strengths, weaknesses, and areas for improvement.

Background

Research on student learning maintains that teachers are the most important school-related factor influencing student achievement (Edutopia, 2008). Teacher education programs train our teachers, providing initial and ongoing support, resources, and hands-on experience, to prepare them for their teaching careers.

These programs face at least two important challenges that call for a more sophisticated education process. On one hand, teachers need an ever-growing set of knowledge, skills, and attitudes to meet their responsibilities; on the other hand, faced with decreased funding, increased regulation, and growing competition for available teaching jobs, they must clearly demonstrate their competencies in enhancing students’ learning (Girod & Girod, 2008).

Teaching Practice and the Practicum

Classroom teaching practice provides most student teachers with their first experience in applying the knowledge and exercising the skills that they study. The practicum is intended to give pre-service teachers the opportunity to develop practical skills and knowledge, receive feedback from experts and professionals, and gain experience with students and the school environment that can directly help them to prepare for classroom teaching. Also, practicum experiences allow teacher candidates to learn and grow in protected settings (Girod & Girod, 2008). Therefore, field experiences are often identified as the most important aspect of teacher education programs (Arnett & Freeburg, 2008; Phillion, Miller, & Lehman, 2005).

However, the practicum is fraught with difficulties, including a lack of appropriate field placements, particularly for rural, special-needs, and rarely found conditions; shortages of host teachers willing to provide their time and expertise; host teachers’ poor teaching practices, particularly with special-needs students; limited opportunity for repeated practice; and poor integration with the university curriculum (Billingsley & Scheuermann, 2014; Howey, 1996; McPherson, Tyler-Wood, Mcenturff, & Peak, 2011; Wilson, Floden, & Ferrini-Mundy, 2001; Young, 1998). It is, therefore, important to consider ways to augment the traditional practicum to enhance both the quantity and quality of students’ pre-service teaching experience.

Simulation-Based Practice

Classroom simulations are starting to offer the possibility of enhancing the practicum by providing new opportunities for pre-service teachers to practice their skills. A simulation is a simplified but accurate, valid, and dynamic model of reality implemented as a system (Sauvé, Renaud, Kaufman, & Marquis, 2007). R. D. Duke (1980), the founder of simulation and gaming as a scientific discipline, noted that the meaning of “to simulate” stems from the Latin simulare, “to imitate,” and defined it as “a conscious endeavor to reproduce the central characteristics of a system in order to understand, experiment with, and/or predict the behavior of that system” (cited in Duke & Geurts, 2004, Section 1.5.2). Simulation involves play, exploration, and discovery, all elements of learning (Huizinga, 1938/1955). It has a long history in adult education, initially in the form of abstract representations using physical components such as paper and pencil or playing boards and, more recently, in many types of computer-based virtual environments (Ramsey, 2000).

Simulations are distinguished from games in that they do not involve explicit competition; instead of trying to “win,” simulation participants take on roles, try out actions, see the results, and try new actions without causing real-life harm. Simulations, when paired with reflection, offer the possibility of experiential learning (Dewey, 1938; Kolb, 1984; Lyons, 2012; Ulrich, 1997). Dieker, Rodriguez, Lignugaris/Kraft, Hynes, and Hughes (2014) pointed out that an effective simulation produces a sense of realism that leads the user to regard the simulated world as real in some sense:

These environments must provide a personalized experience that each teacher believes is real (i.e., the teacher “suspends his/her disbelief”). At the same time, the teacher must feel a sense of personal responsibility for improving his or her practice grounded in a process of critical self-reflection. (p. 22)

Suspension of disbelief and this sense of personal responsibility work together to engage the learner in the simulation process so that it becomes a “live” experience; feedback and reflection complete a cycle so that the learner can conceptualize and ultimately apply the new learning (Kolb, 1984).

Simulations have many advantages for teacher education, particularly now that new technologies support more realistic modeling of classrooms and students. McKeachie (1994) maintained that the main advantage of an effective educational simulation is that students are active participants rather than passive observers, and such a shift in roles motivates students. Simulations can provide a venue for practicing and refining the transfer to the classroom of newly learned theory and skills, based on experimentation, feedback from simulated students, reflection/debriefing, and repetition (Carrington et al., 2011; Crookall, 2010; Parente, 1995). In this way, failure becomes part of an ongoing learning process rather than a block to achievement (Carstens & Beck, 2005). The role-play aspect of classroom simulations supports students in taking on and practicing unfamiliar teaching roles, developing new self-efficacy and professional identity over time (Carrington et al., 2011; Gibson, Christensen, Tyler-Wood, & Knezek, 2011). In simulations, scenarios can be encountered that are ethically or logistically difficult to create in the real world, such as high-stress urban environments or mixed groups of special-needs students; pre-service teachers can begin to prepare for these before they experience them in real life (Dieker, Hynes, Stapleton, & Hughes, 2007; Dieker et al., 2014). With simulations, learners can make mistakes without harming actual students—an advantage that is particularly pronounced when training for work with difficult or special-needs learners (Dieker et al., 2014; Ferry et al., 2004). Finally, simulations using commonly available technologies offer a low-cost alternative to extending teaching practice time in the field.

The Importance of Simulation Design

Realizing a simulation’s potential in any learning situation depends on a range of factors, including its fidelity, usability, relationship to learning goals, and learning processes. As simulations have developed in sophistication and have become accepted teaching tools in business, medicine, and other disciplines, researchers and practitioners have come to agree on a broad set of design and implementation principles for effective learning support.

Drawing on experience in disciplines other than teaching, Dieker et al. (2014) identified three critical components in teaching simulations that affect learning of new behaviors. One is “a sense of real presence” so that the users in some sense “suspend disbelief,” engage with the simulated environment as real, and feel personal responsibility to improve their practice (Dede, 2009). Related but distinct is the concept of fidelity, the validity of the simulation model (the degree to which the simulation represents reality); fidelity ensures that learning from the simulation is valid and transfers to practice in real life (Alessi & Trollip, 2001).

Dieker et al. (2014) also highlighted the importance for simulation-based learning of a cyclical process of action, feedback and debriefing, and modified action. This is known in the military as ARC, or the Action Review Cycle (Holman, Devene, & Cady, 2007). A meta-analysis by Gegenfurtner, Quesada-Pallarès, and Knogler (2014) confirmed that feedback after simulation activity led to greater self-efficacy and skills transfer.

The third component, only partly realized in today’s simulations, is personalized learning through flexible environments that focus on assessing and teaching the specific new skills needed by the learner (Dieker et al., 2014). One aspect of personalized learning that is incorporated into some simulations is user control over levels of difficulty, which increases self-efficacy beliefs and skills transfer (Gegenfurtner et al., 2014). Gibbons, Fairweather, Anderson, and Merrill (1997) maintained that effective simulations should allow learners to change simulation parameters, repeat the experiment, and directly observe the consequences.

Perhaps the most comprehensive set of recommended simulation design features comes from Issenberg, McGaghie, Petrusa, Gordon, and Scalese’s (2005) comprehensive systematic review, covering 34 years and 109 studies of high-fidelity medical simulations. This review identified 10 simulation features that facilitate effective learning (Table 1); the features are applicable outside the medical domain and have been recommended by many simulation experts and education researchers, including Aldrich (2004, 2005), Alessi and Trollip (2001), Duffy and Cunningham (2001), Ferry et al. (2005), and others.

Table 1.

Simulation Conditions for Effective Learning.

Condition	%^a
Educational feedback	47
Repetitive practice	39
Integration into the curriculum	25
Range of difficulty levels	14
Multiple learning strategies	10
Ability to capture variations in clinical conditions	10
Experimentation without adverse consequences	9
Reproducible, standardized experiences, students as active learners	9
Clearly stated goals and defined outcomes	6
Validity of the simulation model	3

Source. Summarized from Issenberg, McGaghie, Petrusa, Gordon, and Scalese (2005, p. 10).

Percentage of 109 reviewed articles reporting evidence of effectiveness.

The above design criteria, focusing on learning processes, are chiefly concerned with whether or not the simulation is effective in terms of producing defined learning outcomes for users. Usability, or the ease with which users are able carry out tasks using the software, is not directly addressed but is a fundamental determinant of the user’s experience with the software. Usability is distinct from utility, or whether the software is capable of carrying out its intended tasks (Microsoft, 2000). Clearly, both are important, if implied, design criteria for an effective simulation.

The simSchool Classroom Simulation

simSchool (www.simschool.org) is a web-based classroom simulation designed to provide pre-service teachers with the opportunity to practice different classroom teaching skills. The player in simSchool has the role of a teacher responsible for teaching and managing a classroom of students, choosing a grade between 7 and 12. simSchool provides student teachers with the opportunity of practicing classroom teaching skills by analyzing student differences, adapting instructions to learners’ needs and characteristics, and getting feedback from the simulation as the results of their teaching actions and choices (simSchool, 2011). Each simStudent (simulated student) has a profile that includes information about personality, academics, and teacher’s reflections; these profiles are modeled on real student profiles in actual teachers’ records. The profile include statements about the simStudent’s behavior and learning preferences. Each has an individual personality with settings on six dimensions: expected academic performance, openness to learning, conscientiousness toward tasks, extroversion or introversion, agreeableness, and emotional stability; settings range from very negative to very positive on each dimension, with about 20 different possible points on each of the six dimensions (Badiee & Kaufman, 2014; Christensen et al., 2011; Deale & Pastore, 2014; Gibson, 2007; Hettler, Gibson, Christensen, & Zibit, 2008).

In the simSchool classroom, the player must select tasks and conversational exchanges that best fit the students’ needs, and simStudents respond with changes in their expressions and responses. The teacher’s choices in interaction with simStudents affect their academic outcomes and behaviors, and the player should make appropriate decisions to help students on their given learning tasks (Zibit & Gibson, 2005).

As the simulation runs, the player is required to make many choices about organizing the lesson, managing the classroom, and interacting with individual students. These issues have been identified as significant areas that underlie the quality of instruction for teachers (Nelson, 2002). Based on their simulation experience, student teachers can practice decision making and refine their classroom teaching strategies (Zibit & Gibson, 2005). simSchool is designed to support the user in developing expertise and thinking like a teacher. Success in the simulation comes through helping simStudents improve, both in their academic performance and their behavior. simSchool is intended to be used on an ongoing basis as part of the pre-service curriculum, with an instructor’s guidance (Deale & Pastore, 2014).

Recent studies have evaluated simSchool’s effectiveness for general teaching practice (Badiee & Kaufman, 2014; Deale & Pastore, 2014), for the development of student teachers’ self-efficacy (Christensen et al., 2011; Gibson et al., 2011), and for learning to work with diverse and special-needs student populations (McPherson et al., 2011; Rayner & Fluck, 2014). These have indicated a range of positive learning outcomes for pre-service teachers after simSchool use. The Rayner and Fluck study also captured, in qualitative comments, users’ general opinions about the simulation’s realism and ease of use. These questioned the simulation’s realism (particularly simulated student responses) and identified difficulties with the user interface (particularly the mechanics of task and response selection). The article also suggested that simSchool’s realism could be improved by extending its virtual classroom to include inter-student interactions and by improving the classroom’s visual realism.

Unlike the above studies, the research reported considers the initial user experience with simSchool based on a series of brief introductory sessions. Rather than introducing the simulation as part of a formal teaching preparation program, this experiment studies initial user perceptions of and experiences with the software, identifying the key factors that support or inhibit its user acceptance, usability, and utility for augmenting the practicum experience.

Method

This research was done as part of a pilot study of the overall effectiveness of simSchool in a pre-service teacher education program in a mid-sized Western Canadian university. Results related to simSchool’s effectiveness for teacher preparation are presented in Badiee and Kaufman (2014) and are not addressed in this article.

Research Questions

The design evaluation, intended for a preliminary user-oriented evaluation of simSchool’s usability and relevance, was guided by two broad questions:

Research Question 1: What do student teachers see as the strengths and weaknesses of simSchool?

Research Question 2: What design features of simSchool need to be improved to meet student teachers’ perceived preparation needs?

The questions were addressed through a combination of quantitative and qualitative methods, as described below.

Participants

Twenty-two student teacher volunteers from a teacher education program at a Western Canadian university took part in the study. Their program is made up of a combination of professional coursework and practicum experience. Because they came from several class cohorts in the program, some participants had participated in a practicum, whereas others had not. For the purposes of this evaluation, the study did not distinguish among cohorts.

Experiment and Instrument

Because the study relied on busy volunteer participants, the experiment was conducted in a single session with breaks between brief experimental tasks. The experiment was conducted outside the education curriculum; in contrast, Rayner and Fluck’s (2014) study embedded the simulation sessions in the formal curriculum, was conducted over a longer period, and included formal training for the simulation facilitator. In this study, the facilitator relied on the simSchool manual and self-study to learn the software prior to the experiment.

The experiment consisted of three sessions with simSchool Version 1. The first session was used simply for practice with one simStudent, and the research assistant circulated and assisted any student teachers who were unclear about what to do. Then, student teachers worked through the simulation “for real” with one simStudent and then with five simStudents. There was a debriefing step after each session. During the debriefings, participants received and were able to discuss their simSchool-generated performance results. The experimental tasks and assigned times are shown in Table 2.

Table 2.

simSchool Experiment Tasks and Approximate Times.

Step	Task	Time (min)
1	Demo and introduction	10
2	Work with one simStudent (practice)	20
3	Debrief Step 2	10
4	Work with one simStudent (experiment)	20
5	Debrief Step 4	10
6	Work with five simStudents (experiment)	30
7	Debrief Step 6	10

Data related to simSchool’s design were collected from a post-experiment questionnaire based on standards of effective simulation design. The questionnaire used five-point scales to rate the simulation’s realism and other features, as well as three open-ended questions about the simulation’s design. SPSS Version 21 was used to produce descriptive statistics (frequencies and percentages). Due to the small number of responses, thematic analysis for the open-ended questions was done manually using Microsoft Word.

Results

Participant Backgrounds

Table 3 shows selected participant background characteristics. The great majority of participants (86.4%) were female. Two thirds rated their computer skills as intermediate. Less than a third (31.8%) had used computer-based simulations for education, but the majority (84.2%) had used computer-based simulations in another context. Following their simSchool use, less than one fifth (18.2%) of student teacher participants indicated that they planned to use what they had learned in the simulation in actual classrooms; however, almost three quarters (72.7%) responded that they were not sure whether they wanted to do so in the future.

Table 3.

Selected Participant Background Characteristics.

Characteristic	n (%)
Gender (n = 22)
Male	3 (13.6)
Female	19 (86.4)
Self-rated computer skill (n = 21)
Novice	2 (9.5)
Intermediate	14 (66.7)
Proficient	5 (23.8)
Have used computer-based simulation for education (n = 22)
No	15 (68.2)
Yes	7 (31.8)
Have used computer-based simulation in other context(s) (n = 19)
No	3 (15.8)
Yes	16 (84.2)
Plan to use what you learned in the simulation in the classroom (n = 22)
No	2 (9.1)
Yes	4 (18.2)
Not sure	16 (72.7)

Quantitative Ratings

Participant ratings of simSchool’s realism (fidelity) are summarized in Table 4. Overall, ratings were moderate, with the mean rating of the highest rated characteristic, “the challenges of a typical teacher in the classroom,” 3.73 out of 5. Of the participants, 68.1% rated these challenges as “realistic” or “very realistic.” The realism of simStudent profiles was rated at a similar level, with a mean of 3.71; 71.4% rated the profiles as “realistic” or “very realistic.” “Characteristics of simSchool students compared to those of real students” received a mean rating of 3.27, with 50.0% of the participants rating these as “realistic or “very realistic.” The lowest rated characteristic was “conversations between you as a teacher and simStudents,” with a mean of 2.23; 72.8% of the participants rated these simulated conversations as “unrealistic” or “very unrealistic.” The remaining three items were rated toward the midpoint of the rating range, with ratings spread across “unrealistic,” “unsure,” and “realistic.” “Options for assigning classroom tasks to simStudents” was rated 2.77 out of 5, with ratings somewhat evenly spread over the middle three rating categories. The remaining three characteristics—classroom design, simStudents’ behavior, and academic performance outcomes—were rated close to the scale midpoint of 3, indicating that participants were divided about these features and on average were unsure about their realism.

Table 4.

Participant Ratings of simSchool Realism (n = 22).

How realistic did you find the following features of simSchool?	M (SD)^a	Very unrealistic (%)	Unrealistic (%)	Unsure (%)	Realistic (%)	Very realistic (%)
The characteristics of simStudents compared with the characteristics of real students	3.27 (1.03)	9.1	9.1	31.8	45.5	4.5
simStudent profiles	3.71 (0.78)	0.0	9.5	19.0	61.9	9.5
The design of the simSchool classroom compared with a real classroom situation	2.91 (1.19)	18.2	18.2	18.2	45.5	0.0
simStudents’ behavior	3.05 (1.11)	4.8	38.1	9.5	42.9	4.8
The outcome of simStudents’ academic performance	3.09 (0.75)	0.0	22.7	45.5	31.8	0.0
Conversations between you as a teacher and simStudents	2.23 (1.15)	27.3	45.5	9.1	13.6	4.5
Options for assigning academic tasks to simStudents	2.77 (1.02)	9.1	36.4	22.7	31.8	0.0
Teachers’ challenges represented in simSchool	3.73 (0.82)	0.0	9.1	22.7	54.5	13.6

Based on a 5-point scale, with 1 = very unrealistic, 5 = very realistic.

Tables 5 through 7 show participant ratings for other aspects of the simulation. Four items (clarity of purpose, educational value, concept coverage, and generalizability) were rated with respect to content and curriculum appropriateness (Table 5); the highest mean rating was for the simulation’s educational value (3.14 out of 5, with 3 = “good”). It is worth noting that 86.3% of the respondents rated this characteristic “good,” or higher. The lowest rating (2.77) was for its generalizability. All four items were rated “good” (the midpoint of the scale) by the largest percentage of participants.

Table 5.

Ratings of simSchool Content and Curriculum (n = 22).

	M (SD)^a	Very poor (%)	Poor (%)	Good (%)	Very good (%)	Excellent (%)
Please rate the following aspects of simSchool:
It has a clear purpose	3.05 (0.78)	0.0	22.7	54.5	18.2	4.5
The content has educational value	3.14 (0.83)	4.5	9.2	59.1	22.7	4.5
It covers key concepts of classroom management	2.95 (0.80)	4.8	14.3	66.7	9.5	4.8
It is generalizable to an appropriate range of situations	2.77 (0.86)	13.6	9.1	63.6	13.6	0.0

Based on a 5-point scale, with 1 = very poor, 5 = excellent.

Table 6.

Ratings of simSchool Appropriateness for Target Users (n = 22).

	M (SD)^a	Very poor (%)	Poor (%)	Good (%)	Very good (%)	Excellent (%)
Please rate the following aspects of simSchool:
It effectively stimulates my creativity	2.41 (1.00)	22.7	27.3	36.4	13.6	0.0
It matches with my previous experiences	2.50 (1.10)	18.2	36.4	27.3	13.6	4.5
It is motivational to use	2.77 (0.92)	4.5	36.4	40.9	13.6	4.5
It is flexible for different users	2.64 (0.90)	4.5	45.5	36.4	9.1	4.5
I find it fun	2.73 (0.98)	9.1	31.8	40.9	13.6	4.5

Based on a 5-point scale with 1 = very poor, 5 = excellent.

Table 7.

Ratings of simSchool User Interaction (n = 22).

	M (SD)^a	Very poor (%)	Poor (%)	Good (%)	Very good (%)	Excellent (%)
Please rate the following aspects of simSchool:
It is easy for me to use	2.50 (0.96)	9.1	50.0	27.3	9.1	4.5
I could use it without help	2.91 (1.15)	9.1	31.8	27.3	22.7	9.1
Feedback on student responses is effectively used	2.36 (0.90)	13.6	50.0	22.7	13.6	0.0
Graphics, color, and sound are used for appropriate instructional reasons	3.23 (0.81)	0.0	18.2	45.5	31.8	4.5
It gives me control over the rate and the sequence of the simulation	2.82 (1.00)	13.6	13.6	54.5	13.6	4.5

Based on a 5-point scale: 1 = very poor, 5 = excellent

Table 6 shows participant ratings for the appropriateness of simSchool for its target users. All mean ratings in this group were below the scale midpoint of 3, between “poor” and “good,” reflecting users’ somewhat negative opinions. The items “It is motivational to use” and “I find it fun” received the highest mean ratings (2.77 and 2.73, respectively). Lowest rated were “It effectively stimulates my creativity” (M = 2.41) and “It matches with my previous experience” (M = 2.50). The item “It is flexible for different users,” which refers to an important criterion for effective simulations, received a “poor” or “very poor” rating by 50.0% of the participants.

When asked about simSchool’s user interaction (Table 7), participants gave a rating over 3 to only one item—the appropriate use of graphics, color, and sound (3.23 out of 5). Other items were rated less than 3.0 (“good”). Participants gave low to moderate ratings (Ms between 2 and 3, or “poor” and “good”) to other items about ease of use and about effective use of feedback and user control.

Finally, one additional item asking about the simulation’s freedom from racial, ethnic, and gender stereotypes received a relatively high mean rating of 3.55 out of 5.

Open-Ended Questions

Three open-ended questions gathered qualitative data from the 22 participants about their opinions and perceptions of simSchool. The following themes in their comments were relevant to the simulation design:

Question 1: What did you like most about simSchool? Respondents identified the following:

The variety of options for interaction and having conversation with simStudents (n = 5)

The variety in responses and attitudes of simStudents and the change and development of their academic performance (n = 5)

Simulation feedback in the form of interim and final results, spreadsheets and graphs, and/or student performance (n = 4)

Some participants appreciated the richness of certain aspects of the simulation, including the interactions with simStudents, the simStudents’ different attitudes, and the responsiveness of their academic achievement to the teacher’s decisions in the simulation. These are all key design features that contribute to simSchool’s effectiveness for pre-service teacher classroom practice. However, these features were only identified by small numbers of participants (four or five for each theme).

Question 2. What did you like least about simSchool? The following themes were reported:

Inappropriate, limited, or unrealistic options for conversation and interaction with simStudents (n = 12)

Difficulty with the interface in navigating through the options for interaction and conversation with simStudents (n = 9)

simStudents’ responses to the chosen tasks/conversation options did not always seem to suit or make sense (n = 5)

A larger number of participants noted difficulties with the realism of the simulation’s conversation and interaction options (n = 12); some also criticized the plausibility of student responses (n = 5). Nine participants noted difficulties with the user interface.

Question 3. Please provide any suggestions you have for improving simSchool and/or its use with student teachers. Respondents provided the following:

Have a clearer, more user friendly, and ordered categorization of comments in the interface for navigation and interaction with simStudents (n = 9)

Allow more realistic options for a variety of interactions, conversations, and teaching styles (n = 6)

Allow users to create their original comments for interaction with simStudents (n = 5)

These suggestions are consistent with the weaknesses identified in response to Question 2 above.

Discussion

Research Question 1: What do student teachers see as the strengths and weaknesses of simSchool?

In general, participant ratings of simSchool varied widely and were moderate rather than highly positive. Regarding the simulation’s fidelity, respondents regarded as most realistic the classroom challenges experienced by the user as a simulated teacher, the simStudent profiles and learning characteristics, their simulated classroom behaviors, and their academic performance outcomes (although the last two only received ratings close to “good”). The realism of simulated conversations between simStudents and the teacher received a low rating, as did correspondence with users’ previous experience. These ratings were consistent with participants’ written comments, which identified “most liked” features as conversation and interaction options, variety in simStudent responses and attitudes, and changes in their academic performance in response to teacher actions.

Overall, ease of use and stimulation of user creativity received low ratings, whereas comments identified the “least liked” features as the user interface for conversation and interaction with simStudents, general navigation in the user interface, the realism of simStudents’ responses, and general ease of use.

These results suggest that users were not quite able to suspend their disbelief and enter fully into their roles as teachers, and that they were not able, given their short exposure to the simulation, to easily choose and carry out required tasks. Results of other studies (e.g., Christensen et al., 2011) suggest that using and believing the simulation might become easier given time and support for new users to become more familiar with the software and with how to respond to its underlying student models. Also, simSchool’s flexibility for different users, an important simulation design criterion, was rated “poor” or “very poor” by 50% of users, indicating that they were not aware of the software’s capabilities for defining multiple student learning needs and for changing the class size and learning requirements; this was probably also due to the short experimental time.

Ratings were above 3 (“good”) for the simulation’s clarity of purpose; the educational value of simSchool content; the appropriate use of graphics, color, and sound; and simSchool’s freedom from racial, ethnic, and gender stereotypes. These, together with the high proportion (86.3%) of participants rating “educational value” as “good” or higher, and positive comments on simSchool’s feedback, suggest that new users in the study recognized the simulation’s potential as a learning tool despite their initial difficulties.

These findings are valuable because they come from the reflection and feedback of student teachers—the main target users of simSchool. However, given the short times available for participants to practice and work with the simulation, they reflect first impressions about the simulation as well as frustrations that might have been mitigated with longer practice time and simulation sessions. For example, the effectiveness of feedback through student responses received a low rating, although the student teachers appreciated receiving feedback and being able to see and compare their performance results. It is worth noting here that in longer experiments, instructors worked with users to help them fully understand and learn from the system’s feedback, suggesting that limited debriefing time might have negatively affected user opinions about feedback.

Research Question 2: What design features of simSchool need to be improved to meet student teachers’ perceived preparation needs?

Through low ratings of some aspects of the program and in written comments, participants argued for improvements in the conversation and interaction options between simStudents and teachers, as well as improvements in the user interface. These comments suggest that improving these aspects of the simulation could lessen initial user frustration and improve its overall effectiveness.

Conclusion

This study looked at users’ initial responses to simSchool based on limited training and brief simulation sessions. Although these initial perceptions and opinions might well change with increased exposure and instructor support, they indicate issues that need to be addressed to use the simulation effectively for pre-service teaching practice. These results are consistent with Rayner and Fluck’s (2014) observations in that both reflect the effects of participants’ limited time working with the simulation. Taken together, these two studies confirm that for effective training, the version of simSchool evaluated in this study requires longer periods of use and stronger instructor support than in their experiments. The results reported in this article do suggest that addressing usability and fidelity issues could reduce these time and resource requirements, encouraging its wider use. Despite the moderate to low ratings, the student teacher participants in this study found overall that simSchool is an instructional program of educational value.

Study Limitations

This design evaluation was conducted within a short time frame. Due to participants’ extremely full schedules, they were only available for one simulation period, which limited the time available for them to practice with the simulation. This did not allow time for them to become comfortable with the user interface, to acclimate to the simulated teacher’s role and required behaviors, or to practice with a more realistic 18-student classroom. Finally, the sample of students involved in the study might be considered biased, because it involved willing volunteer student teachers (primarily female) rather than a randomly selected sample. Therefore, the generalizability of the results is limited.

Further Research

The results of this study suggest several areas for further design evaluation work, beginning with addressing the limitations identified above by providing a longer time frame and a larger participant sample to test whether the negative ratings in this study would lessen with more learning time. Evaluation of specific design criteria could provide more targeted feedback for simSchool developers. Some of these issues have been addressed in Version 2 of the software, so future studies will be conducted with this version.

Using a sample of participants at different stages in their teacher education program could help to evaluate how participants’ prior knowledge and experience affect simSchool’s perceived design strengths and weaknesses and whether practice with the simulation might affect whether or not student teachers plan to use simulations such as simSchool in the future. It would also be useful to evaluate, with teacher educators, the simulation’s content and curriculum appropriateness.

Footnotes

Authors’ Note

Farnaz Badiee is now at the Center for Teaching, Learning, and Technology, University of British Columbia.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research and/or authorship of this article.

Author Biographies

Farnaz Badiee, at the time of this study, was an M.A. candidate in the Educational Technology and Learning Design Program at the Faculty of Education, Simon Fraser University. She is now an instructional designer/project manager at the Centre for Teaching, Learning and Technology at the University of British Columbia.

David Kaufman has served as director of Course Design, BC Open Learning Agency, and professor and director of the Medical Education Unit in Dalhousie University’s Faculty of Medicine. From 2001 to 2008, he was director of the Teaching and Learning Centre at Simon Fraser University (SFU). He is currently a professor in the Faculty of Education at SFU and conducts research on digital games and simulations for learning, and as tools to to enhance the social, emotional and cognitive lives of older adults.

References

Aldrich

(2004). Simulations and the future of learning. San Francisco, CA: Pfeiffer.

Aldrich

(2005). Learning by doing: The essential guide to simulations, computer games, and pedagogy in e-learning and other educational experiences. San Francisco, CA: Jossey-Bass.

Alessi

S. M.

Trollip

S. R.

(2001). Multimedia for learning: Methods and development (3rd ed.). Boston, MA: Allyn & Bacon.

Allen

J. M.

Wright

S. E.

(2013). Integrating theory and practice in the pre-service teacher education practicum. Teachers and Teaching: Theory and Practice, 20, 136-151. doi:10.1080/13540602.2013.848568

Arnett

S. E.

Freeburg

B. W.

(2008). Family and consumer sciences pre-service teachers: Impact of an early field experience. Journal of Family and Consumer Sciences Education, 26, 48-55.

Badiee

Kaufman

(2014). Effectiveness of an online simulation for teacher education. Journal of Technology and Teacher Education, 22, 167-186.

Billingsley

G. M.

Scheuermann

B. K.

(2014). Using virtual technology to enhance field experiences for pre-service special education teachers. Teacher Education and Special Education, 37, 255-272. doi:10.1177/0888406414530413

Bradley

E. G.

Kendall

(2014). A review of computer simulations in teacher education. Journal of Educational Technology Systems, 43, 3-12.

Carrington

Kervin

Ferry

(2011). Enhancing the development of pre-service teacher professional identity via an online classroom simulation. Journal of Technology and Teacher Education, 19, 351-368.

10.

Carstens

Beck

(2005). Get ready for the gamer generation. TechTrends, 49, 22-25.

11.

Christensen

Knezek

Tyler-Wood

Gibson

(2011). simSchool: An online dynamic simulator for enhancing teacher preparation. International Journal of Learning Technology, 6, 201-220. doi:10.1504/IJLT.2011.042649

12.

Crookall

(2010). Serious games, debriefing, and simulation/gaming as a discipline. Simulation & Gaming, 41, 898-920.

13.

Deale

Pastore

(2014). Evaluation of simSchool, an instructional simulation for pre-service teachers. Computers in the Schools, 31, 197-219.

14.

Dede

(2009). Immersive interfaces for engagement and learning. Science, 323, 66-69.

15.

Dewey

(1938). Experience and education. New York, NY: Collier and Kappa Delta Pi.

16.

Dieker

L. A.

Hynes

Stapleton

Hughes

(2007). Virtual classrooms: STAR simulator: Building virtual environments for teacher training in effective classroom management. New Learning Technology SALT, 4, 1-22.

17.

Dieker

L. A.

Rodriguez

J. A.

Lignugaris/Kraft

Hynes

M. C.

Hughes

C. E.

(2014). The potential of simulated environments in teacher education: Current and future possibilities. Teacher Education and Special Education, 37, 21-33.

18.

Drews

F. A.

Backdash

J. D.

(2013). Simulation training in health care. Reviews of Human Factors and Ergonomics, 8, 191-234.

19.

Duffy

Cunningham

(2001). Constructivism: Implications for the design and delivery of instruction. In Jonassen

(Ed.), Handbook of research on educational communications and technology. Bloomington, IN: The Association for Educational Communications and Technology. Retrieved from http://www.aect.org/edtech/ed1/

20.

Duke

R. D.

(1980). A paradigm for game design. Simulation & Gaming, 11, 364-377.

21.

Duke

R. D.

Geurts

J. L. A.

(2004). Policy games for strategic management. Amsterdam, The Netherlands: Rozenberg Publishers.

22.

Edutopia. (2008). Why is teacher development important: Because students deserve the best. Retrieved from http://www.edutopia.org/teacher-development-introduction

23.

Ferry

Kervin

Cambourne

Turbill

Puglisi

Jonassen

Hedberg

(2004). Online classroom simulation: The “next wave” for pre-service teacher education? In Atkinson

McBeath

Jonas-Dwyer

Phillips

(Eds.), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 294-302). Perth: Australian Society for Computers in Learning in Tertiary Education. Retrieved from http://www.ascilite.org.au/conferences/perth04/procs/ferry.html

24.

Ferry

Kervin

Hedberg

J. G.

Turbill

Cambourne

Jonassen

(2005). Operationalizing nine design elements of authentic learning environments in a classroom-based on-line simulation. Journal of Learning Design, 1, 22-31.

25.

Gegenfurtner

Quesada-Pallarès

Knogler

(2014). Digital simulation-based training: A meta-analysis. British Journal of Educational Technology, 45, 1097-1114.

26.

Gibbons

A. S.

Fairweather

P. G.

Anderson

T. A.

Merrill

M. D.

(1997). Simulation and computer-based instruction: A future view. In Dills

C. R.

Romiszowski

A. J.

(Eds.), Instructional development: State of the art (pp. 772-783). Englewood Cliffs, NJ: Instructional Technology Publications.

27.

Gibson

(2007). simSchool and the conceptual assessment framework. In Gibson

Aldrich

Prensky

(Eds.), Games and simulations in online learning: Research and development frameworks (pp. 308-322). Hershey, PA: Idea Group.

28.

Gibson

Christensen

Tyler-Wood

Knezek

(2011). simSchool: Enhancing teacher preparation through simulated classrooms. In Koehler

Mishra

(Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2011 (pp. 1504-1510). Chesapeake, VA: AACE.

29.

Gibson

Halverson

(2004). simSchool: Preparing tomorrow’s teachers to improve student results. In Ferdig

Crawford

Carlsen

Davis

Price

Weber

Willis

D. A.

(Eds.), Proceedings of the Society for Information Technology & Teacher Education Annual Conference 2004 (pp. 3318-3321). Chesapeake, VA: AACE.

30.

Girod

(2008). Simulation and the need for quality practice in teacher preparation. Journal of Technology and Teacher Education, 16, 307-337.

31.

Hettler

Gibson

Christensen

Zibit

(2008). simMentoring: Guiding development from virtual to real teaching! Stowe, VT: CurveShift.

32.

Hixon

H.-J.

(2009). Technology’s role in field experiences for preservice teacher training. Educational Technology & Society, 12, 294-304.

33.

Holman

Devane

Cady

(2007). Change handbook: The definitive resource on today’s best methods for engaging whole systems (2nd ed.). San Francisco, CA: Berrett-Koehler.

34.

Howey

(1996). Designing coherent and effective teacher education programs. In Sikula

Buttert

Guyton

(Eds.), Handbook of research on teacher education (2nd ed., pp. 143-170). New York, NY: Macmillan.

35.

Huizinga

(1955). Homo Ludens: A study of the play element in culture. Boston, MA: Beacon Press. (Original work published 1938)

36.

Issenberg

S. B.

McGaghie

W. C.

Petrusa

E. R.

Gordon

D. L.

Scalese

R. J.

(2005). Features and uses of high-fidelity medical simulations that lead to effective learning: A BEME systematic review. Medical Teacher, 27, 10-28.

37.

Kolb

D. A.

(1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall.

38.

Lyons

(2012). Learning with technology: Theoretical foundations underpinning simulations in higher education. In Brown

Hartnett

Stewart

(Eds.), ascilite 2012 Proceedings. Wellington, New Zealand: Massey University. Retrieved from http://www.ascilite.org.au/conferences/wellington12/2012/images/custom/asclite2012_proceedings.pdf

39.

McKeachie

W. J.

(1994). Teaching tips: Strategies, research, and theory for college and university teachers (9th ed.). Lexington, MA: D.C. Heath.

40.

McPherson

Tyler-Wood

Mcenturff

Peak

(2011). Using a computerized classroom simulation to prepare pre-service teachers. Journal of Technology and Teacher Education, 19, 93-110.

41.

Microsoft. (2000). Usability in software design. Redmond, WA: Microsoft Corporation. Retrieved from https://msdn.microsoft.com/en-us/library/ms997577.aspx

42.

Nelson

(2002). Quality teaching a national priority. Unicorn (Carlton, Vic), 28(2), 23-29.

43.

Parente

D. H.

(1995). A large-scale simulation for teaching business strategy. In Crookall

Arai

(Eds.), Simulation and gaming across disciplines and cultures (pp. 75-82). Thousand Oaks, CA: SAGE.

44.

Phillion

Miller

P. C.

Lehman

J. D.

(2005). Providing field experiences with diverse populations for preservice teachers: Using technology to bridge distances and cultures. Multicultural Perspectives, 7, 3-9.

45.

Ramsey

(2000). Quality matters: Revitalising teaching: Critical times, critical choices. Sydney, Australia: New South Wales Department of Education and Training.

46.

Rayner

Fluck

(2014). Pre-service teacher’s perceptions of simSchool as preparation for inclusive education: A pilot study. Asia-Pacific Journal of Teacher Education, 42, 212-227. doi:10.1080/1359866X.2014.927825

47.

Sauvé

Renaud

Kaufman

Marquis

J.-S.

(2007). Distinguishing between games and simulations: A systematic review. Journal of Educational Technology & Society, 10, 247-256.

48.

simSchool. (2011). About simSchool. Retrieved from www.simschool.org/about

49.

Ulrich

(1997). Links between experiential learning and simulation and gaming. In Guerts

Joldersma

Roelofs

(Eds.), Gaming/simulation for policy development and organizational change: Proceedings of the 28th Annual International Conference of the International Simulation and Gaming Association (ISAGA) (pp. 269-275). Tilburg, The Netherlands: Tilburg University Press.

50.

Wilson

S. M.

Floden

R. F.

Ferrini-Mundy

(2001). Teacher preparation research: Current knowledge, recommendations, and priorities for the future. Seattle: Center for the Study of Teaching Policy, University of Washington.

51.

Young

(1998). Rethinking teacher education for a global future: Lessons from the English. Journal of Education for Teaching, 24, 51-62.

52.

Zibit

Gibson

(2005). simSchool: The game of teaching. Innovate Online, 1(6). Retrieved from http://www.innovateonline.info/pdf/vol1_issue6/simSchool___The_Game_of_Teaching.pdf

Design Evaluation of a Simulation for Teacher Education

Abstract

Keywords

Background

Teaching Practice and the Practicum

Simulation-Based Practice

The Importance of Simulation Design

The simSchool Classroom Simulation

Method

Research Questions

Participants

Experiment and Instrument

Results

Participant Backgrounds

Quantitative Ratings

Open-Ended Questions

Discussion

Conclusion

Study Limitations

Further Research

Footnotes

Authors’ Note

Declaration of Conflicting Interests

Funding

Author Biographies

References