Abstract
In the present study, the Incredible Years Teacher Classroom Management (IY TCM) program was implemented as a universal preventive intervention in a regular, lower primary school setting. Outcomes for teacher’s behavior management practice, problem behavior in the classroom and the school environment, teacher self- and collective efficacy, and classroom climate were examined. Using a quasi-experimental pre–post comparison group design, teacher-reported outcomes were compared between 163 teachers in 21 schools who participated in the IY TCM program and 139 teachers from 23 schools who did not participate in the program. No significant main effects of the IY TCM program on teacher-reported outcomes were revealed by linear mixed model analyses. The findings suggest that further evaluation is warranted. Future research should explore the impact of training teachers more consistently (e.g., provision of additional individual coaching outside the Teacher Classroom Management [TCM] sessions) and over a longer period than provided in the present study.
Keywords
It has been well established that teachers’ classroom management skills are important to support young children’s behavioral, social, and emotional competence (Hamre & Pianta, 2005; Oliver et al., 2011). Classroom management practices have been defined as the actions teachers implement to create an educational environment that supports and facilitates students’ learning opportunities, both academically and socioemotionally. This include nonacademic classroom procedures, such as teaching prosocial behavior, and the use of proactive strategies to prevent and reduce disruptive classroom behavior (Evertson & Weinstein, 2006). Previous findings show that teachers in well-managed classrooms who use proactive teaching strategies, including praise and encouragement for appropriate behaviors, and nonharsh discipline, may foster children’s behavioral, social, and emotional adjustment, skills that are essential to academic learning (Murray et al., 2018; Reinke et al., 2018; Webster-Stratton et al., 2008). Furthermore, classroom management practices, such as monitoring student attention and performance, establishing behavioral expectations, and implementing clear rules and procedures that prevent conflict situations from occurring, may decrease problem behavior in the classroom (Oliver et al., 2011).
Teachers’ self-efficacy is defined as teachers’ beliefs in their own ability to plan, organize, and execute courses of action that are required to successfully accomplish a given educational goal or specific teaching task. Teachers with a high self-efficacy may be characterized by setting more challenging goals for themselves, take more personal responsibility for student outcomes, are more open to new teaching methods, and have a strong belief in their own efficacy (Skaalvik & Skaalvik, 2010; Tschannen-Moran & Hoy, 2001). The teacher may also have specific beliefs in relation to the school’s collective efficacy to execute actions required to produce given attainments. This refers to teachers’ judgment about whether the faculty as a whole can organize and execute the courses of action required to influence positive effect on students (Goddard et al., 2004; Skaalvik & Skaalvik, 2010). Therefore, it is of value to examine teachers’ perceived efficacy at both the individual and collective levels.
The classroom climate may also have an important impact on students’ social and emotional, cognitive, and academic development (Hamre & Pianta, 2010). The quality of the emotional support and closeness that teachers offer to students have been recognized as critical factors for a positive classroom environment, especially for children at risk of developing behavioral problems (McGrath & Van Bergen, 2015; Sabol & Pianta, 2012). Establishing behavioral guidelines in ways that promote student motivation, coaching students through conflict situations, encouraging cooperation among students, and acting as a role model for respectful communication and prosocial behavior are behaviors in teachers associated with optimal classroom climate and desired student outcomes (Jennings & Greenberg, 2009).
Classrooms with high levels of disruptive and aggressive behavior may result in a negative learning environment, which place children at a higher risk of developing more serious academic, behavioral, social, and emotional problems. Teachers’ attempts to control disruptive behavior take time away from academic teaching and inhibit positive relationships between students and teachers, which in turn negatively affect students’ academic performance, school attachment, and social well-being (Conroy et al., 2009; Downer et al., 2010; Simonsen et al., 2008). Managing student misbehavior and time constraints in the classroom have been found to predict dimensions of burnout in teachers, as well as decreased self-efficacy and teacher well-being (Ford et al., 2019; Kokkinos, 2007). Given an increasing expectation toward teachers to identify and manage pupils with mental health problems, interventions that support teachers to manage disruptive behavior and promote socioemotional competence in students may have positive effects not only for the students they teach but also for the teachers themselves (Ford et al., 2019).
The Incredible Years Teacher Classroom Management (IY TCM) Program
The Incredible Years (IY) program series is a comprehensive series of interventions including parent, child, and teacher training components that were developed to prevent problem behavior and promote social skills in young children (Webster-Stratton, 2012). The IY TCM program focuses on creating a positive classroom environment through preventive rather than reactive procedures, and on directing teachers’ attention toward positive rather than negative student behaviors. The IY TCM program have been evaluated when implemented in combination with the IY parent and IY child programs. Findings from these studies showed that both child and teacher behaviors were positively affected by the IY programs (Webster-Stratton et al., 2001, 2004, 2008). The IY TCM program implemented in isolation has also been subject to comprehensive empirical examinations by other independent investigators. The results yielded positive changes in teacher behavior, such as increased use of praise, encouragement, and incentives; and fewer harsh and critical statements (Baker-Henningham et al., 2009; Baker-Henningham & Walker, 2018; Hickey et al., 2017; McGilloway et al., 2010). An evaluation by Leckey et al. (2016) showed significant improvements in teachers’ classroom management strategies, as well as qualitative findings of proactive discipline strategies and higher levels of teacher self-efficacy. After teachers completed the IY TCM program, Hutchings et al. (2013) found significant reductions in the total number of commands (e.g., negative instructions) teachers directed toward the children, which in turn led to an increase in the rate of compliance in children, such as children showing more attention and willingness to cooperate with their teachers. In a recent study by Murray et al. (2018), the IY Teacher Satisfaction Questionnaire was used to determine teacher satisfaction after completing the IY TCM program and a high level of teacher satisfaction was reported. Moreover, a significant effect on teacher-reported classroom climate was found among teachers who had completed the IY TCM program, but the effect was not maintained into the next school year. Previous qualitative findings have also shown that the strategies taught in the IY TCM program were perceived to be useful by teachers (Carlson et al., 2011; McGilloway et al., 2010; Williford & Shelton, 2008).
In general, the average teacher educational level in Norway is high (Nygård, 2014). However, the use of nonacademic classroom procedures, such as the use of proactive preventive teaching strategies to prevent and reduce disruptive classroom behavior, as well as strategies to promote student social and emotional behavior, are teaching strategies that may be lacking or are less emphasized in regular teacher in-service training (Nye et al., 2016; Universitets- og høgskolerådet [UHR], 2019). Therefore, it may be questioned whether teachers may need additional skills to promote students’ social and emotional skills to prevent problem behavior to arise in their classroom.
Purpose of the Study
The present study used a quasi-experimental pre–post comparison group design to examine the effects of the IY TCM program when implemented as a universal preventive intervention among all teachers of first to third grades in a regular school setting, on teacher-reported outcomes: behavior management practice, problem behavior in the classroom and the school environment, teacher self-efficacy, and collective efficacy, and classroom climate. The studies cited above showed a promising effect of the IY TCM program on teacher behavior (Baker-Henningham et al., 2009; Baker-Henningham & Walker, 2018; Carlson et al., 2011; Hickey et al., 2017; Hutchings et al., 2007, 2013; Leckey et al., 2016; McGilloway et al., 2010; Murray et al., 2018; Webster-Stratton et al., 2001, 2004, 2008; Williford & Shelton, 2008). However, the majority of these studies were carried out in “problem” schools, with children from adverse backgrounds or identified risk factors (e.g., Head Start centers, high-poverty schools in urban areas, and schools that receive a higher level of support in terms of pupil–teacher ratios, special school grants, and extra support for students). It is not known whether the IY TCM program can provide comparable benefits for teachers in regular school settings with no specific risk factors. As far as we know, this is the first evaluation of the IY TCM program given as a universal preventive intervention to the entire group of teachers at the lower primary school level simultaneously. Findings from the same data set showed significant results for teacher-reported changes in student problem behavior and social skills after the IY TCM intervention, as well as for change in teacher–student relationship and parent involvement in school/teacher (Aasheim et al., 2018, 2019). Overall findings in these studies were in the small range. All the same, based on the logic assumption that a change in teacher behavior is followed by a change in student behavior, significant results for change in teachers’ behavior management practices, their report of problem behavior, classroom climate, and self- and collective efficacy were expected.
The following research questions were proposed:
Overall, a more favorable development in the intervention group was expected. However, as the IY TCM program was implemented as a universal preventive intervention in a regular school setting with a low-risk student population, large effects were not expected.
Method
Participants
In order to use already available IY group leaders, IY Norway invited municipalities that had already implemented the IY-Parenting program to implement the IY TCM program, and to participate in the research. Extensive predefined study inclusion criteria for IY TCM had to be met prior to study participation: Participating schools had to agree to school-wide implementation of IY TCM in first to third grades, and this implementation had to be approved by at least 80% of the teaching staff. In addition, to examine the organization’s readiness for program implementation, schools and municipalities had to fulfill an Agency Readiness Questionnaire Provided by IY Norway. Provided these criteria were met, the school was enrolled in the study and allocated to the IY TCM group (the intervention group of the study). Of the 25 schools that applied, 21 met the inclusion criteria and were offered the IY TCM program free of charge. The four schools that did not meet these criteria accepted to be allocated to the comparison group, and were offered IY TCM program once the study had ended.
To minimize program contamination, IY Norway contacted education agencies in 12 municipalities that had not yet offered any IY programs, and invited schools to participate as part of the comparison group of the study. These municipalities were strategically selected so that their schools could be matched to schools in the IY TCM group by geographical location and school size (small <200 students, medium 201–350 students, or large 351–780 students; Nygård, 2014). Of the 32 schools invited to be part of the comparison group, 19 accepted. These schools were offered modest financial compensation, and if they wanted to implement the IY TCM program after the study had ended, IY Norway offered them support to do so. In total, there were 21 schools allocated to the IY TCM group and 23 schools allocated to the comparison group (four who did not meet IY TCM criteria and 19 mentioned above).
Procedure
This quasi-experimental pre–post study enrolled schools and teachers across five consecutive years, from the fall of 2009 to the fall of 2013. Before start-up of the IY TCM intervention, teachers and staff, including the headmasters, were informed about the IY TCM program and the data collection procedures. The participants completed questionnaires before the IY TCM intervention took place during fall about 3 weeks before the first TCM workshop (preassessment). Completing of questionnaires after the TCM intervention was carried out in the spring the following semester about 3 weeks after the final TCM workshop (postassessment). The duration between the two assessments was typically 8 to 9 months. Based on the number of teachers in each class in schools, a statistician (fourth author BHH) generated a list of id-codes electronically. The lists of id-codes were distributed by a research coordinator to the headmasters in schools by email, then the headmaster matched the id-codes with teachers in each class. Internet links were prepared of the questionnaires using the internet survey tool Quest Back, and were also distributed by a research coordinator to the headmasters in school by emails, where the headmaster distributed them further to the teachers. Provided teachers wanted to complete the questionnaires on paper, questionnaires in written form with id-codes, as well as prepaid envelopes were sent to the headmasters in schools. The headmaster distributed the questionnaires in written form and the prepaid envelopes further to teachers. The teachers who had daily contact with students were invited to complete questionnaires both prior to the IY TCM program (preassessment) and following the completion of the IY TCM program (postassessment). Respondents received a small financial compensation for the time spent completing the questionnaires.
The study was approved by the Regional Committee for Medical and Health Research Ethics, Norway (approval/reference number: 200,803,705-7/MGA006/400).
The Intervention
The IY TCM program addresses teacher management skills and classroom climate with six, full-day workshops, each dedicated to a specific topic: (a) building positive relationships between teacher and student, and between teacher and parents; (b) teacher attention, coaching, encouragement, and praise; (c) motivating students through incentives; (d) decreasing inappropriate behavior—ignoring and redirecting; (e) decreasing inappropriate behavior—follow through with consequences; and (f) emotional regulation, social skills, and problem solving (Webster-Stratton, 2012).
The prerequisites to become an IY TCM group leader were a bachelor’s or master’s degree in teaching, special education, psychology, health, or social studies, in addition to suitable personal characteristics (e.g., relational competence, experience with and motivation to execute manualized interventions). Mainly, the group leaders were employees from the municipal Educational-Psychological Service. To become an IY TCM group leader, they had to complete a 21-hr mandatory TCM group-leader training course provided by IY Norway before carrying out the training of teachers. The group leaders received supervision from IY TCM mentors, 6 to 10 times a year on average when groups were initiated and during the intervention. The group leaders were required to bring video recordings of their training groups to the supervisions. The supervisions were full days, and consisted typically of approximately 12 to 18 group leaders. These meetings usual had a fixed set of topics (e.g., how to get the most out of the program’s video vignettes, how to direct an effective role-play), in addition to themes presented by the group leaders. Thereafter, to maintain their status as approved and qualified group leaders, they have to deliver the IY TCM program to teachers at least once per year on average (or in one to two schools depending on school size), which also was the requirement before they could run the IY TCM training for this study. All group leaders included in the study were trained by the same two IY TCM mentors (certified by the program originator in both the IY-Parenting and the IY TCM programs), and were supervised by these mentors throughout the data collection period.
In the present study, two group leaders trained groups of 15 to 20 teachers through six full-day workshops, over an 8- to 9-month period, 42 hr in total. The workshops include active learning training methods recommended for teachers (Darling-Hammond et al., 2009), such as video modeling, behavioral rehearsal of key skills through role-play, classroom practice assignments, and teacher goal setting and self-monitoring. Teachers were encouraged to self-reflect on their current practices using program checklists, to set aims for implementing specific program strategies in their classroom, to practice the strategies during the month following each workshop, and to report on their experiences at the start of the following workshop. Between each workshop, the teachers received supervision by the group leaders either by phone, or through a visit from the group leaders in classrooms (on-site) with the aim to provide consultation to support teachers’ implementation of strategies taught in the workshops. On-site consultation between each workshop was based on a written guide that aligned with the workshop topics and was recommended as part of the IY TCM program implementation. The consultation involved giving positive feedback and encouragement for teachers’ observed use of recommended strategies, making suggestions for implementation of additional strategies in specific situations in the classroom and with specific students, and consulting on developmental plans for difficult students. As part of the training, teachers were given the IY TCM program book, How to Promote Social and Emotional Competence in Young Children (Webster-Stratton & Okstad, 2005), and asked to read sections each month as recommended by the program protocol. To ensure evidence-based implementation of the program, fidelity in training was promoted by means of checklists completed by both group leaders and teachers, including user satisfaction questionnaires, which were completed at the end of each workshop (Webster-Stratton et al., 2011).
Measures
Behavior management practices
Teachers’ strategies to promote positive student behavior and manage problem behavior were assessed using a 32-item scale originally developed by Grey and Sime (1989). Previously, the items have been translated into Norwegian, and have been used in a nationally representative study of problem behavior in regular school settings (Ogden, 1998). For this study, exploratory factor analysis on preassessment data revealed a two-factor structure, interpreted as “Positive Behavior Support Strategies,” 12 items, α (predata) = .78, α (postdata) = .81; and “Behavior Correction Strategies,” 15 items, α (predata) = .72, α (postdata) = .79. Teachers were asked how many times in the previous week they had used positive behavioral supportive strategies, such as “reasoning with a student in the classroom setting” and “praised positive student behavior,” and behavioral corrections strategies, such as “threaten with sending student to Principal’s office” and “keeping a student in detention.” Items were rated on a 4-point scale (1 = never, 4 = often).
Problem behavior in the classroom and the school environment
The prevalence of problem behavior was measured using two scales: “Problem Behavior in the Classroom” (20 items) and “Problem Behavior in the School Environment” (15 items), based on Grey and Sime (1989) and Ogden (1998). Teachers and staff were asked how many times in the previous week they had observed various types of problem behavior in the classroom and in the school environment. Item examples include “talking out of turn and making unnecessary (non-verbal) noise,” “physical aggression towards other students,” and “running in corridors.” A 5-point Likert-type scale was applied (1 = not observed, 5 = observed several times per day). Satisfactory psychometric properties in prior Norwegian studies have been shown (Kjøbli & Sørlie, 2008; Sørlie et al., 2016; Sørlie & Ogden, 2007). Based on frequency analysis, five items in the “Problem Behavior in the Classroom” Scale were excluded due to low variation in this study. Furthermore, exploratory factor analyses revealed underlying subfactors for each scale: moderate and severe problem behavior in the classroom (eight and seven items), moderate and severe problem behavior in the school environment (seven and eight items). The subfactors showed acceptable internal reliability for “Moderate Problem Behavior in the Classroom,” α (predata) = .87, α (postdata) = .88; and for “Moderate Problem Behavior in the School Environment,” α (predata) = .82, α (postdata) = .83. Acceptable internal reliability was not found for “Severe Problem Behavior in the Classroom,” α (predata) = .60, α (postdata) = .54; or for “Severe Problem Behavior in the School Environment,” α (predata) = .49, α (postdata) = .48, as values below .70 were considered inadequate (Evers et al., 2013). This also is in line with results from a previous school-based intervention study (Sørlie et al., 2015).
Teacher self-efficacy and collective efficacy
Teachers’ perception of their level of self-efficacy was measured using the Teachers’ Sense of Efficacy Scale (TSES; Tschannen-Moran & Hoy, 2001). Tschannen-Moran and Hoy (2001) developed two versions of this instrument: a long version with 24 items and a short version with 12 items. The instrument consists of three subscales: Efficacy in Student Engagement, Efficacy in Instructional Strategies, and Efficacy in Classroom Management. In this study, the items from the subscale “Efficacy in Classroom Management” in the long version combined with the two other subscales from the short version in the schools was used, including 16 items in total. Teachers rated how competent they felt in managing students’ behavior on a 9-point scale. For example, “How well can you implement alternative strategies in your classroom?” “How much can you control disruptive behavior in the classroom?” “How much can you do to motivate students who show low interest in schoolwork?” The TSES has previously been translated into Norwegian by a professional translator, two experienced language teachers, and two experienced researchers. To avoid errors, the Norwegian version was back-translated into English (Skaalvik & Skaalvik, 2007). In this study, a total score was calculated based on the 16 items used. Cronbach’s alpha for these 16 items was .95 for preassessment data and .94 for postassessment data in the present study.
Teachers’ collective efficacy was measured with the Collective Efficacy Scale (CES) developed by Goddard (2002), which is a revised short version of the CES (Goddard et al., 2000). The CES is a frequently used 12-item instrument that assesses the extent to which teachers believe in their mutual capability to influence students’ learning and school outcomes positively. Responses were given on 6-point scale (1 = totally disagree, 6 = totally agree), and the instrument consists of items such as “teachers here are confident that they will be able to motivate their students” and “teachers in this school are able to get through to difficult students.” In this study, a total score was calculated based on the 12 CES items. Cronbach’s alpha for these 12 items was .81 for preassessment data and .80 for postassessment data.
Classroom climate
To assess the quality of the general learning climate in the classroom the Classroom Environment Scale, based on the instrument created by Moos and Trickett (1974), was used. The Classroom Environment Scale is a 14-item instrument that includes statements such as “the students in this class are good friends” and “the students are active and interested during lessons.” Responses are given on a 4-point scale (1 = does not fit, 4 = fits completely). In this study, a total score was calculated based on all the 12 items. Previously, the instrument have been translated into Norwegian by Ogden (1998), and has been shown to have acceptable internal consistency in prior studies (Ogden, 1998; Sørlie & Ogden, 2007). In this study, Cronbach’s alpha for the Classroom Environment Scale was .79 for preassessment data and .78 for postassessment data.
Statistics
Statistical analyses were conducted with SPSS 24. The independent sample t test and the Pearson’s chi-square test were used to test for group differences on demographic variables. The data were hierarchically organized, with teachers (Level 1) nested within schools (Level 2). Linear mixed models were used to test for group differences in preassessment scores and between score at pre- and postassessment. The dependency in data is handled by the linear mixed models, which is a suitable method for analyzing hierarchical data. Effect sizes (d) were calculated according to Feingold’s (2013) recommendations, where the standardized mean difference was calculated based on the unstandardized mean difference (regression coefficient) divided by the pooled, within-group SD of the raw outcome scores at preassessment. The effects (d) are shown with a positive value when the IY TCM group had a more favorable change than the comparison group. Intraclass correlations were calculated on outcome variables based on the change in scores to estimate the degree of dependency within schools that this clustering causes. Intraclass correlations ranged from .04 to .20. The mean (M) and standard error (SE) we report were obtained from the estimated marginal means table output in SPSS. A significance level of .05 was used for all tests.
Results
The mean size for all 44 schools was 179 students (range = 22–652); the total number of first- to third-grade classes was 225 (124 in the IY TCM group and 101 in the comparison group). Mean class size was 19.5 (SD = 9.1). None of the 44 schools were actively attending or had attended any other evidence-based school intervention programs during the previous year. The flow of participants through each stage of the study is illustrated in Figure 1.
Flowchart of schools and teachers in the study.
Demographic information on the schools and teachers included in the study is presented in Table 1. None of the demographical variables showed significant group differences at the .05 level.
Descriptive Information for Schools and Teachers.
Note. Large school (351–780 students), medium school (201–350 students), small school (<200 students). Teacher = primary and lower secondary school teacher and preschool teacher. IY TCM = Incredible Years Teacher Classroom Management; N = number of schools; n = number of teachers.
Attrition
Of 302 teachers invited to complete questionnaires, 277 (92%) completed the preassessment, 151 (93%) in the IY TCM group and 126 (91%) in the comparison group. The 25 teachers who did not complete the preassessment were excluded: 12 were from the IY TCM group and 13 were from the comparison group. Eleven of the 25 participants were missing due to protocol errors (two in the IY TCM group and nine in the comparison group), and 14 were due to missing replies and insufficiently completed questionnaires (10 in the IY TCM group and four in the comparison group).
Two hundred thirty-five teachers completed the postassessment, 121 (74%) in the IY TCM group and 114 (82%) in the comparison group. Of the 42 teachers who did not complete the postassessment, 31 were from the IY TCM group and 11 were from the comparison group. Teachers did not complete the postassessment for various reasons: One school in the IY TCM group dropped out due to organizational issues (seven teachers), five teachers were lost because they left or changed jobs (three in the IY TCM group and two in the comparison group), and 30 participants had missing replies or insufficiently completed questionnaires (21 in the IY TCM group and nine in the comparison group).
For the pre–post analysis, an additional 28 teachers were excluded: seven in the IY TCM group and one in the comparison group who had participated in TCM training before preassessment, six in the IY TCM group who did not participate in the TCM training at all, and 14 who completed the postassessment only (10 in the IY TCM group and four in the comparison group). The final sample consisted of 98 teachers from the IY TCM group and 109 teachers from the comparison group were included in the pre–post analyses.
With the exception of classroom climate, there were no significant differences between the IY TCM and comparison groups at preassessment (t = 2.06, p = .05; Table 2).
Descriptive Statistics at Pre- and Posttest, Results of Multilevel Analyses Examining Group Differences in Pre–Post Change Scores, Estimates, and Effect Sizes (d).
Note. Estimate = estimated group difference at postassessment obtained from the estimate of fixed effects table; d = effect sizes were calculated using the standardized mean difference based on the unstandardized mean difference divided with the pooled within-groups SD at preassessment, which was based on raw data. The effects are shown with a positive value when the IY TCM group has a more favorable change than the control group. The covariates gender, education, special education, and school size were statistically accounted for in the different multilevel analyses. IY TCM = Incredible Years Teacher Classroom Management.
p < .05.
Change in Teacher-Reported Outcomes From Pre- to Postassessment
The difference in teacher-reported use of positive behavior support strategies (p = .50) and behavior correction strategies (p = .66) from pre to postassessment was nonsignificant between the two groups (Table 2). This was also true for the difference in teacher-reported problem behavior in the classroom (p = .07), and for the subscores moderate (p = .053) and severe problem behavior in the classroom (p = .57; Table 2). Furthermore, none of the group differences in teacher-reported problem behavior in the school environment from pre- to postassessment were significant, neither in total score or in the subscores (see Table 2). Furthermore, no significant effects of the IY TCM were observed for teacher-reported self-efficacy (p = .27) or collective efficacy (p = .46) from pre- to postassessment (see Table 2). When teachers with pre- and postassessment data were compared with those with missing data at postassessment, a significant interaction on self-efficacy emerged in the dropout group. The predicted mean score at preassessment for self-efficacy in dropouts from the comparison group was significantly lower (M = 105.00, SE = 4.01) than that of teachers from the comparison group with complete data (M = 114.49, SE = 1.73), whereas the predicted mean score at preassessment for self-efficacy in dropouts from the IY TCM group did not differ significantly (M = 115.53, SE = 2.61) from that of teachers with complete data (M = 113.44, SE = 1.83). Furthermore, the group difference in pre–post change in teacher-reported classroom climate was not significant (p = .80; Table 2).
Possible moderating effects of sex, age, education, work experience, school size, and class size on all outcome variables were examined; however, no significant moderators were found. For further details about descriptive statistics at pre- and postassessment estimates, group differences in pre–post changes, and effect sizes, see Table 2.
Discussion
The present study evaluated the impact of the IY TCM program on teacher-reported behavior management practice, problem behavior, self- and collective efficacy, and classroom climate, after all teachers of first to third grades in a regular school setting completed the IY TCM as a universal preventive intervention. Because the IY TCM program previously has been examined primarily in disadvantageous or high-risk school settings, the purpose of this study was to determine the program’s impact as a universal preventive intervention in regular school settings. Specifically, we examined the effects on teacher-reported use of positive behavior support and behavior correction strategies in the classroom, teacher-reported problem behavior in the classroom and in other school environments, teacher-reported perception of self- and collective efficacy, and of classroom climate. The changes were small, as indicated by effect sizes ranging from −0.06 to 0.32, and none of the outcome variables were statistically significant at the .05 level.
Teachers in the IY TCM group did not change significantly from pre- to postassessment in their use of positive and negative classroom management strategies, when compared with teachers in the comparison group. To explain these findings, issues of acceptability and appropriateness may need to be addressed when transferring the IY TCM program to Norway. For instance, the standard American vignettes used in the IY TCM program are considered being at odd in a non-American educational contexts, for example, teachers have expressed that the American classrooms portrayed in the vignettes are incongruent with their own (Nye et al., 2018). This may affect how well the teachers adopted the strategies presented in the film vignettes. Furthermore, the premise for participating in the study was that all teachers and staff of first to third grades had to take part in the IY TCM training simultaneously. The teachers had to facilitate and set off time for six full-day workshops, including organize temporary staff while the TCM training was proceeded, find necessary time for preparations and homework, time to put TCM strategies into practice, as well as time to receive guidance from group leaders between the workshops. In addition to an extensive and time-consuming program implementation, the teachers also had to complete questionnaires. The total burden for teachers may have been too high.
The Jamaica studies (Baker-Henningham et al., 2009; Baker-Henningham & Walker, 2018) reported of a significant increase in teachers’ positive behavior and a reduction in negative behavior. The teachers in the Jamaica studies were experienced, but on average less than 10% of them were actually educated teachers (Baker-Henningham & Walker, 2018). This is in contrast to teachers in the present study, where 90% of the teachers in the IY TCM group were teacher educated with an average of more than 12 years’ work experience (M = 12.2 years, SD = 9.4 years), and 97% of teachers in the comparison were educated as teachers and on average had more than 14 years’ work experience (M = 14.4 years, SD = 9.0 years; see Table 1). Taken together, this could suggest a small potential for improvement in teacher’s behavior in the present study. Effective classroom management may also depend on how well teachers command authority. Students tend to give authority to teachers who succeed in building positive relationships, establish a good learning environment, promote autonomy by encouraging the student participation in decisions about behavior, and manage the challenges arising from student problem behavior (Vaaland, 2016). An authoritative teacher is acknowledged as the most effective and preferable teacher in schools; hence, this “teacher model” has served as a basis for other characteristics included in several evidence-based interventions offered to Norwegian schools during the last years (Ertesvåg, 2011). Different cultural or professional beliefs and values may also have affected the teacher’s acceptability and receptiveness to the IY TCM program, such as the specific learning techniques and the manualized nature of the program (Nye et al., 2018).
Positive effects of the IY TCM program were not observed for teacher-reported problem behavior in the classroom or in the school environment, or when exploring the subscores of moderate and severe problem behavior in the classroom and in the school environment. In agreement with our findings, no positive effects on problem behavior in the classroom were found 1 or 3 years after the implementation of the School-Wide Positive Behavior Support (SWPBS) model in Norway (Sørlie et al., 2015; Sørlie & Ogden, 2015). However, contrary to our findings, small to moderate effects on problem behavior in the school environment were found after 1 year, including on moderate and severe problem behavior (Sørlie et al., 2015), and small effects on these variables were also found after 3 years of SWPBS implementation (Sørlie & Ogden, 2015). The SWPBS model and the IY TCM program were both presented to teachers as universal preventive interventions; however, the SWPBS model was directed toward the whole school environment from first to seventh grades, whereas the IY TCM program was directed toward all teachers from first to third grades. Furthermore, in the SWPBS model, a “Response to intervention Framework” was used; hence, students were targeted at the selected and indicated levels too, which not was the case in the present study.
Similar to our findings on teacher-reported self- and collective efficacy, the SWPBS model in Norway failed to verify significant effects on the TSES or the CES after 1 year of implementation (Sørlie et al., 2015, 2016). But a small effect on collective efficacy was confirmed after 3 years with the SWPBS model (Sørlie et al., 2016), which may suggest that the nonsignificant findings in our study could be explained by a need for more than 8 to 9 months for teachers to use the IY TCM strategies into practice. As noted earlier, included teachers in the present study were both well educated and experienced, which may also explain why mean scores on teachers’ report of collective efficacy were high in the first place. In the Sørlie et al. (2016) study, mean baseline scores for collective efficacy ranged between 55.0 and 57.8, whereas mean scores at preassessment in the present study were 60.2 and 60.4 in the IY TCM and the comparison group, respectively. For the teachers’ report of collective efficacy, a ceiling effect may have occurred, thereby decreasing the likelihood that the instrument has accurately measured the intended domain (Taylor, 2010).
Furthermore, no positive intervention effect was found on the Classroom Environment Scale in the present study. In a recent study by Murray et al. (2018), a positive effect on the classroom climate was found 1 year after the IY TCM program (d = 0.45). In the Murray study, the classroom climate was assessed using the Classroom Assessment Scoring System (Pianta et al., 2008), which classifies the quality of classroom interactions according to three overarching domains, for example, emotional support, classroom organization, and instructional support (Westergård et al., 2019). In contrast, the Classroom Environment Scale by Moos and Trickett (1974) assesses the quality of general learning conditions in the classroom. Hence, the two measurements may capture different elements of the classroom environment. As cited, in a teacher sample within the same data set as in the present study, positive effects on teacher–student conflict and closeness were revealed (dw = 0.15–0.22; Aasheim et al., 2018). Taken together, these findings may suggest that teachers in the present study did in fact experience some advantageous changes in their classroom interactions with students.
Strengths and Limitations
This study has some limitations that should be pointed out. First, a randomized controlled trial would have been the preferred design for the study. Due to practical obstacles related to the recruitment of schools, a quasi-experimental pre–post design with continuous enrolment of intervention and comparison schools was chosen. The study was dependent on the presence of qualified IY group leaders in municipalities with participating schools; therefore, recruitment to the intervention group had to be carried out in these municipalities. Furthermore, because extensive predefined criteria for program implementation had to be fulfilled before study inclusion, the schools had to apply to IY Norway to prove they met these criteria before program implementation, and to be included in the study. A strength of the study is that the implementation of the intervention was naturalistic, carried out under real-world conditions, and incorporated into routine educational practice by regular school teachers.
Second, the Norwegian Directorate of Health funds the IY Norway, and the authorities meet expenses in connection with organizing curriculums, groups, and training of group leaders. Therefore, at the time when the study was conducted, the funder wanted there to be clear barriers between the implementation of IY TCM in Norway and the research, to facilitate independence between research and implementation. The implementation process was handed over to the local municipalities involved. Fidelity information was given through checklists and workshop evaluations by group leaders and teachers; however, getting access to these assessments for the researchers was problematic due to practical causes. Therefore, at the time of planning the study, program and implementation fidelity data were not included in the study.
Third, students and teachers from 21 intervention and 23 comparison schools constitute the entire study sample. The power calculations for this study were originally based on the number of randomized students included, not the number of teachers. Power calculations for a two-stage sample during study planning estimated that the number of teachers had to be around 100 to 200 in each group. This study included 99 and 109 teachers in the IY TCM group and comparison group, respectively. It is possible that these numbers constitute a sample of the overall teacher population that is too restricted. Restricting range of data may cause less variation in outcome variables, as the overall proportion of variation is reduced (Bland & Altman, 2011).
Finally, the reliability of the subscales Severe Problem Behavior in the Classroom and for Severe Problem Behavior in the School Environment was inadequate. Collectively, the measures used were not designed specifically to capture the core skills taught in the IY TCM program and might have lacked sufficient sensitivity to capture changes in teachers’ behavior management practices.
Conclusion and Implications for Practice and Research
Given the effects for change in teacher-reported student behavior and teacher–student relationship found in the same data set after the IY TCM intervention (Aasheim et al., 2018, 2019), and the logic assumption that a change in teacher behavior is followed by a change in student behavior, group differences in pre–post change on teacher behavior were expected in the study. No statistically significant main effects of the IY TCM program on teacher-reported outcomes were found when the program was implemented over an 8 to 9 months period as a universal preventive intervention in a regular school setting. Findings in the present study suggest that further evaluation is warranted. Based on previous research, some of the components taught in the IY TCM program have been assessed as more valuable by teachers, such as strategies to strengthen the relationships with students, the use of proactive strategies, and how to coach students’ social and emotional skills, as well as the use of developmental plans for students (Murray et al., 2018; Nye et al., 2018; Reinke et al., 2018). Implications for future practice and research may be to implement and explore whether an isolated combination of these components is more practical to adopt and put into practice. Targeting these components and provide them as shorter in-service trainings to all teachers or to beginning teachers in schools could increase program feasibility and focus for schools wanting to target universal preventive areas of need within their school (Murray et al., 2018). Furthermore, evaluations of program and implementation fidelity are recommended in future practice and research. To consolidate changes in teacher behavior through universal preventive interventions, a more consistent implementation practice should be explored (e.g., such as the provision of additional individual coaching to teachers outside the TCM sessions), in addition to identifying barriers to implementation and features of successful implementation, for example, the use of process evaluation to identify moderating effects of implementation quality on intervention outcomes.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
