Abstract
Childhood emotional and behavioural problems can indicate a higher risk of developing mental illness in adolescence and beyond. Schools provide an appropriate setting in which to deliver universal preventions to improve well-being and protect against early risks for mental health disorder. However, interventions can often be difficult to implement in schools due to their perceived or actual burdensomeness and implementation challenges can affect effectiveness. Teacher-delivered well-being interventions that can be integrated into usual classroom teaching practices provide a potential solution. This study examines the efficacy of a teacher-delivered programme, the Good Behaviour Game, in Australian primary schools. The programme significantly reduced emotional and behavioural problems as measured by the Strengths and Difficulties Questionnaire from baseline to post-intervention, suggesting that it may be an effective teaching practice for managing students in the classroom and for promoting well-being and development.
Introduction
The importance of psychological well-being for children’s healthy emotional, social and physical development is well recognised in the research and in current directives from policy makers that schools prioritise well-being. There is clear evidence that emotional and behavioural problems (EBPs) which emerge during childhood are important antecedents for mental and physical health inequities, poorer educational attainment, criminality and lower economic productivity across the life course (Dua et al., 2016; Fergusson, Horwood, & Ridder, 2005; Jacka et al., 2013). In Australia, it is estimated that one in six children will experience an emotional or behavioural disorder by 11 years of age (Lawrence et al., 2015), and that boys aged 4–11 years have a 64% increased risk of meeting criteria for a mental disorder (e.g., conduct disorder, attention-deficit hyperactivity disorder (ADHD), anxiety, depression) compared to girls of the same age (Lawrence et al., 2015), suggesting early-life well-being initiatives targeting boys may be of particular importance.
During early-mid childhood, EBPs typically manifest as disruptive and aggressive behaviours, shyness and lower academic capability (Sutherland, Lewis-Palmer, Stichter, & Morgan, 2008; Weeks, Coplan, & Kingsbury, 2009), and schools, a key microsystem of this life stage, are host to a multitude of socialisation risks (e.g., bullying, peer exclusion, depression contagion, peer deviancy training) that can contribute to the development or worsening of EBPs (Guy, Furber, Leach, & Segal, 2016; Jacka et al., 2013). Children’s experiences in the school or classroom can alter early developmental trajectories and lead to chronic, complex maladaptation which is costly and difficult to treat (Bayer et al., 2009; Hamre & Pianta, 2005; Sanson, Havighurst, & Zubrick, 2011). As such, schools are an important setting for prevention efforts.
While there is mounting evidence that EBPs can be effectively mitigated, particularly through universal prevention strategies which target age-relevant developmental factors (i.e., self-regulation, peer connectedness and resilience) (Sanson et al., 2011; Wise, 2005), identifying effective well-being interventions that are available for implementation in schools is a major challenge. Even where effective programmes are available, schools can be reluctant to implement programmes that might place additional burden on already overburdened staff and which might take away from students’ class time (Jaycox et al., 2006). Identifying best practice programmes that work well within the natural school system is important in order to meaningfully improve students’ well-being and to protect against the development of EBPs.
A universal prevention programme known as the Good Behaviour Game (Kellam et al., 2011) provides a potential solution to school-relevant implementation challenges. The programme is teacher-delivered and is designed to be integrated into the existing curricula by providing behavioural management strategies to enhance students’ self-regulation and learning. The programme is based on life course and social field theory principles and aims to strengthen emotional- and behavioural self-regulation by improving adaptation and socialisation (hierarchical and peer-based) in the classroom (Kellam, Reid, & Balster, 2008).
Randomised controlled trial (RCT) evaluations conducted in the United States and Europe have found that, with as little as 12 weeks delivery, the programme is associated with reductions in disruptive or aggressive behaviours, improvements in on-task behaviours and increased academic (literacy, numeracy) outcomes at up to 12 months post-baseline (Flower, McKenna, Bunuan, Muething, & Vega, 2014; Kellam et al., 2011, 2008). Significant short-term effects in respect to disruptive, aggressive and shy behaviours have been reported for both girls and boys (Dolan et al., 1993; Ialongo et al., 1999). Notable longer term outcomes of the programme (up to 60 months after the programme) include reduced rates of drug and alcohol use disorders, delayed onset of regular smoking, fewer antisocial personality disorder diagnoses and reduced rates of suicide ideation and attempt (Kellam et al., 2008), however, these effects have largely only been observed for males and for longer term implementation (up to 12 months). The available evidence suggests that the programme has significant, small-to-moderate transdiagnostic prevention effects that are sustained over time.
The current study is a pilot examination of whether the Good Behaviour Game reduces EBPs among students after 12 weeks of exposure, relative to baseline. This is the first time that this programme has been tested in the Australian context. This study also examines whether fidelity of programme delivery and dose of the intervention moderate the effects of the Good Behaviour Game on EBPs and whether gender effects are observed. This study establishes preliminary evidence for whether the programme translates effectively to the Australian setting and importantly contributes to the cross-cultural evidence base for effective, scalable prevention programmes that can be integrated into schools.
Methods
Design
A non-experimental design (Blundell & Costa Dias, 2000), in which all schools received the Good Behaviour Game intervention, was used to examine change in severity of EBPs between baseline and post-intervention assessments using the total difficulties scale of the Strengths and Difficulties Questionnaire (SDQ) (R. Goodman, 2001) as the primary outcome. We also examined change on the externalising and internalising difficulties sub-scales of the SDQ over the same time points as secondary outcomes. This information can be used to estimate the effect size in a fully powered trial (Stuart, Marcus, Horvitz-Lennon, Gibbons, & Normand, 2009).
Setting
Four independent (non-government) primary schools in Sydney, New South Wales (NSW) and two schools (one government and one Catholic) in Warrnambool, Victoria, Australia participated. In respect to socioeconomic status (SES), two schools had Index of Community Socio-Educational Advantage (ICSEA) scores well above the national mean score of 1000, two schools had scores commensurate with the national average, and two schools below the average. This means that the sample was evenly distributed across low, medium, and high SES categories. Non-probability sampling of schools was used, where a range of schools in the Sydney and Warrnambool areas were contacted, and schools volunteered to participate in the study.
Intervention
The PAX Good Behaviour Game is made up of nine ‘kernels’ (i.e., evidence-based behavioural contingencies) including the Classroom Vision (sets out the agreed desirable and undesirable behaviours), PAX Quiet (use of a harmonica to gain the children’s attention and achieve silence), Granny’s Wacky Prizes (rewards for desirable behaviours), Beat the Timer (the primary part of the programme) and Tootle Notes (to reinforce positive relations between students). The kernels are designed to be used every day, at various times, individually or together to encourage desirable behaviour and to teach children to make positive choices.
Beat the Timer is the key programme component of the intervention and is delivered by grouping students into teams, balanced for gender and behaviour. A timer is set for a time specified by the teacher, and while the timer is going, the teams encourage, influence and help each other to avoid engaging in off-task or undesirable behaviours. For each off-task/undesired behaviour engaged in, teams receive one point. At the end of the playing period, teams with three or fewer points are rewarded. In this way, prosocial behaviour and academic progress are rewarded in an inclusive group-orientated system, leading to internalisation of norms for appropriate behaviours. This part of the programme is played every day for the 12 weeks, up to three times per day, from 1 to 40 minutes (games get longer and more frequent over time), as the children’s self-regulation and behaviour improves.
Participants and eligibility
Students from Grade 1 through to Grade 5 were eligible to participate, and two classrooms per school were chosen (from the same grade where possible to control for grade effects), to have a pair of teachers who could provide support to each other. This approach yielded a sample of n = 230 students. All students were exposed to the Good Behaviour Game given it is a universal prevention programme. Universal programmes are designed to be implemented in populations which are not identified based on potential risk or symptoms, which, in this case, would be EBPs as measured by SDQ scores assigned by teachers to students. While parental consent was not required to participate in the programme because no data were collected from students or parents, passive information and consent forms were sent home to parents/guardians of the students in the participating classrooms, providing the opportunity for parents to object to their child’s participation in the study. However, no objections were received.
Procedure
All trial procedures were approved by the University of New South Wales Human Research Ethics Committee (HREC) (protocol # HC180034), and this ethics approval was ratified by Deakin University HREC and the Victorian Department of Education and Training. All schools were contacted via email, and followed up via phone, to determine interest in participating in the study. Recruitment took place between March and May 2018. School principals were required to provide active, written consent to the PAX Good Behaviour Game being delivered to students in their schools. Teachers were required to provide individual written consent to participate in data collection. Consent forms were submitted directly to the research team once completed. Participating teachers were then enrolled in the trial and required to participate in a two-day training programme held at Black Dog Institute and delivered by a trainer from the PAXIS Institute, United States.
After completing the training, and prior to delivering the programme, all teachers completed a baseline questionnaire which included the SDQ (R. Goodman, 2001) for each student in their class, which has been found to accurately identify EBPs in school-age children (Silva, Osório, & Loureiro, 2015). Data for each student were collected face-to-face from teachers in the schools in NSW and via telephone for the schools in Victoria. Teachers then delivered the intervention, daily, for 12 weeks in their classrooms, excluding school holidays. Schools were staggered in their start dates for feasibility reasons, particularly so that not all assessments needed to be undertaken simultaneously. The active intervention began in the first school on 7 May 2018 and the final school completing on 30 November 2018. Data collection completed on 14 December 2018. During the active implementation period, two classroom observations of between 60 and 90 minutes each were carried out per classroom by a research team member (at week 4 and week 8) to confirm programme fidelity. Observations were taken at times which were most convenient to teachers and therefore did not always occur at the same time of day. However, given the programme is meant to be implemented at different times of day, this is unlikely to have had an effect on the fidelity of implementation. The observation was captured on a standardised observation form (see Online Appendix 1). These visits also provided an opportunity to provide teachers with feedback regarding their implementation of the programme.
At the end of the 12-week active intervention period, teachers were contacted by the research team to organise a date to complete post-intervention questionnaires. This included teachers providing information about the experience of programme delivery and SDQs for each student (see Online Appendix 2). As with baseline questionnaires, all SDQ forms were completed by teachers in their own time and posted or collected from the school, while the programme delivery data were collected face-to-face in NSW schools and via telephone elsewhere.
Data collection and measures
Questionnaires were teacher self-report and administered via paper and pen. The information collected is described below.
Student demographics
Information about student gender and age was collected as part of the SDQ.
Primary outcome
The primary outcome was the reduction of EBPs, assessed by the total difficulties score on the Teacher-Rated 25-item SDQ (R. Goodman, 2001) for children aged 4–10 years. The SDQ was rated by teachers at baseline and post-intervention for each student in their class. The total difficulties score was generated by summing the four sub-scales (emotional symptoms, conduct problems, hyperactivity/inattention and peer relationships problems) and scores ranged from 0 to 40 whereby higher scores indicated more severe EBPs. The total difficulties score has been used as an outcome measure in the previous studies of mental health and well-being in primary school-age children (Bunketorp Käll, Malmgren, Olsson, Lindén, & Nilsson, 2015; Muris, Meesters, Eijkelenboom, & Vincken, 2004; Van Leeuwen, Meerschaert, Bosmans, De Medts, & Braet, 2006). The internal consistency of the SDQ total difficulties score was good at baseline (Cronbach’s α = 0.85) and at post-intervention ((Cronbach’s α = 0.81) in this study.
Secondary outcomes
SDQ sub-scales
An ‘internalising difficulties’ sub-scale was created by summing the emotional and peer relationship scales, while an ‘externalising difficulties’ score was created by summing the conduct and hyperactivity scales. Total scores on both of these sub-scales ranged from 0 to 20, and these scales have been reported as having better validity for community samples than the five-item sub-scales of which the SDQ is comprised (A. Goodman, Lamping, & Ploubidis, 2010).
Fidelity
A six-item Intervention Fidelity Scale was developed by the research team to assess to what extent the teachers’ delivery of the intervention adhered to the programme model. The fidelity checklist was based on observation data collected during two visits to the schools, and these were mapped to what were considered to be key fidelity criteria based on the expectations described in the programme’s manual. The six items that overall fidelity was rated on were as follows: (i) adherence to the behavioural contingencies (‘kernels’; were all nine components delivered each week), (ii) adherence to the rule infraction system (only negative behaviours are counted), (iii) adherence to the reward system (only teams with scores of <= 3 are rewarded), (iv) timing of delivery (‘beat the timer’ was played at least once per day, for greater lengths of time), (v) level of comprehension (teachers provided clear directions of what would happen, and students were able to demonstrate their understanding of what was expected of them, e.g., ‘What PAX voices will we use in the literacy task?’; ‘What PAX behaviours do we want to see when reading on the floor?’) and (vi) consistency in delivery (the extent to which teachers deliberately delivered the ‘beat the timer’ during different types of lessons, consistent with the programme model). Each domain was rated on a five-point Likert-type scale (range: 0 (‘strongly disagree’) to 4 (‘strongly agree’)). For some items, only a single factor was considered in determining a rating, for others, multiple factors were considered, but a single overall rating provided. Rating agreements were made by discussion between two researchers (MT and VR). The intervention fidelity scale was used at each observation visit. Scores were averaged across the two time points to provide a global fidelity score (maximum score 24). Higher scores indicated greater fidelity. The Intervention Fidelity Scale had good internal consistency (Cronbach’s α = 0.89).
Dose
Although the programme is designed as a 12-week intervention, actual dose varied and was calculated based on the teacher self-reported total number of weeks the programme was delivered.
Power analyses
Sample size calculations were based on the expected effect of the intervention on the reduction of EBPs. In order to be able to detect an effect size of 0.30 between pre- and post-intervention outcomes with α = 0.05 and power = 0.80, 119 student participants were needed. Including an expected attrition rate of 20% due to students leaving the school or teachers not completing questionnaires for all children, the minimum sample size needed was n = 149.
Statistical analyses
All data were analysed using SPSS v.24.0 (IBM Corp., 2016). Data were analysed at the unit (individual) level. Descriptive statistics were used to describe the sample and to summarise the scores on the SDQ, dose and fidelity measures. Means and standard deviations (SDs) are presented for continuous variables. Independent samples t tests were used to compare baseline (T0) SDQ scores by gender (male vs. female). The primary outcome measure was analysed using linear mixed modelling (LMM), which can account for missing data, to model change in the SDQ global difficulties scores from pre- to post-intervention. The time variable was coded such that pre-intervention = 0 and post-intervention = 1. Random effects were modelled for intercept and time. Subsequent analyses examined the potential moderating effect of gender, age, dosage and fidelity on the pre- to post-intervention change on the SDQ outcome variable. Separate models were analysed for gender, age, dose and fidelity, with focus on the Time × Gender, Time × Age, Time × Dose and Time × Fidelity interaction terms in the respective models. Proposed moderating variables that were continuous were mean-centred before computing the interaction term. The α level was set at 0.05 for all analyses.
Sample description
The baseline sample size was n = 230 students divided across 11 classrooms, and post-intervention data were completed for n = 225 students (97.8% completion rate). At baseline, just over half of the students were females (n = 121; 52.6%), and the average age was 7.96 years (SD: 1.76, range 6–13 years). Most students were in Grade 2 (n = 141, 61.3%), while 18.7% (n = 41) were in Grade 1. The remaining 20% were in Grade 5. Average SDQ total difficulties scores of the whole sample at baseline reveal moderate levels of EBPs (mean score = 6.63, SD = 5.87, range: 0–28). No significant gender differences on SDQ total difficulties scores were observed at baseline (female: mean = 6.07, SD = 5.77 vs. male: mean = 7.30, SD = 5.93; t(228) = 1.66, p = .10). There were significant gender differences at baseline on the externalising difficulties, hyperactive and prosocial sub-scales, with males exhibiting higher scores on these sub-scales than females (Table 1). This indicates that teachers observed more behavioural problems among male students compared with female students.
Strengths and Difficulties Questionnaire characteristics at baseline by gender.
M: mean; SD: standard deviation.
Results
Primary outcome
The mean change on all continuous outcome measures related to the SDQ is displayed in Table 2. The results for SDQ total difficulty scores demonstrated an improvement in scores over time from baseline to post-intervention (modelled means = 6.63 vs. 4.16, B = −2.46, 95% confidence interval (CI) (−3.09, −1.84), t(223.84) = −7.80, p < .001). In addition, improvements over time were detected on the externalising difficulties sub-scale (modelled means = 3.63 vs. 2.47, B = −1.16, 95% CI (−1.53, −0.79), t(225.96) = −6.18, p < .001), and the internalising difficulties sub-scale (modelled means = 3.00 vs. 1.69, B = −1.30, 95% CI (−1.70, −0.91), t(222.71) = −6.47, p < .001). Associated effect sizes are reported in Table 2.
Mean change from baseline to post-test (T1) and effect sizes for total sample.
Note: Raw means (M) and standard deviations (SDs) presented. Cohen’s d values are time effects for pre-intervention to post-intervention using the modelled mean difference divided by the sample pre-intervention SD. CI: confidence interval; SDQ: Strengths and Difficulties Questionnaire.
Moderator analysis
Gender, dose and fidelity were examined as potential moderators of the pre- to post-intervention changes in SDQ total difficulties scores, as were externalising difficulties scores and internalising difficulties scores.
Gender
The results showed that gender did not moderate the time effect on SDQ global difficulties scores (B = 0.05, 95% CI (−1.20, 1.30), t(224.42) = 0.08, p = .938), indicating that both males and females had a similar decrease in SDQ global difficulties scores from pre- to post-intervention. Similarly, there were also non-significant Gender × Time interactions for SDQ internalising difficulties scores (B = −0.50, 95% CI (−1.29, 0.29), t(222.11) = −1.24, p = .217) and SDQ externalising difficulties scores (B = 0.55, 95% CI (−0.19, 1.30), t(226.47) = 1.48, p = .141), suggesting that both males and females had similar decreases in SDQ internalising difficulties scores and SDQ externalising difficulties scores from pre- to post-intervention.
Fidelity
The average fidelity score was 17.1 (SD = 4.0, range 11–23.5). The average scores for each of the six fidelity categories are presented in Table 3. Scores, on average, were lowest for the adherence to the rule count system (i.e., only negative or off-task behaviours should incur a point) and the adherence to the reward system (i.e., teams who score less than ‘3’ should be rewarded, rewards should be given to the whole class rather than individualised).
Averaged scores for fidelity scale categories across the two observation points.
SD: standard deviation.
The LMM procedure also confirmed a significant Fidelity × Time effect (B = 0.19, 95% CI (0.03, 0.34), t(225.56) = 2.39, p = .018). This interaction reflected a pre- to post-intervention decrease on the SDQ global difficulties score when fidelity was high (i.e., one SD above the mean on the fidelity variable), B = −1.72, 95% CI (−2.58, −0.86), z = −3.92, p < .001, but an even greater pre- to post-intervention decrease on the SDQ global difficulties score when fidelity scores were lower (i.e., one SD below the mean on the fidelity variable), B = −3.23, 95% CI (−4.11, −2.35), z = −7.19, p < .001, see Figure 1). Based on the modelled mean difference between pre- and post-intervention, and the SD at pre-intervention, the decrease in SDQ global difficulties scores when the intervention was delivered with high fidelity and when it was delivered with low fidelity corresponded to effect sizes (Cohen’s d) of 0.34 and 0.63, respectively.
Fidelity by time interaction on SDQ ‘total difficulties’ scores. SD: standard deviation; SDQ: Strengths and Difficulties Questionnaire.
Dose
The mean number of weeks that the Good Behaviour Game was delivered for was 11.1 weeks (SD = 1.3, range: 8–12 weeks). A significant Dose × Time effect was found (B = −0.86, 95% CI (−1.33, −0.40), t(223.04) = −3.66, p < .001). This interaction reflected a significant pre- to post-intervention decrease on SDQ global difficulties scores when dose was high (i.e., one SD above the mean on the dose variable), B = −3.60, 95% CI (−4.46, −2.74), z = −8.24, p < .001, and a smaller yet still significant pre- to post-intervention decrease on SDQ global difficulties scores when dose was low (i.e., one SD below the mean on the dose variable), B = −1.35, 95% CI (−2.20, −0.51), z = −3.14, p = .002 (Figure 2). Based on the modelled mean difference between pre- and post-intervention, and the SD at pre-intervention, the decrease in SDQ global difficulties scores when dose was high and when dose was low corresponded to effect sizes (Cohen’s d) of 0.71 and 0.27, respectively.
Dose by time interaction on SDQ ‘total difficulties’ scores. SD: standard deviation; SDQ: Strengths and Difficulties Questionnaire.
Discussion
The aim of this study was to establish whether the Good Behaviour Game could be delivered by teachers in Australian primary schools to improve EBPs among students. The high average dose (11.1 of 12 weeks) and fidelity scores (17.1/24) suggest that teachers were able to successfully implement the programme, and this is reflected in the primary outcome finding that within 12 weeks, the programme was able to significantly lower EBPs. Whether these gains were maintained over time is unknown, but previous evaluations of this programme indicated that improvements across indicators of EBPs are evident at up to 60 or more months post-implementation (Kellam et al., 2011). Moreover, while the absence of an RCT design limits our ability to causally attribute the observed effects to the programme, these findings are consistent with what has previously been reported in other RCTs of the Good Behaviour Game (Flower et al., 2014). When considered in the context of broader evidence base, the current study suggests that the Good Behaviour Game is a strategy that teachers can use to promote students’ healthy development and well-being, at least in the short term, while providing additional evidence that the effects of the programme seem robust and resilient to context-specific factors.
Improvements in EBPs were driven by reductions across externalising and internalising difficulties domains. These findings are consistent with the previous studies (Dolan et al., 1993; Flower et al., 2014; Vuijk, van Lier, Crijnen, & Huizink, 2007), where particularly robust evidence has been established for reductions in early externalising behavioural deficits (e.g., hyperactivity) (Flower et al., 2014). Programmes which can protect against externalising difficulties during childhood appear to be particularly important in the Australian context. For example, ADHD has been identified as the most common disorder among Australian children aged 4–11 years (prevalence of 8.2%), accounting for 60% of the mental health burden in this age-group in Australia (Lawrence et al., 2015). Moreover, rates of ADHD are two-fold higher among boys aged 4–11 years than females (10.9% and 5.4% respectively) (Lawrence et al., 2015), signalling that programmes which can prevent emerging externalising problems from establishing are likely to enable healthy developmental trajectories for males.
Prior studies suggest that Good Behaviour Game has a less lasting impact on internalising difficulties, with some studies reporting short-term improvements in shy or anxious behaviours (Dolan et al., 1993; Vuijk et al., 2007), but no evidence that these effects are maintained beyond 12 months (Kellam et al., 2011). Anxiety in the school setting can have particularly negative effects, hindering socioemotional development, which can lead to lower perceptions of social acceptance, lower self-esteem, difficulty forming friendships and difficulties in academic achievement (Weeks et al., 2009). Programmes that effectively target early emotional problems (e.g., shyness) to prevent anxiety from establishing are critical, but particularly so in the Australian context as anxiety is the second most common mental disorder among Australian children aged 4–11 years (7.6% in males and 6.1% in females) (Lawrence et al., 2015). Given that childhood anxiety elevates the risk for anxiety in adulthood (Vallance & Fernandez, 2016), it is unsurprising that it is the most common mental health disorder among Australian adults (Slade et al., 2009), Accordingly, interventions other than the Good Behaviour Game – which has yet been unable to demonstrate lasting impact on internalising problems – which target anxiety may need to be introduced at multiple points during a child’s developmental trajectory (i.e., primary and secondary school) to prevent the maladaptive functioning associated with anxiety that can lead problems later.
Examination of potential teacher-controlled moderators (fidelity and dosage) identified an anomalous finding, that lower fidelity had a greater impact on efficacy. The most likely explanation for this inverse association is that teachers modified aspects of the programme, which would be considered a deviation from the manual and resulted in lower fidelity scores. Certainly, the average fidelity scores were lower on indicators that were more prone to modification (i.e., modifying the scoring system, individualising rewards rather than delivery whole-of-classroom rewards), consistent with classroom observations of teacher’s delivery. These findings suggest that modifications may have been intentional, potentially informed by teachers’ practice wisdom, understanding of classroom dynamics or knowledge of student’s personalities. Given that modifications could potentially improve student and/or teacher engagement with the programme, it is important that future research explore the need for, and impact of, programme modification as it is likely to be of relevance when considering scaling up this programme in differing cultural settings.
Implications
This study provides preliminary evidence that the Good Behaviour Game could be effectively delivered by teachers in Australian schools as part of primary prevention efforts related to healthy development and well-being. The small scale of this study, however, means that it remains to be understood how the Good Behaviour Game could be embedded into teaching practices, at scale, to achieve broader prevention impact. Certainly, scaling interventions across schools, nationally, is the necessary to provide upstream protection against the mental disorder burden (Wyman, 2014). To progress towards scalable, evidence-based teaching practices for students’ well-being, training and delivery models that can enable integration of well-being approaches into usual classroom practices are needed. An example of this may be that universities start to incorporate best practice prevention programmes, like the Good Behaviour Game, into teaching degrees so that well-being strategies are embedded in standard teaching practices.
To enable effective interventions to be implemented in primary schools, information on best practice programmes and their availability (access to training, cost) needs to be easily accessible by key decision makers. Such information is not readily available, or held within a single repository, in Australia. Consideration should be given to developing a consolidated online register for teachers, principals, school administrators and educational stakeholders to facilitation knowledge transfer and to provide information as to what programmes have been rigorously evaluated, versus those which have not, and programmes which may even be harmful.
Finally, as implementation factors (fidelity and dose) were identified as mediating factors in the improvement of EBPs, consideration needs to be given to what fidelity looks like in the Australian context and how prescriptive delivery needs to be, given the anomalous finding that lower implementation fidelity correlated with greater reductions in EBPs. Going forward, it would be important to discern what are the ‘core’ components of programme implementation that need to be adhered to with high fidelity and where modifications could be pursued without jeopardising the programme’s effectiveness. Once a more nuanced understanding of what programme components require high fidelity implementation, consideration should be given to how support this. This may consist of training a cohort of individuals (‘coaches’) in the programme whose role is to observe delivery and provide implementation support while teachers are learning to embed the programme in their classrooms. Moreover, teacher training may need to be modified to indicate where deviation is not acceptable and where modifications are appropriate.
Limitations
As with all studies, the findings of this research should be considered in the context of certain limitations. Firstly, while a high rate of follow-up data was collected, SDQ data were only collected from teachers and were not cross-validated with other sources (e.g., parents), potentially biasing the results. However, it has been shown that the internal consistency, test–retest reliability and inter-rater agreement are satisfactory for parent and teacher versions of the SDQ and that at the sub-scale level, the reliability of the teacher version was found to be stronger compared to that of the parent version (Stone, Otten, Engels, Vermulst, & Janssens, 2010). Secondly, in respect to the observation of implementation, the research staff had not only to observe what teachers were doing but also to simultaneously evaluate teacher performance at the same time. The complexity of this cognitive task may potentially account for anomalies in the fidelity scoring if aspects of delivery were missed or misinterpreted. Thirdly, this trial used a non-experimental design and did not have a control arm nor randomisation characteristics. As such, causality cannot be directly attributed to the intervention. Fourth, as this study only used a pre–post design, it is unclear whether the improvements in EBPs observed at post-intervention were maintained longer term. Longer term follow-up is needed. Finally, the differences in data collection methods (face-to-face vs. telephone) mean that data could be vulnerable to subjectivity. However, as the data collected was collected using a structured survey instrument, any effect of the collection method is likely to be minimal.
Conclusion
This study indicates that the Good Behaviour Game could be integrated into standard teaching practice as part of school-based upstream prevention efforts to promote children’s development and well-being. Incorporated well, and at scale, this programme has potential to provide transdiagnostic protection against mental disorder in later developmental stages by reducing early EBPs. However, further rigorous implementation research is needed to identify local social, cultural and other factors that may inhibit adoption and implementation fidelity and how such factors may impact the long-term effectiveness of the programme.
Footnotes
Acknowledgements
The authors would like to thank the teachers and their students for their enthusiasm in this project as well as Mark Brennan and Tina Hosseini for supporting the Victorian schools to participate in the trial.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article:
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by funding from the Society for Mental Health Research and the Perpetual IMPACT Funding. The funders had no involvement in the study design, collection, analysis, or interpretation of the data. Torok is supported by an Australian NHMRC Early Career Fellowship, Calear is supported by an Australian NHMRC Career Development Fellowship and Werner-Seidler is supported by an NSW Health Early-to-Mid Career Fellowship.