Sage Journals: Discover world-class research

Abstract

Check-In Check-Out (CICO) is a widely used intervention for students who need more intensive behavior support in school. Although CICO support has been extensively studied, there is still a lack of large-scale studies on the effectiveness of the CICO support as well as studies of moderators of the intervention effect. In this study, we evaluated the outcomes of the CICO support in Finnish schools and the effects of different moderators on behavioral change. Fifty-one students participated in the study. Positive and problem behaviors of the participating students were followed before, during and after CICO support. Moderating variables were individual-level factors. At the group level, the effect sizes for change in behavior were large. Individual-level factors were not connected to changes in behaviors. CICO support appeared effective across all grade levels, regardless of the pupil’s need for pedagogical support, behavioral diagnosis, or preintervention problem behavior level.

Keywords

problem behavior behavioral intervention Check-In Check-Out School-wide Positive Behavioral Support intervention moderators

The problem behavior of the students is one of the big challenges in schools. Behaviors such as talking out of turn, getting out of one’s seat, difficulties in task completion, noncompliance, and defiance, may exert a detrimental influence on the classroom climate and personnel’s and students’ overall well-being. In addition, students demonstrating problem behavior confront both social and academic challenges (Sortkær & Reimer, 2018).

Behavioral Support Intensification

School-wide Positive Behavioral Interventions and Support (SWPBIS) builds on universal support aimed at preventing school-wide problem behavior (Horner et al., 2010). The whole school commits to shared behavioral expectations, which are routinely taught in classrooms and in all other areas of the school. Expected behaviors are reinforced by active targeted positive feedback, which is often connected to a reward system. Key components of SWPBIS include an agreement on consequences for unwanted behaviors and having a system of a continuum of more intensive supports that are easily available after it becomes evident that universal support is not enough for some individuals (Sugai & Horner, 2020).

One method to intensify behavior support that is widely used in SWPBIS schools is Check-In Check-Out (CICO; Hawken et al., 2020). The key theoretical elements in CICO support are antecedent strategies, such as precorrection, systematic positive feedback, and reinforcement when the goals are met (Bundock et al., 2020). In CICO support, the goals set for individuals play a particularly important role. Needs-based and concrete goals linked with universal behavior expectations support the students’ ability to perform meaningful tasks and change behavior (Pfiffner et al., 2006). Moreover, the specificity of the goals ensures that students have a clear understanding of what is expected of them, facilitating their engagement and commitment to the actions (Bandura, 1997). Targeted intervention provides immediate positive feedback and praise, and together with positive adult interactions promotes positive change toward the expected behaviors (Hawken & Horner, 2003).

The key practical elements of CICO support can be divided into five sections (see Hawken et al., 2020). (a) Students start their morning with a short check-in meeting with the CICO coordinator, during which the coordinator reminds them of the behavior expectations. (b) Throughout the day, teachers provide feedback to students, both verbally and on the Daily Report Card (DRC), on the target behavior they are practicing. (c) At the end of the day, students meet with the CICO coordinator to determine whether the daily goals have been met and to receive a reinforcement for their efforts, and finally, (d) the DRC is sent home for guardians to review. (e) Finally, the CICO processes include systematic data collection that enables information-based decision-making and progress-monitoring of the intervention. Delivery of all these key practical features of the CICO support is necessary for achieving desired outcomes (Filter et al., 2022). The Finnish version of CICO is contextualized to Finnish schools and support system. However, all the necessary elements are present. Finnish contextualized CICO support begins with a student-specific behavior assessment, similar to Carr et al.’s (2002) modified functional behavior assessment. This is done by a multiprofessional evaluation that includes the views of teachers and special education teachers (Björn et al., 2016). Each student participating CICO support in Finland has one to two individual behavior goals that are tailored to his or her classroom environment. The individual goals may be specific and verbally different from the universal behavior expectations at the school level but are set in accordance with the universal behavior expectations.

The three-tiered support system in Finland is an administrative framework to coordinate and structure support measures for students in basic education (Björn et al., 2016). In Finland, the three tiers are known as universal, intensified, and special support (Basic Education Act, 2010). A key difference from the tier model in many other countries, such as the United States, is that special education teachers in Finland can provide special education support at all three tiers (Björn et al., 2016). While CICO support is a form of additional support for some individuals, it was never officially tied to any specific Tier of the Finnish support system. Consequently, CICO support can be provided at any level of the three-tiered Finnish educational support system.

Research on the Effectiveness of CICO Support

Evidence from previous research largely suggests that CICO support is effective for students demonstrating various problem behaviors, such as attention seeking (McIntosh et al., 2008), escape (Drevon et al., 2019), and off-task behaviors (Miller et al., 2015). A large number of studies are single case studies with different outcome measures: changes in problem behavior, improvements in academic engagement and achievement (Drevon et al., 2019; Hawken et al., 2014; Maggin et al., 2015), and also on social skills (Park & Blair, 2020). In their review article, Hawken et al. (2014) reported median percentage nonoverlapping data points (PND) for single single-case studies. The review results revealed that, across 20 single-case studies, the overall median Percentage of Nonoverlapping Data (PND) was 68%, indicating questionable effectiveness. In the review of Maggin et al. (2015) both visual analysis and nonparametric quantitative methods were applied to evaluate effectiveness of the CICO support. The results revealed that 80% of the single cases included in the review exhibited a positive intervention effect as determined through visual analysis. Nonparametric quantitative analysis corroborated these findings by demonstrating statistically significant intervention effects, aligning with the outcomes of visual analysis.

In their meta-analysis Drevon et al. (2019) synthesized the effect sizes from 32 CICO studies, encompassing both single-case and between-group experimental designs. Using robust variance estimation, their results showed that CICO support had effects on student behavior in both single-case and between-group experimental conditions with improvements of more than one standard deviation. A meta-analysis conducted by Park and Blair (2020), which focused on between-group experimental studies, reported medium effect sizes (g = 0.42). Conversely, Hawken et al. (2014), in their research involving group designs, reported effect sizes spanning from small to moderate (median d = .40; range, .25 to .60). However, in their systematic review, comprising only two studies, Maggin et al. (2015) identified no significant intervention effects in between-group experimental studies. Maggin et al. (2015) suggested that the inclusion of participants with varying behavioral functions may have contributed to the limited intervention effects. A recent study of CICO support for elementary school students with autism spectrum disorder (ASD) found no significant reduction in problem behaviors and only a limited increase in compliance with school expectations, suggesting that some adaptations beyond traditional CICO support may be needed for individuals with ASD (Carpenter et al., 2023).

The change observed has not always been just a reduction in problem behavior but also an increase in academic engagement (Drevon et al., 2019; Miller et al., 2015; Park & Blair, 2020). However, there are also contrasting views on the impact of CICO support in this regard. According to Wolfe et al. (2016), CICO support did little to increase desirable or appropriate behavior. Their systematic review found little positive impact on academic engagement.

A few studies have examined the potential moderating factors, including student-related factors, such as grade level, special education status and behavior type, that influence the effectiveness of CICO interventions on various outcome measures, including reductions in problem behavior, improvements in academic engagement and achievement, and social skills (Drevon et al., 2019; Hawken et al., 2014; Park & Blair, 2020). A review study by Hawken et al. (2014) and a meta-analysis by Park and Blair (2020), which included a sample of studies conducted with elementary and middle school students suggested that the effectiveness of CICO support may vary by student grade level of students and may be more effective in elementary schools than in secondary schools. However, a meta-analysis of systematic reviews by Drevon et al. (2019), which included samples of students both from elementary and secondary school, found conflicting results, suggesting that age did not significantly explain the heterogeneity in intervention outcomes. Furthermore, the proportion of students receiving special education in the samples did not moderate the outcomes of CICO support (Drevon et al., 2019). Previous findings on other psychosocial interventions, such as parenting programs for child conduct problems (Leijten et al., 2020) and cognitive behavior therapy for adolescents, suggest that child diagnoses, such as oppositional defiant disorder and conduct disorder with comorbid attention deficit hyperactivity disorder (ADHD), do not moderate the intervention effects (Leijten et al., 2020; Smeets et al., 2015). However, there is evidence that children with more severe conduct problems may benefit even more from behavior intervention programs (Leijten et al., 2020). Given that previous research on how the characteristics of students moderate the effectiveness of CICO support is scarce and inconclusive, in this study, we examined whether intervention effects vary according to grade level and initial level of problem behavior. In addition, we compared the intervention effects between participants with or without special educational support or neuropsychiatric diagnoses (ADHD and ASD).

Present Study

Check-In Check-Out support has been quite widely studied (Drevon et al., 2019); however, only a few large-scale studies have been conducted (Hawken et al., 2014, 2015), while none have been conducted in Finland. Therefore, knowledge regarding the overall effectiveness of CICO is insufficient, and specific knowledge on how effective CICO support is in Finland for different groups of students is lacking. To address the question of whether CICO support increases desirable behavior and decreases problem behavior, in this study, a change in both set target behaviors and in the prevalence of problem behaviors in different school situations and different groups of students was estimated. In practice, changes in target behaviors were assessed with DRC and changes in behavior in different locations in school and generalization of behavior change were evaluated with the School Situations Questionnaire (SSQ; Barkley, 1981). The following research questions were set:

Research Question 1 (RQ1): Does the CICO support lead to a positive change in the outcomes set as goals for the individuals and do the changes generalize to other school situations?

Research Question 2 (RQ2): Do student-related factors, students’ grade level or tier level or having a neuropsychiatric diagnosis moderate the effect of the intervention?

Research Question 3 (RQ3): Is the level of problem behavior observed at baseline related to the results of the intervention?

Method

Participants

Fifty-one students and their teachers from 11 schools in Eastern and Central Finland participated in the study, which was carried out as part of a larger Erasmus research project (https://www.pbiseurope.org). The culturally modified Tiered Fidelity Assessment (Algozzine et al., 2019) was used to assess the fidelity of the universal SWPBIS in the 11 participating schools twice a year. The level of fidelity of the participating schools ranged from 30% to 96.7% (average 78.8%; for three schools, fidelity was less than 70%). The inclusion criteria that the schools used for the student participant selection were as follows: Students having problem behaviors in the school environment to such a degree that they impaired student’s social adaptation and academic progress. The final decision concerning students’ inclusion in the intervention was entrusted to the team responsible for executing the CICO support in each school. The behavioral goals delineated by the school staff to facilitate students enrolled in the CICO support could be classified into three principal categories: (a) fostering students’ orientation toward academic learning and tasks (e.g., “I complete the tasks assigned to me”) (b) upholding respectful behavior and language (e.g., “I give others space and time to work.”), and (c) maintaining positive interaction within the classroom (e.g., “I listen without interrupting).

The criteria for selecting students for CICO support were aligned with the normative practices of need-based learning support in the Finnish three-tiered support system (Finnish Ministry of Culture and Education, 2010). Learning support in Finnish schools is flexible and in addition to traditional special class instruction, so called part-time special education teachers support students in everyday schoolwork. This support can be provided to students at any of the three tiers (Björn et al., 2016). Preintervention assessments using the problem behavior inventory, School Situation Questionnaire, revealed that participants had on the average high levels of problem behavior (M = 5.35, SD = 1.32), with scores ranging from 1.33 to 8.00 (with a maximum possible score of 9), and only three participants had a mean problem behavior score below 3.00. It has been proposed that a score of 5 to 6 on the SSQ represents the threshold for clinical significance (Altepeter & Breen, 1989).

Students receiving CICO support were from Grades 1 to 6 and the age ranged from 6 to 13 years (M = 8.87, SD = 1.72; see Table 1). CICO support was offered mostly for boys (86.5%; see Table 1). Fifty-six percent of the students had previously been identified by the school personnel as needing Tier 2 (intensified) or Tier 3 (special) level support. In most cases, the support had been targeted at both academic learning and behavior. At the same time 34% of all the students had a neuropsychiatric diagnosis (ADHD or ASD). All participants expect one had Finnish as their first language. Participation in the study was voluntary, and consent was received from the students’ guardians and school personnel.

Table 1.

Distribution of Students in Check-In Check-Out Support by Grade-Level, Age, and Gender.

Grade	Number of pupils	Proportion	Age (years), M (SD)
First	11	22.0	7.1 (0.74)
Second	11	22.0	7.9 (0.71)
Third	10	18.0	9.00 (0.71)
Fourth	10	20.0	10.2. (0.79)
Fifth	3	6.0	11.5 (0.71)
Sixth	6	12.0	12.3 (0.50)
Gender
Girls	6	11.8
Boys	45	88.2
N	51

Procedures

School personnel from the participating schools were provided with two to three training sessions by the researchers, and training attendees received a handbook with detailed instructions for conducting the intervention. The training was attended by both experienced and novice CICO support practitioners. In six schools, staff members had prior experience with CICO support, as they had been implementing CICO support for several years. For the school personnel who were new to CICO support, they participated in two extra training sessions and were provided with coaching to assist them in implementing and organizing CICO support within their respective schools. A total of 61 school staff members, including 34 teachers and 27 CICO coaches, participated in the implementation of CICO support.

Each school organized behavioral support to align with their unique special education activities. Despite the differences in the organization of special education, it was a requirement for all schools to establish a dedicated CICO team that included the teacher coordinating CICO support activities, special education teacher or one classroom teacher and CICO coach. The team was responsible for the selection of students for CICO support, in collaboration with the students’ classroom teachers, and the team provided support for high-fidelity implementation of CICO support. A student background information survey was completed by teachers and parents. The teachers were asked to provide information on their students’ special educational needs. Information was collected from parents on whether their children had a neuropsychiatric diagnosis. The teachers were also asked to describe the students’ problem behavior and the situations in which the problematic behavior occurred. The students’ own behavioral goals were set based on the problem description. Behavioral goals were to be one to three at a time for each student. Check-In Check-Out teams ensured that the goals were aligned and closely linked to the school’s universal level behavioral expectations.

At the beginning of the support period, the teachers were instructed on how to use the DRC and reminded to use verbal feedback alongside the DRC. When using the DRC, teachers were instructed to mark the score after each lesson to indicate the degree to which behavior expectations were met, and to provide encouraging verbal feedback while marking the score. CICO coaches, who were usually paraprofessionals, took care of daily check-in and check-out meetings. Together with the students’ teachers, the teams evaluated students’ response to CICO support and made decisions about when the CICO support period could end. These decisions were made individually, and the duration of the CICO support period ranged from 5–17 weeks. Some of the CICO support processes were long due to willingness to ensure maintenance of the positive intervention effects.

Procedural fidelity of the intervention was assessed daily with a checklist. The checklist was filled in by the CICO coach after each school day. The fidelity checklist that consisted of the implementation components and practices was given to CICO coaches to be completed daily during the intervention period: check-in, check-out, active use of DRC, recording and follow-up of the DRC scores, and parents’ involvement (parents’ signings of the DRC). For each day, five was the maximum value of procedural fidelity. Proportions of the maximum number of checklist counts were calculated for each participant. Overall fidelity in the present sample was high (mean 91.56%, SD = 0.09), and only four participants had lower than 80% fidelity (ranging from 66% to 77%). These four low fidelity intervention periods occurred in two different schools and the reasons for low fidelity were lack of information on daily parent involvement (lack of parent signature on the DRC) and occasional lack of regular team or teacher follow-up on DRC scores.

Measurements

The outcomes of the CICO support period were followed and investigated using two assessment tools and repeated measurements. First, students’ progress in individually set goals was followed by the DRC. At the end of each lesson, students received feedback, and by using the DRC, their behavior was evaluated in relation to individual behavioral goals. The DRC had a three-point scoring criterion ranging from 0 to 2, (0 = “expectations not met,” 1 = “expectations partly met” and 2 = “expectations fully met”). Each student had one to three goals and the relative proportions of the maximum points converted into percentages indicated daily scores. The DRC has been found to be sensitive to change in classroom intervention studies (e.g., Jurbergs et al., 2007). Repeated measures were completed at the baseline, intervention, and follow-up phases. Daily percentages were obtained for a minimum of 3 days before the intervention phase.

Second, changes in students’ problem behaviors were assessed using the SSQ (Barkley, 1981; Appendix). The SSQ was filled in by the teacher at the end of each school week. The SSQ was used to measure problem behavior (a) in classroom situations and (b) outside the classroom in other areas and situations of a school. In the SSQ, school personnel are required to answer the question “Does this child present any behavior problems for you in any of these situations?” altogether in 12 different situations (e.g., “During individual desk work” and “During small group activities”) that are assessed with a Likert-type scale ranging from 0 (no problems) to 9 (severe problems). During the data collection, due to the COVID-19 pandemic, schools did not organize field trips and special assemblies; therefore, items concerning these situations were excluded from the data. In addition, transportation on the bus to schools is not a common practice in Finland; therefore, the item “while on the bus,” which is present in the original SSQ scale, was excluded. In this study, nine items were used: five items describing classroom situations and four items referring to other situations in school. Cronbach’s alpha in school-age samples has shown good internal consistency for the SSQ, with alphas ranging from .84 to .91 (Pelletier et al., 2006). In the present sample, Cronbach’s alphas were .91 for classroom situations and .80 for other situations in schools. Repeated measures were completed at the baseline, intervention, and follow-up phases. In the baseline phase, the SSQs were to be completed 3 times within 2 to 3 weeks. The final data revealed that out of 51 participants, 10 participants had two baseline assessments and six participants had one baseline assessment. The data were collected during the COVID-19 pandemic, and due to the pandemic and the restrictions imposed, the use of systematic observations or multidata methods was not possible.

Statistical Analyses

The first aim was to estimate changes in the level of outcome measures (behavior indicators). To do this, mean values of the outcome indicators, DRC and SSQ (classroom and other areas), were calculated for baseline, intervention-end phase (two last assessment weeks) and follow-up phase (at least three assessments). These mean values were used to analyze behavior change between phases using a paired sample t-test; the first contrasted the baseline and intervention-end phases, and the next contrasted the baseline and follow-up phases. Therefore, the primary measure of impact was the change from baseline.

To answer the second research question, the differences in degree of change in DRC and SSQ (classroom and other areas) were analyzed between grade levels, two-tier level groups and diagnosis groups. This was done using repeated measure analysis of variance (ANOVA); the baseline phase, intervention end-phase (two last assessment weeks), and follow-up phase were set as the within-subject factors, and grade-levels, two tier-level groups and diagnosis groups as between-subjects factors.

To answer the third research question, whether the initial level of problem behavior was related to behavior change, the repeated-measure ANOVA was used. The baseline phase and intervention-end phase were set as within-subject factors depicting behavior change over time, and the baseline values were set as covariates. Effect sizes were computed as Cohen’s d; positive effect size represented improvement in DRC, and negative effect sizes represented decrease in problem behaviors. The analysis was conducted using SPSS 28 software.

Results

Effects of the Intervention

Group-level analysis showed that the students benefited from the CICO support, and statistically significant changes in behaviors between baseline and intervention-end phase and between baseline and follow-up phase were detected (see Table 2). The results indicated that the targeted behaviors set for students saw improvement during the intervention period, with a concurrent decrease in problem behaviors. These positive changes persisted through the follow-up period. The shift in daily scores was noteworthy, with a 17 % change between the baseline and intervention end-phase and a 14 % change between the baseline and follow-up phase. The SSQ score indicating problem behavior in class situations fell by two points and by around one and a half points in other situations (see Table 2). The effect sizes for change were large between baseline and intervention-end phase (except SSQ other areas), as well as between baseline and follow-up phase: for DRC d = 1.08 and d = 0.90, respectively; for SSQ classroom, d = −1.26 and d = −1.30, respectively and for SSQ other areas, d = −0.65 and d = −0.81, respectively.

Table 2.

Paired-Sample t-Test Results for Comparison of the Baseline Values to the Intervention and Follow-Up Phases.

Variable	Baseline (SD)	Intervention end (SD)	Follow-up (SD)	Baseline vs. intervention end			Baseline vs. follow-up
Variable	Baseline (SD)	Intervention end (SD)	Follow-up (SD)	t-value (df)	P	d ^a	t-value (df)	p	d
DRC	65.39 (14.43)	82.44 (11.24)	79.30 (12.26)	7.39 (46)	<.001	1.08	5.30 (34)	<.001	0.90
SSQ classroom	5.34 (1.34)	3.19 (2.08)	3.30 (2.10)	8.64 (49)	<.001	−1.26	7.42 (49)	<.001	−1.30
SSQ other areas	3.92 (1.99)	2.50 (2.39)	2.38 (2.16)	4.84 (49)	<.001	−0.65	5.28 (49)	<.001	−0.81

Note. DRC = Daily Report Card; SSQ = School Situation Questionnaire.

Cohen’s d.

Moderator Effects

The analysis of the moderator effects revealed that grade level had no influence on the outcomes of the intervention, F(1, 44) = 0.78, p = .568. Across all grade levels, from 1 to 6, students exhibited positive changes in DRC values and a concurrent decrease in SSQ values. Similarly, the individual-level factors, tier level and neuropsychiatric diagnosis, had no statistically significant association with the outcomes of the intervention—tier-level F(1, 46) = 0.04, p = .845; diagnosis F(1, 48) = 0.61, p = .607. The DRC and SSQ values at different time points for the two diagnosis groups and two tier-level groups are presented in Table 3. Both DRC and SSQ values indicated similar progress for students in different tier levels and for students with or without neuropsychiatric diagnosis. Similarly, the variance of problem behavior at the baseline was not associated with the intervention effects, F(1, 48) = 0.53, p = .551.

Table 3.

DRC and SSQ Scores at Pre-, Post-, and Follow-Up Assessments for Tier-Level and Diagnosed and Nondiagnosed Groups.

Variable	Comparison	n	Baseline (SD)	Intervention end (SD)	Follow-up (SD)^a
DRC	Tier 1	20	64.34 (12.70)	82.30 (11.11)	78.91 (9.28)
	Tier 2/3	25	66.11 (16.43)	82.51 (12.01)	80.34 (15.62)
SSQ classroom	Tier 1	22	5.32 (1.35)	3.19 (1.75)	3.57 (1.92)
	Tier 2/3	26	5.20 (1.39)	3.16 (2.24)	2.86 (2.11)
SSQ other areas	Tier 1	22	4.03 (1.94)	2.40 (1.84)	2.68 (2.00)
	Tier 2/3	26	3.54 (1.95)	2.56 (2.79)	1.99 (2.28)
DRC	Dg^b No	31	64.84 (14.66)	80.57 (11.76)	78.18 (11.57)
	Dg Yes	15	66.57 (14.36)	86.43 (9.15)	81.44 (13.75)
SSQ classroom	Dg No	34	5.33 (1.47)	3.38 (2.25)	3.37 (2.30)
	Dg Yes	16	5.16 (1.10)	2.80 (1.66)	3.16 (1.17)
SSQ other areas	Dg No	34	4.05 (2.03)	2.73 (2.48)	2.45 (2.28)
	Dg Yes	16	3.40 (1.78)	2.01 (2.19)	2.23 (1.95)

Note. DRC = Daily Record Card. SSQ = School Situation Questionnaire. No statistically significant differences were found between tier levels, diagnosed and no-diagnose groups.

n = 39 for SSQ and n = 33 for DRC. ^bNeuropsychiatric diagnosis.

Discussion

Effects of the CICO Support

The first aim of this study was to examine the outcomes of CICO support for students with problem behavior. The group-level results showed that students’ individually set target behaviors increased and problem behavior decreased during the intervention period, and clear differences remained until follow-up. The effect sizes for changes in behaviors in the classrooms and in other situations outside classroom were large, consistent with the findings of the Drevon et al. (2019) meta-analysis, and larger effects than those reported in the Hawken et al. review and the Park and Blair meta-analysis. Change in problem behavior indicates a less severe manifestation of problem behavior in the classroom. Furthermore, as the questions on the classroom environment in the SSQ questionnaire mainly relate to academic learning situations, it is likely that students’ engagement in academic work has increased.

The results also demonstrated a decrease in problem behavior in other situations outside classroom (SSQ other areas) indicating that CICO support appeared to have generalized effects on behavior in other school areas. Establishing clear and agreed-upon behavioral goals, along with reminders and a well-structured support system, can promote positive behavioral changes in various situations (Pfiffner et al., 2016). Reduced problem behaviors in other settings outside the classroom is relevant to a child’s social functioning and peer relations. Positive behavior changes are likely to reduce peer conflicts and decrease stressful situations during social interactions (Ursache et al., 2012). Furthermore, success in achieving target behaviors and a reduction in problem behavior may also positively contribute to teacher–child interactions and relationships (Sutherland et al., 2018). Engaging in sustained positive interactions with peers and teachers may have a lasting impact on both behavioral and academic outcomes (Kiuru et al., 2020).

Examination of Potential Moderators

Second, we investigated whether any of the different student-related factors moderated the impact of the intervention. Neither factor related to students’ difficulties—the identified special educational need or behavioral diagnosis—was associated with the intervention outcomes. This result is in line with previous findings (Drevon et al., 2019; Smeets et al., 2015) and shows that students’ learning difficulties or severe problem behavior leading to a diagnosis do not impede the benefits of the intervention. It is worth noting that only two of the participants had a diagnosis of ASD, and their results were inconsistent: one benefited from the intervention, while the other did not. Consequently, we lack strong evidence regarding the effectiveness of CICO support specifically for students with ASD (Carpenter et al., 2023).

Since the problem scores (SSQ) among the subjects in the baseline phase were considerably high (mean classroom situation score 5.33, range of scores 1.33–8.00), it can be assumed that the CICO support was mainly started when the behavioral problems were relatively severe. However, the results showed that the severity of the problems was not related to the effect of the intervention, which also shows that the CICO is beneficial for both low level and more severe problems.

The results also gave no indication that grade level would have had any impact on the results of the intervention. Previous research has found variation in intervention responses between students in primary and secondary schools (Hawken et al., 2014; Park & Blair, 2020). Based on our findings, we can assume that, in Finnish schools, the intervention can be beneficial at all grade levels in elementary school. Although the intervention had a positive impact at the group level, there was still variation between individuals in the extent to which they benefited from the intervention. Notably, this variability was found to be unrelated to the student-related factors designated as moderator variables in our study. Future research should explore which factors, other than those in this study, may contribute to the variation in intervention outcomes.

Limitations

The study was conducted in a regular school environment and as part of everyday school life, which exposed it to some limitations. A major limitation was that the teachers rated the assessments, both the DRC and the problem behavior data, and that the data on interobserver agreement (IOA) were missing. This limitation regarding IOA also applies to the CICO fidelity checklists completed by the CICO coaches. In the future, the use of systematic observations or multiinformant methods is recommended to diminish possible biases in the data. Second, the fidelity of the intervention was evaluated using a checklist to assess procedural fidelity. However, variations in intervention fidelity can be attributed to the quality of interaction, which is an independent variable distinct from procedural fidelity. Consequently, future studies may benefit from the incorporation of observational techniques to assess fidelity. Third, participating schools had the freedom to organize behavior support in line with their own special education support activities, which may have contributed to some variation in implementation between schools. As the number of students receiving CICO support varied significantly between schools (ranging from 1 to 13 students), it was not possible to test for within-class variation.

Conclusions

Hawken et al.’s (2015) vision for the CICO support method was that “CICO is designed to provide students engaging in minor problem behaviors (e.g., off-task behavior in class, talking back to teachers, etc.) with a structured way to learn appropriate behaviors through positive reinforcement.” (p. 305). Even so, the findings of this study revealed that the severity of the problems did not influence the outcomes. Thus, a key implication of the study is that the methods employed in CICO support demonstrate effectiveness for students with diverse problem behavior. However, it is recommended that systematic behavior support is initiated early enough to provide effective support for students that prevents the escalation of problem behavior and supports academic learning. In addition, if a sufficient intervention response is not achieved, incorporating social, learning and executive skills practices can help to promote positive school and classroom behavior and increase school engagement (Cheney et al., 2010; Karhu et al., 2020). This may be particularly relevant in the context of students exhibiting symptoms associated with ASD. For instance, in this study, two students with ASD demonstrated disparate outcomes following the intervention. In addition, in the study conducted by Carpenter et al. (2023), the application of traditional CICO support did not yield the anticipated intervention effects. Further research is required to elucidate the efficacy of school-based interventions for children exhibiting symptoms of ASD.

The other significant finding of the study was that the intervention had a generalized effect. Although the behavioral objectives are explicitly linked to specific behaviors in classroom activities, the results suggest that the effects of the intervention generalized to other behaviors and environments in school. It is possible that during the intervention period students learned behaviors that are beneficial in multiple situations. In addition, successes in target behaviors may have created positive cycles that increased positive interactions with adults and more positive perceptions of students who received CICO support. It is important to recognize that the favorable outcomes of behavioral interventions may result in additional positive consequences. As problem behavior are also known to have many negative consequences for other students and teachers, individual level preventive measures are likely to benefit the whole class and contribute to improving the classroom climate and teacher well-being. This underscores the significance of implementing efficacious behavioral support strategies within the school setting.

Footnotes

Appendix

School Situations Questionnaire (Barkley, 1981).

Does this child present any behavior problems for you in any of these situations? If so, indicate how severe the problems are.

Authors’ Note

Location of the study: Finland.

ORCID iDs

Mika Paananen

Anne Karhu

Ethical Considerations

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Prior to commencement, the study was evaluated by the Ethical Committee of the University of Jyväskylä.

Consent to Participate

Written informed consent was obtained from all legal guardians of the children and from the teachers included in the study.

Funding

This study was funded by the European Commission (Erasmus+ nos. 606687).

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Algozzine

Barrett

Eber

George

Horner

Lewis

Putnam

Swain-Bradway

McIntosh

Sugai

(2019). School-wide PBIS tiered fidelity inventory. OSEP technical assistance center on positive behavioral interventions and supports. https://www.pbis.org

Altepeter

T. S.

Breen

M. J.

(1989). The Home Situations Questionnaire (HSQ) and the School Situations Questionnaire (SSQ): Normative data and an evaluation of psychometric properties. Journal of Psychoeducational Assessment, 7(4), 312–322. https://doi.org/10.1177/073428298900700404

Barkley

R. A.

(1981). Hyperactive children: A handbook for diagnosis and treatment. The Guilford Press.

Bandura

(1997). Self-efficacy: The exercise of control. Freeman.

Björn

P. M.

Aro

M. T.

Koponen

T. K.

Fuchs

L. S.

Fuchs

D. H.

(2016). The many faces of special education within RTI frameworks in the United States and Finland. Learning Disability Quarterly, 39, 58–66. https://doi.org/10.1177/0731948715594787

Bundock

Hawken

L. S.

Kladis

Breen

(2020). Innovating the check-in, check-out intervention: A process for creating adaptations. Intervention in School and Clinic, 55(3), 169–177. https://doi.org/10.1177/1053451219842206

Carpenter

M. E.

Y. Y.

Walker

V. L.

Masud

A. B.

Tapp

M. C.

(2023). Effects of check-in/check-out on the behavior of students with autism spectrum disorder who have extensive support needs. Psychology in the Schools, 60(9), 3504–3519. https://doi.org/10.1002/pits.22934

Carr

E. G.

Dunlap

Horner

R. H.

Koegel

R. L.

Turnbull

A. P.

Sailor

Anderson

J. L.

Albin

L. K.

Koegel

L. K.

Fox

(2002). Positive behavior support: Evolution of an applied science. Journal of Positive Behavior Interventions, 4, 4–16.

Cheney

Lynass

Flower

Waugh

Iwaszuk

Mielenz

Hawken

(2010). The check, connect, and expect program: A targeted, tier 2 intervention in the schoolwide positive behavior support model. Preventing School Failure: Alternative Education for Children and Youth, 54(3), 152–158. https://doi.org/10.1080/10459880903492742

10.

Drevon

D. D.

Hixson

M. D.

Wyse

R. D.

Rigney

A. M.

(2019). A meta-analytic review of the evidence for check-in check-out. Psychology in the Schools, 56(3), 393–412. https://doi.org/10.1002/pits.22195

11.

Filter

K. J.

Ford

A. L.

Bullard

S. J.

Cook

C. R.

Sowle

C. A.

Johnson

L. D.

Kloos

Dupuis

(2022). Distilling check-in/check-out into its core practice elements through an expert consensus process. School Mental Health, 14, 695–708. https://doi.org/10.1007/s12310-021-09495-x

12.

Finnish Ministry of Culture and Education (2010). Basic Education Act: Perusopetuslaki 2010. http://www.finlex.fi/fi/laki/alkup/2010/20100642

13.

Hawken

L. S.

Bundock

Barrett

C. A.

Eber

Breen

Phillips

(2015). Large-scale implementation of check-in, check-out: A descriptive study. Canadian Journal of School Psychology, 30(4), 304–319. https://doi.org/10.1177/0829573515601005

14.

Hawken

L. S.

Bundock

Kladis

O’Keeffe

Barrett

C. A.

(2014). Systematic review of the check-in, check-out intervention for students at risk for emotional and behavioral disorders. Education and Treatment of Children, 37(4), 635–658. https://doi.org/10.1353/etc.2014.0030

15.

Hawken

L. S.

Crone

D. A.

Bundock

Horner

R. H.

(2020). Responding to problem behavior in schools: The check-in, check-out intervention (3rd ed.). The Guilford Press.

16.

Hawken

L. S.

Horner

R. H.

(2003). Evaluation of a targeted intervention within a schoolwide system of behavior support. Journal of Behavioral Education, 12, 225–240. https://doi.org/10.1023/A:1025512411930

17.

Horner

R. H.

Sugai

Anderson

C. M.

(2010). Examining the evidence base for school-wide positive behavior support. Focus on Exceptional Children, 42(8), 1–14. https://doi.org/10.17161/fec.v42i8.6906

18.

Jurbergs

Palcic

Kelley

M. L.

(2007). School-home notes with and without response cost: Increasing attention and academic performance in low-income children with attention-deficit/hyperactivity disorder. School Psychology Quarterly, 22, 358–379. https://doi.org/10.1037/1045-3830.22.3.358

19.

Karhu

Paananen

Närhi

Savolainen

(2020). Implementation of the inclusive CICO Plus intervention for pupil at risk of severe behaviour problems in SWPBS schools. European Journal of Special Needs Education, 36(5), 758–772. https://doi.org/10.1080/08856257.2020.1809801

20.

Kiuru

Wang

M. T.

Salmela-Aro

Kannas

Ahonen

Hirvonen

(2020). Associations between adolescents’ interpersonal relationships, school well-being, and academic achievement during educational transitions. Journal of Youth and Adolescence, 49(5), 1057–1072. https://doi.org/10.1007/s10964-019-01184-y

21.

Leijten

Scott

Landau

Harris

Mann

Hutchings

Beecham

Gardner

(2020). Individual participant data meta-analysis: Impact of conduct problem severity, comorbid attention-deficit/hyperactivity disorder and emotional problems, and maternal depression on parenting program effects. Journal of the American Academy of Child & Adolescent Psychiatry, 59(8), 933–943. https://doi.org/10.1016/j.jaac.2020.01.023

22.

Maggin

D. M.

Zurheide

Pickett

K. C.

Baillie

S. J.

(2015). A systematic evidence review of the check-in/check-out program for reducing student challenging behaviors. Journal of Positive Behavior Interventions, 17(4), 197–208. https://doi.org/10.1177/1098300715573630

23.

McIntosh

Campbell

A. L.

Carter

D. R.

Rossetto Dickey

(2008). Differential effects of a tier two behavior intervention based on function of problem behavior. Journal of Positive Behavior Interventions, 11(2), 82–93. https://doi.org/10.1177/1098300708319127

24.

Miller

L. M.

Dufrene

B. A.

Sterling

H. E.

Olmi

D. J.

Bachmayer

(2015). The effects of check-in/check-out on problem behavior and academic engagement in elementary school students. Journal of Positive Behavior Interventions, 17(1), 28–38. https://doi.org/10.1177/1098300713517141

25.

Park

E.-Y.

Blair

K.-S. C.

(2020). Check-in/Check-out implementation in schools: A meta-analysis of group design studies. Education & Treatment of Children, 43(4), 361–375. https://doi.org/10.1007/s43494-020-00030-2

26.

Pelletier

Collett

Gimpel

Crowley

(2006). Assessment of disruptive behaviors in preschoolers: Psychometric properties of the disruptive behavior disorders rating scale and school situations questionnaire. Journal of Psychoeducational Assessment, 24(1), 3–18. https://doi.org/10.1177/0734282905285235

27.

Pfiffner

L. J.

Barkley

R. A.

DuPaul

G. J.

(2006). Treatment of ADHD in school settings. In Barkley

R. A.

(Ed.), Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (3rd ed., pp. 547–589). The Guilford Press.

28.

Pfiffner

L. J.

Rooney

Haack

Villodas

Delucchi

McBurnett

(2016). A randomized controlled trial of a school-implemented school–home intervention for attention-deficit/hyperactivity disorder symptoms and impairment. Journal of the American Academy of Child & Adolescent Psychiatry, 55(9), 762–770. https://doi.org/10.1016/j.jaac.2016.05.023

29.

Smeets

K. C.

Leeijen

A. A.

van der Molen

M. J.

Scheepers

F. E.

Buitelaar

J. K.

Rommelse

N. N.

(2015). Treatment moderators of cognitive behavior therapy to reduce aggressive behavior: A meta-analysis. European Child & Adolescent Psychiatry, 24, 255–264. https://doi.org/10.1007/s00787-014-0592-1

30.

Sortkær

Reimer

(2018). Classroom disciplinary climate of schools and gender—Evidence from the Nordic countries. School Effectiveness and School Improvement, 29(4), 511–528. https://doi.org/10.1080/09243453.2018.1460382

31.

Sugai

Horner

R. H.

(2020). Sustaining and scaling positive behavioral interventions and supports: Implementation drivers, outcomes, and considerations. Exceptional Children, 86(2), 120–136. https://doi.org/10.1177/0014402919855331

32.

Sutherland

K. S.

Conroy

M. A.

Algina

Ladwig

Jessee

Gyure

(2018). Reducing child problem behaviors and improving teacher-child interactions and relationships: A randomized controlled trial of BEST in CLASS. Early Childhood Research Quarterly, 42, 31–43. https://doi.org/10.1016/j.ecresq.2017.08.001

33.

Ursache

Blair

Raver

C. C.

(2012). The promotion of self-regulation as a means of enhancing school readiness and early achievement in children at risk for school failure. Child Development Perspectives, 6(2), 122–128. https://doi.org/10.1111/j.1750-8606.2011.00209.x

34.

Wolfe

Pyle

Charlton

C. T.

Sabey

C. V.

Lund

E. M.

Ross

S. W.

(2016). A systematic review of the empirical support for check-in check-out. Journal of Positive Behavior Interventions, 18(2), 74–88. https://doi.org/10.1177/1098300715595957

Check-In Check-Out Behavior Support in Mainstream Schools: Intervention Response and Moderators

Abstract

Keywords

Behavioral Support Intensification

Research on the Effectiveness of CICO Support

Present Study

Method

Participants

Procedures

Measurements

Statistical Analyses

Results

Effects of the Intervention

Moderator Effects

Discussion

Effects of the CICO Support

Examination of Potential Moderators

Limitations

Conclusions

Footnotes

Appendix

Authors’ Note

ORCID iDs

Ethical Considerations

Consent to Participate

Funding

Declaration of Conflicting Interests

References