Abstract
Check-In, Check-Out (CICO) is a targeted behavioral intervention designed for students with more significant social and academic needs, who may not respond as effectively to broader preventive programs like School-Wide Positive Behavioral Supports. This study aimed to (i) assess the effectiveness of CICO at the primary student level as part of a multi-tiered support system that addresses and reduces disruptive behaviors, (ii) determine whether these adaptations could be implemented with high fidelity and evaluate if high fidelity was associated with improvements in student behavior. Using a single-case research methodology design, the study involved three Romanian schools, and eight students, supported by three teachers, in the implementation of Tier 2 CICO interventions. Structured daily routines and clear behavioral expectations were established. Findings suggested that CICO was particularly effective during the first 3 weeks, with most children meeting behavioral expectations, likely due to the intensified positive behavior support provided by teachers. The paper discusses the implementation experience of CICO in Romanian primary schools and its implications for future practice.
Keywords
Introduction
Multi-tiered behavior support systems, such as School-Wide Positive Behavioral Supports (SWPBS), are being implemented by primary schools as a preventive framework to address disruptive behaviors, promote a positive learning environment, and support the overall well-being of students (Borgen et al., 2020; Nitz et al., 2023; Şerbănică & Drăgan, 2024). Developed by George Sugai and Rob Horner in the 1990s, SWPBS offers an alternative to reactive approaches for managing disruptive behaviors, which have often failed to produce positive outcomes in both the short and long term (Sugai et al., 2000; Sugai & Horner, 2006; Walker et al., 1996). Studies indicate that the use of suspension and expulsion as disciplinary measures in schools rarely leads to a reduction in overall problematic behaviors (Conroy et al., 1999; Lewis et al., 1998; Walker et al., 1996).
The shift toward using more positive behavioral support and the least restrictive treatment model began with researchers’ efforts to implement practical solutions for managing extremely challenging behaviors (Beamish & Bryer, 2019). These solutions valued humane, effective, and person-centered planning interventions, particularly for individuals with significant disabilities, including multiple disabilities (LaVigna & Willis, 2012). Originating from the work of researchers such as Evans and Meyer (1985), and LaVigna et al. (1989) in the 1980s, these approaches sought to provide practical solutions for individuals with significant disabilities. Later, Horner et al. (1990) integrated individualized behavioral assessment and planning techniques with person-centered planning, coining the term “positive behavioral support” to highlight non-aversive principles (Horner et al., 1990, p. 126).
SWPBS focuses on preventive strategies to create a school-wide system that reinforces positive behaviors, establishes clear expectations, and provides interventions to address challenging behaviors (OSEP Center on Positive Behavioral Interventions and Supports, n.d.; Sugai et al., 2000; Sugai & Horner, 2006). The approach involves collaboration among school staff, students, and families to foster a supportive learning environment that enhances both academic and social success (Flannery et al., 2009; Power et al., 2022; Sugai & Horner, 2006). This framework uses evidence-based interventions to encourage positive behavior in the general student population, particularly those who do not exhibit chronic behavior problems, as part of a preventive model (Tier 1). The second tier (Tier 2) targets students at risk of developing chronic behavior problems, emphasizing secondary prevention. The third tier addresses students who face significant behavioral challenges, focusing on tertiary prevention. This tiered approach ensures that interventions are tailored to the varying levels of behavioral needs within the school community (Medley et al., 2008).
Check-In Check-Out (CICO)
Tier 2 support, also known as secondary prevention, involves targeted interventions designed for students whose social and academic needs are more intense than those addressed by Tier 1 support. The Tier 2 behavioral intervention specifically targets students with chronic disruptive behaviors, who are resistant to Tier 1 strategies by implementing the Check-In Check-Out (CICO) program. The aim of CICO is to provide frequent instruction on expected behaviors through structured student-adult interactions, a formal mechanism for both positive and corrective feedback, and reinforcement opportunities for demonstrating expected behaviors (Kato et al., 2023). The intervention includes six key procedures: checking in with a professional (such as a therapist or a teacher) in the morning, use of a daily point card to track progress on school-wide expectations, receiving praise and specific feedback from school staff, reviewing goals and progress at the end of the school day, taking home a daily card for caregiver signature and positive feedback, and continuously reviewing student data to make necessary adjustments (Hawken et al., 2014).
A recommendation from practical implementation is the gradual fading of CICO, where successful students begin to self-assess their performance and seek teacher feedback to align their self-evaluations with teacher expectations. This self-monitoring skill, demonstrated by accurate self-assessment, is essential for the students before fully transitioning out of the intervention (Hawken et al., 2021). CICO support is a low-intensity, time-efficient resource that can be easily adapted to meet students’ individual needs and address various behavioral functions (Cain Swoszowski & Hart Rollins, 2019). It is widely implemented in schools in Western countries (Karhu et al., 2021; Kato et al., 2023); however, it has lower adoption rates and less empirical support in Romanian primary schools. Adaptations to fit the local school contexts are often necessary, yet the fidelity of these adaptations remains understudied. Implementation fidelity, defined as delivering an intervention as planned, is crucial and includes adaptation as a natural feature rather than a deviation. Fidelity is measured using various strategies, such as procedural checklists, self-reports, and observations, to ensure both system-level infrastructure and procedural adherence (Kato et al., 2023).
Purpose of the Study
The purpose of this study was to evaluate the implementation fidelity and effectiveness of the CICO intervention, incorporating strategic contextual adaptations, in three primary schools from Arges county, Romania. Specifically, the study aimed to examine the effectiveness of Tier 2 interventions at the student level and determine whether these adaptations could be implemented with high fidelity and if high fidelity was associated with improved student behavior. The research addressed the following research questions:
RQ1. How is Check-In/Check-Out (CICO) implemented in Tier 2 settings, and what components contribute to the effectiveness of these interventions?
RQ2. To what extent are the effects of the CICO intervention maintained over time following the conclusion of the intervention period?
RQ3. How does the method used to conclude the CICO intervention influence the long-term maintenance of its effects?
Method
Participants and settings
Participants in the study included three primary schools (CICO coordinators, students, and teachers, see Table 1) located in Arges County, Romania. All three schools had previously implemented Tier 1 School-Wide Positive Behavioral Supports (SWPBS) with high fidelity, demonstrating readiness for Tier 2 interventions. The candidate schools were selected based on several pre-determined criteria that served as quality factors (1) satisfactory Tiered Fidelity Inventory (TFI) scores, indicating gradual and steady progress, measured by the School-Wide Evaluation Tool (Horner et al., 2004; see Table 2), (2) strong and supportive leadership, (3) good collaboration among teachers, (4) small school size (easier to manage and transmit the SWPBS philosophy), and (5) schools, teachers, students and families eager to implement Tier 2 support. Generally, SWPBS Tiers 2 and 3 are to be implemented at schools that have demonstrated at least 80% adherence to the experimental protocol (i.e. fidelity score). High quality implementation is associated with better school outcomes, including fewer problem behaviors, increased social competencies, and a more inclusive and positive school environment (Kim et al., 2018). Additionally, the research team identified a small group of students non-responsive to Tier 1 support and provided them with additional Tiers 2 support (e.g. small group social skills instruction and CICO). Participating schools signed a partnership agreement with the national SWPBS leadership team at the beginning of the school year, demonstrating their commitment to the project. As part of their participation, each school established a team responsible for training and leading SWPBS action planning and implementation. In line with this, the school team agreed to pilot the Tier 2 intervention with a small group of students.
Schools’ demographics.
Tiered Fidelity Inventory scores of participant schools.
The study involved eight primary school students who participated in the CICO program during the 2021 to 2022 school year. Demographic details of the students are provided in Table 3.
CICO participation, demographics.
All CICO coordinators underwent a training session conducted by an external coach from the research team before beginning the intervention. The training consisted of four sessions, a total of 11 hr and covered CICO protocols, including procedures and a review of expected behaviors. Three CICO coordinators participated in the study, each responsible for conducting morning check-ins and afternoon check-outs with students, as well as providing coaching and support to both students and teachers. Eight primary school students with chronic disruptive behaviors participated in the CICO program. These students received a brief orientation (approximately 30 min) to help them identify their behavioral goals and articulate them in their own words.
Implementation design
For this study, we conducted a quantitative experimental single-case design in three primary schools where SWPBS Tier 1 had been implemented with high fidelity (e.g. 80% or above). All participating schools were in their third year of SWPBIS implementation. Single-case design experiments were used to examine the effectiveness of Tier 2 interventions at the student level. After building capacity during the first 2 years of Tier 1 SWPBS framework implementation, the Romanian team proceeded with pilot Tier 2 support in three schools from Cohort 1 (experimental group) all of which had been implementing Tier 1 for 3 years and had achieved at least an 80% fidelity score of Tier 1. Current fidelity data showed that these schools exhibit high-quality implementation and met the selection criteria (as shown in Table 1). Students requiring additional behavioral support were identified by school teams, based on Functional Behavioral Assessments (FBA; O’Neill et al., 1997), with teacher observations also contributing to the identification of support needs for Tier 2 interventions.
This research used a concurrent multiple baseline design across participants to assess the effectiveness of the CICO support, focusing on its impact on Daily Report Card (DRC) scores and students’ educational success skills. The DRC scores reflect whether a student met the objective for a specific period (one academic hour). Student progress was monitored (by CICO coaches) using a scoring system that evaluated their performance over 6-hr periods each day, across an 8-week period. Improvement was measured by tracking the percentage of objectives met, with daily scores of 0 (objective not met), 1 (objective in progress), and 2 (objective met). This system allowed both CICO coaches and the research team to monitor student progress and clearly identify which objectives were met, in progress, or unmet, providing a clear and measurable method to track academic performance and behavior. The CICO support consisted of three phases: baseline (B1), intervention (I2–6), and maintenance (M7–8). In the baseline phase, which covered the first week of data collection, students participated in daily academic activities (e.g. checking their binder for assignments, bringing materials to class, and appropriately discussing behavioral issues) without receiving specific instructions on how to perform these tasks. CICO coaches completed DRC scores at the end of each day. Other responsibilities of the CICO coaches included: (1) accurately following checklist steps, (2) assisting students in setting daily goals, and (3) conducting mini social skills activities based on FBA data. Based on the baseline data, the participant with the lowest and most stable level of performance was the first to enter the intervention phase, followed by the second participant selected using the same criteria. The CICO coach and the research team identified the function of challenging behaviors using FBA data and set two social skills objectives for each participant. The CICO Coach implemented the intervention over 5 weeks (I2–6), followed by the maintenance phase during the last 2 weeks (M7–8), which included social skills instruction and specific feedback. Figure 1 illustrates the key phases of the implementation process of the CICO.
Romanian application of CICO support.
Each participating school implemented the CICO support for selected students, ensuring that regular support and daily routines were established and organized with high fidelity. The schools systematically collected data during the intervention’s such as age, grade, and gender; (2) Daily Report Card (DRC), collected each school day, including a baseline period before the intervention phase; and (3) weekly report, compiled at the end of each week, which included fidelity information detailing the activities of each day. At the end of the intervention phase, teachers, CICO coaches, parents, and students assessed the fidelity and acceptability of Tier 2 CICO support to evaluate its effectiveness and satisfaction with the intervention.
Data collection and analysis
The study used checklists, self-reports, and observations to measure implementation fidelity. This multi-faceted approach ensured comprehensive monitoring of both system-level and procedural aspects of the intervention. The research team analyzed the data using a piecewise latent growth curve methodology, which allowed them to statistically test and analyze the data. This methodology provided a nuanced understanding of the intervention’s impact over a 3-week period, helping to assess when significant changes occurred and how the trajectory of improvement varied across different phases of the intervention. The aim was to determine whether the interventions had a statistically significant average effect, assess the magnitude of the effect size, and evaluate the duration of implementation required to achieve meaningful outcomes. Additionally, the Tier 2 studies focused on the sustainability of the intervention’s effects, exploring strategies for gradually fading the intervention (Rodriguez et al., 2015). In this context, students receiving CICO support were randomly assigned to different fading procedures once they demonstrated consistent performance on the CICO program. The primary success criterion for these procedures was whether the students could consistently meet their behavioral goals for three consecutive weeks.
Results
Three Romanian primary schools participated in the Tier 2 CICO support, involving eight students and three teachers. Structured daily routines and clear behavioral expectations were established. In all three schools, CICO support was provided to students in grades 3 and 4.
Prior to the implementation of CICO, the SWPBS framework had been implemented as a whole-school approach for 2 years (2019–2021). The participant group consisted of five female and three male preservice teachers, reflecting the composition of the sample without suggesting any notable gender-based difference. All students who received CICO support were considered at risk and had been identified by the school counselor as needing individualized behavioral support before the intervention began. Additionally, five students, two-thirds of the participants, had behavior-related disabilities. Most of the students recommended for CICO support were identified as having special education needs. The evaluation of social validity did not differ significantly between schools; therefore, a single-level analysis was applied.
Direct observation data
Figure 2 presents direct observation data for students’ DRC and positive behavior learning. Table 4 displays mean scores and standard deviation of the overall progress of students during the CICO support implementation, based on DRC scores for each phase.
Overall progress of study participant receiving CICO support (n = 8).
Overall students’ progress during CICO support implementation based on DRC scores.
Note. B1 represents Baseline Phase, week 1; I2–I6 represent Implementation Phase, weeks 2–6; and M7–M8 represent Maintenance Phase, weeks 7 and 8. S1–S2 are the study participants (n = 8).
Table 4 presents the results of students’ progress during the implementation of the CICO support program, as measured by DRC scores. In the table includes the mean (M) and standard deviation (SD) of DRC scores across different phases: Baseline (B1), Implementation (I2–I6), and Maintenance (M7–M8) for eight participants (S1–S8).
Baseline scores for all participants (S1–S8) ranged from 8.33 to 24.28, with varying levels of variability as indicated by the standard deviations. This phase established the baseline performance prior to the intervention. Even in the second week, a noticeable improvement in scores was observed for all students, with means ranging from 21.66 to 38.57, reflecting the early impact of the CICO intervention. Continued improvement was seen during the following week of the CICO intervention. The scores peaked during the 6 weeks of the implementation phase (I6), with means ranging from 54.28 to 77.85, indicating substantial progress for most participants. The standard deviations suggest that some students demonstrated more consistent performance. In the maintenance phase, scores remained generally high, ranging from 60.00 to 95.00, indicating that the improvements achieved during the implementation phase were largely sustained. Similar high scores were observed in the final week (M8), ranging from 60.71 to 100.00, with maximum scores achieved by some students (S2 and S3), indicating sustained progress. However, some variability persisted, as reflected by the standard deviations.
In summary, the data highlights the significant positive impact of the CICO intervention on students’ academic planning and social skills, as evidenced by the consistent improvement in DRC scores from the baseline through both the implementation and maintenance phases. The high scores observed during the maintenance phase suggest that the gains achieved were sustained over time. However, the variability in standard deviations indicates differences in individual responses to the intervention, emphasizing the need for personalized approaches to maximize effectiveness for all students, as illustrated in Figure 3.
Overall students’ progress based on DRC reports (n = 8).
Discussion
The study aimed to assess the effectiveness of Tier 2 interventions at the student level using a quantitative, experimental, single-case research methodology. The findings of this research examine the implementation of CICO support in three Romanian primary schools for students in third and fourth grades. In this study, more boys than girls were recommended for CICO support, a pattern that is consistent with findings from other research (Paananen et al., 2023), though no specific gender-based disparities were analyzed. Among the students recommended for CICO, boys’ disruptive behaviors were more easily identified. However, all the girls in the study were previously identified as having specific needs for pedagogical support or disabilities before the start of CICO support, while only three boys had a disability in addition to disruptive behaviors.
These findings raise several important questions about the role of CICO as a preventive intervention. Should it be considered part of the special education support provided in Romanian schools, or should it be treated as a separate entity? Furthermore, these observations suggest a need to evaluate how CICO is integrated within the broader educational support framework and its effectiveness in addressing the diverse needs of students, especially those with pre-existing conditions that require pedagogical intervention. The results of this study suggest a significant functional relationship between the CICO intervention and improvements in reducing disruptive behaviors while promoting appropriate social skills, as evidenced by increases in DRC scores. Students’ average correct response rate for task completion increased from 16.78 (1.79) during the baseline condition to 82.08 (1.51) during the intervention phase. Our findings support the integration of positive behavior instruction with the CICO support for students, aligning with previous research by Paananen et al. (2023), Cain Swoszowski and Hart Rollins (2019), Hawken et al. (2014), and Karhu et al. (2021).
Each participant successfully completed the planning checklist, articulated the steps of social skills beneficial in the classroom and school environment, and improved their DRC scores. Findings indicated CICO’s effectiveness, particularly during the first 3 weeks, with most children meeting behavioral expectations due to increased teacher support. However, data showed fluctuations in progress, which may have been influenced by factors such as a lack of parental encouragement, domestic issues, and interruptions like the Easter break.
Social validity outcomes from students, parents, and classroom teachers involved in the study were also positive. Students reported that they found the daily coaching sessions helpful and beneficial, expressing a sense of inclusion and acceptance. Based on student perceptions, performance, and teacher feedback, the CICO components effectively reduced disruptive behavior among primary school students. These findings align with previous studies (Peart et al., 2024), particularly those reporting a stronger impact of CICO when there is increased teacher attention directed toward CICO students (Chauvin et al., 2024). Teachers found the CICO program to be worth recommending despite challenges in its implementation, primarily due to the absence of school psychologists or counselors.
Teachers reported that one of the key challenges they faced was the limited time available for conducting daily check-ins and check-outs with students. This issue was particularly pronounced in schools where teachers had heavy teaching loads and lacked the dedicated time necessary to provide consistent, high-quality support. The absence of trained school psychologists or counselors in some schools added to the difficulty, forcing teachers to take on roles outside of their traditional responsibilities, such as conducting behavioral assessments and administering social skills training. This situation has been shown to strain available resources and undermine the fidelity of the intervention (Smith et al., 2020). Additionally, as previously mentioned, the lack of consistent parental involvement presented another significant challenge. Many parents did not regularly review their children’s DRCs or reinforce the goals at home, which led to inconsistency in support and contributed to variability in student progress, ultimately affecting the overall success of the intervention.
The implementation of the CICO intervention in Romanian schools presented additional challenges, primarily due to the need to adapt the program to fit the local educational context and culture. While CICO has proven effective in other countries, applying it in a different cultural setting required careful adjustments to ensure it aligned with local practices, policies, and available resources. These challenges highlight the importance of thorough preparation, comprehensive training, and adequate resource allocation when implementing behavioral interventions like CICO. The following discussion explores strategies to address these challenges in future implementations, including enhancing teacher training, increasing resources for behavioral support staff, and strengthening parental involvement strategies.
This gap required teachers to adjust their schedules for daily meetings, which posed a significant challenge for both teachers and school principals. However, teachers acknowledge that, with the implementation of CICO, student engagement and involvement in school activities increased significantly, leading to a notable reduction in disruptive behaviors. As O’Brien et al. (2024) stated, “Overwhelming evidence supports the benefits of active pedagogy and student participation in lessons” (p. 860). Children recognized their teachers’ support in achieving CICO goals, becoming more aware of these goals, and receiving rewards and encouragement. However, many reported that their parents did not consistently check their daily cards or offer encouragement upon goal completion which impacted the fidelity of the program’s implementation. Parents viewed CICO positively, noting improvements in their children’s behavior and a reduction in problematic behaviors. They considered the program worthwhile and would recommend it to others.
Overall, this study contributes to the understanding of how structured mentoring programs, which incorporate both academic planning and social skills training, can positively impact student behavior and engagement. It reinforces the importance of stakeholder feedback in evaluating the effectiveness of behavioral interventions and highlights the potential of CICO as a valuable tool for reducing disruptive behaviors across diverse student populations.
Conclusion
Exclusionary practices and punitive measures significantly impact at-risk students, particularly those in special education (Losen et al., 2015). This study makes two key contributions to educational practice. First, the results support previous findings that CICO positively influences student behavior (Cain Swoszowski & Hart Rollins, 2019; Paananen et al., 2023; Rodriguez et al., 2015). Second, it provides new insights into how the intervention can be effectively implemented and adapted in different educational settings, particularly within primary schools in the Romanian context, thereby enhancing our understanding of its potential benefits and limitations. The findings indicated that CICO was particularly effective, during the first 3 weeks, with most children meeting behavioral expectations due to increased teacher support. However, the data also showed fluctuations in progress. Tier 2 interventions, such as CICO, focus on students with more intense social and academic needs, offering additional instructional support. These low-intensity interventions, coordinated by a Tier 2 school team, are efficiently administered by school personnel. Parental involvement is essential for both consent and participation in the intervention.
Overall, the study supports the effectiveness of CICO in Romanian schools and highlights the potential need for Tier 3 support for students who are unresponsive to Tier 2 interventions. School-wide positive behavior support programs, such as SWPBS, could further enhance the benefits for the entire school community. Successful implementation relies on improved collaboration among program developers, researchers, practitioners, policymakers, teachers, and parents.
Limitations
The current study has several limitations. Firstly, since it was conducted in a specific Romanian county, the findings may not be applicable to students in other geographical areas. Secondly, the results may not apply to general education settings, as the research was conducted in primary classrooms. Thirdly, single-case research design studies typically have limited external validity due to their small sample sizes and the lack of randomization. Therefore, further replications with larger samples and more rigorous designs are needed to confirm these findings.
Implications and future research
Understanding the fidelity of CICO implementation in primary schools and its impact on student behavior is critical for developing effective behavioral interventions, particularly for supporting at-risk students and those receiving special education services. This study presented a model for implementing CICO within a SWPBS framework, demonstrating positive outcomes within a specific school setting. However, the generalizability of these results is limited due to the unique context and existing SWPBS features within the Romanian educational system, along with the availability of adequate resources.
The findings highlight the importance of resources such as CICO coaches, time, and funding for the systematic implementation of Tier 2 interventions. Future research should explore the generalizability of these findings in diverse educational contexts and further investigate the social validity of CICO from the perspectives of various stakeholders, including teachers, administrators, parents, and students. The results from this study could significantly have practical implications for the implementation and scaling of Tier 2 interventions in educational settings. Understanding the optimal duration and intensity of interventions, as well as effective strategies for fading support, and will help educators design more effective and sustainable programs. Future research could expand on these findings by exploring the long-term outcomes for students who transition from intensive support to faded intervention, as well as assessing the impact of different contextual factors on the success of these strategies.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been carried out within the project SWPBS: a KA3/ Support for Policy Reform Project (ref. no. 606687-EPP-1-2018-2-CY-EPPKA3-PI-POLICY), financially supported by the European Commission through the Erasmus+ Programme. The project coordinator was CARDET - Centre for Advancement of Research and Development in Educational Technology (CY) and the partners were Universitatea din Pitești (RO); Inspectoratul Școlar Județean Argeș (RO); Paidagogiko Instituto Kyprou, Ministry of Education (CY); Innovade Ltd (CY); Jyvaskylan Yliopisto (FI); Kontiolahden kunta (FI); City of Varkaus (FI); Lappeenrannan Kaupunki (FI); Aristotelio Panepistimio Thessalonikis (GR); Perifereiaki Diefthinsi Protovathmias Kai Defterovathmias Ekpaidefsis (GR). The Open Access was provided based on ANELIS transformative Agreement.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.
Informed consent
Participation in the study was entirely voluntary. Participants were provided with an information letter and consent form. Written informed consent was obtained from all legal guardians of the children and from the teachers and schools staff included in this study. All participants, including the pupils’ parents and support teachers, signed the consent statement, and none opted to withdraw from the study.
