Sage Journals: Discover world-class research

Abstract

Within the context of a multi-tiered framework for behavior support, intervention intensification is warranted when a Tier II intervention fails to adequately address a targeted behavior. A traditional approach to intensification is to shift from standardized supports at Tier II to individualized and more resource-intensive supports at Tier III. Recent conceptualizations, however, reflect the use of a process-oriented, adaptive approach in which a continuum of responsive intensifications connects the framework’s tiers. Adaptive intervention design relies on the identification of critical factors to inform responsive adaptations to a standard treatment protocol. This study examined the distinct and relative influence of adaptations to Check-In/Check-Out (CICO), a targeted intervention, informed by two critical factors—behavioral function and teachers’ use of behavior-specific praise. Results indicated that, relative to traditional CICO, CICO adapted to address behavioral function was less effective in producing differentiated levels of disruptive behavior. However, CICO adapted to strengthen a teaching practice—the delivery of behavior-specific praise—was effective in producing differentiated levels of disruptive behavior relative to the standard treatment protocol. Additional research on treatment adaptations informed by behavioral function and evidence-based teacher practices is needed to evaluate whether the variables may serve as critical factors within an adaptive approach to building treatment intensity at Tier II.

Keywords

positive behavior support Tier II check-in check-out intervention adaptation behavior-specific praise

Multi-tiered systems of support (MTSS) in education, based on a public health model of support, represent frameworks for organizing, aligning, and integrating school-based approaches to preventing and addressing social–emotional, behavioral, and academic problems (Bruns et al., 2016). While the prevention focus of MTSS models varies, a critical feature is the accessibility of a continuum of support to all students such that the intensity of delivered support aims to match student need and a problem’s resistance to intervention efforts (Sugai & Horner, 2006).

Consonant with the logic of MTSS, intensified support is warranted when instruction or intervention fails to adequately address a targeted behavior or skill. Early conceptualizations of intervention intensification grouped students along a severity-of-risk continuum, matching the groups to a distinct level of intervention (see Baker, 2005). In these early models, intervention intensification occurred through a static change from one level of support to the next. Recent conceptualizations focus on methods of intensification through dynamic change, using a process-oriented, adaptive approach that connects and builds treatment intensity within and across the tiers (e.g., Fuchs et al., 2017; Wehby & Kern, 2014). Within MTSS, a logical place to focus on intensification may be Tier II. The rationale is that Tier II interventions are most amendable to intensification efforts to prevent students from being referred to more expensive Tier III interventions or referral to special education services. Within Tier II, Check-In/Check-Out (CICO), a commonly reported Tier II intervention, has been the target of some intensification efforts (see Cain Swoszowski & Hart Rollings, 2019). Missing from this research is the presence of a systematic framework for the creation of intensification plans. One such framework uses an adaptive approach to plot intensification efforts (Wehby & Kern, 2014).

Adaptive Intervention

An adaptive approach to intensification aims to increase the effectiveness of an intervention by considering the treatment recipient (e.g., student) throughout a systematic, data-based decision-making process (August et al., 2010). Akin to data-based individualization (DBI; Deno & Mirkin, 1977), adaptive intervention utilizes assessment data, validated interventions, and research-based adaptation strategies to tailor an intervention to accommodate the needs, preferences, and responses of the treatment recipient (August et al., 2010). Using adaptive intervention models outlined in applied, preventative, and clinical psychology literature as its basis, Wehby and Kern (2014) detailed four requisite elements of a systematic adaptation process for Tier II behavior intervention: (a) critical factors (i.e., characteristics of a student or environment that inform adaptations), (b) adaptations (i.e., changes to a standard treatment protocol), (c) tailoring variables (i.e., measurements to determine the effectiveness of an adapted treatment), and (d) decision rules (i.e., rules to determine whether to continue or further adapt a treatment). Together, the elements support an iterative, data-driven process of intervention intensification (Wehby & Kern, 2014); when applied to multi-tiered models of behavior support, the process bridges the gap between Tier II and III levels of support, creating a natural progression of intensification. Consideration of critical factors is a first and most important step of an adaptive intensification process.

Critical Factors

Critical factors include stable or malleable characteristics of a student or environment that moderate the influence of treatment and provide a pretext for treatment adaptation before or as an intervention is received (Noser et al., 2017). When identified in an adaptive intervention model, they inform systematic variations of a standard treatment protocol, allowing interventionists to address heterogeneity in treatment recipients’ need for and response to treatment (Bierman et al., 2006; Noser et al., 2017). Examples include parental functioning (e.g., Bierman et al., 2006), academic performance and individual growth trajectories (Collins et al., 2004), and medication adherence (e.g., Modi et al., 2011). In this study, we focus on two specific critical factors—behavioral function and teaching practices—given the strong evidence substantiating their use to develop school-based interventions and their common inclusion within diagnostic assessments used to improve instructional practices for struggling students within an MTSS framework (e.g. teacher observations and functional behavior assessments).

Behavioral Function as a Critical Factor

Behavioral function characterizes the effect of behavior on the environment (Carr, 1977). Behavior maintained or strengthened by the delivery of socially mediated consequences is categorized as having a social positive reinforcement function, while behavior maintained or strengthened by the contingent removal of socially mediated reinforcement is categorized as having a social negative reinforcement function. Across multiple evidence-based targeted interventions, evidence suggests an understanding of the functional relationships between environmental events and challenging behavior can improve the potency of the standard treatment protocol when used to inform treatment decisions (e.g., Kilgus et al., 2016).

Teacher Practices as a Critical Factor

While existing evidence points to the utility of behavioral function as a critical factor, one may also think of instructional practices as antecedents or setting events that may influence the effectiveness of Tier II interventions, regardless of function. That is, teacher practices are part of a complex treatment environment, with the environment being a potential moderator of treatment effect and an influential factor in taxonomies for treatment intensification (e.g., Fuchs et al., 2017). While there are a number of high-leverage practices that have been recommended for improving student performance (McLeskey et al., 2022), behavior-specific praise (BSP) has been consistently shown to improve student behavior in a number of research studies across a range of student status and grade level (see Royer et al., 2019).

Purpose

Adaptive intervention design relies on the identification of critical factors to inform adaptations to a standard treatment protocol. Evidence suggests behavioral function and teacher practices have the potential to bolster the effects of targeted behavioral interventions for students whose behavior is inadequately responsive to a standard protocol. While these critical factors have been studied individually, their effectiveness has not been directly compared. The purpose of the current study is to examine the distinct and relative influence of Tier II treatment adaptations informed by two critical factors, behavioral function, and teacher practices; implications of these comparisons can inform and potentially increase the efficiency of adaptive intensifications. Using CICO as a model Tier II intervention and BSP as the distinguished teacher practice, the study will address the following research questions:

Research Question 1: Relative to the standard CICO protocol, is there a functional relation between CICO with a function-based adaptation and a decrease in disruptive behavior of students at risk for emotional/behavioral disorders (EBD)?

Research Question 2: Relative to the standard CICO protocol, is there a functional relation between increased use of BSP with implementation of CICO and a decrease in disruptive behavior of students at risk for EBD?

Research Question 3: What are the relative effects of CICO adapted to address behavioral function and CICO implemented with an enhanced teacher practice, BSP?

Method

Setting

The study was approved by the institutional review board and conducted in two public elementary schools within an urban, public school district in the southeastern region of the United States. According to parent-reported demographic information collected by the district, economically disadvantaged students represented at least 50% of each schools’ student population, and at least 68% of the schools’ students were identified as belonging to a racial and ethnic minority group. In one school, 9% of the population was identified as having a disability, with 20% of the population in the second school receiving special education services. To participate, schools were required to demonstrate the use of a behavior-focused MTSS (e.g., Positive Behavioral Intervention and Supports; PBIS), with implementation of school-wide practices assessed through the (a) School-wide Evaluation Tool (SET; Sugai et al., 2001) or (b) School-wide PBIS Tiered Fidelity Inventory (TFI; Algozzine et al., 2014). At a minimum, schools were required to evidence a current SET total score and Expectations Taught subscore ≥80% or a TFI Tier I subscore greater than or equal to 70%; this requirement confirmed potential student participants had access to existing universal behavior supports before the consideration of targeted support. Both schools reported a TFI Tier I subscore >70% (School A, reported a subscore = 90%; School B, reported a subscore = 83%).

Participants

Participants were three elementary students meeting the following inclusion criteria: (a) the student’s teacher or administrator nominated the student for participation in the study based on an observed pattern of persistent and disruptive behavior that interfered with the student’s learning or the learning of others; (b) parent consent and student assent for the student’s participation in the study were obtained; (c) English was reported to be the student’s primary language; (d) the student received instruction from his or her primary teacher for at least 75% of academic instructional blocks; and (e) the student was identified as at risk of challenging, anti-social behavior based on a Student Risk Screening Scale (SRSS; Drummond, 1994) categorization of Moderate or High Risk for externalizing behaviors. Additionally, prior to the study onset, event-based observational recording of the student’s disruptive behavior using observation methods identical to those used throughout the study confirmed the presence of persistent and elevated rates of disruptive behavior. Table 1 provides student characteristics.

Table 1.

Student Characteristics, Hypothesized Behavior Function, and Treatment Adaptations.

Student	Grade	Race	Sex	SRSS-E total score	Hypothesized function	Treatment adaptation
Student	Grade	Race	Sex	SRSS-E total score	Hypothesized function	CICO + F	CICO + TP
Participant 1	First	African American	Female	9	Access adult attn.	Teacher helper (15 min.)	Increase BSP rate by .25, from 6 to 8
Participant 2	K	White	Male	10	Access peer attn.	Choice activity with preferred peer (15 min.)	Increase BSP rate from .2 to 6
Participant 3	Fourth	African American	Female	12	Access peer attn.	Lunch with preferred peer in school library	Increase BSP from 1 to 6

Note. Student Risk Screening Scale–Externalizing (SRSS-E; Drummond, 1994); a total score within the 9–21 range constitutes a “high” risk categorization. For Participants 1 and 2, the earned rewards were provided at the end of the day; for Participant 3, earned rewards were announced at the end of the day but provided the next day. The CICO + TP adaptation indicates how praise rates were increased from baseline (i.e., naturally occurring rates) to the comparison phase (i.e., programmed rates). BSP = behavior-specific praise; CICO + TP = Check-In/Check-Out teacher practice; CICO + F = Check-In/Check-Out.

Measures

Student Risk Screening Scale

The SRSS (Drummond, 1994) was used to identify participating students. The teacher-completed screener requires the respondent to rate the frequency with which a student displays seven indicators of externalizing problems across a four-point Likert-type scale. Item ratings are summed to produce a total score and determine a student’s level of risk (low, moderate, or high). The SRSS has good internal consistency (α = .83) and is a valid indicator of both social and behavioral outcomes (r = .52; Kilgus et al., 2018).

Functional Assessment Checklist for Teachers and Staff

Prior to the onset of the study, participating teachers completed the Functional Assessment Checklist for Teachers and Staff interview (FACTS; March et al., 2000). The FACTS is a semi-structured functional behavior assessment interview designed to identify specific problem behaviors, contexts in which the behavior is most and least likely to occur, and possible maintaining behavioral functions. Data from the FACTS guided the development of function-based adaptations evaluated during the intervention phase of the study.

Direct Observation

Research assistants enrolled in graduate-level and special education-focused coursework collected direct observation data across conditions for formative and summative evaluation purposes. Data collection occurred three to four times per week during all phases using continuous, event-based recording methods. For each data collection session, student- and teacher-level behaviors were measured during the instructional block indicated by the teacher as most problematic for the participating student. Behaviors of the participating student and teacher were collected concurrently across 15-min observation sessions.

Event-Based Recording

The Multiple Option Observation System for Experimental Studies software program (MOOSES; Tapp et al., 1995) was used to quantify two variables, disruptive behavior and BSP, during the 15-min observation sessions. The authors selected event-based recording methods on the basis of prior and similar research using MOOSES and demonstrating strong interobserver agreement (e.g., Wills et al., 2018; Van Camp et al., 2021). Disruptive behavior, a student-related dependent variable, was operationalized as a verbal or physical action that interfered with classroom participation or productivity. Behavior-specific praise, a teacher-related variable, was operationalized as a verbal statement from the teacher to the participating student or a group inclusive of the participating student that indicated approval of academic or social behavior beyond the acknowledgment of adequacy or accuracy; the variable was monitored to ensure it was held constant at programmed levels during comparison conditions. Data collectors received training using a three-step process like the one described by Majeika et al. (2020).

Interobserver Agreement

To evaluate interobserver agreement, primary and secondary data collectors simultaneously collected observational data using identical measurement procedures during 40.2% of sessions across conditions and participants. Agreement estimates were obtained using a point-by-point method and procedures outlined by MacLean et al. (1985), with 5-s agreement windows around variables coded by the primary data collector. The mean percentages of the agreement for disruptive behavior and BSP across participants, sessions, and observers were 82% (range = 56–100%) and 89% (range = 0–100%), respectively.

Intervention Rating Profile-15

At the conclusion of the study, participating teachers completed the Intervention Rating Profile-15 (IRP-15; Martens et al., 1985) to assess the acceptability of each of the three compared treatments. The IRP-15 consists of 15 items scaled on a six-point Likert format ranging from strongly disagree to strongly agree, with higher scores indicating higher social validity. Treatments rated above 52.5 are considered acceptable (von Brock & Elliott, 1987). The instrument has documented strong internal consistency (α = .98; Martens et al., 1985). Each participating teacher the rating scale three times, once for each compared treatment. All rating scales were completed before the results of the study were provided to the participating teachers.

Social Validity Interview

In addition to completing the IRP-15, teachers participated in a researcher-developed interview at the conclusion of the study. The first author conducted the interviews during a brief (approximately 20 min.), one-on-one meeting with each teacher. The interview included seven questions that assessed beliefs on the efficacy of the compared treatments and preferences toward the adaptations. Four interview questions were posed before the intervention; the remaining three questions were asked after the results from the study were shared.

Experimental Design

An alternating treatments single-case design (Barlow & Hayes, 1979) was used to provide a direct comparison of the effects of a standard Tier II treatment, CICO, and two adapted versions of the treatment on participating students’ disruptive behavior. Across conditions, condition-correlated stimuli were used to facilitate students,’ teachers,’ and CICO coordinators’ discrimination of the in-effect condition for any given session. Stimuli included visual and verbal cues; these are described in the sections below.

Two schools with adequate Tier 1 supports based on the TFI or SET were selected for participation. Principals at these schools identified teachers who had requested behavior support for students. Upon consent for participation, teachers nominated student participants based on the study’s inclusion criteria, and parent consent and child assent was obtained. Subsequently, participating teachers nominated a prospective CICO coordinator in the school for each participating student. The participating teachers and CICO coordinators were interventionists for the study; their experience working for the school ranged from 1 to 10 years, and their experience implementing CICO ranged from 0 to 6 years.

Pre-Baseline

Prior to the baseline phase, researchers measured all dependent variables using event-based recording procedures. No changes to the classroom context or behavioral supports received by the participating students were made. Disruptive behavior measurements were used confirm participating students’ excesses of externalizing challenging behavior.

Baseline

During the baseline phase, the standard CICO protocol was implemented across consecutive school days. The intervention included the following core elements implemented within a daily “cycle”: (a) a morning check-in with the CICO coordinator, (b) a daily progress report (DPR) to rate student behavior, (c) teacher feedback delivered regularly via the DPR and verbal interaction, (d) an afternoon check-out with the CICO coordinator and positive feedback contingent on the student meeting a pre-established behavior goal (i.e., 80% of possible points), and (e) home-school collaboration. Repeated measurement of dependent variables remained in effect until (a) the direction of the student’s disruptive behavior indicated a zero celerating trend along the ordinate scale or (b) an accelerating trend with stable data (i.e., 80% of values were within ± 25% of the median value of the condition; Lane & Gast, 2014).

Check-In/Check-Out Procedures

Check-In

Upon arrival to school, each participating student briefly met with their CICO coordinator to initiate the CICO cycle. The CICO coordinator directed the meeting using an implementation guide that outlined essential components of the check-in process; the coordinator (a) initiated the check-in process, (b) retrieved the home-school communication form sent home with student at the end of the previous school day; (c) provided the student with a DPR printed on white paper (a condition-correlated stimulus) for use during the school day); (d) indicated to the student the intervention in effect for the day by stating the color of paper for the DPR (e.g., Today, you’ll be using a white CICO sheet; a condition-correlated verbal cue); (e) reviewed with the student behavioral expectations outlined on the DPR; (f) reviewed with the student how DPR ratings were earned based on the student meeting the behavioral expectations; (g) informed the student of the student’s point goal; (h) checked to see if student had all necessary materials for the day; and (i) ended the meeting with a positive statement that encouraged appropriate behavior. The CICO coordinator’s implementation guide, like the student’s DPR form, was printed on white paper. For each student participant, we tailored the student’s DPR to include the school-wide expectations unique to the student’s school and the teacher-selected instructional blocks or natural transition periods (e.g., end of recess) in which the teacher the DPR.

DPR Completion

During the school day, the participating teacher used an implementation guide printed on white paper outlining the treatment components essential to the completion of the DPR throughout the day. In accordance with the guide and at the end of each instructional block, the student’s teacher (a) rated the student’s behavior performance on the DPR and (b) praised the student if behavioral expectations were met or provided neutral feedback, reminding the student of how they could earn points in the future if they did not meet the behavioral expectations.

Check-Out Procedures

Upon completion of all instructional blocks, participating students returned to their CICO coordinator for a brief meeting to conclude the CICO cycle. The CICO coordinator directed the meeting using an implementation guide that outlined the essential components of the check-out process on white paper. The CICO coordinator (a) reviewed the student’s DPR for the day, determining with the student whether the student met their daily point goal, (b) praised the student if they met their point goal (but did not provide a reward) or provided neutral feedback, reminding the student how they could earn points in the future, if they did not meet the point goal, and (c) sent a completed home-school report home with the student for a parent or guardian signature. While parent participation in the treatment via signed home-school reports was encouraged, it was not included as a procedural component of the CICO treatment. At the end of the check-out meeting, the CICO coordinator returned the student to his or her classroom.

Check-In/Check-Out Training

Prior to the baseline phase, CICO coordinators and teachers participated in a scripted, one-on-one training session with the first author. The script provided an overview of the CICO cycle and instruction, modeling, rehearsal, and feedback opportunities for each component of the treatment. Training lasted approximately 30 min. Similarly, students participated in a 10-min, one-on-one training session prior to the student’s initial check-in meeting. The student’s CICO coordinator led the session using a training script. The script provided an overview of CICO and an opportunity to check the student’s understanding of the treatment.

Comparison Phase

Upon conclusion of the baseline phase, traditional CICO was compared with two variations of the treatment through rapid and repeated alternation of three conditions. CICO procedures were identical to baseline procedures, while procedures for the comparison treatments were adapted to address distinct critical factors. One variation of the standard CICO protocol addressed a teacher practice (CICO + TP), while the other addressed behavioral function (CICO + F). CICO coordinators and teachers facilitated the change of conditions (CICO, CICO + TP, and CICO + F) using a schedule with randomly sequenced condition changes.

Check-In/Check-Out Procedures

During the comparison phase, CICO procedures were nearly identical to the standard CICO procedures implemented during the baseline phase, with the exception pertaining to the addition of procedures designed to maintain teachers’ baseline rates of BSP during instruction. During the CICO condition, self-management procedures were used to hold constant participating teachers’ rates of BSP to levels not exceeding those measured during baseline. Across sessions, each participating teacher wore a MotivAider® programmed to cue the delivery of praise at regular intervals during the entire instructional block; cues were brief vibrations emitted by the device and only discernable to the teacher. Interval lengths varied by teacher and were based on the number of praise statements needed during a 15-min period to demonstrate continued baseline rates of the instructional practice. When cued by the device, the teacher was asked to deliver one BSP statement to the participating student or a group containing the participating student; the teacher was asked to withhold the delivery of praise to the student at all other times. Condition-correlated stimuli (i.e., color of implementation guides and DPR; verbal cue provided to student during check-in; condition-identifying label adhered to MotivAider®) matched those used during baseline.

Check-In/Check-Out + Teacher Practice Procedures

Procedures of the CICO + TP condition were nearly identical to CICO procedures implemented during the baseline phase, with the exception pertaining to procedures designed to enhance the rate of teacher praise. Prior to the onset of the comparison phase, teachers’ baseline rates of BSP were evaluated against recommended criteria (i.e., six praise statements within a 15-min period; Sutherland et al., 2000). During the CICO + TP condition, self-management procedures were used to increase participating teachers’ delivery of BSP to recommended rates or beyond. As in the CICO comparison condition, an electronic device cued the delivery of praise at regular intervals, with the interval length varying by the teacher. In the CICO + TP condition, however, interval lengths were based on the number of praise statements needed during a 15-min period to demonstrate (a) recommended rates or (b) a 25% rate increase given an optimal average baseline rate. When cued by the device, teachers were asked to deliver one praise statement pertaining to any observable behavior to the participating student or a group containing the participating student; additionally, and in contrast to the other comparison conditions, teachers were told they could praise freely during the instructional block, or at any time, in addition to when cued. Table 1 indicates the CICO + TP adaptations for each participant (i.e., programmed increases in praise delivery to enhance the teachers’ naturally occurring rates). For Participant 1, the MotivAider® increased praise statements by 25%, from 6 to 8 during a 15-min period; for Participants 2 and 3, the device increased praise to the recommended rate—from .2 and 1, respectively, to 6 praise statements per 15 min.

To support discrimination between conditions, condition-correlated stimuli were changed from baseline; CICO + TP intervention materials (i.e., implementation guides, DPRs, and condition-identifying label on the MotivAidor®) were yellow, and the CICO coordinator provided a verbal cue during check-ins to highlight this change (i.e., Today, we’re using a yellow daily behavior report).

Check-In/Check-Out + Function Procedures

Procedures of the CICO + F condition were nearly identical to those implemented during the CICO condition, with the exception pertaining to the addition of a procedural adaptation designed to address the behavioral function of participating students. Prior to the onset of the comparison phase, the first author evaluated the FBA results for each participating student and developed a hypothesis of the primary consequence of maintaining the student’s disruptive behavior; only one function emerged for each participant. The first author, a board-certified behavior analyst, then determined an adaptation to the standard CICO protocol designed to address the student’s behavioral function; selected adaptations were evidenced in extant literature examining the effects of function-based adaptations to CICO (e.g., Campbell & Anderson, 2008; MacLeod et al., 2016; March & Horner, 2002). Across participants, the function-based procedural adaptation consisted of providing the student the opportunity to earn a function-based reward contingent on the student meeting his or her daily point goal (i.e., 80% of possible points). Table 1 provides the hypothesized behavioral function for each participant and the corresponding function-based adaptation made to the standard CICO treatment protocol during the CICO + F condition. For Participant 1, FBA results suggested access to adult attention maintained the student’s disruptive behavior; the student earned an opportunity to serve as ‘teacher’s helper’ for 15 min at the end of days during which they met their daily point goal. For Participants 2 and 3, results suggested access to peer attention maintained their disruptive behavior. Participant 2 earned access to access to a preferred activity with a peer for 15 min at the end of days during which they met their daily point goal. On days Participant 3 met the point goal; she earned the opportunity to have lunch with a preferred peer in the school library the following day.

As with the compared conditions, condition-correlated stimuli in the CICO + F were distinct. Intervention materials were blue, and the CICO coordinator provided a verbal cue during CICO + F check-ins to distinguish the condition (i.e., Today, we’re using a blue daily behavior report).

Training

Prior to the onset of the comparison phase, each participating CICO coordinator and teacher participated in a one-on-one training session. The first author led the training using a script that provided an overview of the procedural differences across conditions as well as instruction, modeling, rehearsal, and feedback opportunities for all essential components of the comparison treatments. Training lasted approximately 30 min. Each participating student also participated in a brief one-on-one training with his or her CICO coordinator prior to the first session of the comparison phase. The coordinator followed a training script to provide the student with an overview of the three comparison treatments and an opportunity to “check for understanding” at the closing of the meeting.

Procedural Fidelity

Research assistants conducted direct observations to evaluate treatment implementation across four broad procedural areas: (a) check-in, (b) DPR completion and feedback, (c) check-out, and (d) adaptation delivery. They used a treatment integrity checklist to assess the presence or absence of a total of 19 treatment components, each corresponding to one of the broad procedural areas; one of these components was adherence to programmed rates of praise delivery, which are provided in Table 1 and displayed in Figures 1 to 3. To evaluate each of the four areas and 19 corresponding treatment components, they conducted treatment integrity observations at three distinct time points in a participating student’s school day: during morning check-ins, during the instructional block in which we measured the student’s disruptive behavior (i.e., the most problematic instructional block for the student), and during afternoon check-outs. Across participants, each of the 19 treatment components was assessed for at least 20% of sessions in each phase and condition.

Figure 1.

Observed rates of behavior-specific praise and disruptive behavior for Participant 1.

Figure 2.

Observed rates of behavior-specific praise and disruptive behavior for Participant 2.

Figure 3.

Observed rates of behavior-specific praise and disruptive behavior for Participant 3.

The fidelity of implementation was calculated by dividing the total number of observed treatment components by the total number of assessed treatment components and multiplying the quotient by 100. Overall, the fidelity of implementation was 97% for Participant 1, 99% for Participant 2, and 92% for Participant 3.

Results

Visual inspection of data using guidelines outlined by Barton et al. (2018) was used for formative and summative analyses of data. They are displayed in Figures 1, 2, 3, for Participants 1, 2, and 3, respectively.

Participant 1

For Participant 1, rates of behavior-specific praise exceeded programmed praise rates during the initial CICO and CICO + F sessions (Sessions 12 and 13, respectively, see Figure 1). Thereafter, we implemented a procedural change during the conditions, such that we provided a visual cue (i.e., a stop sign printed on a 6.5'' × 3.5'' piece of paper) to the teacher to withhold further praise once the teacher’s delivery of praise reached the programmed rate. With the visual cue in place, praise rates matched or remained below the programmed rate for all CICO and CICO +F sessions and exceeded the programmed rate for all CICO + TP sessions.

In the pre-baseline phase, Participant 1 displayed high levels of disruptive behavior, with rates of the behavior ranging from 30 to 57 occurrences per 15-min session with 83% of values falling within ± 25% of the median value of the condition. During the baseline phase, levels of the behavior remained high and a zero celerating trend with increased stability was observed (range = 36–56%; 100% of values fell within ±25% of the median value of the condition). During the comparison phase, levels of disruptive behavior remained high with the continued implementation of CICO. However, the stability of the data values decreased (40% of values fell within ± 25% of the median value of the condition), and disruptive behavior ranged from 22 to 69 occurrences per 15 min. Relative to the CICO condition, levels of disruptive behavior in the CICO + F condition were consistently higher; contratherapeutic changes in levels were observed for four of five adjacent data points of the compared conditions. Levels of disruptive behavior in the CICO + TP condition were consistently lower than observed levels in both the CICO and the CICO + F conditions. Differentiation in responding was immediate and consistent across sessions, producing at least five demonstrations of effect. Thus, a functional relation was demonstrated between CICO + TP and decreased rates of disruptive behavior relative to the implementation of CICO and CICO + F, with evidence supporting CICO + TP as the superior adapted treatment.

Participant 2

For Participant 2, rates of behavior-specific praise matched or remained below the programmed rate for all CICO sessions and all but one CICO + F session (session 19). Rates of praise delivery met or exceeded the programmed rate across CICO +TP sessions. During the pre-baseline phase, Participant 2 displayed high levels of disruptive behavior, with rates ranging from 19 to 50 occurrences per session. Levels of the behavior remained high and increased stability was observed during the baseline phase (range = 25–48%; 80% of values fell within ± 25% of the median value of the condition). During the comparison phase, levels of disruptive behavior decreased with continued implementation of the standard CICO protocol, while variability in the data values increased (67% of values fell within ± 25% of the median value of the condition). Levels of disruptive behavior observed during the CICO + F condition were consistently higher than those of the CICO condition; as contra-therapeutic changes in levels were observed for five of six adjacent data points of the compared conditions, a functional relation between CICO + F and decreased disruptive behavior was not demonstrated. Likewise, a functional relation between CICO + TP and decreased rates of disruptive behavior was not demonstrated, given the inconsistent differentiation between the data paths of the CICO + TP and CICO conditions. A functional relation was demonstrated, however, between disruptive behavior and the type of adaptation made to the standard CICO protocol; levels of disruptive behavior in the CICO + TP condition were consistently lower than those observed in the CICO + F condition, with differentiation observed for five of six adjacent data points of the compared conditions.

Participant 3

For Participant 3, rates of behavior-specific praise matched or remained below the programmed rate for all CICO and CICO + F sessions. Rates of praise delivery met or exceeded the programmed rate across CICO + TP sessions. During the pre-baseline phase, levels were highly variable, with response rates ranging from 15 to 38 occurrences per session and 0% of data values falling within ± 25% of the median value of the condition. During the baseline phase, levels remained variable, and response rates ranged from 4 to 36 occurrences per session with 71% of data values falling within ± 25% of the median value of the condition (median = 30). During the comparison phase, levels of disruptive behavior decreased and stabilized with continued implementation of CICO (100% of range =12–17%; 100% of values fell within ± 25% of the median value of the condition). Inconsistent differentiation (i.e., percentage of nonoverlapping data; PND) between levels of disruptive behavior in the CICO and CICO + F conditions occurred, providing insufficient evidence of a functional relation between the conditions (PND = 40%). However, levels of disruptive behavior in the CICO + TP condition were consistently lower than observed levels in the CICO and CICO + F conditions. Five demonstrations of effect were observed between CICO and each of the compared conditions, providing clear evidence of a functional relation between CICO + TP and decreased disruptive behavior.

Social Validity

Intervention Rating Profile-15

CICO + TP was rated as acceptable by all teachers, while CICO and CICO + F were rated as acceptable by two of the three teachers. CICO + TP was the highest rated treatment, on average (average total score = 73.7; range = 54–86); average total scores for CICO and CICO + F were slightly lower and nearly identical (CICO average total score = 62; CICO + F average total score = 62.3; CICO range = 50–79; CICO + F range = 51–70).

Social Validity Interview

Prior to receiving results from the present study, the teachers of Participants 1 and 2 reported a preference for CICO + F, both indicating their students seemed more “motivated” when provided the opportunity to work toward a reward. To the contrary, the teacher of Participant 3 favored CICO + TP, attributing her preference to her unfavorable attitude toward the provision of rewards. The teacher of Participant 1 believed CICO + TP most effective for her student, while the teachers of Participants 2 and 3 believed CICO + F most effective. All three teachers hypothesized CICO was the least effective treatment, with the teacher of Participant 2 attributing her belief to the fact that her student “didn’t get anything out of it.” Participant 3’s teacher also hypothesized CICO + F was ineffective. After result of the study were presented, all three teachers indicated they would choose to implement CICO + TP over CICO and CICO + F due to its effectiveness. However, two of the three teachers indicated difficulty implementing the treatment, with one describing the associated praise rate as “unrealistic” and the other citing difficulty delivering praise to an individual student when working directly with other students. When asked why they believed CICO + F was less effective relative to CICO + TP, the teachers provided divergent beliefs. The teacher of Participant 1 believed the reward opportunity provided with CICO + F was too delayed, while the teacher of Participant 2 believed her student became discouraged and “gave-up” midday when he missed the programmed reward. The teacher of Participant 3 believed the treatment was less effective due to its lower programmed praise rate.

Discussion

An adaptive approach to intervention intensification considers variables that may moderate the influence of a standard treatment—critical factors—and builds treatment intensity through treatment adaptations that accommodate the treatment recipient’s (i.e., student’s) resulting needs, preferences, or treatment response. The identification of adaptations that address critical factors commonly explored in applied behavioral research has the potential to bridge the gap between evidence-based, targeted treatment options and highly intensive interventions for students with persistent behavior problems.

The present study evaluated the influence of treatment adaptations to CICO informed by two critical factors, behavioral function, and teacher practice. Specifically, this evaluation was conducted with students rated as at high risk for behavior difficulties and whose disruptive behavior was persistent or elevated and nonresponsive to traditional CICO. Relative to traditional CICO, when CICO was adapted to address behavioral function, it was ineffective in producing differentiated levels of disruptive behavior. On the contrary, CICO adapted to strengthen a high-leverage teaching practice—the delivery of behavior of BSP—was effective in producing differentiated levels of disruptive behavior relative to the standard treatment protocol for two of three participants. When compared with the CICO + F, results indicated CICO + TP was the superior adapted treatment across all participants.

Interpretation of Findings

These results suggest a number of possible interpretations. First, the finding that traditional CICO was ineffective may be related to the high-risk status of the participants. Despite the general positive evidence for CICO, it is unclear whether this intervention is more or less effective for different types of students, particularly given that most published literature on CICO is related to an adaptive version of the intervention (Majeika et al., 2020). Second, the finding of a stronger impact of CICO-TP may be related to the presence of increased teacher attention toward the target students, whose functions were attention based. Relatedly, the immediacy of teacher praise as compared with the absence or delay of a programmed reward based on function in the other conditions could explain these findings. However, we did not design the current study to directly answer this particular question, and confirming this hypothesis would require additional research.

Third, while function-based adaptations to CICO have previously been associated with improved behavior outcomes for students (Klingbeil et al., 2019), results of the current study deviate from previous findings. As noted by Majeika et al. (2020), most function based adaptations of CICO often involved multiple adaptations to the standard protocol. Such concurrent changes make unclear which adaptation resulted in behavior change and limit conclusions regarding the efficacy of function-based adaptations.

Finally, researchers should consider the indirect assessment approach applied in the present study when evaluating the relative ineffectiveness of the CICO + F treatment. Although administration of the FACTS alone is a recommended, abbreviated approach to functional behavior assessment at Tier II (e.g., McIntosh et al., 2009), evidence suggests indirect assessments may be an unreliable method of determining behavioral function (see Oliver et al., 2015). It is possible the hypothesized functions generated by the FACTS were inaccurate, leading to misalignment between the adaptations of the CICO + F condition and actual behavioral functions.

Treatment Adaptations Informed by Teacher Practice

While the present study builds upon previous research examining Tier II treatment adaptations informed by behavioral function, it is unique in its evaluation of intensification through enhancement of evidence-based teaching practice, the delivery of BSP. For two of three participants, Participant 1 and Participant 3, increased behavior-specific praise while implementing CICO resulted in superior treatment effects relative to the implementation of the standard protocol. For all three participants, CICO with increased BSP proved more effective than variations informed by behavioral function. Taken together, these findings are noteworthy as they provide evidence in support of CEC recommendations for high-leverage practices (McLeskey et al., 2022) and recent frameworks (Fuchs et al., 2017) suggesting the amplification of teacher practices to adapt interventions before more intensive behavior support is provided.

In summary, we designed this study to compare two distinct adaptations based on critical factors—intensification of a teacher practice compared to a function-based adaptation—before determining function of behavior. Given that the identified function for all participants was attention based, the resulting comparisons could be viewed as a comparisons of immediate versus delayed attention. Our assessments results determined only one student was reinforced by adult attention; yet, all three responded to the increase of BSP. This could point to weaknesses in assessment approaches used at Tier II.

Limitations

The results of the present study should be interpreted with consideration of the following limitations. First, multi-treatment interference and separation of treatments (Wolery et al., 2018), may have impacted observed levels of disruptive behavior in the comparison phase. Although clear differences in response patterns of the compared conditions were observed for each participant during the comparison phase, the examined treatments may have functioned differently when used alone in contrast to when rapidly alternated with other conditions. Second, the delivery of BSP across conditions contributed to strengthened experimental control; however, the utilized procedures may have inadvertently changed the tendency of a teacher to use praise to prevent or address disruptive behavior. Finally, we suggest caution in interpreting adaptation superiority from these preliminary findings. For all participants, FBA results yielded an attention-related hypothesis. Assuming the hypotheses were accurate, it is possible increased rates of praise in the CICO + TP condition inadvertently decreased the value of other forms of attention available in the learning environment, thus impacting rates of disruptive behavior.

Conclusion

Additional research is needed to determine how critical factors can build treatment intensity at Tier II through effective and responsive treatment adaptations. Given the preliminary nature of the present findings, replication of the current study is needed to strengthen their validity; replication might include a “best alone” condition to examine data of the superior treatment when used alone. Researchers might examine the use of other evidence-based teacher practices (e.g., opportunities for student response; precorrection; instructional scaffolding), other student characteristics (e.g., student preference) as critical factors within adaptive intervention design. Comparative research examining the efficacy of other forms of function-based adaptation relative to both the standard CICO protocol and the CICO adapted to enhance evidence-based teacher practices is needed to more clearly understand if and how function-based adaptations contribute to treatment intensity at Tier II. Finally, CICO was adapted in the present study after implementation of the standard protocol was deemed inadequately effective. Future research should explore how critical factors can contribute to adjusting a treatment for persistent behavior problem before the treatment is initially applied.

Footnotes

Action Editor: Todd Haydon

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This research was supported by an Institute of Education Sciences Development and Innovation Education Research Grant, grant R324A160096.

References

Algozzine

Barrett

Eber

George

Horner

Lewis

Putnam, B., Swain-Bradway, J., McIntosh, K., & Sugai

(2014). School-wide PBIS Tiered Fidelity Inventory. OSEP Technical Assistance Center on Positive Behavioral Interventions and Supports. www.pbis.org

August

G. J.

Gewirtz

Realmuto

G. M.

(2010). Moving the field of prevention from science to service: Integrating evidence-based preventive interventions into community practice through adapted and adaptive models. Applied and Preventive Psychology, 14(1–4), 72–85. https://doi.org/10.1016/j.appsy.2008.11.001

Baker

C. K.

(2005). The PBS triangle: Does it fit as a heuristic? A reflection on the first international conference on positive behavior support. Journal of Positive Behavior Interventions, 7(2), 120–123. https://doi.org/10.1177/10983007050070020601

Barlow

D. H.

Hayes

S. C.

(1979). Alternating treatments design: One strategy for comparing the effects of two treatments in a single subject. Journal of Applied Behavior Analysis, 12(2), 199–210. https://doi.org/10.1901/jaba.1979.12-199

Barton

E. E.

Lloyd

B. P.

Spriggs

A. D.

Gast

D. L.

(2018). Visual analysis of graphic data. In Ledford

J. R.

Gast

D. L.

(Eds.), Single case research methodology: Applications in special education and behavioral sciences (3rd ed., pp. 179–214). Routledge.

Bierman

K. L.

Nix

R. L.

Maples

J. J.

Murphy

S. A.

(2006). Examining clinical judgment in an adaptive intervention design: The fast track program. Journal of Consulting and Clinical Psychology, 74(3), 468–448. https://doi.org/10.1037/0022-006X.74.3.468

Bruns

E. J.

Duong

M. T.

Lyon

A. R.

Pullman

M. D.

Cook

Cheney

McCauley

(2016). Fostering SMART partnerships to develop an effective continuum of behavioral health services and supports in schools. American Journal of Orthopsychiatry, 86(2), 156–170. https://doi.org/10.1037/ort0000083

Cain Swoszowski

Hart Rollins

. (2019). Varying intensities and configurations for CICO and CICUCO. Preventing School Failure, 63(3), 203–210. https://doi.org/10.1080/1045988X.2018.1561409

Campbell

Anderson

C. M.

(2008). Enhancing effects of Check-in/Check-out with function-based support. Behavioral Disorders, 33(4), 233–243. https://doi.org/10.1177/019874290803300404

10.

Carr

E. G.

(1977). The motivation of self-injurious behavior: A review of some hypotheses. Psychological Bulletin, 84, 800–816. https://doi.org/10.1037/0033-2909.84.4.800

11.

Collins

L. M. M.

urphy

S. A.

Bierman

K. L.

(2004). A conceptual framework for adaptive and preventative interventions. Prevention Science, 5(3), 185–196. https://doi.org/10.1023/B:PREV.0000037641.26017.00

12.

Deno

S. L.

Mirkin

P. K.

(1977). Data-based program modification: A manual. Leadership Training Institute for Special Education.

13.

Drummond

(1994). The Student Risk Screening Scale (SRSS). Josephine County Mental Health Program.

14.

Fuchs

L. S.

Fuchs

Malone

A. S.

(2017). The taxonomy of intervention intensity. TEACHING Exceptional Children, 50, 35–43. https://doi.org/10.1177/004005991875816

15.

Kilgus

S. P.

Eklund

Maggin

D. M.

Taylor

C. N.

Allen

A. N.

(2018). The Student Risk Screening Scale: A reliability and validity generalization meta-analysis. Journal of Emotional and Behavioral Disorders, 26(3), 143–155. https://doi.org/10.1177/1063426617710207

16.

Kilgus

S. P.

Fallon

L. M.

Feinberg

A. B.

(2016). Function-based modification of Check-in/Check-Out to influence escape-maintained behavior. Journal of Applied School Psychology, 32(1), 24–45. https://doi.org/10.1080/15377903.2015.1084965

17.

Klingbeil

D. A.

Hart

E. H.

Schramm

A. L.

(2019). A systematic review of function-modified check-in/check-out. Journal of Positive Behavior Interventions, 21(2), 77–92. https://doi.org/10.1177/1098300718778032

18.

Lane

J. D.

Gast

D. L.

(2014). Visual analysis in single case experimental design studies: Brief review and guidelines. Neuropsychological Rehabilitation, 24(3–4), 445–463. https://doi.org/10.1080/09602011.2013.815636

19.

Maclean

W. E.

Tapp

J. T.

Johnson

W. L.

(1985). Alternative methods and software for calculating interobserver agreement for continuous observation data. Journal of Psychopathology and Behavioral Assessment, 7, 5–73. https://doi.org/10.1007/BF00961847

20.

MacLeod

K. S.

Hawken

L. S.

O’Neill

R. E.

Bundock

(2016). Combining tier 2 and tier 3 supports for students with disabilities in general education settings. Journal of Educational Issues, 2, 1–21. https://doi.org/10.5296/jei.v2i2.10183

21.

Majeika

C. E.

Van Camp

A. M.

Wehby

J. H.

Kern

Commisso

Gaier

(2020). An evaluation of the adaptations made to check-in check-out. Journal of Positive Behavior Supports, 22(1), 25–37. https://doi.org/10.1177/1098300719860131

22.

March

R. E.

Horner

R. H.

(2002). Feasibility of contributions of functional behavioral assessment in schools. Journal of Emotional and Behavioral Disorders, 10, 158–171. https://doi.org/10.1177/10634266020100030401

23.

March

R. E.

Lewis-Palmer

Brown

Crone

Todd

A. W.

Carr

(2000). Functional assessment checklist for teachers and staff (FACTS) [Education and community supports]. University of Oregon.

24.

Martens

B. K.

Witt

J. C.

Elliott

S. N.

Darveaux

D. X.

(1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16, 191–198. https://doi.org/10.1037/0735-7028.16.2.191

25.

McIntosh

Campbell

A. L.

Carter

D. R.

Dickey

C. R.

(2009). Differential effects of a tier two intervention based on function of problem behavior. Journal of Positive Behavior Intervention, 11(2), 82–93. https://doi.org/10.1177/1098300708319127

26.

McLeskey

Maheady

Billingsley

Brownell

M. T.

Lewis

T. J.

(Eds.). (2022). High leverage practices for inclusive classrooms (2nd ed.). Routledge.

27.

Modi

A. C.

Rausch

J. R.

Glauser

T. A.

(2011). Patterns of nonadherence to antiepileptic drug therapy in children with newly diagnosed epilepsy. Journal of the American Medical Association, 305, 1669–1676. https://doi.org/10.1001/jama.2011.506

28.

Noser

A. E.

Cushing

C. C.

McGrady

M. E.

Amaro

C. M.

Huffhines

L. P.

(2017). Adaptive intervention designs in pediatric psychology: The promise of sequential multiple assignment randomized trials of pediatric interventions. Clinical Practice in Pediatric Psychology, 5(2), 170–179. https://doi.org/10.1037/cpp0000185

29.

Oliver

A. C.

Pratt

L. A.

Normand

M. P.

(2015). A survey of functional behavior assessment methods used by behavior analysts in practice. Journal of Applied Behavior Analysis, 48(4), 817–829. https://doi.org/10.1002/jaba.256

30.

Royer

D. J.

Lane

K. L.

Dunlap

K. D.

Ennis

R. P.

(2019). A systematic review of teacher-delivered behavior-specific praise on K–12 student performance. Remedial and Special Education, 40(2), 112–128. https://doi.org/10.1177/0741932517751054

31.

Sugai

Horner

R. R.

(2006). A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review, 35(2), 245–259. https://doi.org/10.1080/02796015.2006.12087989

32.

Sugai

Lewis-Palmer

Todd

Horner

R. H.

(2001). School-wide evaluation tool. University of Oregon. https://doi.org/10.1177/10983007040060010201

33.

Sutherland

K. S.

Wehby

J. H.

Copeland

S. R.

(2000). Effect of varying rates of behavior-specific praise on the on-task behavior of students with EBD. Journal of Emotional and Behavioral Disorders, 8(1), 2–8. https://doi.org/10.1177/106342660000800101

34.

Tapp

J. T.

Wehby

J. H.

Ellis

(1995). MOOSES: A multi-option observation system for experimental studies. Behavior Research Methods, Instruments, & Computers, 27(1), 25–31. https://doi.org/10.3758/BF03203616

35.

Van Camp

A. M.

Wehby

J. H.

Copeland

B. A.

Bruhn

A. L

. (2021). Building from the bottom up: The importance of Tier 1 supports in the context of Tier 2 interventions. Journal of Positive Behavior Interventions, 23(1), 53–64. https://doi.org/10.1177/1098300720916716

36.

von Brock

M. B.

Elliott

S. N

. (1987). Influence of treatment effectiveness information on the acceptability of classroom interventions. Journal of School Psychology, 25(2), 131–144. http://doi.org./10.1016/0022-4405(87)90022-7

37.

Wehby

J. H.

Kern

(2014). Intensive behavior intervention: “What is it, what is its evidence base, and why do we need to implement now?” TEACHING Exceptional Children, 46, 38–44. https://doi.org/10.1177/0040059914523956

38.

Wills

Wehby

Caldarella

Kamps

Swinburne Romine

(2018). Classroom management that works: A replication trial of the CW-FIT program. Exceptional Children, 84(4), 437–456. https://doi.org/10.1177/0014402918771321

39.

Wolery

Gast

Ledford

(2018). Comparative designs. In Ledford

Gast

(Eds.), Single case research methodology: Applications in special education and behavioral sciences (pp. 283–334). Routledge. https://doi.org/10.4324/978131515066611

Use of Behavior Function and Teacher Practices to Intensify Tier II Intervention: A Comparison of Check-In,Check-Out Adaptations

Abstract

Keywords

Adaptive Intervention

Critical Factors

Behavioral Function as a Critical Factor

Teacher Practices as a Critical Factor

Purpose

Method

Setting

Participants

Measures

Student Risk Screening Scale

Functional Assessment Checklist for Teachers and Staff

Direct Observation

Event-Based Recording

Interobserver Agreement

Intervention Rating Profile-15

Social Validity Interview

Experimental Design

Pre-Baseline

Baseline

Check-In/Check-Out Procedures

Check-In

DPR Completion

Check-Out Procedures

Check-In/Check-Out Training

Comparison Phase

Check-In/Check-Out Procedures

Check-In/Check-Out + Teacher Practice Procedures

Check-In/Check-Out + Function Procedures

Training

Procedural Fidelity

Results

Participant 1

Participant 2

Participant 3

Social Validity

Intervention Rating Profile-15

Social Validity Interview

Discussion

Interpretation of Findings

Treatment Adaptations Informed by Teacher Practice

Limitations

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

References