Abstract
This study investigated whether a restorative practice initiative improved school climate. The study presents findings from a cluster randomized, controlled trial conducted with 2,248 students from grades 5 to 12 (38% Black, 32% Hispanic) attending 18 schools in the U.S. Northeast. After 1 year of implementation, the experimental analyses of the whole-school initiative did not result in more positive perceptions of school climate for students in the intervention schools compared with students in the control schools. Correlational analyses were also conducted with student surveys in both intervention and control schools. Based on student report, restorative practice exposure in all 18 schools was associated with positive perceptions of the school climate. This suggests that initiatives may need to focus on fidelity of implementation and consistent implementation across more years for substantive school climate gains.
Keywords
Momentum for comprehensive school discipline reform has grown with the increasing recognition of the long-term negative effects of school suspension (e.g., Bacher-Hicks et al., 2019) and the persistent racial disparities in exclusionary discipline (Gage et al., 2021). Both internationally and in the United States, schools have turned to restorative practices to reduce the use of exclusionary discipline (e.g., González, 2016). In fact, there is an emerging body of evidence from experimental trials that suggests that restorative practice initiatives can reduce exclusionary discipline (Augustine et al., 2018). Theories of action would indicate that prevention- and intervention-oriented restorative practice activities also could improve school climate given that these practices aim to strengthen relationships, repair harm, and increase equity (Evans & Vaandering, 2022). Yet, the few experimental studies addressing the multifaceted theorized outcomes of restorative practices that go beyond school discipline records have provided mixed results (e.g., Acosta et al., 2019; Bonell et al., 2018; Grant et al., 2022; for a review, see Zakszeski & Rutherford, 2021). This study addresses the need for more research by investigating a restorative practice initiative’s impact on school climate and the climate of bullying and teasing based on a cluster randomized, controlled trial using student-reported measures after 1 year of implementation. It also examines the degree to which student-reported exposure to restorative practices was associated with positive outcomes across both intervention and control schools.
This study builds directly on prior findings using data from the same experimental trial of the restorative practice project (Gregory et al., 2022; Huang et al., 2023). In the first study, we showed that students were less likely to be in a discipline incident in the intervention schools than the control schools. The impacts held when accounting for school and student characteristics (students’ prior discipline incidents, sex, race, disability status, and income status; Gregory et al., 2022). In the second study, we found that, in the intervention schools, previously-suspended students had a reduced likelihood of receiving another suspension (11% probability), relative to similar peers in the control schools (33% probability, Huang et al., 2023). Taken together, the studies indicate positive effects on school discipline — RP may help reduce cycles of re-suspension and prevent incidents from arising in the first place. Both studies, however, narrowly focus on school discipline records and do not examine effects on school climate.
School Climate
School climate is conceptualized in multifaceted ways. Most definitions emphasize the shared norms and values related to student and staff social, emotional, academic, and physical well-being (Cohen et al., 2009). A critique of school climate research is that it lacks clear theoretical underpinnings without specific examination of positive factors that are attuned to the developmental needs of students. To sharpen conceptualizations of school climate, scholars extended Baumrind’s home-based parenting styles with adolescents to educators in the school setting (Gregory & Cornell, 2009; Gregory & Weinstein, 2004). Similar to the authoritative parenting style, these scholars postulated that youth excel the most in schools offering fairness, consistency, and clear behavioral expectations (structure) and warmth, acceptance, and respect for autonomy (support) (Gregory & Cornell, 2009). Evidence for this assertion has accrued for almost two decades. For example, an early study using a national sample of high schools showed that for low-income adolescents, teacher connection accompanied by teacher structure jointly predicted gains in math achievement by 12th grade (Gregory & Weinstein, 2004).
Measurement and research in this area have advanced through empirical testing of the Authoritative School Climate Survey (ASCS), which has been studied extensively in schools throughout Virginia. Across different years and cohorts of students, disciplinary structure and student support have been found to predict positive outcomes in Virginia secondary schools, including student academic engagement, grades, and educational aspirations (Cornell et al., 2016). These dimensions of authoritative school climate also have been associated with lower negative outcomes, such as less peer victimization (Gregory et al., 2010), reduced suspension rates (Gregory et al., 2011), and lower prevalence of teasing and bullying (Konold et al., 2014).
Theorizing About Restorative Practices and Authoritative School Climate
Restorative practices have their origins in Indigenous traditions. For example, practices draw from Navajo peacemaking, family group conferencing of the Maori in New Zealand, and the circle process of Yukon’s Tagish and Tlingit people (McCaslin, 2005; Native American Rights Fund, n.d.). Restorative practices focus on strengthening the collective, healing relationships, and repairing harm (Zehr, 2014). In schools, this means shifting away from a punitive approach to misconduct and toward a restorative approach. It also means an explicit focus on belonging and connection so that the community can better weather rupture and conflict (Thorsborne & Blood, 2013). Further, there is an emphasis on engaging student voice in a fair process of decision making (T. Wachtel, 2016).
McCold and Wachtel (2003) directly linked restorative practices to an authoritative manner of interacting with students. Drawing on Glaser’s (1964) theorizing, they offer four quadrants in the social discipline window, with the restorative quadrant combining both high limit setting and high support (an authoritative approach) versus the other three quadrants (low/low and low/high limit setting/support). Vaandering (2013) offers more relational language, shifting to the terms high expectations and high support, which, she argues, supports humanizing interactions between educators and students. The notion is that developmentally attuned adults engage youth with consistent high expectations along with support—they further honor youth’s needs for autonomy and do things with students (rather than to them or for them).
Given the theoretical alignment, restorative practices could be a means through which to foster an authoritative school climate. Various restorative practices can be theorized to nurture students’ experiences of support and structure in their schools. Restorative practices include prevention- and intervention-oriented activities. On the prevention end, most well known are community-building circles typically used during advisory, academic instruction, or small-group meetings among students and/or staff. Boyes-Watson and Pranis (2020) described circles as a structured process of communication where each participant has a chance to speak. The circle facilitator offers a circle prompt that is meaningful to participants and elicits personal sharing—each person has an opportunity to share, including the educators. Sharing, it is theorized, strengthens relationships and a sense of community—thereby increasing a sense of support among peers and between teachers and students. Another restorative practice theorized to improve support is teaching and using affective language skills, defined as verbal and nonverbal emotion-focused communication that demonstrates empathy, openness, and respect (T. Wachtel, 2016). This type of communication may facilitate understanding and therefore stronger supportive relationships.
Restorative practice activities include problem-solving circles, informal restorative conversations, and formal restorative conferences to address serious incidents. Students and staff involved in disputes jointly share their experiences of the incidents, identify the harmed parties and underlying contributors to the problem, and seek solutions to repair the harm. A series of questions often guides the problem solving with those involved (e.g., “What were you thinking/feeling at the time?” “Who has been impacted?” “What impact has this incident had on you and others?” “What do you think needs to happen to make things right?”) (B. Wachtel et al., 2010). This structured dialogue, if done with regularity in response to discipline incidents and conflict, could foster a sense of consistency and fairness (i.e., structure) in the school climate. The forum for conflict resolution offers students (those harmed and those doing the harm) a chance to explain their perspectives, which honors adolescents’ need for autonomy.
Also related to structure, a sense of fairness and justice may be nurtured through equity-focused restorative practices, which have an explicit focus on social and racial justice (Davis, 2019; Manassah et al., 2018). The circle process can offer a format to nurture critical consciousness when using circle discussion prompts that address systemic racism and oppression (Winn, 2018). Especially when restorative practices explicitly address social justice issues such as racial and ethnic discrimination, they may foster a sense of fair treatment of minoritized groups in their school (Huguley et al., 2022). Scholars also note that restorative practices inherently draw on equitable principles through the circle process, which elicits student voice and disrupts typical school hierarchies by valuing multiple perspectives (Evans & Vaandering, 2022; Zehr, 2014). Through dialogue in a circle/conference, adults practicing perspective taking, empathy, and individuation may counter their own racial bias that contributes to the overrepresentation of students of color in discipline (Okonofua et al., 2020). Adults countering their biases then may facilitate disciplinary decisions based on students’ individual situations rather than subconsciously held negative stereotypes based on social group or race.
Theorizing About Restorative Practices and Perceptions of Teasing/Bullying Climate
Just as restorative practices may be associated with the promotion of positive aspects of the school climate, they also may help prevent a climate of negative exchanges among peers and teachers. Specifically, restorative practices’ focus on community-building, perspective-taking, and student input in decision making may strengthen bonds among peers and between teachers and their students. By doing so, they may help prevent aversive school climates that are characterized by teasing and bullying. With stronger affiliative connections, students may be more likely to perceive a positive interpersonal climate (e.g., lower climate of teasing/bullying). Theorizing along these lines (Markham & Aveyard, 2003), Bonell et al. (2018) evaluated a 3-year experimental trial in 40 middle schools (20 intervention and 20 control schools) in England. Intervention school staff received training in both restorative practices and a social–emotional learning (SEL) curriculum. The evaluators detected positive effects on student-reported surveys such that students in the intervention schools reported less bullying victimization than students in the control schools.
Restorative Practice Research and School Climate
Whereas the bulk of research on restorative practices in schools has been qualitative without control schools, the few experimental trials that examine whether restorative practice programs caused an improvement in school climate offer mixed findings (Zakszeski & Rutherford, 2021). Grant et al. (2022) examined the effects of a schoolwide dropout-prevention program combined with the Institute for International Restorative Practices’ (IIRP) training. In the 25-school experimental trial, they found small but positive effects on school climate. Specifically, improvements occurred in student-perceived support (e.g., “My teachers really listen to what I have to say.”), problematic behaviors (e.g., “How much of a problem is . . . students bringing weapons like knives and guns to school?”), and teacher-perceived professional learning and collaboration.
In another experimental trial, Augustine et al. (2018) detected positive effects based on teacher and staff surveys, but not student surveys. In their study of 44 schools in Pittsburgh, teachers in schools that implemented the IIRP 2-year program reported better student relationships, more positive working conditions, and improved climate managing student behavior (e.g., schools had clearer policies and had a safer environment) relative to teachers in the comparison schools. This positive effect on the climate, however, was not found when analyzing student surveys. The authors found no meaningful effect on a student-perceived school climate scale that measured feelings of safety and security in school and perceptions of teacher–student and student–student relationships.
Similarly, another experimental trial of the IIRP program showed that restorative practices did not cause improvements in student-perceived school climate (Acosta et al., 2019). In a 2-year experimental trial with 13 middle schools in Maine, Acosta et al. (2019) reported no statistically significant differences in student-perceived school climate. As an explanation for the null findings, the authors noted a minimal treatment contrast across the intervention and comparison schools, which included student-reported exposure to restorative practices (e.g., restorative conferences, restorative questions, and circles). Undertaking correlational analyses across both intervention and comparison schools, the authors found that when students reported that their teachers used restorative practices (e.g., restorative questions) more often, they also reported a more positive school climate (e.g., consistency and clarity of rules/expectations, teacher support, positive peer interactions, and student input) relative to students indicating less frequent restorative practices use. The authors concluded that restorative practice initiatives have promise for fostering positive climates if used consistently.
Relatedly, Darling-Hammond (2023) pointed out that “practices (rather than programs) are the drivers of outcomes” (p. 15). He called for researchers to track student exposure to restorative practices and not assume that it occurs after professional development. Using data from 485 California middle schools, Darling-Hammond showed that when schools increased their use of restorative approaches over years, as measured by increased positive student reports that their schools engage students and staff in repairing conflict, building community, and fostering cohesion across diversity, their perceptions of school climate improved. This suggests that increasing restorative practices exposure may leverage improvements in school climate.
This Study
Too few experimental studies have examined the impact of restorative practices on school climate (Zakszeski & Rutherford, 2021). The results from experimental trials on school climate have been mixed, with two showing positive effects (Bonell et al., 2018; Grant et al., 2022) and another showing null effects (Acosta et al., 2019). Further, recent research indicates the importance of not just examining effects based on staff trainings but also effects based on student exposure to restorative practices (Darling-Hammond, 2023). This study builds on prior studies and examines the effects of a single year of the Whole School Restorative Practices Project (RP Project) initiative on student perceptions of the school climate. Aiming to advance differentiation between training and student exposure to practices, this study examines the degree to which restorative practices exposure has positive school climate correlates. The research questions are as follows:
Methods
A cluster randomized, controlled trial was conducted to investigate the impact of the RP Project on school climate where schools were randomly assigned to be in either the treatment/intervention or control condition. Twenty-three schools from one large school district with similar demographic characteristics (i.e., student enrollment based on gender, race, language learner status, disability status, percent poverty, and grade levels offered) in one high-poverty school district were invited to participate. Of the 23 schools invited, 18 schools agreed to participate; schools were assigned to the intervention or control condition on June 1, 2018. Prior to collecting any data, the study was preregistered with Abt Associates, the technical advisor of the funding agency.
The study received institutional review board approval through both the school district and Rutgers University. All procedures performed involving human participants were in accordance with the Code of Ethics of the American Educational Research Association and the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Families were given an opportunity to review the survey in advance and to opt their student out of the survey. Students also had the opportunity to decline or assent to participation on the opening page of the survey. We collected surveys in spring 2019 during year 1 of the initiative. We halted data collection in spring 2020 due to the COVID-19 pandemic.
Participants
Of the 18 schools, six were elementary, six were middle, four were high, and two were combined middle–high schools. Project staff and evaluators met with all 18 principals to discuss the randomization process. Blocking by school type (e.g., middle schools) resulted in an equal number of elementary, middle, combined, and high schools to be randomized into the control and intervention conditions.
Based on administrative records, the 18 schools have an average enrollment of 366 students. The racial/ethnic composition, on average, was Black (55%), Hispanic (38%), and white (2%). Most students (91%) qualified for free or reduced-price meals (FRPMs), a proxy for socioeconomic status. Only fifth graders were surveyed at the elementary schools given that the prior validation of selected scales typically did not include earlier grades (Konold et al., 2014). The survey response rate was 83% of enrolled students.
Of the 2,674 students surveyed, assenting 5th through 12th graders, 1,393 students were in the control group and 1,281 students were in the intervention group. Following the What Works Clearinghouse (2020) guidelines, students (n = 235) who enrolled in the school a few weeks after the start of the 2018–2019 school year (referred to as late joiners) were not included in the analytic sample. Because the use of validity screening measures has been shown to improve overall survey quality with adolescents (Cornell et al., 2012), students who had indicated that they were not telling the truth (n = 106) and those that responded in less than 5 minutes (n = 46) also were excluded. Finally, students with missing data on all outcome measures (n = 39) were excluded, resulting in an analytic sample of 2,248 students (nControl = 1,145; nTreatment = 1,103).
Of the analytic sample (male = 48%), the majority self-identified as Black/African American (38%), followed by Hispanic/Latinx (32%). Twenty-nine percent indicated some other race/ethnicity (e.g., American Indian or Alaska Native, Asian, Native Hawaiian or Pacific Islander, Other, or two or more of the listed racial groups), and 1% indicated white. Eighteen percent were eligible for special education services, and 16% were English language learners. The largest percentage of students reported that their parents graduated from high school without further formal education (38%). Comparing both school- and student-level characteristics showed minimal differences across all characteristics (see Table 1).
Descriptive statistics by condition
FRPM. free or reduced-price meals
Notes. P values for χ2 (for categorical) or t tests (for continuous variables) accounting for clustering within schools. d = Cohen’s d or standardized mean differences.
Intervention Condition
The RP Project was comprised of five components, which included professional development and coaching with school leaders and staff. Restorative practices coordinators worked for 1–3 full days per week in the intervention schools in the 2018–2019 school year. They served many roles, including as lead trainers, individual consultants, and coaches. The restorative practices coordinators had decades of experience working in education (mean = 22.6 years; min = 14 years; max = 37 years) and substantial experience advancing SEL and/or restorative practices in consultant/coach roles (mean = 11.5 years; min = 3 years; max = 27 years).
The four components for year 1 were (1) restorative practices leadership team development, (2) restorative interventions, (3) schoolwide staff development, and (4) family restorative practices opportunities. The comprehensive nature of the restorative practices, SEL, and racial equity project was designed to include explicit social–emotional skill development and address equity through a restorative format. Across the grade levels, the schoolwide staff training (e.g., restorative mindset, understanding race, racism, and oppression) was similar. Teacher circle facilitators were trained in grade-level specific SEL curriculum integrated into weekly restorative community-building circles during advisory or English language arts periods. Further, each program component was designed to address equity through adults’ intrapersonal exploration about bias and privilege and through their critical examination of policies/practices that marginalize students (for more information on the intervention, see Gregory et al., 2022). The schools implemented the trainings at adequate levels as determined by fidelity standards established in collaboration with project designers and the research team based on a pilot year of implementation (with three different schools; for details, see Online Supplementary Materials and Table S1).
Control Condition
Control schools were provided $5,000 and could engage in the RP Project training at the end of year 2 of implementation. Principal interviews in control schools indicated that none were implementing integrated restorative practices initiatives that focused on capacity building of the school leadership, taught SEL skills, and had school-based teams develop and deliver training programs using the circle process. Yet, in three of the nine schools, principals noted some level of restorative practices programming. During the first year of the project, one school was inconsistently implementing circles in advisory and another school had an option for students to self-refer to conferences for minor discipline incidents. Finally, one middle school was doing both circles in advisory and conferences for more serious incidents.
Measures
In each school, a team of university students and the project coordinator administered the 30-minute student and staff surveys in-person and assisted individuals as needed. They used a standardized set of instructions for the online, anonymous Qualtrics surveys (Qualtrics, Provo, UT). Three intact school climate scales (i.e., student support, disciplinary structure, and the prevalence of teasing and bullying) were drawn from the Authoritative School Climate Survey (ASCS; Cornell et al., 2017). Studies have shown strong psychometric support for the ASCS, whose factor structure has been investigated using multilevel exploratory and confirmatory factor analyses (e.g., Konold et al., 2014) and has been used in various studies on school climate (e.g., Cornell & Huang, 2016; Huang & Cornell, 2018). Response options ranged from “Strongly disagree” (1) to “Strongly agree” (4). For all measures, reliability is shown using categorical omega (ω), which can be interpreted in the same manner as Cronbach’s alpha (Hayes & Coutts, 2020). For all measures, summed scores are used.
School Climate: Student Support
An eight-item scale measured perceived supportiveness of student–teacher relationships and included such items as, “Most teachers and other adults at this school care about all students” and “There are adults at this school I could talk with if I had a personal problem.” In this study, scores ranged from 8 to 32 (mean = 24.91; SD = 4.33; skew = −0.57, and ω = .85).
School Climate: Disciplinary Structure
The scale consisted of seven items measuring perceived fairness and consistency of school rules using such items as, “The school rules are fair” and “The punishment for breaking school rules is the same for all students.” Based on the current sample, scores ranged from 7 to 28 (mean = 19.63; SD = 3.60; skew = −0.38, and ω = 70).
School Climate: Racial/Ethnic Equal Status
A limitation of the ASCS is its lack of focus on perceived fairness as relates to race/ethnicity. Selected as a validated subscale from the School Racial Climate Survey (Byrd, 2017), we included the three-item racial/ethnic equal status subscale. Its focus on fair treatment across racial and ethnic groups fits with the racial equity emphasis of the initiative. Items included “Students of all races/ethnicities are treated equally at your school”; “The principal/school leaders treat students of all races/ethnicities fairly”; and “At your school, teachers are fair.” Response options ranged from 1 = ”Not at all true” to 5 = ”Completely true.” Based on the current sample, scores ranged from 3 to 15 (mean = 12.04; SD = 2.86; skew = −0.97; and ω = .84).
School Climate: Teasing and Bullying
Five items from the ASCS were used to measure the prevalence of teasing and bullying subscale. The well-established scale measures the perceived climate of aversive peer interactions, not personal experience with being bullied. Items such as, “Students in this school are teased about their clothing or physical appearance” and “Students in this school are teased or put down because of their race or ethnicity” were included. In this study, scores ranged from 5 to 20 (mean = 11.77; SD = 3.58; skew = 0.20; and ω = .83). We also included a measure of the perceived negative climate of teacher–student interactions or the teacher bullying climate scale from the ASCS with the following three items: “There are teachers or other adults at this school who make fun of students”; “Some teachers or other adults at this school say things that make students feel badly”; “Some teachers or other adults at this school pick on certain students.” Scores ranged from 3 to 12 (mean = 6.17; SD = 2.36; skew = 0.35; and ω = .82). This negative teacher–student interaction scale measures the perceived climate of bullying/teasing, not personal victimization.
Restorative Practices Use
We measured student exposure to restorative practices based on their self-report on seven items. Students indicated the frequency with which they were exposed, including “Not at all,” “Rarely,” “Sometimes,” “Often,” and “Always.” The restorative practices use scale was shortened by Gregory and Clawson (2016) from a longer version of the IIRP self-assessment scale. With the current sample, factor analysis yielded two factors. The first factor focused on engaging student voice through restorative practices, as seen in the four items, “My teachers ask students to express their feelings, ideas, and experiences”; “My teachers use circles as a time for students to share feelings, ideas, and experiences”; “My teachers take students’ thoughts and ideas into account when making decisions”; and “The administration (principal, vice principal) listens to my side of the story.” We named the factor RP voice (ω = .79). The second factor focused on engaging students in problem solving and repairing harm, as measured by three items: “When someone misbehaves, my teachers ask students questions about their side of the story”; “When someone misbehaves, my teachers have that person talk to who they hurt and ask them to make things right” and “When someone misbehaves, my teachers have those who were hurt have a say in what needs to happen to make things right.” We named the factor RP repair (ω = .83). With the current dataset, the two-factor measurement model, using cluster robust confirmatory factor analysis, fit the data well (route mean square error of approximation [RMSEA] = .05; comparative fit index [CFI] = 1.00; Tucker–Lewis index [TLI] = .99). Although the two subscales were highly correlated (r = .73), a likelihood-ratio test indicated that a two-correlated-factor model fit better than a one-factor model (χ2(1) = 88.48; P < .001).
Student and School Sociodemographic Characteristics
Students self-reported their race and were able to select “All that applied.” This resulted in two larger groups: Hispanic/Latinx and Black/African American. It also resulted in a range of groups of a smaller size including American Indian or Alaska Native, Asian, Native Hawaiian or Pacific Islander, white, other, or one or more of the listed racial groups. We combined small subgroups of students into a heterogeneous “other” category. Students reported on their gender identity (i.e., male, female, or transgender/gender nonconforming/questioning/other). Students also self-reported on whether they had an individualized education plan or 504 plan for special education services. Further, students indicated the highest level of parental education, a proxy for socioeconomic status. They also answered, “Are you learning English as a new language?” Finally, we obtained publicly accessible sociodemographic information about each school (e.g., percentage of students eligible for FRPM).
Analytic Strategy
We examined whether assignment to the restorative practices intervention condition resulted in more positive school climate (Research Question 1). Because the intervention status was assigned at the school level, to account for the nesting of students within schools, two-level random intercept models were used. Overall, 88% of the respondents had complete data, and the outcomes and the survey-based covariates (e.g., gender, race) all had minimal missing data (<2%). The variables with the most missing data were the RP voice (7.0%) and RP repair (6.4%) subscales. To account for missing data, we used multilevel multiple imputation with a joint-modeling approach employing the jomo (Quartagno & Carpenter, 2019) and the mitml (Grund et al., 2019) packages in R. Based on the recommendations of White et al. (2011), 15 multiple imputations were used. Imputed data were analyzed using multilevel linear models estimated using restricted maximum likelihood with the lmerTest (Kuznetsova et al., 2016) package in R version 4.1 (R Core Team, 2022). Multiply imputed results were pooled using Rubin’s (2004) rules.
The student-level covariates included dummy codes for race (Latinx, other, and Black as the reference group), gender (male, other, and female as the reference group), special education status, and English language learner status. The highest level of parental education, a proxy for socioeconomic status, also was included. At the school level, the percentage of students eligible for FRPM and the percentage of Black students enrolled were included as covariates. The continuous school-level covariates were grand mean centered. The predictor of interest was intervention status (at the school level; 1 = treatment; 0 = control; the variable of interest). School type was included as a series of dummy codes with middle schools as the reference group. The combined impact evaluation formula can be expressed as
where Yij represents the outcome (i.e., the school climate measure) of student i in school j, and δ represents the intent-to-treat effect. All outcomes were standardized (i.e., mean = 0; SD = 1), and the treatment effect can be interpreted as standardized mean differences.
For the second research question, which investigated if higher restorative practices use was associated with improved school climate outcomes, the two restorative practices subscales were standardized and entered into the model together. Outcomes and restorative practices use scales can be interpreted as standardized regression coefficients. To account for multiple outcomes, Bonferroni adjustments were used.
Results
Intervention Effect on School Climate
Unconditional multilevel models (i.e., models with no predictors) were fit to assess the intraclass correlation coefficients for the different outcome measures. Intraclass correlation coefficients ranged from .02 for equal status to .07 for both disciplinary structure and the prevalence of teasing and bullying. Next, random intercept models were fit for each of the five outcomes of interest and included student- and school-level covariates (see Table 2). All the intent-to-treat coefficients were small (|d| = .03 to .13), and no intervention effects were found for any outcomes (P = .25 to .84). Given that one control middle school indicated that it was implementing both circles and conferences, we reran the preregistered analyses without that control school and found no substantive differences in the findings.
Student outcomes predicted by intervention condition (n = 2,248 students in 18 schools)
Restorative Practices Use Subscales and School Climate
We examined the restorative practices use measures by testing whether there were statistically significant differences in both restorative practices use scales in the intervention and control schools. Results indicated that there were no statistically significant differences (bVoice = 0.172, P = .13 and bRepair = −0.065, P = .42) between the two conditions. Visualizations of the restorative practices use scales using density plots (which show the proportion of values in each range) are shown in Figure 1. In both the intervention and control conditions, the plots are very similar, as shown by the gray overlap across conditions.
Student self-reported experiences with restorative practice by intervention condition.
We then entered the restorative practices use scales as predictors of the school climate measures as the outcomes. Results show that both scales had a beneficial association with each of the outcomes (see Table 3; all P values are <.01, except for RP repair as a predictor of the climate of teasing and bullying scale). For example, a 1 SD increase in the RP voice subscale was associated with a 0.25 SD increase in disciplinary structure. A 1 SD increase in RP repair was associated with a −0.11 SD decrease in a climate of teacher bullying. We investigated whether the restorative practices subscales were moderated by intervention condition or school type; all interactions were not statistically significant (all P values >.05).
Student outcomes predicted by restorative practices (RP) use scales (n = 2,248 students in 18 schools)
Notes. Results from two-level random intercept models including student- and school-level covariates (not shown) with both restorative practices use subscales together in the model. All outcomes and restorative practices use scales standardized. Based on 15 multiply imputed datasets. Statistical significance can be evaluated using a Bonferonni adjusted alpha of .01.
Discussion
Despite the popularity of restorative practices in schools, there are only a handful of experimental studies that have examined the effects of restorative practices on student perceptions of school climate, highlighting the need to build the experimental evidence base. Our preregistered cluster randomized, controlled trial examined the impact of the RP Project on school climate. By the spring of the first year of implementation, no intervention effects were found comparing student survey responses in schools randomly assigned to the intervention or control condition. This indicates that restorative practices did not positively (or negatively) change the school climate.
We also found that across experimental conditions, students reported a similar frequency in their teachers eliciting their voices and engaging them in repairing harm. The variability in self-reported exposure to restorative practices within schools, however, was associated with student outcomes. Students who reported greater exposure to restorative practices, compared with those reporting less exposure, perceived greater structure, support, and racial/ethnic equality status in their school climate. Further, greater restorative practices exposure was associated with lower perceptions of a climate of bullying and teasing from students and from teachers.
Understanding the Experimental Findings
The null experimental findings differ from other experimental trials that found effects of restorative initiatives on school climate (e.g., Grant et al., 2022). Our current initiative differs from these prior restorative practice projects that implemented their programs for 2 years or more. Thus, the null results may reflect that the RP Project was truncated to a single year given the switch to remote schooling after the onset of COVID-19. Some scholars have suggested that restorative practices can take several years for uptake (Mansfield et al., 2018). The field would benefit from greater research differentiating effect sizes among 1 versus 2 years or more of program implementation—a differentiation that is missing from recent meta-analytic studies of comprehensive school initiatives (Goldberg et al., 2019; Melendez-Torres et al., 2023).
In assessing experimental treatment contrast through principal interviews, we found that control schools were not using a comprehensive, racial equity restorative practice and SEL initiative. Further, six control schools had no restorative practices, but three control school principals reported some level of restorative practices in their schools, although the scope of implementation varied. Despite this variable implementation of restorative practices in control schools, this study showed that in both experimental conditions, students similarly perceived that their teachers were engaging their voice, facilitating problem solving, and repairing harm when conflict arose. This suggests that across schools even without comprehensive restorative practices initiatives, individual teachers may be integrating restorative practices approaches into their personal classroom practices. In fact, uptake of principles of restorative practices may be more diffuse in districts with widespread restorative practices trainings (Diliberti et al., 2019). Diffusion of practices pose real challenges for conducting gold standard experimental trials given that a stark contrast across treatment and control schools cannot be assumed. This calls for researchers to closely examine innovations in both treatment and control conditions, ideally over multiple years of an initiative.
It also might be the case that the RP Project professional development training did not lead to widespread changes in educator–student interactions. We cannot assume that training leads to increased student restorative practices exposure (Darling-Hammond, 2023). A limitation of this study included not deploying outside observers to record whether, after training, teachers more frequently used specific restorative practices (e.g., restorative questions).
Restorative Practices Exposure
In their Maine experimental trial, Acosta et al. (2019) also did not find an effect of restorative practices on the school climate. Yet, also similar to this study, they found that greater frequency of restorative practice–related experiences was associated with more positive student-reported school climate in both control and intervention conditions. Understanding why exposure to restorative practices is associated with more positive perceptions of the school climate requires additional research. In this study, when students reported that their teachers frequently solicited their opinions and feelings (RP voice), they also reported greater support from adults in the school and fair treatment across racial and ethnic groups. Future research might examine whether an explanatory mechanism is that when individuals feel respected by authority figures, they tend to view their use of power as legitimate and trustworthy (Huguley et al., 2022; Tyler, 2006).
This study also found that when students reported that teachers frequently facilitated student problem solving among both disputants and those harmed (RP repair), they also reported a lower climate of teacher bullying. An explanation might be that when students perceive that youth voice is elicited regarding their perspectives about conflict, it generally strengthens bonds between teachers and students. Said differently, teachers’ concerted effort to seek student input may prevent aversive teacher–student interactions from occurring in the first place, although this is said speculatively given the correlational nature of the findings.
Limitations
These experimental results need to be viewed with caution given that the RP Project was not completed due to COVID-19 even though it was designed for 2 full years. A single year of an initiative may not be enough to detect widespread effects on student perceptions of the school. Also, a noteworthy limitation is that the control condition included some schools with staff trained in restorative practices. Thus, a limitation of the experimental analyses was that from the student perspective, a sharp treatment contrast was not found. In many districts, restorative practices principles may be widespread, with some staff regularly using student-centered approaches (e.g., engaging student voice). Future studies may need to better differentiate restorative practices that integrate a focus on both racial equity and SEL—as was conducted in this project. In some states and districts, it may be easier to find stark contrasts in restorative practices exposure across schools. For example, Nevada recently minimized restorative justice requirements (e.g., Zalaznick, 2023). Finally, the restorative practices exposure and school climate correlational findings require caution in interpretation. Both perceived practices and outcomes were based on student surveys. Thus, common-method bias limits the strength of the findings. Future studies might draw on different informants (e.g., teachers and students) and methods (e.g., observer ratings of restorative practices exposure) in predicting student-perceived school climate.
Conclusion
The study adds to the few experimental studies examining the effects of restorative practices on school climate (e.g., Acosta et al., 2019). Recent articles in psychology and education sciences have highlighted the importance of replicating studies in order for research to influence educational policy and practice (Makel & Plucker, 2014). This study did not find meaningful effects of the restorative practices intervention on measures of authoritative school climate and the climate of teasing and bullying. This may be due to the truncated initiative and analyses after only a single year of implementation. Similar to Acosta et al. (2019), we found that student reports of higher restorative practices exposure predicted positive perceptions of authoritative school climates. When students perceived that their teachers elicited their voice and engaged them in resolving conflict and repairing harm, they tended to also perceive adult support, structure, and fair treatment across racial and ethnic groups. Our null experimental findings, together with the correlational analyses, are important to consider (for a perspective on interpreting null findings, see Herrington & Maynard, 2019) and may highlight the need to allow for a longer period of time to assess the impact of complex, whole-school reform initiatives that involve a change in mindsets.
Supplemental Material
sj-docx-1-ero-10.1177_23328584241288525 – Supplemental material for Adolescent Exposure to Restorative Practices and Their Perceptions of Support, Structure, and Bullying in the School Climate
Supplemental material, sj-docx-1-ero-10.1177_23328584241288525 for Adolescent Exposure to Restorative Practices and Their Perceptions of Support, Structure, and Bullying in the School Climate by Anne Gregory, Francis L. Huang and Allison Rae Ward-Seidel in AERA Open
Footnotes
Declaration of Conflicting Interests
The authors declare no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This research was supported by funding from the U.S. Department of Education Investing in Innovation (i3 Grant) through Grant Award U411C160035 to the Morningside Center for Teaching Social Responsibility.
Authors
ANNE GREGORY, Ph.D., is a professor in the Graduate School of Applied and Professional Psychology at Rutgers University. Her research aims to strengthen characteristics of teachers, classrooms, and schools associated with the successful schooling of minoritized and marginalized students.
FRANCIS L. HUANG, Ph.D. is a professor in the Statistics, Measurement, and Evaluation in Education Program in the Department of Educational, School, and Counseling Psychology at the University of Missouri. He is an applied quantitative methodologist with current substantive interests in school climate and the disproportional use of exclusionary disciplinary sanctions.
ALLISON RAE WARD-SEIDEL is a doctoral student at the University of Virginia. She is a former teacher and currently studying restorative practices in education as a mechanism to improve school climate and address inequity in school discipline.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
