Sage Journals: Discover world-class research

Abstract

This manuscript describes our effort to apply a novel approach to understanding student outcomes associated with a schoolwide antiracist intervention. We report a multimethod quantitative approach to evaluate a 10-week antiracist intervention designed and implemented by school staff by examining patterns of student intervention engagement and measures of key constructs that connect to antiracism, psychological well-being, and school connectedness. Our novel approach combines schoolwide surveys with smaller samples of daily diary participants, documenting variation in intervention engagement and examining postintervention outcomes. Our findings are limited by high attrition rates, small sample size, and data collection during the COVID-19 pandemic; however, our methods offer a promising transferable approach to evaluate school-based antiracist interventions by examining patterns and predictors of intervention engagement, as well as daily fluctuations in student experience throughout the intervention period.

Keywords

adolescence antiracist education belonging critical consciousness dual pandemics evaluation high schools school-led intervention stress

In response to the co-occurrence of dual pandemics (i.e., COVID-19 and racial injustice) in 2020, many educators in the United States adopted antiracist curricula to help students process current events and the history of anti-Black racism in the United States (Agarwal & Sen, 2022; Scheyder, 2020). Several education associations published statements of support for antiracist education, arguing that all students need and can benefit from these practices (Alvarez, 2021; Myers, 2021; Torres, 2020). However, learning about the history of racism is currently a contested and politicized issue in U.S. K–12 education. As of April 2023, 88% of states adopted or introduced laws or policies to either expand or restrict the teaching of race/racism (Stout & Wilburn, 2022). Of these states, 27 passed or had pending legislation, state policies, or executive orders that sought to limit how K–12 educators could discuss race in their classrooms (America, 2023; Stout & Wilburn, 2022), with critics arguing that talking about racism in schools can harm White students’ psychological well-being and lead to negative interracial dynamics in school (Lewis, 2022; Sawchuk, 2021). Empirical support for legislative efforts on both sides of this issue is lacking, given limited research regarding student outcomes of school-based antiracist educational practices.

In fall 2020, our partner school implemented schoolwide advisory periods to help educators identify and support students’ socioemotional needs and build positive relationships. In the context of these advisory periods, the school also designed and implemented a 10-session antiracist intervention (the Stamped Intervention), which involved students and staff reading and discussing Stamped: Racism, Antiracism, and You (Reynolds & Kendi, 2020). This book encourages readers to develop an antiracist mindset by identifying examples of racism and antiracism throughout history. The goal was for students and staff, through text and dialogue, to develop awareness of historical and current racism in the United States.

This study describes a research–practice partnership between developmental scientists and partner school leaders to explore student outcomes associated with the Stamped Intervention. We present a novel quantitative approach that expands on methods typically used in prior literature (e.g., pretest/posttest design, measures of outgroup attitudes). In particular, the novelty of our approach is grounded in (a) examining underlying patterns of student intervention engagement, (b) combining schoolwide surveys with smaller samples of daily diary participants, and (c) using measures of key constructs that connect to antiracism, psychological well-being, and school connectedness to examine these constructs as both predictors and outcomes of student intervention engagement. Although our findings are limited by high attrition rates and small sample sizes due to data collection occurring during COVID-19 with fully virtual schooling, our methods offer a promising approach to evaluate school-based antiracist interventions by examining patterns and predictors of intervention engagement, as well as daily fluctuations in student experience throughout the intervention period. In sharing our work, we offer a multimethod approach that others might adopt to evaluate school-based antiracist curricula, thereby informing education practice and policy.

Antiracist Education and Critical Consciousness Development

Critical consciousness (CC) is a developmental process that involves gaining awareness of social inequality and taking action to reduce inequality (Freire, 1970). CC is thought to include three components: critical reflection, critical motivation, and critical action (Diemer et al., 2017; Rapa et al., 2020; Watts et al., 2011). Critical reflection refers to an individual’s awareness of social inequality, which results from analyses of structural and historical roots of social inequities (e.g., racism, classism, sexism, ableism). Critical motivation involves individuals’ political efficacy or their perceived agency to affect political and social change either as an individual or as part of a collective. Critical action involves sociopolitical participation aimed at reducing inequities. Freire (1970) argued that CC development is best encouraged through a critical pedagogy in which individuals learn about systemic oppression through problem-posing texts and open dialogue between teachers and students.

Antiracist education is a form of critical pedagogy (Freire, 1970, 1973; Monchinski, 2008) that promotes race-based CC development by emphasizing critical analysis of the world through the lens of racial privilege and oppression and critical action to reduce racial injustice. Antiracist education focuses on identifying systemic racism, challenging complicity in racial oppression, and transforming structural inequalities (Lynch et al., 2017). As such, antiracist education promotes CC socialization in schools by encouraging students to recognize and address differences between racial groups’ power and privilege (Byrd, 2017).

Research suggests that exposure to antiracist education and other forms of critical pedagogy can benefit students’ development (Cabrera et al., 2014; Pinedo et al., 2021), especially for racially marginalized youth (Bañales et al., 2019; Gray et al., 2020). Most critical pedagogy literature focuses on racially marginalized (i.e., Black and Latino/a/e/x) students, suggesting that exposure to critical pedagogy facilitates CC and ethnic-racial identity development, which serve as protective factors that help racially marginalized students navigate structural inequalities (Pinedo et al., 2021). Studies focused on antiracist education for White students generally report positive outcomes such as reduced racial bias (Hughes et al., 2007) and greater appreciation of Black, Latino/a/e/x, and Asian American contributions (Nelsen, 2020). Together, prior research suggests that racially marginalized and privileged students might experience antiracist curricula differently, although positive outcomes are generally seen for both.

Methodological Features of Antiracist Interventions

Studies exploring outcomes of antiracist curricula have employed a variety of methods and span in-school and out-of-school contexts. Further, prior research examines a broad range of antiracist efforts. While the Stamped Intervention is best conceptualized as antiracist education in that it involves helping students to both identify and challenge systemic forms of racism they observe (Lynch et al., 2017), prior work has considered antiracist programming to be any technique that attempts to reduce racist thinking (McGregor, 1993; Weems et al., 2022). An early meta-analysis of instructional practices implemented in the 1970s revealed that antiracist teaching practices reduce students’ racist attitudes (McGregor, 1993). More recently, a systematic review of 23 experimental studies evaluating various school-based antiracist interventions concluded that interventions that leverage cognitive and educational techniques to encourage positive outgroup contact seemed the most promising (Weems et al., 2022). As noted, however, they conceptualized antiracism as any technique seeking to remediate racist ideas, which differs from our definition of antiracist education as identifying and challenging systemic racism (Lynch et al., 2017). Weems et al. (2022) proposed several dimensions on which future experimental tests of school-based antiracist interventions can be improved, including the use of treatment manuals, fidelity checks, outcome measures with strong psychometric properties, preintervention and postintervention assessments, longer-term follow-up assessments, and a focus on outcomes directly associated with antiracist attitudes, beliefs, and actions. Nonexperimental, quantitative evaluations of school-based antiracist programming tend to focus on interpersonal racism, with immediate posttests demonstrating improvements in youth’s understanding of interpersonal racism and outgroup attitudes (Kingett & Abrams, 2017; Turner & Brown, 2008). Our approach builds upon this work by applying methods appropriate to a universal, school-developed and implemented intervention employing a nonexperimental design. Specifically, we collected data on student experiences before, during, and after the intervention using measures with sound psychometric properties and explored outcomes of antiracist interventions beyond outgroup attitudes and behavioral intentions, including psychological well-being, school connectedness, and CC.

Prior work has explored CC as an outcome of antiracist interventions in the context of extracurricular programs. For example, adolescents who participated in an 8-week intergroup dialogue program in Detroit demonstrated greater awareness of racism and took action to address issues of racism in their communities (Aldana et al., 2012). More recently, Burgess et al. (2021) found that adolescents who participated in a 5-day intensive peer advocate program demonstrated greater awareness of racism immediately following the intervention. Together, these findings highlight how out-of-school antiracist programming might facilitate adolescents’ CC development. However, the effectiveness of schoolwide antiracist programming is less clear. Some case studies (de los Ríos et al., 2015; McLean et al., 2017) document outcomes of school-based antiracist curricula for particular groups of students (e.g., those enrolled in an ethnic studies course), offering rich descriptions of the journey to connect antiracist work beyond the classroom to the larger community. One study found that students in a Mexican American studies course had better academic outcomes than comparable students not in the course (Cabrera et al., 2014). Thus, adolescents exposed to school-based antiracist curricula tend to experience positive outcomes; however, these studies do not consider adolescents’ level of engagement or how their experiences might vary throughout the course. The current study seeks to expand this work by exploring patterns of student engagement in an antiracist intervention, predictors of their engagement patterns, and concurrent experiences of belonging and stress throughout exposure to antiracist curricula.

Partner School Intervention Context and the Evaluation Approach

The Stamped Intervention occurred during the 2020–2021 academic year when schooling was entirely virtual. Before the school year began, the school assigned advisory groups, which consisted of one to two staff members (teachers, instructional coaches, and/or support staff) and approximately 15 students of mixed grade levels. Copies of Stamped (Reynolds & Kendi, 2020) were distributed to all students and staff before school began. The Stamped Intervention occurred during 10 weekly, 45-minute advisory periods (October–December 2020). During each period, staff facilitated virtual lessons within their groups and encouraged discussions centered on understanding the weekly reading and connecting it to current events. Prepared lessons designed by the school’s Advisory and Equity Committees were available to staff. Lessons were designed to foster dialogue and to allow multiple entry points for students to engage (see supplemental materials for unit guides and a description of staff training). Advisors were given autonomy to structure Stamped lessons/discussions to meet their groups’ needs. Most advisory leaders used or modified the prepared lessons to facilitate Stamped discussions, and few deviated from the prepared materials to address the same topics. The Stamped Intervention aligns well with Freire’s (1970) approach to critical pedagogy by centering on a problem-posing text (Stamped) and encouraging open, critical dialogue between teachers and students.

Several features of the Stamped Intervention are relatively novel (i.e., virtual, schoolwide implementation; designed by school leaders; prolonged programming), which led us to take a multipronged evaluation approach. Expanding upon pretest/posttest designs traditionally used to evaluate antiracist interventions (Aldana et al., 2012; Burgess et al., 2021; Kingett & Abrams, 2017; McGregor, 1993; Nelsen, 2021), we used multiple methods to understand student engagement and change in key outcomes over time, including a whole-school data collection before and after the Stamped Intervention and a daily diary component tracking daily fluctuations in belonging and stress throughout the intervention period, conducted with a small subsample of students. Given that students were encouraged but not required to attend advisory periods and participate in the Stamped Intervention, we employed latent class analyses to explore patterns of engagement across three possible intervention components: (a) reading the book, (b) attending the Stamped-related advisory periods, and (c) contributing to Stamped-related advisory discussions. Thus, our approach pairs traditional pretest/posttest assessments with latent class analyses designed to assess variation in intervention engagement and a daily diary component to explore fluctuation in student experience across the intervention to address the following research questions:

Are there distinct patterns of student engagement across the intervention components (i.e., reading the book, attending advisory periods, and participating in discussions)?

Do students’ preintervention school connectedness (feelings of belonging, student–staff relationships), overall stress, and/or CC predict their patterns of engagement with the Stamped Intervention?

Do patterns of engagement and student racial identity predict changes in school connectedness, overall stress, and/or CC after the intervention period?

During the intervention period, do students’ daily experiences of belonging and stress differ on days when the intervention occurs? Do these effects vary as a function of students’ racial identity and patterns of engagement?

Given that prior studies evaluating outcomes of school-based antiracist curricula did not factor in student engagement or concurrent experiences, our exploratory research did not have specific hypotheses. Our decision to include belonging, quality of student–staff relationships, and stress as predictors and outcomes of intervention engagement was informed by our conversations with the school and literature demonstrating that these constructs are important indicators of adolescent socioemotional well-being in school (Eccles & Roeser, 2013). Our choice to include CC as a predictor and outcome of intervention engagement was informed by the theoretical goals of critical pedagogy and antiracist education (Freire, 1973; Lynch et al., 2017; Monchinski, 2008). Thus, the current study provides a novel approach for scholarship centered on understanding the impact of school-based interventions focused on race.

Method

Design

The design of this research study involves a preintervention/postintervention survey and daily diary surveys. The preintervention survey occurred two weeks before the Stamped Intervention began. The postintervention survey occurred three months after the Stamped Intervention concluded. A subsample of students completed daily diary surveys, which occurred over the course of three nonconsecutive weeks for a total of 15 days, thereby assessing student experiences at the beginning, middle, and end of the Stamped Intervention (see supplemental materials for calendar details).

Participants

Our partner school is a public high school located in the Southeastern United States. Although all students were invited to participate in the study, the virtual learning environment during COVID-19 limited participation; thus, the sample for the current analyses does not reflect the whole school population (see supplemental materials for schoolwide demographics and attrition analyses). Further, our partner school is in a majority-White, politically liberal community near a university; thus, findings may not generalize to other contexts.

A total of 584 students (68.8% of enrolled students) completed the preintervention and/or postintervention surveys, but 325 students were excluded because they only completed one survey, thus missing key study information. The final sample included students (N = 227, 26.7% of enrolled students) who participated in both surveys and completed all measures of interest. See Table 1 for racial/ethnic (approximately 70% white students, 30% minoritized students) and gender identity of participants.

Table 1

Participant Demographics

	Final Sample ParticipantsN = 227	Daily Check-in Subsample Participantsn = 67
Age	15.99 (1.15)	15.99 (1.07)
Grade	27.3% 9th grade	22.4% 9th grade
	25.1% 10th grade	28.4% 10th grade
	26.9% 11th grade	29.9% 11th grade
	20.7% 12th grade	19.4% 12th grade
Gender	41.4% male	28.4% male
	52.9% female	71.6% female
	5.7% nonbinary, prefer to self-describe, prefer not to say
Race/ethnicity	67.4% majoritized (White)	70.1% majoritized (White)
Race/ethnicity	31.2% minoritized (12.3% multiracial, 7.0% Latinx/Hispanic, 6.2% Black/African American, 5.7% Asian/Asian American, 1.3% did not respond)	29.9% minoritized (11.9% multiracial, 6.0% Latinx/Hispanic, 7.5% Black/African American, 4.5% Asian/Asian American)

A total of 91 students participated in a “Daily Check-In Study” and completed at least one daily survey; however, only 67 students (29.5% of final sample) also participated in the postintervention survey and were included in this study (see supplemental materials for details).

Procedures

Study procedures were approved by the Institutional Review Board at Duke University with a reliance agreement with the IRB at North Carolina State University. Two weeks before pre- and post-Stamped data collection, the school informed all parents/guardians of the surveys and included information on how to opt their child(ren) out of participation; approximately 30 students were opted out of each survey by their parents. Students who did not opt out of participation and assented had approximately 45 minutes to complete the surveys during an advisory period (pretest) and an extended class period (posttest). Teachers introduced the study by reading a script and directing students to their school email for the survey links. Following assent, students completed the survey (in English or Spanish). If they did not finish during the allotted period, students had one week to complete the survey in their own time. The research team donated $50 per grade per timepoint to a charity determined by student vote. To further incentivize participation in the postintervention survey, students who completed the survey were entered into a drawing for one of 25 electronic $50 gift cards.

One item on the preintervention survey described an upcoming “Daily Check-in Study.” Students who indicated interest in learning more about the study were asked to provide their parent contact information, and those whose parents did not opt them out of participation were emailed an assent form. Students who agreed to participate before the first “check-in” received daily survey links via email. Students who completed at least four of the five surveys per week received a $15 electronic gift card.

Measures

Stamped Intervention Engagement (postintervention survey)

We developed three items asking students to retrospectively report their level of engagement in each intervention component throughout the intervention period. Specifically, students were asked: (a) how much of the book they read during the fall semester on a scale of 1 (none of it) to 5 (all of it), (b) how many of the Stamped-related advisory periods they attended on a scale of 1 (none of them) to 5 (all of them), and (c) how much they participated in the Stamped-related advisory discussions on a scale of 1 (not at all) to 5 (a great deal).

Student–Staff Relationships (Preintervention/Postintervention Surveys)

To understand the quality of student–staff relationships at school, students responded yes (1) or no (0) to three items developed for this study. Items asked students if they had: (a) a positive relationship with at least one staff member, (b) at least one staff member that they could talk to about a personal matter, and (c) at least one staff member that knows them well. Items were summed to create a composite score representing the quality of student–staff relationships ranging from (0) poor to (3) strong (α_{pre and post} = .71).

Critical Consciousness (Preintervention/Postintervention Surveys)

Students responded to two subscales included in the Short Critical Consciousness Scale (CCS-S, Rapa et al., 2020). The CCS-S has demonstrated good reliability and validity as well as measurement invariance across adolescent ethnic/racial, age, and gender groups (Rapa et al., 2020). Our study focused on the Critical Reflection (CR): Perceived Inequality subscale (three items, e.g., “Certain racial or ethnic groups have fewer chances to get ahead”; α_pre = .93, α_post = .91) and on the Critical Motivation (CM) subscale (four items, e.g., “It is important to correct social and economic inequality”; α_pre = .83, α_post = .81). Students responded to items on a scale from 1 (strongly disagree) to 6 (strongly agree), which were averaged into subscale composites.

We also collected data on Critical Reflection: Egalitarianism but chose not to include this subscale in analyses because recent research demonstrates unexpected negative or weak correlations with other CC indices (Diemer & Rapa, 2016; Diemer et al., 2017, 2022). We also did not include the Critical Action subscale because we suspected that factors outside of the intervention might have shaped opportunities for sociopolitical participation.

School Belonging (Preintervention/Postintervention and Daily Diary Surveys)

Belonging Preintervention and Postintervention

To understand students’ sense of belonging in school, participants responded to a measure that was adapted from the Asher and Weeks’s (2014) college belongingness measure. The measure contained six items, such as “I feel like I belong at [school name]” and “I feel welcome at [school name],” on a scale of 1 (strongly disagree) to 5 (strongly agree), which were averaged into a composite belonging score (α_pre = .88, α_post = .89).

Daily Belonging

To understand students’ belonging in school on a day-to-day basis, in the daily diary component, participants responded to three items adapted from the longer scale that was included in the preintervention and postintervention. Items were slightly revised to reference feelings that day (e.g., “Today, I felt like I belonged at [school name]”), and students responded on a scale of 1 (strongly disagree) to 5 (strongly agree). Responses to the three items were averaged into a daily belonging score.

Stress (Preintervention/Postintervention and Daily Diary Surveys)

Pre/Poststress

Students responded to the Perceived Stress Scale (Cohen et al., 1983) on both preintervention and postintervention surveys. This scale has demonstrated good reliability and content validity among adolescent samples (Kechter et al., 2019; Lee, 2012). Students responded to the 10-item measure, which included statements about their frequency of stress over the course of the past month on a scale of 0 (never) to 4 (very often). For example, “In the last month, how often have you been upset because of something that happened unexpectedly?” At each timepoint, items were summed to create a perceived stress composite, which could range from 0–40 (α_pre = .88, α_post = .90).

Daily Stress

To capture the daily experience of stress, daily diary participants responded to three items adapted from the Perceived Stress Scale (Cohen et al., 1983). Items asked about the participant’s frequency of stress over the past 24 hours on a scale of 0 (never) to 4 (very often), including, “In the last 24 hours, how often have you felt that you were unable to control the important things in your life?” Items were summed to create a daily perceived stress score with a range of 0–12.

Data Analytic Plan

First, we checked for outliers and examined descriptive statistics and correlations among variables. To address our first question, we conducted a latent class analysis (LCA, using Mplus version 8.6) to estimate distinct underlying patterns of engagement in the Stamped Intervention using the z-transformed mean scores of the three intervention components (i.e., book reading, attendance, and discussion participation) as indicators. LCA is a person-centered approach (Bergman & Trost, 2006; Oberski, 2016) that enabled us to identify common patterns of student participation across the three components. Based on our conversations with school partners, we anticipated between-student differences in overall levels of intervention engagement and within-student differences in engagement with each component (e.g., some students may not have read the book but still attended advisory and engaged in discussions).

To address our second question, we estimated a series of multinomial logistic regressions to explore predictors of latent class engagement group membership, controlling for students’ age, gender, and racial/ethnic identity. Pre-intervention reports of school belonging, quality of student – staff relationships, perceived stress, CR, and CM were examined as predictors of intervention engagement.

To address our third question, we explored potential differences in preintervention to postintervention change in school belonging, student – staff relationships, stress, CR, and CM as a function of intervention engagement and racial/ethnic identity using repeated measures of ANOVA. Data for questions 2 and 3 were analyzed using SPSS (version 27).

To address our fourth question, multilevel models (MLM, see supplemental materials for equations) were estimated to explore potential differences in students’ daily sense of belonging and stress as a function of intervention day, students’ racial/ethnic identity, and intervention engagement. To compare belonging and stress between days when the intervention occurred versus did not, we created a dichotomous “intervention day” variable such that the four days in which the Stamped Intervention occurred during data collection were coded as “1” and the other 11 days were coded as “0” (see supplemental materials for calendar). Students’ intervention engagement was derived from their latent class membership. Class assignment was a categorical variable that was further dummy coded. Within our multilevel model, daily belonging/stress was associated with Level 1, whereas variables describing students (e.g., engagement grouping, race) were associated with Level 2. MLM analyses were conducted using SAS^® Studio software. Variances were calculated using the Snijders and Bosker (2011) method.

Multilevel models with belonging as the dependent variable were labeled with “b” (e.g., Model 1b) whereas models with stress as the dependent variable were labeled with “s” (e.g., Model 1s). First, the fully unconditional models (Model 1b/s) were used to calculate the intraclass correlation coefficient (ICC) for daily belonging/stress. Next, Model 2b/s explored whether daily belonging differed on days when the intervention did and did not occur by adding a dichotomous Level 1 predictor of intervention day. Model 3b/s explored whether racial/ethnic background moderated the relationship between belonging/stress and intervention day by adding a dichotomous Level 2 predictor of race and the day*race interaction. Finally, Models 4b/s and 5b/s explored whether students’ level of intervention engagement moderated the relationship between daily belonging/stress and intervention day. These models involved the same equation but included different referent groups.

Results

Table 2 includes descriptive statistics and Pearson correlation matrix for the total sample. Descriptive statistics showed that all variables had a level of skew and kurtosis within the range reported in Blanca et al. (2017) that demonstrated the robustness of F-tests to Type 1 errors.

Table 2

Study 1 Descriptive Statistics and Correlation Matrix for Preintervention and Postintervention Surveys

	M (SD)	1	2	3	4	5	6	7	8	9	10	11	12
1. Read^b	3.31 (1.41)	–
2. Attend^b	3.93 (1.21)	.43***	–
3. Discuss^b	3.12 (1.27)	.59***	.57***	–
4. Belong^a	3.72 (0.76)	.13*	.17*	.21***	–
5. Belong^b	3.66 (0.78)	.18**	.18**	.24***	.79***	–
6. SR^a	1.91 (1.08)	.16*	.24***	.30***	.24***	.23***	–
7. SR^b	2.22 (1.01)	.24***	.28***	.35***	.26***	.32***	.64***	–
8. Stress^a	20.37 (7.97)	−.05	−.13	−.05	−.27***	−.31***	−.07	−.09	–
9. Stress^b	21.05 (8.40)	−.13	−.17*	−.12	−.27***	−.35***	−.10	−.13*	.66***	–
10. CR^a	4.48 (1.43)	.00	.02	.04	−.13	−.25***	.13	.05	.26***	.25***	–
11. CR^b	4.77 (1.23)	.13*	.07	.10	−.13*	−.19**	.19**	.09	.25***	.15*	.70***	–
12. CM^a	5.05 (0.87)	.21**	.13	.20**	.03	.04	.02	.05	.23***	.17*	.38***	.46***	–
13. CM^b	5.08 (0.83)	.24***	.18**	.28***	.14*	.11	.09	.17*	.16*	.11	.34***	.46***	.71***

Note. Scale 1–5 for variables 1–5; scale 0–3 for variables 6–7; scale 0–40 for variables 8–9; scale 1–6 for variables 10–13. SR = student—staff relationships; CR = critical reflection: perceived inequality; CM = critical motivation.

Preintervention, ^bpostintervention.

p < .05, **p < .01, ***p < .001.

Patterns of Stamped Intervention Engagement

To explore underlying patterns in students’ intervention engagement, we used latent class analysis. To determine the best-fitting model, Bayesian Information Criterion (BIC) values and Lo-Mendell-Rubin (LMR) and bootstrapped likelihood ratio tests (BLRT) were used given that these metrics have performed well in simulations (Nylund et al., 2007). Lower BIC values indicate a better-fitting model, and significant (p < .05) LMR and BLRT tests indicate improvement over a solution with k-1 classes (Asparouhov & Muthén, 2012). Theoretical considerations, class size and interpretability, and parsimony also factored into model selection (Collins & Lanza, 2010). We examined one to four classes (Table 3). The BIC values indicated that a three-class model fit best. The BLRT test with 10,000 bootstrap draws indicated that a three-class solution was preferred over a two-class solution (BLRT p = .0051), although the adjusted LMR was nonsignificant (adjusted LMR = 61.18, p = 0.44), suggesting a two-class model was preferred. Examination of the two- and three-class models indicated that a two-class model had one very large class (74.4% of the sample) whereas, in the three-class model, this class was separated into two conceptually distinct and theoretically meaningful classes. Based on statistical and theoretical rationale, we selected the three-class solution.

Table 3

Latent Class Analysis Model Summary Statistics

Classes	N	Parameters	LL	Entropy	AIC	BIC	CAIC	ssBIC
1	227	9	−870.071	0.00	1758.14	1788.97	1797.97	1760.44
2	227	19	−809.964	18.88	1657.93	1723.00	1742.00	1662.79
3	227	29	−778.809	32.67	1615.62	1714.94	1743.94	1623.03
4	227	39	−931.290	66.71	1940.58	2074.15	2113.15	1950.55

Note. LL = log-likelihood; AIC = Akaike’s Information Criterion; BIC = Bayesian Information Criterion; CAIC = Consistent AIC; ssBIC = Sample Size Adjusted BIC. Smaller information criterion values indicate a better-fitting model (bolded).

The first class (59.9%; n = 136) was labeled the “Highly Engaged” group because participants in this class reported above-average rates of participation across all three components (see Figure 1 and Table 4). The second class (21.1%; n = 48) was labeled the “Attender” group, because students in this class reported above-average rates of attending Stamped-related advisory sessions but reported below-average levels of reading and discussing the book. Finally, the third class (18.9%; n = 43) was labeled the “Disengaged” group because students in this class reported below-average levels of participation across the three components. The three engagement groups did not differ by gender, race/ethnicity, age, or grade (see supplemental materials for full analyses).

Figure 1.

Standardized means for Stamped engagement groups

Table 4

Raw Means and Standard Deviations by Stamped Engagement Groups

Variables	Highly Engagedn = 136	Attendern = 48	Disengagedn = 43
Stamped-Related
Read^b	4.27 (0.82)	2.04 (0.71)	1.70 (0.67)
Attend^b	4.11 (0.97)	4.81 (0.39)	2.40 (1.09)
Discuss^b	3.66 (1.04)	3.00 (1.20)	1.53 (0.51)
School-Related
Belong^a	3.76 (0.69)	3.65 (0.93)	3.67 (0.74)
Belong^b	3.74 (0.75)	3.54 (0.84)	3.53 (0.79)
SR^a	2.00 (1.09)	2.02 (0.99)	1.49 (1.08)
SR^b	2.37 (0.94)	2.31 (0.93)	1.63 (1.13)
Stress^a	20.49 (7.66)	20.68 (8.23)	19.64 (8.76)
Stress^b	20.70 (7.68)	21.42 (8.99)	21.74 (9.93)
Critical Consciousness
CR^a	4.45 (1.51)	4.48 (1.32)	4.58 (1.34)
CR^b	4.91 (1.20)	4.57 (0.57)	4.54 (1.32)
CM^a	5.16 (0.81)	4.88 (0.84)	4.88 (1.02)
CM^b	5.24 (0.74)	4.91 (0.77)	4.77 (1.02)

Note. SR = student—staff relationships; CR = critical reflection: perceived inequality; CM = critical motivation.

Preintervention, ^bpostintervention.

Predictors of Stamped Intervention Engagement

To examine predictors of Stamped Intervention engagement, we estimated multinomial logistic regressions predicting engagement-class membership from preintervention measures of school belonging, student–staff relationships, stress, CR, and CM. Age, race, gender, and grade were not included since our preliminary analyses revealed that engagement class membership did not differ by these demographics (see supplemental materials). The first regression explored differences between the Highly Engaged group and the two other groups, whereas the second regression explored differences between the Disengaged and Attenders groups (see Table 5). Preintervention quality of student – staff relationships significantly predicted membership in the Highly Engaged and Attenders classes compared to the Disengaged class, such that students with lower-quality staff relationships were more likely to not participate in the intervention. Additionally, CM predicted membership in the Highly Engaged class compared to the Attenders and Disengaged classes, such that students who were more critically motivated at preintervention were more likely to be highly engaged in the intervention.

Table 5

Predictors of Group Membership: Highly Engaged Referent Group

	B	SE	Wald	df	p	Exp(b)
Highly Engaged vs. Attender
Belong	−0.17	0.24	0.46	1	.499	0.85
SR	0.05	0.17	0.09	1	.764	1.05
Stress	0.01	0.02	0.07	1	.798	1.01
CR	0.09	0.14	0.43	1	.510	1.10
CM	−0.56	0.23	5.85	1	.016*	0.57
Highly Engaged vs. Disengaged
Belong	0.10	0.28	0.00	1	.971	1.01
SR	−0.48	0.18	6.87	1	.009**	0.62
Stress	−0.02	0.03	0.31	1	.581	0.99
CR	0.28	0.16	3.01	1	.083	1.32
CM	−0.60	0.25	5.80	1	.016*	0.55
Disengaged vs. Attender
Belong	−0.18	0.31	0.31	1	.576	0.84
SR	0.53	0.21	6.27	1	.012*	1.70
Stress	0.02	0.03	0.47	1	.493	1.02
CR	−0.19	0.19	1.03	1	.310	0.83
CM	0.04	0.27	0.02	1	.889	1.04

Note. SR = student—staff relationships; CR = critical reflection: perceived inequality; CM = critical motivation. Exp(b) can be interpreted as the odds ratio.

p < .05, **p < .01.

Outcomes of Stamped Intervention Engagement

To investigate potential differences in intervention outcomes between Stamped engagement classes, we conducted a series of 2 (time: preintervention, postintervention) × 3 (class: Highly Engaged, Attender, Disengaged) × 2 (race: majoritized, minoritized) repeated measures ANOVAs for the following outcomes: school belonging, quality of student–staff relationships, perceived stress, CR, and CM. Doing so allowed us to examine changes across the intervention period and whether changes differed as a function of patterns of engagement and racial/ethnic identity. As stated in Research Question 3, we were most interested in interpreting within-subject differences and the interaction between time and engagement class (see the supplemental materials for other results). All post-hoc analyses were conducted using the Bonferroni correction. Missing data were handled using listwise deletion.

Belonging

Results for belonging revealed a significant main effect of time (F(1, 218) = 4.72, p = .031, η_p² = .02) such that belonging decreased slightly from preintervention (M = 3.73, SD = 0.76) to postintervention (M = 3.67, SD = 0.78, N = 224). The time × class, time × race, and three-way interactions were all nonsignificant. The total model accounted for approximately 3% of the variance in belonging.

Student–Staff Relationship

Results for quality of student–staff relationships revealed a significant main effect of time (F(1, 207) = 18.30, p < .001, η_p² = .08) such that relationships increased in strength from preintervention (M = 1.92, SD = 1.08) to postintervention (M = 2.23, SD = 1.01, N = 213). The time × class, time × race, and three-way interactions were all nonsignificant. The total model accounted for 9% of the variance in student–staff relationships.

Stress

Results for perceived stress revealed a significant main effect of time (F(1, 209) = 6.05, p = .015, η_p² = .03), which was qualified by a significant time × race interaction (F(1, 209) = 7.81, p = .006, η_p² = .04). The time × class and three-way interactions were nonsignificant. The total model accounted for 11% of the variance in perceived stress.

Post-hoc analyses of the time × race interaction revealed that racially minoritized students’ (n = 67) stress increased from intervention to postintervention (M_pre = 20.01, SD_pre = 8.31, M_post = 22.81, SD_post = 8.92, p = .001). However, majoritized students’ (n = 148) stress did not differ between timepoints (M_pre = 20.48, SD_pre = 7.81, M_post = 20.13, SD_post = 8.15, p = .782). Racially majoritized and minoritized students’ stress did not differ from each other at either timepoint.

Critical Reflection

Results for CR (perceived inequality) revealed a significant main effect of time (F(1, 205) = 4.71, p = .031, η_p² = .02) and significant time × class (F(2, 205) = 5.90, p = .003, η_p² = .05) and three-way interactions (F(2, 205) = 4.03, p = .019, η_p² = .04). The time × race interaction was not significant. The total model accounted for 11% of the variance in CR.

We report results of the time × class × race interaction because it qualifies main and lower-order interaction effects. Perceived inequality increased from intervention to postintervention for both racially majoritized and minoritized students in the Highly Engaged class (ps < .01, Figure 2, Table 6); however, CR did not differ between engagement classes at either timepoint. Racially majoritized students perceived greater inequality than their minoritized counterparts in the Highly Engaged group at both timepoints (ps < .05) and in the Disengaged group at postintervention (p = .021).

Figure 2.

Changes in critical reflection over time by Stamped engagement group and racial identity

Table 6

Descriptive Statistics for Critical Reflection Over Time by Engagement Class and Racial Identity

	Highly Engaged		Attenders		Disengaged
	Pre	Post	Pre	Post	Pre	Post
Majoritized	4.72 (1.30)	5.03 (1.06)	4.60 (1.16)	4.59 (1.01)	4.72 (1.10)	5.00 (1.06)
Minoritized	3.74 (1.78)	4.51 (1.55)	4.30 (1.55)	4.46 (1.42)	4.45 (1.47)	4.05 (1.44)

Note. Means are presented with standard deviations in parentheses.

Critical Motivation

Results for CM revealed a nonsignificant main effect of time and nonsignificant time × class, time × race, and three-way interactions, accounting for 3% of the variance in CM.

Daily Experiences

Daily Belonging

Model 1b revealed that, on average, students reported a moderate sense of belonging in school (see Table 7). The ICC for belonging indicated that 28% of the variance was within-person and 72% was between-person. Model 2b revealed a nonsignificant main effect of intervention day (γ₁₀), indicating that, on average, students’ sense of belonging did not differ on intervention days compared to nonintervention days. This association did not vary significantly across participants (τ₁₁). Model 2b accounted for 0% of the within- and between-person variances in belonging.

Table 7

Multilevel Model Parameters for Daily School Belonging

Effect	Parameter	Model 1b	Model 2b	Model 3b	Model 4b	Model 5b
		Fixed Effects
Outcome, β₀
Intercept	γ₀₀	3.40 (0.10)***	3.41 (0.10)***	3.52 (0.12)***	3.40 (0.14)***	3.32 (0.27)***
Race	γ₀₁			−0.37 (0.23)
Attender	γ₀₁				0.08 (0.25)	0.17 (0.34)
Third eng group	γ₀₂				−0.09 (0.31)	0.09 (0.31)
Linear Relationship, β₁
Intervention day	γ₁₀		−0.04 (0.05)	−0.02 (0.05)	−0.02 (0.06)	−0.11 (0.12)
Day * race	γ₁₁			−0.07 (0.10)
Day * attender	γ₁₁				−0.01 (0.11)	0.08 (0.15)
Day * third eng group	γ₁₂				−0.09 (0.14)	0.09 (0.14)
		Random Effects
Variance components
Within-person fluctuation	σ₂	0.27 (0.01)***	0.27 (0.01)***	0.27 (0.01)***	0.27 (0.01)***	0.27 (0.01)***
Between-person fluctuation	τ₀₀	0.69 (0.13)***	0.68 (0.13)***	0.66 (0.12)***	0.70 (0.13)***	0.70 (0.13)***
Day-intercept covariance	τ₁₀		0.02 (0.04)	0.01 (0.04)	0.02 (0.04)	0.02 (0.04)
Day-belonging slope	τ₁₁		0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)
		Effect Size
R² within		28%	0%	2%	0%	0%
R² between		72%	0%	3%	0%	0%

Note. Standard errors are in parentheses. Belonging composite scores ranged from 1–5. The Highly Engaged and Disengaged groups were the referent groups for Model 4b and 5b, respectively. Therefore, the “Third Eng Group” refers to the difference between the Highly Engaged and Disengaged groups.

***

p < .001

Model 3b revealed nonsignificant main effects of intervention day (γ₁₀) and race (γ₀₁) as well as a nonsignificant day × race interaction (γ₁₁), indicating that students’ sense of belonging did not differ between racially majoritized and minoritized students on days when the intervention did and did not occur, which did not differ between participants (τ₁₁). Model 3b accounted for 2% of the within-person variance in belonging and 3% of the between-person variance in belonging, which was not a significant improvement from Model 2b.

Given that race was not a significant predictor in Model 3b, it was dropped from Models 4b and 5b. The Highly Engaged group (59.7% of the subsample) was the referent group for Models 4b/s, whereas the Disengaged group (14.9% of the subsample) was the referent group for Models 5b/s. Model 4b revealed nonsignificant main effects of intervention day (γ₁₀), Attenders group (γ₀₂), and Disengaged group (γ₀₁), as well as nonsignificant Attenders × day (γ₁₁) and Disengaged × day interactions (γ₁₂). Similarly, Model 5b revealed nonsignificant main effects of intervention day (γ₁₀), Attenders group (γ₀₁), and Highly Engaged group (γ₀₂), as well as nonsignificant Attenders × day (γ₁₁) and Highly Engaged × day interactions (γ₁₂). Models 4b and 5b accounted for 0% of the within- and between-person variances in belonging. Overall, Models 4b and 5b indicated that students’ level of engagement in the Stamped Intervention did not impact their sense of belonging throughout the intervention period.

Daily Stress

Model 1s showed that, on average, diary participants reported a moderate level of stress (γ₀₀; Table 8). The ICC indicated that 36% of the variance in daily stress was within-person and 64% was between-person.

Table 8

Multilevel Model Parameters for Daily Perceived Stress

Effect	Parameter	Model 1s	Model 2s	Model 3s	Model 4s	Model 5s
		Fixed Effects
Outcome, β₀
Intercept	γ₀₀	6.45 (0.27)***	6.39 (0.28)***	6.16 (0.33)***	6.18 (0.36)***	6.66 (0.73)***
Race	γ₀₁			0.79 (0.61)
Attender	γ₀₁				0.56 (0.67)	0.07 (0.92)
Third eng group	γ₀₂				0.48 (0.82)	−0.48 (0.82)
Linear Relationship, β₁
Intervention Day	γ₁₀		0.24 (0.14)	0.26 (0.16)	0.24 (0.18)	0.49 (0.38)
Day * race	γ₁₁			−0.06 (0.32)
Day * attender	γ₁₁				−0.14 (0.33)	−0.39 (0.47)
Day * third eng group	γ₁₂				0.25 (0.42)	−0.25 (0.42)
		Random Effects
Variance Components
Within-person fluctuation	σ₂	2.60 (0.14)***	2.59 (0.14)***	2.60 (0.14)***	2.60 (0.14)***	2.60 (0.14)***
Between-person fluctuation	τ₀₀	4.59 (0.85)***	4.78 (0.90)***	4.71 (0.90)***	4.87 (0.93)***	4.87 (0.93)***
Day-intercept covariance	τ₁₀		−0.37 (0.30)	−0.36 (0.31)	−0.38 (0.31)	−0.38 (0.31)
Day-belonging slope	τ₁₁		0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)
		Effect Size
R² within		36%	0%	1%	0%	0%
R² between		64%	0%	1%	0%	0%

Note. Standard errors are in parentheses. Daily stress scores ranged from 0–12. R² was calculated using Snijders & Bosker’s (2011) recommended method. Highly Engaged and Disengaged groups were the referent groups for Model 4s and 5s, respectively. Therefore, the “Third Eng Group” refers to the difference between the Highly Engaged and Disengaged groups.

***

p < .001.

Next, Model 2s revealed a nonsignificant main effect of intervention day (γ₁₀), indicating that students’ stress did not significantly differ between days when the intervention did and did not occur, which did not differ between participants (τ₁₁). Model 2s accounted for 0% of the within- and between-person variance in daily stress.

Model 3s revealed nonsignificant main effects of intervention day (γ₁₀) and race (γ₀₁), as well as a nonsignificant day × race interaction (γ₁₁), indicating that students’ stress did not differ as a function of race on days when the intervention did and did not occur, which did not differ between participants (τ₁₁). This model accounted for 1% of the within-person variance and 1% of the between-person variance in daily stress.

Model 4s revealed nonsignificant main effects of intervention day (γ₁₀), Attenders group (γ₀₁), and Disengaged group (γ₀₂) as well as nonsignificant Attenders × day (γ₁₁) and Disengaged × day interactions (γ₁₂). Similarly, Model 5s revealed nonsignificant main effects of intervention day (γ₁₀), Attenders group (γ₀₁), and Highly Engaged group (γ₀₂), as well as nonsignificant Attenders × day (γ₁₁) and Highly Engaged × day interactions (γ₁₂). Models 4s and 5s accounted for 0% of the within- and between-person variances in stress. Together, Models 4s and 5s show that students’ level of engagement in the intervention did not impact their perceived stress throughout the intervention.

Discussion

This study offers a transferable approach to evaluate longer-term, school-based antiracist interventions that expand traditional pretest/posttest approaches by combining schoolwide surveys with smaller samples of daily diaries to consider how key constructs related to antiracism, psychological well-being, and school connectedness function as predictors and outcomes of student intervention engagement. We also examined how outcomes of the intervention might differ by students’ racial/ethnic identity. Notably, our findings are limited by high attrition rates and small sample sizes at a single school, but they do highlight the promise of using a research approach that considers intervention engagement patterns and student experiences during and after the intervention period.

Patterns of Stamped Engagement

To address our first research question, we explored underlying patterns of students’ intervention engagement. To our knowledge, our study is the first to consider student engagement patterns in evaluating antiracist interventions. Doing so is important because, especially in the context of schoolwide implementation, it is unreasonable to assume that all students engage in all aspects of the intervention equally. Understanding patterns of engagement enables researchers and educators to identify the level of exposure and which components of the intervention are related to student outcomes. In our sample, most participants were either highly engaged or completely disengaged with all intervention components, whereas a smaller group participated in one component but not others (i.e., attended without reading or discussing the book). Thus, exploring patterns of engagement allowed us to capture a nuanced understanding of adolescents’ participation as opposed to including participation components separately in our analyses. Future research–practice partnerships should continue to explore patterns of engagement when evaluating antiracist interventions. Our findings also have implications for practice: we document that some students may be exposed to the curriculum through attendance but not fully engaged. Teachers and staff implementing antiracist curricula may wish to target these “attenders” as specific efforts may help move them from ambiently experiencing the intervention to fully participating.

Future interventions should consider ways to increase engagement in the intervention itself and in efforts to evaluate those interventions. In particular, future research–practice partnerships might take a youth participatory action research (yPAR) approach to democratize research and ground research processes in students’ lived experiences and local knowledge (Ozer, 2016). Formative feedback loops throughout the intervention period could also inform ways to modify the intervention and research design as it progresses. These modifications may help to maintain engagement or reengage students and teachers in antiracist curricula.

Predictors of Stamped Engagement

To address our second research question, we examined preintervention belonging, quality of student–staff relationships, stress, CR, and CM as predictors of student engagement in the intervention. Although studies involving antiracist intervention evaluation have included pretests (Aldana et al., 2012; Burgess et al., 2021; Hughes et al., 2007; Nelsen, 2021), our study extends this work to explore how preintervention factors predict students’ engagement with the intervention. Prior research has not explicitly examined engagement: our findings suggest the benefits of directly exploring engagement rather than assuming that all adolescents are willing or able to engage in antiracist work in the same ways, as we reveal differing outcomes depending on engagement. Our findings highlight the role that high-quality student–staff relationships play in supporting adolescent engagement in antiracist curricula and that students who are more critically motivated are more likely to be highly engaged. Notably, among students who participated, stress and belonging were not predictors of engagement in the virtual context in which the intervention occurred. Schools interested in implementing antiracist curricula might first consider ways to build rapport between students and staff and enhance adolescents’ beliefs about the importance of and responsibility to change social inequality before beginning the challenging work of engaging in antiracist interventions.

Longer-Term Outcomes of Stamped Intervention Engagement

To address our third research question, we examined whether changes in students’ belonging, quality of student–staff relationships, stress, CR, and CM occurred as a function of their level of Stamped engagement and racial/ethnic identity. To our knowledge, our study is the first to consider socioemotional outcomes of antiracist interventions, in addition to components of CC, which are important to consider as critics of antiracist education argue that learning about racism may be psychologically harmful to White students (Sawchuk, 2021). We found that, over time, participants reported reduced belonging in school, more stress (especially for racially minoritized students), and higher-quality relationships with staff; however, these changes did not vary as a function of intervention engagement. This finding indicates that factors outside of the intervention likely played a role in shaping adolescents’ school connectedness and stress during the dual pandemics of COVID-19 and racial injustice.

Concerningly, racially minoritized students’ perceived stress increased, whereas racially majoritized students’ stress remained stable between fall 2020 and spring 2021, although stress was unrelated to intervention engagement. While the current study cannot explain with certainty why racially minoritized students reported increased stress postintervention, it has long been documented that experiencing racism is associated with adverse psychological outcomes, including increased stress (Pieterse & Powell, 2016). Disproportionate mortality rates and targeted racial violence during the dual pandemics increased experiences of racial trauma for Asian, Black, and Latino/a/e/x communities (Liu & Modir, 2020), which may be reflected in the increased stress these students reported. French et al. (2019) posit that developing CC can help communities of color heal from racial trauma, which supports implementing antiracist and critical pedagogy in schools.

Critical Reflection

Aligned with prior antiracist intervention work, which found that adolescents who self-selected to participate in out-of-school antiracist programming demonstrated increased ethnic-racial identity and racism awareness (Aldana et al., 2012) and antiracist attitudes (Burgess et al., 2021), we found that changes in students’ CR over time differed based on their level of intervention engagement and racial/ethnic identity. Racially majoritized and minoritized students who were highly engaged in the intervention demonstrated growth in their perceptions of social inequality, whereas perceptions of inequality remained stable for Attenders and Disengaged students. Our finding suggests that adolescents may need opportunities to discuss and debate ideas to change their awareness rather than passive exposure to antiracist curricula. This finding also indicates that perhaps greater engagement with the intervention might have been most effective for students with moderate CR, given that racially minoritized students in the Highly Engaged group reported the lowest perceptions of social inequality at pretest and, therefore, had the most statistical room for growth.

Our findings regarding perceived inequality were particularly limited by small cell sizes for racially majoritized (n = 23) and minoritized (n = 14) students in the Disengaged group, which limited our power to detect changes over time. Given that students could choose to attend the advisory periods and that both the intervention and the pretest survey occurred during the advisory periods, we may have differentially missed the participation of disengaged students in the school population. Although racially majoritized students in Highly Engaged and Disengaged groups reported similar rates of perceived inequality at both timepoints, Highly Engaged students’ growth was statistically significant, whereas the Disengaged students’ growth was not. In fact, overall rates of perceived inequality did not differ between any engagement group at either timepoint, suggesting that, despite differing levels of intervention engagement, students’ perceptions of social inequality were similar. Future research–practice partnerships should consider how exposure to news and social media might influence adolescent CR outside of the antiracist intervention. In addition, future work should attempt to understand why some adolescents disengage from antiracist curricula.

Critical Reflection by Racial Identity

We found that, when significant group differences occurred, racially majoritized (White) students reported greater CR than their racially minoritized counterparts, contributing to mixed evidence on the extent to which marginalized or privileged youth report higher CC on quantitative measures. For example, Diemer et al. (2019) found that White youth reported greater CR than youth of color, whereas Bañales et al. (2019) found no differences in CR between these groups. Of particular interest for our research purposes, in the context of antiracist programming, Burgess et al. (2021) also found that White adolescents reported higher CC than participants of color. They argued that participants of color in their sample may have limited exposure to positive messages about race and, therefore, limited opportunities to develop CC (Alemán & Gaytán, 2017), given their context, a predominantly White college town. Our partner school is also located in a majority White, politically liberal community near a university. School administrators encouraged teachers to engage in open dialogue with students about social inequality years before the Stamped Intervention began. Encouraging racially privileged youth to investigate systems of racial oppression and privilege might support their early CR development. Indeed, recent work with more privileged White young adults demonstrates that individuals’ heightened CR on privilege and oppression, White identity exploration, and egalitarian beliefs are associated with endorsing commitments to social justice (i.e., allyship; Frisby, 2022). Future work should (a) consider ways in which antiracist curricula shape White adolescents’ recognition of racial privilege and orientations towards allyship and (b) examine CR developmental trajectories for adolescents of color who experience antiracist curricula in predominantly White settings.

Critical Motivation

We did not find differences between engagement groups’ reported CM at either timepoint nor significant changes in CM over time. It may be that feeling responsible and committed to making social change takes longer than 3 months postintervention to develop. Prior work examining underlying patterns of early adolescents’ CC development demonstrates complex profiles, especially regarding perceptions of internal and external political efficacy (Godfrey et al., 2019), suggesting that CM may take time to mature. Future research might consider assessing both political efficacy and CM at more follow-up postassessments to allow for the possibility that effects might emerge over a longer period and only after students have lived experiences to create change.

Furthermore, our partner school designed the Stamped Intervention with the goals of enhancing CR, not necessarily CM. Although scholars posit that CM serves as an essential bridge between CR and critical action (Diemer et al., 2017; Watts et al., 2011), our preliminary findings indicate that adolescents’ CM may not develop from advancing CR alone. Future interventions might benefit from designing instructional materials to purposefully target CM by creating opportunities for dialogue about why it is important for youth to correct social inequality and opportunities for adolescents to learn about successful youth initiatives in areas of social oppression that speak to them (e.g., LGBTQ rights, disability policy).

Daily Stress and Belonging Experiences

To address our final research question, we explored, with a subsample of students, whether daily fluctuations in belonging and stress occurred as a function of students’ intervention engagement and their racial/ethnic identity. We found evidence for significant within-person variability (i.e., daily fluctuations) in students’ school belonging and perceived stress. This aligns with prior work using college samples, which documents daily fluctuations in belonging (accounting for 21.5 to 39% of within-person variance in belonging; Gillen-O’Neel, 2021; Harris et al., 2022; Walton & Cohen, 2007). Notably, our study is the first to provide evidence for daily fluctuations in school belonging in an adolescent sample. Additionally, our finding that adolescents experienced significant daily fluctuations in perceived stress expands prior work with similar findings (Debeuf et al., 2018; Hsu & Raposa, 2021) in the sociohistorical context of dual pandemics.

We also found that students’ sense of belonging and perceived stress did not differ based on days the intervention sessions occurred, level of intervention engagement, or racial/ethnic identity, suggesting that exposure to antiracist curricula did not influence this subsample’s experiences of belonging or stress. This finding provides preliminary evidence that, counter to the arguments made by communities resistant to teaching about race in school (Lewis, 2022; Sawchuk, 2021), our subset of students who participated in the daily diary study did not experience negative outcomes related to stress or belonging on days they participated in the intervention. To our knowledge, our study is the first to analyze concurrent outcomes of antiracist interventions, and our findings challenge the assumption that intervention outcomes are best measured after programming concludes. We encourage future research–practice partnerships to use daily diary designs when evaluating antiracist interventions because doing so provides insight into students’ experiences throughout the intervention and may identify when and for whom intervention exposure impacts outcomes. Importantly, future work should focus on recruiting a daily diary subsample that is representative of the larger school population.

Limitations

Findings from this evaluation highlight the benefits of employing multiple methods to evaluate antiracist interventions. However, our findings are not generalizable to other student populations or even to the full population of our partner school because of the limited participation rate and high attrition in the context of online learning and because of the unique school community and sociohistorical context. The school implemented the Stamped Intervention amid an online learning environment, which may have shaped students’ sense of belonging and staff relationships, as well as their willingness and ability to participate in the research. Teachers had some autonomy over Stamped-related lessons; as such, discussion participation and topics likely differed between groups. The broader sociohistorical context of dual pandemics in fall 2020 also likely influenced participation and outcomes. Although our findings cannot be generalized, educators and researchers can still consider ways in which our evaluation methods might transfer to their partnerships.

There were several limitations that we encourage future research–practice partnerships to consider in their attempts to evaluate antiracist interventions. Most importantly, our understanding of racially minoritized students was limited because we combined all students of color into a single group for analysis purposes. Future partnerships should take steps to recruit and retain larger samples of racially minoritized participants. Second, our measurement of intervention engagement was limited to three items that asked students to retrospectively recall their participation across the intervention, three months after it concluded. Thus, it is possible that adolescents’ engagement reporting was somewhat inaccurate. Adding a measure of daily intervention engagement to daily diary surveys, as well as collecting instructor reports of intervention engagement, could provide more valid and reliable data. Third, we did not account for how the intervention was implemented in each advisory group, which is a common limitation in school-based antiracist intervention evaluation (Weems et al., 2022). Given our partner school’s encouragement for advisory instructors to modify Stamped-related lessons to fit their group’s needs, it is likely that students were exposed to different materials and conversations. Future research should account for fidelity to intervention protocols by observing advisory sessions and having educators complete self-evaluations/reflections throughout the intervention period (Weems et al., 2022). Finally, we may have encountered ceiling effects for some of the CC-related subscales, making it difficult to identify possible growth that may have occurred over time. Quantitative measures might also fail to capture change in CC over time if adolescents initially overestimate their CC. Future research should incorporate retrospective pretest/posttest designs (Geldhof et al., 2018) or mixed methods that allow participants to expand on ways that their intervention engagement has impacted their critical reflection, motivation, and action.

Conclusions

Our evaluation of a schoolwide antiracist intervention offers a novel approach that examines (a) underlying profiles of intervention engagement, (b) predictors and outcomes of engagement, and (c) concurrent student experiences throughout the intervention. Our findings suggest that, in this school context, among students who participated, the Stamped Intervention did not impact belonging, stress, quality of student–staff relationships, or critical motivation. Rather, high levels of intervention engagement enhanced students’ awareness of social inequality. Legislation that restricts critical discussions of and lessons about race/racism may be inappropriate for schools that are ready, willing, and able to engage in antiracist efforts. To inform evidence-based educational policy and practice, future research should continue to employ multimethod quantitative approaches such as those used in this study to examine predictors and outcomes of student engagement in antiracist curricula.

Supplemental Material

sj-docx-1-ero-10.1177_23328584231223476 – Supplemental material for A Novel Approach for Evaluating a Schoolwide Antiracist Curriculum Intervention

Supplemental material, sj-docx-1-ero-10.1177_23328584231223476 for A Novel Approach for Evaluating a Schoolwide Antiracist Curriculum Intervention by Jacqueline Cerda-Smith, Paula K.S. Yust, Molly S. Weeks, Steven R. Asher and Kelly Lynn Mulvey in AERA Open

Footnotes

Acknowledgements

We acknowledge our partner school’s initiative and commitment to instituting an antiracist intervention during the dual pandemics and thank them for their trust in partnering with us to evaluate it and explore student experiences over time.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a grant from the Education Research Service Projects Program of the American Education Research Association.

ORCID iDs

Jacqueline Cerda-Smith

Molly S. Weeks

Kelly Lynn Mulvey

Open Practices Statement

The syntax and output files that support the findings of this study are available at . The data are not publicly available due to their containing information that could compromise the privacy of research participants. Click the above link to learn how to request the data files.

Supplemental Material

Supplemental material for this article is available online.

Authors

JACQUELINE CERDA-SMITH is a doctoral candidate at North Carolina State University; email: jcerdas@ncsu.edu. Her research focuses on how schools support equity, adolescent well-being, and critical consciousness development.

PAULA K.S. YUST is an assistant professor of psychology at Dickinson College; email: yustp@dickinson.edu. She studies friendships from preschool to college with a focus on friendship quality and the beliefs people hold about friendships.

MOLLY S. WEEKS is the director of research for the Office of Undergraduate Education at Duke University; email: molly.weeks@duke.edu. She studies the factors that contribute to emotional well-being and academic adjustment during childhood, adolescence, and young adulthood, with a focus on the college years.

STEVEN R. ASHER is a professor of psychology and neuroscience at Duke University; email: asher@duke.edu. His research focuses on social development in childhood, adolescence, and the college years with a focus on the conceptualization and assessment of peer relationship competence and peer relationship outcomes, including friendship quality and loneliness.

KELLY LYNN MULVEY is an associate professor of psychology at North Carolina State University; email: klmulvey1@ncsu.edu. Her research focuses on social-cognitive development, in particular moral and social development.

References

Agarwal

Sen

(2022). Antiracist curriculum and digital platforms: Evidence from Black Lives Matter. Management Science, 68(4), 2932–2948. https://doi.org/10.1287/mnsc.2021.4281

Aldana

Rowley

S. J.

Checkoway

Richards-Schuster

(2012). Raising ethnic-racial consciousness: The relationship between intergroup dialogues and adolescents’ ethnic-racial identity and racism awareness. Equity & Excellence in Education, 45(1), 120–137. https://doi.org/10.1080/10665684.2012.641863

Alemán

S. M.

Gaytán

(2017). ‘It doesn’t speak to me’: Understanding student of color resistance to critical race pedagogy. International Journal of Qualitative Studies in Education, 30, 128–146. https://doi.org/10.1080/09518398.2016.1242801

Alvarez

(2021). Teaching with an anti-racist lens. National Education Association. https://www.nea.org/advocating-for-change/new-from-nea/teaching-anti-racist-lens

America

(2023). Educational Censorship. Retrieved April 12 from https://pen.org/issue/educational-censorship/

Asher

S. R.

Weeks

M. S.

(2014). Loneliness and belongingness in the college years. In Coplan

R. J.

Bowker

J. C.

(Eds.), The handbook of solitude (pp. 283–301). https://doi.org/10.1002/9781118427378.ch16

Asparouhov

Muthén

. (2012, May 22). Using Mplus TECH 11 and TECH 14 to test the number of latent classes. Mplus Web Notes: No. 14. https://www.statmodel.com/examples/webnotes/webnote14.pdf

Bañales

Aldana

Richards-Schuster

Flanagan

C. A.

Diemer

M. A.

Rowley

S. J.

(2019). Youth anti-racism action: Contributions of youth perceptions of school racial messages and critical consciousness. Journal of Community Psychology, 49(8), 3079–3100. https://doi.org/https://doi.org/10.1002/jcop.22266

Bergman

L. R.

Trost

(2006). The person-oriented versus the variable-oriented approach: Are they complementary, opposites, or exploring different worlds? Merrill-Palmer Quarterly, 52(3), 601–632. https://doi.org/10.1353/mpq.2006.0023

10.

Blanca

M. J.

Alarcón

Arnau

Bono

Bendayan

(2017, Nov). Non-normal data: Is ANOVA still a valid option? Psicothema, 29(4), 552–557. https://doi.org/10.7334/psicothema2016.383

11.

Burgess

Prescod

D. J.

Bryan

Chatters

(2021). Raising youth critical consciousness: Exploring critical race pedagogy as a framework for anti-racist programming. Journal of School Counseling, 19(34).

12.

Byrd

C. M.

(2017). The complexity of school racial climate: Reliability and validity of a new measure for secondary students. British Journal of Educational Psychology, 87(4), 700–721. https://doi.org/10.1111/bjep.12179

13.

Cabrera

N. L.

Milem

J. F.

Jaquette

Marx

R. W.

(2014). Missing the (student achievement) forest for all the (political) trees: Empiricism and the Mexican American studies controversy in Tucson. American Educational Research Journal, 51(6), 1084–1118. http://www.jstor.org.prox.lib.ncsu.edu/stable/24546712

14.

Cohen

Kamarck

Mermelstein

(1983). A global measure of perceived stress. Journal of Health and Social Behavior, 24(4), 385–396. https://doi.org/10.2307/2136404

15.

Collins

L. M.

Lanza

S. T

. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Wiley & Sons

16.

de los Ríos

C. V.

López

Morrell

. (2015). Toward a critical pedagogy of race: Ethnic studies and literacies of power in high school classrooms. Race and Social Problems, 7(1), 84–96. https://doi.org/10.1007/s12552-014-9142-1

17.

Debeuf

Verbeken

Van Beveren

M. L.

Michels

Braet

(2018). Stress and eating behavior: A daily diary study in youngsters. Frontiers in Psychology, 9, 2657. https://doi.org/10.3389/fpsyg.2018.02657

18.

Diemer

M. A.

Frisby

M. B.

Pinedo

Bardelli

Elliot

Harris

McAlister

Voight

A. M.

(2022). Development of the Short Critical Consciousness Scale (ShoCCS). Applied Developmental Science, 26(3), 409–425. https://doi.org/10.1080/10888691.2020.1834394

19.

Diemer

M. A.

Rapa

L. J.

(2016). Unraveling the complexity of critical consciousness, political efficacy, and political action among marginalized adolescents. Child Development, 87(1), 221–238. https://doi.org/10.1111/cdev.12446

20.

Diemer

M. A.

Rapa

L. J.

Park

C. J.

Perry

J. C.

(2017). Development and validation of the Critical Consciousness Scale. Youth & Society, 49(4), 461–483. https://doi.org/10.1177/0044118X14538289

21.

Diemer

M. A.

Voight

A. M.

Marchand

A. D.

Bañales

(2019). Political identification, political ideology, and critical social analysis of inequality among marginalized youth. Developmental Psychology, 55(3), 538–549. https://doi.org/10.1037/dev0000559

22.

Eccles

J. S.

Roeser

R. W.

(2013). Schools as developmental contexts during adolescence. In Lerner

R. M.

Easterbrooks

M. A.

Mistry

Weiner

I. B.

(Eds.), Handbook of psychology: Developmental psychology, Vol. 6, 2nd ed. (pp. 321–337). John Wiley & Sons, Inc.

23.

Freire

(1970). Pedagogy of the oppressed. Seabury Press.

24.

Freire

(1973). Education for critical consciousness. Continuum.

25.

French

B. H.

Lewis

J. A.

Mosley

D. V.

Adames

H. Y.

Chavez-Dueñas

N. Y.

Chen

G. A.

Neville

H. A.

(2019). Toward a psychological framework of radical healing in communities of color. The Counseling Psychologist, 48(1), 14–46. https://doi.org/10.1177/0011000019843506

26.

Frisby

(2022). Critical consciousness and the social justice engagement of White young adults [Thesis]. https://doi.org/10.7302/4601

27.

Geldhof

G. J.

Warner

D. A.

Finders

J. K.

Thogmartin

A. A.

Clark

Longway

K. A.

(2018). Revisiting the utility of retrospective pre-post designs: The need for mixed-method pilot data. Evaluation and Program Planning, 70, 83–89. https://doi.org/https://doi.org/10.1016/j.evalprogplan.2018.05.002

28.

Gillen-O’Neel

(2021). Sense of belonging and student engagement: A daily study of first- and continuing-generation college students. Research in Higher Education, 62(1), 45–71. https://doi.org/10.1007/s11162-019-09570-y

29.

Godfrey

E. B.

Burson

E. L.

Yanisch

T. M.

Hughes

Way

(2019). A bitter pill to swallow? Patterns of critical consciousness and socioemotional and academic well-being in early adolescence. Developmental Psychology, 55(3), 525–537. https://doi.org/10.1037/dev0000558

30.

Gray

D. L.

Hope

E. C.

Byrd

C. M.

(2020). Why Black adolescents are vulnerable at school and how schools can provide opportunities to belong to fix it. Policy Insights from the Behavioral and Brain Sciences, 7(1), 3–9. https://doi.org/10.1177/2372732219868744

31.

Harris

P. E.

Gordon

A. M.

Dover

T. L.

Small

P. A.

Collins

N. L.

Major

(2022). Sleep, emotions, and sense of belonging: A daily experience study. Affective Science, 3(2), 295–306. https://doi.org/10.1007/s42761-021-00088-0

32.

Hsu

Raposa

E. B.

(2021). Effects of stress on eating behaviours in adolescents: A daily diary investigation. Psychology & Health, 36(2), 236–251. https://doi.org/10.1080/08870446.2020.1766041

33.

Hughes

J. M.

Bigler

R. S.

Levy

S. R.

(2007). Consequences of learning about historical racism among European American and African American children. Child Development, 78(6), 1689–1705. https://doi.org/10.1111/j.1467-8624.2007.01096.x

34.

Kechter

Black

D. S.

Riggs

N. R.

Warren

C. M.

Ritt-Olson

Chou

C.-P.

Pentz

M. A.

(2019). Factors in the perceived stress scale differentially associate with mindfulness disposition and executive function among early adolescents. Journal of Child and Family Studies, 28(3), 814–821. https://doi.org/10.1007/s10826-018-01313-4

35.

Kingett

Abrams

(2017). Show racism the red card: Evaluating the impact of anti-racism educational interventions on the attitudes of young people in secondary school education. Equality and Human Rights Commission.

36.

Lee

E.-H.

(2012). Review of the psychometric evidence of the perceived stress scale. Asian Nursing Research, 6(4), 121–127. https://doi.org/https://doi.org/10.1016/j.anr.2012.08.004

37.

Lewis

(2022). What happens when American children learn about racism. FiveThirtyEight. https://fivethirtyeight.com/features/what-happens-when-american-children-learn-about-racism/

38.

Liu

S. R.

Modir

(2020). The outbreak that was always here: Racial trauma in the context of COVID-19 and implications for mental health providers. Psychological Trauma, 12(5), 439–442. https://doi.org/10.1037/tra0000784

39.

Lynch

Swartz

Isaacs

(2017). Anti-racist moral education: A review of approaches, impact and theoretical underpinnings from 2000 to 2015. Journal of Moral Education, 46(2), 129–144. https://doi.org/10.1080/03057240.2016.1273825

40.

McGregor

(1993). Effectiveness of role playing and antiracist teaching in reducing student prejudice. The Journal of Educational Research, 86(4), 215–226. http://www.jstor.org.prox.lib.ncsu.edu/stable/27541868

41.

McLean

Wilson

Lee

(2017). The Whiteness of Redmen: Indigenous mascots, social media and an antiracist intervention. Australasian Journal of Information Systems, 21. https://doi.org/10.3127/ajis.v21i0.1590

42.

Monchinski

(2008). Critical pedagogy and the everyday classroom. In Critical pedagogy and the everyday classroom: Explorations of educational purpose (Vol. 3, pp. 115–139). Springer. https://doi.org/10.1007/978-1-4020-8463-8_4

43.

Myers

(2021). Anti-racist education benefits all of us. American Federation of Teachers. https://www.aft.org/news/anti-racist-education-benefits-all-us

44.

Nelsen

M. D.

(2020). Educating for empowerment: Race, socialization, and reimagining civic education [unpublished dissertation]. Northwestern University.

45.

Nelsen

M. D.

(2021). Cultivating youth engagement: Race & the behavioral effects of critical pedagogy. Political Behavior, 43(2), 751–784. https://doi.org/10.1007/s11109-019-09573-6

46.

Nylund

K. L.

Asparouhov

Muthén

B. O

. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A monte carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535–569. https://doi.org/10.1080/10705510701575396

47.

Oberski

(2016). Mixture models: Latent profile and latent class analysis. In Robertson

Kaptein

(Eds.), Modern statistical methods for HCI (pp. 275–287). Springer International Publishing. https://doi.org/10.1007/978-3-319-26633-6_12

48.

Ozer

E. J.

(2016). Youth-led participatory action research. In Jason

L. A.

Glenwick

D. S.

(Eds.), Handbook of methodological approaches to community-based research: Qualitative, quantitative, and mixed methods (pp. 263–272). Oxford University Press.

49.

Pieterse

Powell

(2016). A theoretical overview of the impact of racism on people of color. In The cost of racism for people of color: Contextualizing experiences of discrimination (pp. 11–30). American Psychological Association. https://doi.org/10.1037/14852-002

50.

Pinedo

Vossoughi

Lewis

N. A.

(2021). Critical pedagogy and children’s beneficial development. Policy Insights from the Behavioral and Brain Sciences, 8(2), 183–191. https://doi.org/10.1177/23727322211033000

51.

Rapa

L. J.

Bolding

C. W.

Jamil

F. M.

(2020). Development and initial validation of the short critical consciousness scale (CCS-S). Journal of Applied Developmental Psychology, 70, 101164. https://doi.org/10.1016/j.appdev.2020.101164

52.

Reynolds

Kendi

I. X.

(2020). Stamped: Racism, antiracism, and you. Little, Brown and Company.

53.

Sawchuk

(2021). What is critical race theory, and why is it under attack?. EducationWeek. https://www.edweek.org/leadership/what-is-critical-race-theory-and-why-is-it-under-attack/2021/05

54.

Scheyder

(2020). U.S. schools revamp curricula in response to Black Lives Matter. Reuters. https://www.reuters.com/article/us-global-race-usa-textbooks/u-s-schools-revamp-curricula-in-response-to-black-lives-matter-idUSKBN25H1DF

55.

Snijders

T. A.

Bosker

R. J.

(2011). Multilevel analysis: An introduction to basic and advanced multilevel modeling (2nd ed.). SAGE.

56.

Stout

Wilburn

(2022). CRT Map: Efforts to restrict teaching racism and bias have multiplied across the U.S. Chalkbeat. Retrieved April 12 from https://www.chalkbeat.org/22525983/map-critical-race-theory-legislation-teaching-racism

57.

Torres

(2020). All students need anti-racism education. Learning for Justice. https://www.learningforjustice.org/magazine/all-students-need-antiracism-education

58.

Turner

R. N.

Brown

(2008). Improving children’s attitudes toward refugees: An evaluation of a school-based multicultural curriculum and an anti-racist intervention. Journal of Applied Social Psychology, 38(5), 1295–1328. https://doi.org/10.1111/j.1559-1816.2008.00349.x

59.

Walton

G. M.

Cohen

G. L.

(2007). A question of belonging: Race, social fit, and achievement. Journal of Personality and Social Psychology, 92(1), 82–96. https://doi.org/10.1037/0022-3514.92.1.82

60.

Watts

R. J.

Diemer

M. A.

Voight

A. M.

(2011). Critical consciousness: Current status and future directions. New Directions for Child and Adolescent Development, 2011(134), 43–57. https://doi.org/10.1002/cd.310

61.

Weems

McCurdy

B. H.

Scozzafava

M. D.

Pina

A. A.

Varela

R. E.

(2022). A systematic review of experimental studies evaluating anti-racist program techniques for children and adolescents. Merrill-Palmer Quarterly, 68(4), 323–367. https://doi.org/10.31234/osf.io/632tm

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB