Abstract
The ACLU’s National Advocacy Institute (NAI) is a one-week residential program designed to build youth’s capacities to advocate for civil rights and challenge injustices. This study assessed its role in fostering adolescents’ critical consciousness. We surveyed NAI participants and a comparison group of engaged youth before, one week after (N = 58(NAI)/166(comparison)), and six months after the program (N = 47(NAI)/165(comparison)). Using difference-in-differences analyses with covariates, NAI youth increased their likelihood of taking low- and high-risk political actions after one week and high-risk action frequency after six months, relative to the comparison group. NAI youth increased three forms of critical agency one week and, marginally, six months post-program. No associations were found for critical reflection. As a robustness check, propensity score models mostly replicated results. Findings show the promise of short, intensive programs for sparking growth in critical agency and action and highlight the need for more critical consciousness-raising spaces.
Keywords
A
This study examined a unique opportunity structure—a national, residential one-week program for high school students focused on civil rights and liberties, led by the American Civil Liberties Union (ACLU). The program aims to build young people’s knowledge and skills to advocate for civil rights and challenge injustices in a supportive, communal environment. The program includes key ingredients for fostering critical consciousness, such as opportunities for reflection on social injustices, community-building, and practicing skills and actions to challenge injustices (Watts & Guessous, 2006). We expected that program participants would increase in critical reflection, agency, and action one week and six months after the program, relative to a comparison group of young advocates.
Critical Consciousness Development
Critical consciousness is based on the theory and pedagogy of Paulo Freire, who argued that when oppressed people have opportunities to examine their social conditions in complex ways, they develop agency and act to critique and change oppressive social conditions (Freire, 1970). Critical consciousness has three main components (Diemer et al., 2017). Critical reflection consists of analyzing inequality and critiquing social structures that maintain the power of dominant groups and subjugate other groups. Critical agency, also known as critical motivation or efficacy, is the belief that one can be part of social justice change individually or collectively. Critical action, also known as sociopolitical or civic action, refers to individual and collective behaviors that target social change to reduce and eliminate oppression (Watts & Hipolito-Delgado, 2015). These actions can be done within or outside of traditional political structures and take many forms. Not all civic engagement is critically conscious: For actions to be critical, they must be aimed towards liberation and focused on justice and equity.
Scholars have called for better alignment between measurement and conceptualizations of critical consciousness (Rapa & Godfrey, 2023), which includes considering each dimension as multifaceted. Regarding critical reflection, in Freire’s (1970) conceptualization, critiquing systemic oppressions is an ongoing meaning-making process that can always be deepened and expanded. In sociopolitical development theory, Watts et al. 2002 named multiple facets of critical reflection, including recognizing injustice and oppression, understanding social and structural forces, causally attributing oppressions to root causes using a historical lens, and identifying solutions that address those causes. Measures of critical reflection vary in how fully they assess young people’s analysis of systems and understanding of structural roots of inequalities (Burson et al., 2023; Jemal, 2017). For example, Diemer et al.’s critical reflection measure of perceived inequality, commonly used in the field, assesses young people’s beliefs that particular groups (e.g., women, people in poverty, certain racial and ethnic groups) have fewer chances to succeed in society (Diemer et al., 2017, 2020; Rapa et al., 2020). Awareness of inequality is an important aspect of critical reflection that includes some recognition of disparate opportunities, yet this measure stops short of fully assessing youth’s structural explanations for these inequalities. Indeed, often youth are aware of inequalities, yet offer individual attributions for these issues (Godfrey & Wolf, 2016; Hope & Bañales, 2019). Other critical reflection measures more explicitly include structural attributions of inequalities, yet are domain-specific (Shin et al., 2016) or blend reflection and action (Thomas et al., 2014). In addition, domain-general assessments can be complemented by assessing domain-specific reflection on systems of oppression such as racism, classism, cisheterosexism, and ableism (Bañales et al., 2023; Seider et al., 2020; Shin et al., 2016, 2018). Critical reflection measures may be further expanded by considering youth’s intersectional awareness of unequal systems, i.e., understanding that systems of oppression exist in a “matrix of domination” (Hill Collins, 2000), where oppression is upheld via overlapping and interconnected domains of power (Burson et al., 2023). To achieve stronger theory-measure alignment, our critical reflection measures emphasize structural thinking, domain specificity, and intersectionality.
Critical agency has been theorized as a catalyst of critical consciousness development, a motivator of critical action, and a mechanism connecting youth’s critical reflection with actions to challenge inequalities (Bañales, Aldana et al., 2021; Uriostegui et al., 2021). In sociopolitical development theory, Watts and Guessous (2006) drew from efficacy (Bandura, 1997) and empowerment (Zimmerman, 2000) theories to frame critical agency as encompassing self-efficacy, sociopolitical control, skills, and voice. Likewise, empowerment, a multidimensional construct similar to critical consciousness (Christens et al., 2016), conceptualizes emotional empowerment (the dimension that aligns with agency) as including sociopolitical control, self-efficacy, competence, and motivation (Christens, 2012; 2019). In defining critical agency, other scholars have invoked various aspects of the concept, such as feeling that one can and should influence political systems (Seider et al., 2023) and as ability and responsibility to enact social change (Diemer et al., 2020). Likewise, critical agency measures tend to emphasize different facets, such as competence (Diemer & Rapa, 2016), motivation (Rapa et al., 2020), and power to influence change (Christens et al., 2023). Moreover, critical consciousness hinges on individual and collective pursuit of liberation (Freire, 1970), and likewise, measures should have individual and collective components (Rapa & Godfrey, 2023). Drawing from this literature, we assessed four components of critical agency: competence (capabilities to enact social change), drive (motivation to enact social change), and individual and collective power (perceived impact of individual and collective actions, respectively).
Young people take many actions to challenge inequalities, including advocacy (efforts to influence laws, policies, cultural understandings, and institutional practices), activism (raising awareness and demanding change through collective action), and organizing (building grassroots power and social movements; Ekman & Amnå, 2012; Watts et al., 2002). These are not the only action strategies that challenge inequalities and these broad categories overlap. Critical actions can also be classified according to risk (Hope et al., 2019). Low-risk actions (e.g., signing a petition, sharing information about an issue) entail behaviors that are relatively safe and come with less uncertainty, fewer social and personal costs, and less risk of harm or danger. In contrast, high-risk actions (e.g., attending a protest, public speaking) come with significant physical, mental, social, or legal risks. Young people gravitate to different critical actions based on their interests, identities, opportunities, and comfortability with risk (Hope et al., 2019). We assessed low-risk and high-risk actions to more fully capture critical actions. Overall, a multidimensional approach provides a more holistic picture of critical consciousness and offers a comprehensive test of whether and how programs increase critical consciousness.
Programs That Support Critical Consciousness Development
Sociopolitical development and critical consciousness are analogous and often described similarly or interchangeably; sociopolitical development situates critical consciousness in a developmental framework and considers antecedents and processes of critical consciousness development (Hope et al., 2023; Watts et al., 2011). In sociopolitical development theory, opportunity structures are programs, practices, and resources that give youth chances to critically reflect on inequalities, develop critical agency, and take critical action, often in scaffolded and collective ways (Watts & Hipolito-Delgado, 2015). For opportunity structures to effectively promote critical consciousness development, programming must engage deeply with power, oppression, and liberation for disenfranchised communities (Hope et al., 2023).
Several studies have used pre-post test designs to examine change in young people’s critical consciousness after participating in out-of-school time critical consciousness-raising programs. Young people increased their likelihood of taking political action, including activism, after participating in iEngage, a week-long U.S. summer program (Blevins et al., 2021), and increased in critical reflection and collective empowerment after participating in a two-week residential Youth Global Awareness Program in South Africa (Jay et al., 2023). A few studies included comparison groups, offering a benchmark of how critical consciousness develops in the absence of the program and thus more precisely evaluating program-related effects. A California study found that youth organizers reported more protest attendance and critical action online relative to youth in student government (Terriquez, 2015). After a summer and afterschool youth action research program, urban youth of color increased individual and collective efficacy to engage in community change, relative to a matched comparison group (Berg et al., 2009). LGBT and ally youth increased advocacy actions after a community leadership program, but not one year later, relative to a comparison group (Diaz & Kosciw, 2012). Research in schools has identified longitudinal changes in youth of color’s critical reflection, agency, and action in response to taking ethnic studies courses (Gillespie et al., 2025; Pinedo et al., 2025) and in schools with a more progressive, consciousness-raising approach (Seider et al., 2023). Broadening beyond critical consciousness, a wealth of studies have found that civic education in classrooms and other in-school programs have resulted in growth in young people’s civic engagement (e.g., knowledge, skills, efficacy, actions), with a systematic review finding that only 14 of 109 reviewed studies employed a comparison group (Jerome et al., 2024).
Research on critical consciousness–raising programs offers insights into how opportunity structures operate to support adolescents’ critical consciousness development. For example, critical pedagogies that interrogate the structural roots of social inequalities, provide historical knowledge about social problems, and pose critical questions and encourage dialogue support critical reflection development (Casanova, 2024; Hope et al., 2015; Nojan, 2020; Pender et al., 2022; Seider et al., 2023). Qualitative research among youth of color has documented youth’s progression from critical dialogues to deeper critical reflection, from which youth develop agency and identify and implement action strategies (Bloomer & Brown, 2024; Watts et al., 2002). Youth’s critical consciousness development during program participation may also be non-linear; Anderson et al. (2024) highlighted the complexity of adolescents’ development of critical reflection, which takes time and continued exposure to critical pedagogies to unfold.
Other key elements of critical consciousness–raising programs include opportunities for voice, action, and community-building. In qualitative and quantitative research, programs that offer opportunities for adolescents to exercise voice and practice action strategies that challenge injustice have been linked to higher critical reflection, agency, and action (Forenza, 2018; Hipolito-Delgado et al., 2022). Across programs, research documents how adults’ encouragement, role modeling, and mentoring support youth’s critical consciousness development (Christens & Dolan, 2011; Sánchez et al., 2020; Wray-Lake et al., 2024). Critical consciousness development is most likely when youth and adults work in partnership and youth’s voices are heard and respected (Kirshner, 2015; Oto, 2023). Moreover, social justice–focused programs that offer space for young people to build relationships with peers support critical consciousness, as youth benefit from learning from peers and building solidarity around shared interests (Christens et al., 2022; Nicholas et al., 2019; Pender et al., 2022; Woods-Jaeger et al., 2020). Some consciousness-raising spaces are designed for young people who share a marginalized identity, such as programs for Black youth (Brown et al., 2018; Sulé et al., 2021), Vietnamese youth (Nguyen & Quinn, 2018), Latinx immigrant youth (Terriquez & Milkman, 2021), and LGBTQ+ youth (Poteat et al., 2020). In such spaces, pedagogies center the specific group’s history of oppression, allowing youth to connect personal experiences to systemic injustice and build shared visions for liberation (Terriquez & Milkman, 2021).
Overall, research spanning different program models and populations suggests components of promising critical consciousness–raising programs, including critical pedagogical practices and opportunities for exercising voice, taking action, and building relationships. Adding to this research, we evaluate an out-of-school time program with a comparison group to assess the program’s role in enhancing multiple aspects of critical consciousness.
Focal Program: ACLU’s National Advocacy Institute
We focus on the ACLU’s National Advocacy Institute (NAI), a one-week residential experience during the summer for U.S. youth interested in social justice advocacy. The ACLU’s mission is to protect civil rights and liberties in the U.S. through advocacy, organizing, litigation, and education. Consistent with their mission and commitment to supporting the next generation of advocates, the ACLU has offered the NAI to approximately 5,000 high school students and young adults annually since 2016; we focused on high school–aged youth. The short program duration, residential experience, and diversity of the youth’s backgrounds and identities make this program unique relative to other programs in the published literature.
Although the NAI was not explicitly framed as a critical consciousness development program, the program’s theory of change articulates program outcomes that represent the three critical consciousness dimensions: deepening understanding of systemic, historical, and institutional causes of injustice (critical reflection); increasing advocacy and organizing skills and self-efficacy to engage in social change work and capacity to advocate equitably and inclusively (critical agency); and increasing advocacy, organizing, and activism (critical action). Moreover, four core program components align with promising practices for critical consciousness development identified from extant literature. First, NAI’s program content included keynote addresses, panel discussions, workshops, and electives led by issue experts, organizers, and lawyers. These sessions aim to build the youth’s critical historical and contemporary knowledge of social justice issues and develop their capacities to challenge injustices and advocate for civil rights and liberties. Thus, program content had elements of critical pedagogies that support critical reflection and develop agency and capacity for action (e.g., Casanova, 2024). Second, the NAI offered small group sessions, including daily homerooms and optional affinity groups. Affinity groups were youth-selected spaces where youth with shared identities (e.g., race, gender, sexual orientation) gathered for discussions. These sessions align with the practice of offering identity-centered spaces for community-building and critical questioning and dialogue, which support critical consciousness (e.g., Nguyen & Quinn, 2018; Sulé et al., 2021). Third, the NAI includes a day of action related to a sociopolitical issue being actively considered by policymakers. Actions have included lobbying visits, text/phone banking on an issue, and holding a rally. The program provides training to build youth’s knowledge and skills leading up to the action, aligning with research showing that opportunities to practice skills and take scaffolded action support critical consciousness development (e.g., Forenza, 2018; Hipolito-Delgado et al., 2022). Finally, social events offer space for fun and relationship building with peers, which fits with evidence that community-building activities are important features of consciousness-raising programs (Woods-Jaeger et al., 2020; Wray-Lake et al., 2024).
Each year, the NAI includes these same program features, although some topics and content vary based on salient issues and policy context. In 2023, the program was delivered in three settings—Washington, DC, New York City, and Atlanta. The program models were largely the same and emphasized racial justice with keynotes on racial equity and inclusion and sessions on ACLU’s racial justice legal efforts. Electives focused on various issues such as educational equity, Indigenous justice, free speech, transgender justice, and voting rights.
The Current Study
The ACLU’s NAI program may support adolescents’ critical reflection, agency, and action, although these potential impacts have not yet been empirically examined. We tested the extent to which participating in the NAI increased adolescents’ critical consciousness using a pretest-posttest non-equivalent groups design. After recruiting a comparison sample of youth advocates via social media, we surveyed NAI and comparison participants prior to the program and one week and six months after the program. We hypothesized that NAI participants would increase in critical reflection, agency, and action one week and six months after the program, relative to the comparison group. We utilized difference-in-differences (DiD) analyses, designed to account for different starting points between groups and naturally occurring development in critical consciousness. Comparing pretest-posttest difference scores for NAI and comparison participants assesses whether NAI youth showed more growth in critical consciousness than comparison youth, after accounting for covariates of age, race, gender, financial security, political orientation, and length of time as an activist. Multiple indicators of critical reflection (critical systems thinking, intersectional awareness, critical race analysis), critical agency (competence, power, drive, collective power), and action (low-risk, high-risk likelihood and frequency) offered a robust test of the extent to which the NAI program supports critical consciousness development. As an additional robustness check, given the non-equivalent treatment and comparison groups and lack of random assignment, supplemental analyses re-estimated the same DiD models using propensity score matching.
Method
Study Design and Recruitment
We recruited NAI participants and a comparison sample of young social change agents in summer 2023 and surveyed them one week before the NAI and one week and six months after the program. We recruited and surveyed high school–aged NAI participants (including recent spring 2023 graduates): ACLU program staff sent survey invitations via email to enrolled youth one week before the institute and closed the survey before programming began. The DC Program was held the first week of July, the Atlanta program the second week of July, and the New York City program the first week of August. Programs were very similar, with the DC Institute enrolling high school students 15–18 nationwide, the New York City Institute enrolling college students (including recent high school graduates), with more career-focused programming, and the Atlanta Institute enrolling 15–18 year-olds from southern states, with more racial justice-focused programming. All programs are geared toward youth interested in civil liberties issues and social justice advocacy, yet prior experience is not required. Youth submit an application, essay, and letter of recommendation. Most who apply are selected, subject to available space, and approximately 60% receive tuition assistance. In total, 166 youth participated in the DC program, 74 in NYC (but few were 18 and recent high school graduates), and 54 participated in the Atlanta program. Survey occasions were the same distance apart for all sessions.
We recruited a comparison group via Instagram advertisements. We sought 14–18-year-olds who self-identified as advocating for change on issues they care about. To supplement this primary recruitment method, we invited age-eligible youth who participated in a prior pilot survey for a different study. Youth completed an interest form to verify their eligibility and authenticity as high school–aged and interested in social change. Interest form responses (N = 2,255) were excluded from recruitment when youth were 19 or older, did not identify as an advocate or activist, did not complete the form, or responses were inconsistent, non-coherent, or duplicative. Eligible authenticated participants (N = 621, 27% of responses) were sent survey links on the same timeline as NAI participants, at two intervals. NAI and comparison groups received $5 for the pretest, $10 for the one-week posttest, and $20 for the six-month posttest.
Sample
The flow diagram in Figure 1 indicates eligible cases on each occasion. The NAI sample had a 66.3% response rate at pretest, 64.4% retention at one week, and 50.0% retention at six months; these rates exclude youth who participated in a posttest only. The comparison sample had a 39.6% response rate at pretest and 69.1% retention at one week and six months. Other inclusion criteria were (1) being enrolled in high school or spring 2023 graduate and (2) being 18 or younger at pretest; we applied these inclusion criteria after data collection to maximize sample comparability on age and life stage. Separate analytic samples compared pretest to one-week responses and pretest to six-month posttest responses. Final analytic samples were 58 (NAI participants) and 166 (comparison) for the one-week analyses and 47 (NAI participants) and 165 (comparison) for the six-month analyses. Using Power Panel in R (Schochet, 2022), a minimum detectable effect (MDE) for difference-in-difference analyses with our design, sample size, and a power of .80 is .60, a medium to large effect. In analytic samples, 42 young people were from the DC Institute, 11 were from Atlanta, and five were from NYC, but were treated as one sample for analyses.
Flow Diagram of Study Participants.
Table 1 displays sample characteristics. NAI participants were older (range 15–18, M = 16.69) than the comparison group (14–18 (range 14-18, M = 16.19), χ2(4), 23.65, p < .001. Similarly, NAI youth were more likely to be spring graduates (NAI: 23.0%, Comparison: 4.1%) than current high school students (NAI: 77.0%, Comparison: 95.9%), χ2(1), 24.16, p < .001. NAI youth were more likely to be very liberal and less likely to be moderate, conservative, or not sure politically than the comparison group, χ2(5), 12.86, p = .025. Samples were similarly diverse on race and ethnicity (largest groups were multiracial, white, and Asian) and gender (the majority were cisgender female). Similar proportions were immigrant-origin (~11% non-U.S. born and 48–54% children of immigrants). About half of both samples reported having family financial security. Analyses account for demographic differences between samples.
Descriptives for Treatment and Comparison Samples
Note. Descriptive statistics are reported from pretest. Chi square tests (for demographics) and t-tests (for critical consciousness) compared treatment and comparison group. Youth who selected more than one race/ethnicity were coded as Multiracial. “Other” race/ethnicity includes participants who selected “Arab or Middle Eastern,” “American Indian, Native American, Alaska Native, Indigenous,” or “Native Hawaiian or other Pacific Islander.”
Measures
Scale items are shown in Supplemental Tables S1–S3 and reliabilities in Table S4, which were nearly all .70 and higher. Items were averaged to create scales, and response scales vary, as described below. Correlations between constructs are shown in Table S5.
Critical Reflection
Three critical reflection scales captured critical systems thinking (4 items), critical race analysis (6 items), and intersectional awareness (5 items). Critical systems thinking assessed views on systemic oppression, e.g., “Many problems in our society can be attributed to systems of oppression.” Items were adapted from existing measures (Pratto et al., 1994; Shin et al., 2016; Windsor et al., 2022). To maximize variability, a slider scale captured responses ranging from strongly disagree (0) to strongly agree (100), with disagree, neither agree nor disagree, and agree labeled at 25-point intervals. Critical race analysis was measured with a color-blind racial ideology subscale (Neville et al., 2000). Response options were strongly disagree (1), disagree (2), slightly disagree (3), slightly agree (4), agree (5), and strongly agree (6); higher values indicated greater awareness of structural racism. Intersectional awareness, written for this study, captured awareness of oppression operating across multiple systems, e.g., “Understanding experiences of people with disabilities from different racial and ethnic groups is important.” Response options were strongly disagree (1), disagree (2), neither agree nor disagree (3), agree (4), and strongly agree (5). Confirmatory factor analysis showed that the three-factor structure adequately fit the data χ2(87) = 277.44, RMSEA = 0.089, CFI = 0.901, and fit better than a one-factor model, Δχ2(3) = 192.81, p < .001, ΔCFI = .099], CFI = 0.802. All loadings were significant (p < .001) and constructs were positively correlated (r range .53-.63, ps < .001).
Critical Agency
Critical agency measures were developed and validated based on theory and research (Wegemer et al., 2025). Among 1,120 adolescents, factor analyses identified a 4-factor structure as best-fitting (CFI = .978, RMSEA = .048, SRMR = .026, χ²(98) = 341, p < .001), with all factor loadings above .70. Competence (4 items) captured feelings of capability to make social change, e.g., “I can work effectively to improve an injustice facing my community.” Drive (4 items) assessed determination, urgency, and motivation to enact social change, e.g., “I feel driven by a sense of urgency to address social injustices.” Individual power (4 items) measured perceived impact in making social change, e.g., “I have the power to address injustices in my community.” Collective power (4 items) assessed perceived impact of group action, e.g., “Organized groups of young people like me can effectively fight against social and economic inequalities in my community.” Response options were not at all true (1), a little bit true (2), somewhat true (3), mostly true (4), to completely true (5). Correlations ranged from .56 to .78, with the highest association between competence and drive.
Critical Action
Likelihood of low- and high-risk actions was assessed at pretest and one-week posttest. Using identical items, low- and high-risk political action frequency was assessed at pretest and six-month posttest. Thus, all scales were assessed at pretest. We made these decisions to match the study’s theory of change and to appropriately align measures to the study time frame, as one week offers sufficient time to change one’s intentions to act, which are proximal predictors of future action (Ajzen & Fishbein, 1980), whereas six months offers a longer window over which changes in action frequency are feasible. Items were adapted from existing political activism orientation scales (Corning & Myers, 2002; Hope et al., 2019). Low-risk actions (12 items), requiring less risk and effort, included going out of the way to learn about a political or social issue and signing a petition for a political or social cause. High-risk actions (11 items), requiring more risk and effort, included organizing a political or social issue-related event and taking political action that risked personal safety. Youth rated their likelihood of future action from definitely not (1), probably not (2) possibly (3), probably (4), very probably (5), and definitely (6) and frequency of actions over the past six months from never (1), very rarely (2), rarely (3), occasionally (4), frequently (5), and very frequently (6). Scale correlations ranged from .66 to .78, except for low-risk likelihood and high-risk frequency (r = .47).
Control Variables
Models included time-invariant control variables to improve the precision of estimation and better isolate the program effect from other reasons for pre-posttest group differences. Age was a continuous variable from 14–18. Student status was dichotomous: current high school student (1) or recent graduate (0). Race was drawn from a single item asking whether participants identify as a person of color (no = 0, yes = 1); racial groups could not be specified in analyses due to small sample sizes. Gender was dummy-coded into female (no = 0, yes = 1) and nonbinary (no = 0, yes = 1), with male as the reference group. Financial security was on a 3-point scale: We cannot buy the things we need sometimes (1), We have just enough money for the things we need (2), and We have no problem buying the things we need (3). Political orientation was dummy-coded into liberal (liberal & very liberal = 1, else = 0), conservative (conservative & very conservative = 1, else = 0), moderate (no = 0, yes = 1), and not sure (no = 0, yes = 1), with liberal as the reference group. Length of time as an activist was continuous (less than a year = 1, 1-2 years = 2, 3-4 years = 3, 5 or more years = 4).
Youth Advisory Board
Our study involved a Youth Advisory Board (YAB), composed of recent NAI alumni in high school and college, to center young people’s perspectives and ensure data interpretations were informed by their insights. Ten members informed the survey design and recruitment in 2023, and nine members (with continuing and new members) provided feedback and interpretation of findings in summer and fall of 2024. Two members coauthored this paper, contributing writing and feedback. YAB members were compensated for participating.
Attrition Analyses and Multiple Imputation
Data were not completely missing at random, according to a significant Little’s MCAR test (p < .001). We conducted attrition analyses, comparing youth who remained to youth who dropped out at each wave, separately for the treatment and comparison groups (see Tables S6–S7). Regarding demographics, there were no differences between youth who remained versus attrited in the comparison group, and one marginally significant finding for the treatment group, such that liberal youth were most likely to drop out of the treatment group at the one-week posttest (χ2(5) = 6.80, p = .09). Regarding critical consciousness, there were no differences on critical consciousness for youth who remained versus attrited in the comparison group and two marginal findings for the treatment group: NAI youth who dropped out of the survey after one week had marginally lower intersectional awareness (M = 4.51) than youth who remained (M = 4.68; t = 1.81, p = .08), and NAI youth who dropped out of the survey after six months had marginally higher drive (M = 4.71) than youth who remained (M = 4.53; t = −1.32, p = .07). Overall, there were few observable differences in the sample due to attrition.
We used multiple imputation to account for missing data due to item non-response, using all study variables. Imputation models could not account for attrition, given a lack of model convergence, likely due to more parameters than sample size. The proportion of missingness ranged from 0% for age and student status at pretest to 16.5% for one high-risk action item at six-month posttest. We performed separate imputations for program and comparison groups in each pair of data waves (pretest-one week; pretest-six months) to maximize sample size, and likewise estimated change across each interval separately. Using R software (RStudio Pro v2024.04.2), we generated 30 datasets, each with 10 iterations, using chained equations (MICE; White et al., 2011). Relative efficiency estimates for both MI models were .999, indicating that imputation of 30 datasets provided highly efficient estimates relative to an infinite number. We applied Rubin’s (1987) rules to combine parameter estimates across imputed datasets. We used a p < .05 criterion for determining significant differences, yet given our small sample size, we report marginally significant differences (p < .10) and interpret them with caution.
Analytic Plan
We used difference-in-differences (DiD) analysis to examine the role of NAI participation on critical consciousness. As a quasi-experimental method, DiD quantitatively estimates program effects by comparing pretest-posttest differences in each construct between program and comparison groups (Huntington-Klein, 2021). The method aims to approximate causal impacts, but given the lack of random assignment, findings are not causal. The focal parameter is a program × time (pretest to posttest) interaction, estimated in a regression model. The interaction term estimates whether slopes for critical consciousness indicators differ for NAI participants. DiD models have a parallel trend assumption, meaning that any differences between the program and comparison groups are assumed to remain constant over time in the absence of the program. With this assumption, DiD models aim to account for selection due to observed and unobserved variables by allowing for different starting points at pretest. As shown in Table 1, youth enrolled in the NAI were higher on all indicators of critical consciousness than youth in the comparison group. We estimated a DiD model for each of the nine constructs at each posttest, totaling 18 models. Individual fixed effects accounted for unobserved time-invariant characteristics. Models included controls to account for potential confounds of age, student status, race, gender, financial security, political orientation, and time as an activist.
Lack of random assignment is a threat to internal validity. For a robustness check, supplemental analyses used propensity score matching and weighting to improve balance between groups. We employed 3:1 nearest neighbor matching and applied weights from covariate balancing propensity scores (CBPS) to enhance balance (Imai & Ratkovic, 2014), making group comparisons more akin to randomization and estimates more precise. This approach is presented as supplemental, given that smaller matched sample sizes reduced power.
Test of Parallel Trends Assumption
More than one pretest survey occasion is required to test the parallel trends assumption, and thus our study cannot test this assumption. However, the treatment group was more likely to be older and very liberal relative to the comparison group, and if youth with these characteristics also showed different patterns of change over time, then such differences would threaten the parallel trends assumption. Thus, we used variation in the comparison group to examine whether older and more liberal youth had different patterns of change over time, absent participation in the treatment. Repeated measures ANOVAs identified no differences in changes over time for the comparison group on any critical consciousness indicator by age or political orientation (Table S8), offering tentative evidence of parallel trends for these demographic groups.
Results
The program × time interaction is the focal parameter that tests the hypothesized program role on critical consciousness. For each critical consciousness construct, we present program × time findings and then describe main effects for program (i.e., pretest differences between groups) and time (i.e., change across both groups), which offer context for the study. In the figures, the dotted line represents the counterfactual, i.e., the parallel trends comparison line, which is the expected change for NAI youth had they not participated in the program.
NAI and Critical Reflection
Table 2 and Figure 2 display results for critical reflection. There were no significant program × time interactions across the three indicators of critical reflection, suggesting that the program had no notable effect on critical reflection as assessed here.
Differences-in-Differences Models for Critical Reflection
Note. Standardized coefficients and Cohen’s d effect sizes are presented. Program × time interaction is the change in each DV (in SD) from pretest to posttest that is attributable to the program, relative to the comparison group, accounting for covariates. Program effect reflects pretest differences between treatment and comparison groups (treatment = 1). Time represents the change over time from pretest to posttest for both groups combined, except in the context of a significant program × time interaction, where the coefficient represents change over time for the comparison group.
p < .10; *p < .05; **p < .01; ***p < .001.
Critical Reflection Difference-in-Differences Models.
Several main effects were evident for critical reflection. The program main effect captures pretest group differences and was significant for critical race analysis (β1week = 0.30, p = .003, β6months = 0.24, p = .043) and approached significance for intersectional awareness (β1week = 0.22, p = .068; β6months = 0.27, p = .062): NAI youth had higher starting levels of critical race analysis and marginally higher intersectional awareness at pretest than comparison youth, after accounting for controls (Figures 2a & 2b, 2e & 2f). For critical systems thinking, a significant one-week time effect showed that, on average, both groups increased their critical systems thinking by 0.15 standard deviations one week post-program (β = 0.15, p = .005; Figure 2c).
NAI and Critical Agency
As shown in Table 3 and Figure 3, program × time interactions showed that at one week post program, the NAI had a positive association with competence (β = 0.35, p = .016; Figure 3a), individual power (β = 0.35, p = .007; Figure 3e), and collective power (β = 0.32, p = .028; Figure 3g), but not drive. NAI youth increased 0.32-0.35 standard deviations more than comparison youth on these aspects of agency. Program × time interactions at six months approached significance for competence (β = 0.22, p = .098), individual power (β = 0.25, p = .055), and collective power (β = 0.28, p = .072), and were in the expected direction of increased critical agency for NAI participants over time (Figures 3b, f, h).
Differences-in-Differences Models for Critical Agency with Covariates
Note. Standardized coefficients and Cohen’s d effect sizes are presented. Program × time interaction is the change in each DV (in SD) from pretest to posttest that is attributable to the program, relative to the comparison group, accounting for covariates. Program effect reflects pretest differences between treatment and comparison groups (treatment = 1). Time represents the change over time from pretest to posttest for both groups combined, except in the context of a significant program × time interaction, where the coefficient represents change over time for the comparison group.
p < .10; *p < .05; **p < .01; ***p < .001.
Critical Agency Difference-in-Differences Models.
Turning to main effects, pretest differences were found for competence, drive, and individual power (one week: βcomp = 0.40, p < .016; βdrive = 0.67, p < .001, βind power = 0.56, p < .001; six months: βcomp = 0.35, p =.019; βdrive = 0.53, p < .001, βind power = 0.47, p = .001): NAI participants started higher on these forms of critical agency at pretest (Figure 3a-f). Pretest differences approached significance for collective power in the six-month models (β = 0.27, p = .074; Figure 3h), suggesting that NAI participants were marginally higher at pretest, after accounting for controls. A significant time effect showed one-week declines in individual power (β = −0.13, p = .032): in the context of the program × time interaction, this parameter applies to the reference group, indicating a decline for comparison youth (Figure 3e). Drive declined for all participants over six months (β = −0.18, p = .016; Figure 3d) and marginally over one week (β = −0.10, p = .096; Figure 3c). Collective power marginally declined over six months for the comparison group, in the context of the program × time interaction (β = −0.17, p = .054; Figure 3h).
NAI and Critical Action
Action models (Table 4, Figure 4) examined likelihood of actions over one week and frequency of actions over six months. Findings showed significant program × time interactions one week post-program for likelihood of high-risk (β = 0.30, p = .001; Figure 4a) and low-risk actions (β = 0.25, p = .005; Figure 4c), indicating 0.30 and 0.25 standard deviations greater increase in likelihood of high-risk and low-risk actions, respectively, for NAI youth relative to comparison youth. A significant program × time interaction for frequency of high-risk actions over six months (β = 0.40, p = .014) showed that NAI youth had a 0.40 standard deviation increase in high-risk actions relative to comparison youth (Figure 4b).
Differences-in-Differences Models for Critical Action with Covariates
Note. Standardized coefficients and Cohen’s d effect sizes are presented. Program × time interaction is the change in each DV (in SD) from pretest to posttest that is attributable to the program, relative to the comparison group, accounting for covariates. Program effect reflects pretest differences between treatment and comparison groups (treatment = 1). Time represents the change over time from pretest to posttest for both groups combined, except in the context of a significant program × time interaction, where the coefficient represents change over time for the comparison group.
p < .10; *p < .05; **p < .01; ***p < .001.
Critical Action Difference-in-Differences Model.
Regarding main effects, at pretest, NAI youth endorsed higher likelihood of high-risk (β = 0.63, p < .001; Figure 4a) and low-risk actions (β = 0.53, p < .001; Figure 4c) and frequency of low-risk actions (β = 0.35, p = .007; Figure 4d), after accounting for controls. As shown by the time main effect, likelihood of low-risk actions declined over one week for comparison youth (β = −0.12, p = .019), in the context of the significant program × time interaction (Figure 4c).
Covariates
Associations between covariates and critical consciousness were not central to our inquiry but are shown in Tables 2–4. Overall patterns indicated that more time as an activist was consistently related to higher pretest levels of all critical agency and action indicators. Relative to politically liberal youth, politically moderate and conservative youth had lower pretest levels of critical race analysis, critical systems thinking, intersectional awareness, drive, and likelihood of low and high-risk actions. Financial security was related to lower likelihood and frequency of low- and high-risk actions. There were few significant findings for age, race, and gender.
Supplemental Analyses
For a robustness check, we conducted supplemental analyses using propensity score matching and weighting to improve balance between NAI and comparison groups (Tables S9–S11). Two program × time interactions became significant that were marginally significant in covariate-controlled models (individual power at six months, β = 0.37, p = .022; collective power at six months, β = 0.55, p = .018). One significant program × time interaction became marginal after matching and weighting: competence at one week, β = 0.32, p = .057. No parameters went from significant (p < .05) to non-significant (p > .10) or vice versa. Substantial alignment across the models supports the idea that observed group differences in changes over time are more likely attributable to the program than to preexisting group differences.
Sample sizes were too small to conduct significance testing of differential patterns of change across the three program sites. However, for full data transparency, we report means for critical consciousness constructs separately by NAI site in Tables S12-S13. Visual inspection of these patterns, which should be interpreted with caution, suggested that youth in the Atlanta program started higher on critical agency and action and may have shown less change over time.
Discussion
Findings demonstrated that the ACLU’s National Advocacy Institute (NAI) effectively supports critical consciousness development by increasing youth’s critical agency and action immediately after the program. NAI participants increased in three forms of critical agency and in their likelihood of taking low- and high-risk political actions one week post-program. After six months, NAI youth had marginally greater increases in competence, individual power, collective power, and significantly greater increase in high-risk political actions, relative to the comparison group. No program effects for critical reflection were detected. This study builds on a growing body of research documenting the role of school- and community-based programming in cultivating young people’s critical consciousness (e.g., Gillespie et al., 2025; Jay et al., 2023; Pinedo et al., 2025; Terriquez, 2015). We advance this research by assessing multiple dimensions of critical consciousness and examining an intensive one-week residential program. Including a comparison group of youth advocates helped us account for naturally occurring changes in critical consciousness over the timeframe, and difference-in-differences analyses and propensity matching models aimed to account for preexisting group differences. Findings highlight the importance of cultivated opportunities for critical consciousness development and the value of a one-week intensive advocacy program for stimulating critical consciousness.
Critical Agency
NAI participation boosted competence and individual and collective power after one week, and these increases were sustained, albeit marginally, over six months. Findings were most robust for critical agency compared to other dimensions of critical consciousness, pointing to the value of a one-week intensive advocacy program for enhancing youth’s capacity to effect meaningful social change. Agency is widely understood as an essential ingredient to sustained civic action (Anyiwo et al., 2020); critical agency is a consistent precursor to critical actions that challenge inequalities (Diemer & Rapa, 2016; Uriostegui et al., 2021), and a mechanism connecting youth’s beliefs about societal inequities to their actions to challenge these inequalities (Hope et al., 2020). Surprisingly, we still know relatively little about how youth develop and sustain their critical agency. Our study suggests that programs offering opportunities to build skills and knowledge, develop relationships, and take collective action may be instrumental in supporting growth in critical agency. These findings contribute to the evidence on opportunities that grow youth’s agency, including school-based action civics programs (Blevins et al., 2016), long-term participation in community-based organizations and organizing (Turner, 2021), and everyday political discussions and exposure to civic role models (Bañales, Hope, et al., 2021). Community contexts can foster and hinder young people’s critical agency (Wray-Lake & Abrams, 2020), and even highly civically engaged young people do not always feel agentic (Briggs et al., 2023). Indeed, our findings showed declining trends in some facets of critical agency over time. The upward and downward malleability of agency underscores the importance of investing in opportunities that maintain and build youth’s critical agency.
Past research on critical agency has sometimes produced unexpected findings (e.g., Aldana et al., 2019; Diemer & Rapa, 2016; Olle & Fouad, 2015), which may be, in part, due to lack of conceptual clarity and measurement precision. Our study advances the field by examining multiple facets of critical agency, offering a more robust test of whether programs can foster critical consciousness. Findings identified some distinctions in how the NAI program shaped each dimension. Drive to make change started higher among NAI participants and declined over six months for both groups, but showed no difference due to program participation. Motivation for taking critical actions may come from more deeply held values or personal experiences of injustice (Pinedo et al., 2024) and may be less amenable to change due to short-term programming, an idea to test in future research. Yet, drive also declined over six months for both groups, and may reflect the developmental timing of the study period, in that youth may be less motivated to enact social change in winter months or during the school year relative to the summer. The reasons for this decline merit further investigation so that programs can consider how to mitigate potential losses in young people’s drive for enacting social change.
Critical Action
Over the short term, NAI participation increased the likelihood of low-risk and high-risk political actions, demonstrating that the program may help young people recognize their action potential. Because intentions to act are proximal predictors of taking action (Ajzen & Fishbein, 1980), the program’s role in helping young people see themselves as future actors may meaningfully translate into actions later. Given that political engagement fluctuates over short-term time scales based on circumstances and opportunities (Oosterhoff et al., 2022; Wilf et al., 2024), programs that boost youth’s likelihood of action may help buffer some of this natural ebb and flow of critical actions. However, intentions to act tend to be more highly estimated than reported actions, and do not always translate into tangible actions, given various individual and contextual barriers (Eckstein et al., 2013). For this reason, a strength of our study is moving beyond intentions to also assess frequency of actions over six months.
The NAI increased in frequency of high-risk actions six months later, relative to the comparison group. The program may have prepared young people to lean into high-risk actions, like public speaking and protesting, over time. More research is needed that specifies which program components help youth sustain critical action over the long term, which likely include scaffolded actions that enable youth to see tangible achievements, supportive youth-adult partnerships, and a sense of community belonging (Oto, 2023; Sánchez et al., 2020; Wilf et al., 2024; Wray-Lake et al., 2024). Building multiple facets of agency may have also contributed to enhancing high-risk critical actions. For politically active young people, program goals may not need to be increasing frequency of different actions, but instead, programs may aim to deepen and sustain youth’s chosen action strategies and enhance their effectiveness (Terriquez, 2015). Programs like the NAI may also increase young people’s readiness to act in response to a social movement. For example, in this study, a major event between the NAI and the six-month follow-up was the October 7th Hamas attack on Israel, which fomented Israel’s attacks on Gaza and prompted a wave of pro-Palestinian protests across the U.S. Youth may have felt more equipped to join these or other kinds of protests following NAI program participation. Overall, examining low- and high-risk actions is a study strength, as youth have different interests, preferences, and comfort levels for political actions. Low-risk actions may be more easily accessible whereas high-risk actions may not be equally feasible for all youth. It would be strategic and inclusive for social justice-focused programs to support the development of diverse low- and high-risk critical actions.
Critical Reflection
The NAI yielded no discernible impact on young people’s critical reflection. Critical reflection may take time to develop. Critical pedagogies tend to be offered over longer time periods, such as in curricula across a school year, long-term participatory action research projects, and continuous engagement in community-based programs and organizing (Anderson et al., 2024; Bloomer & Brown, 2024; Christens et al., 2023; Pinedo et al., 2024; Seider & Graves, 2020). One-week intensive programs like the NAI may plant the seeds for heightened critical reflection, but these understandings may take time to grow. Christens et al. (2023) found that critical reflection was more pronounced and elaborated for young people who had been activists for a long period of time. Anderson et al. (2024) documented how critical reflection development may be non-linear across a program as young people wrestle with complex ideas and learn systems thinking that challenge individual attributions about social problems. More program touchpoints after one week may foster continued critical reflection dialogues that began during the NAI, and more research is needed on the specific experiences and practices within programs that spark different forms of critical reflection. NAI youth started higher than comparison youth on all measures of critical reflection, and it may be difficult for programs to exert meaningful change on critical reflection when young people start so high. Despite the lack of findings across reflection measures, we would still argue that there are benefits of evaluating programs’ effects on different aspects of critical reflection. We also found that critical systems thinking increased for both program and comparison groups over one week, which is a reminder that critical reflection and other aspects of critical consciousness are continually developing and may be sparked by other proximal contexts or political events.
Limitations
Given our small sample size, our study was underpowered for identifying smaller effects. Detecting small to moderately sized effects despite low power could signal that we underestimated program effects overall, yet also means our study was not designed to reliably test small effects. Because our study was underpowered and thus had a higher Type II error risk, we chose not to correct for heightened Type I error risk that came from conducting multiple significance tests. Yet, our large number of models could mean that some significant findings were detected by chance. A small sample size also created tradeoffs between the covariate-controlled models, which had more power but less balance across groups, and the propensity score models, which had greater precision yet less power. The largely consistent findings across these two approaches gave us more confidence in our findings, yet a larger pool of NAI participants would be useful for replicating findings and exploring variations in outcomes for youth across NAI sites. Visual exploration of critical consciousness across the NAI sites suggested some variation in starting points and change over time. A larger sample would have enabled us to test whether the program more strongly impacted certain young people more than others. For example, previous work showed that an arts program fostered critical action for youth of color more than for white youth (Ibrahim et al., 2022). However, achieving sufficient sample sizes to examine heterogeneous program effects is a significant practical challenge, given that program enrollments are often small. Future research might address this issue by pooling data across multiple years of program implementation.
As we identified few observable differences due to attrition on demographics and critical consciousness, missingness could be MNAR in our data, which has unknown implications for the interpretation of results (Graham, 2009). For example, NAI youth attrition may have been due to demands of being highly politically active as a result of the program, suggesting underestimated program effects. Alternatively, NAI youth attrition may have been due to lack of learning in the program, suggesting overestimated program effects. Another limitation was the preexisting group differences on critical consciousness constructs, age, and political orientation. We did not find differences in family financial security, yet the program’s financial cost may have been prohibitive for some, even with financial aid, which may have added to selection bias. Other unobserved variables could have led to selection bias, such as young people’s access to other advocacy opportunities, levels of family and peers’ critical consciousness and related socialization, or social desirability bias, which all may have been more present for NAI participants. These and other alternative explanations for findings cannot be ruled out. Furthermore, young people with higher critical consciousness may naturally increase more over time, yet if anything, one might expect slower growth for youth starting high on a scale, due to regression to the mean. Our analyses tried to account for selection effects using multiple strategies, but selection effects remain a threat to internal validity. Analyses tried to rule out age and political orientation as factors that might have compromised the parallel trends assumption, yet this assumption could have been violated for unknown or unobserved reasons, biasing estimation of treatment effects. Assessing treatment and comparison groups more than once before the program would have enabled a true test of the parallel trends assumption. Random assignment is not feasible in this kind of program, especially given the high acceptance rate, yet if possible, a comparison group of waitlisted, non-admitted, or soon-to-be-enrolled applicants would enable a stronger comparison.
Although we view our multidimensional measures as a study strength, some measures were new and require further evidence of validity. Factor analyses supported distinct dimensions of critical reflection and agency, yet high correlations between some constructs suggest that some scales have shared variance. The study was unable to determine which program components were most effective in increasing critical agency and action, nor how they were implemented. Further research could better delineate how the program operates and what makes it effective, to enable replication. Political climate and sociohistorical moments such as elections, wars, disasters, or social movements may shape young people’s experience of the NAI and related programs in unidentified ways, as well as influence the broader context of youth critical consciousness development. Sociopolitical contexts are important to consider when evaluating the impacts of advocacy training programs and when aiming to replicate prior findings.
Implications
This study offers several implications for applied research on youth critical consciousness development. Our approach illustrates the value of empirically documenting program outcomes, so that researchers and practitioners can continue to build stronger programs. We contribute to sociopolitical development theory by demonstrating that a one-week residential program for youth advocates can be a viable opportunity structure for supporting short-term and some long-term growth in critical agency and action. We further advance critical consciousness research through our use of multidimensional measures, which offer stronger theory-measurement alignment than many prior studies and may spark other basic and applied research to recognize dimensions of critical consciousness as multifaceted. Deeper conceptualization of the multifaceted nature of critical reflection, agency, and action is an important next step for theory and research. Additionally, comparison groups add rigor to program evaluation research, and following up with young people long after the program is over can shed light on the durability of effects. Future research could benefit from longer-term longitudinal studies, past six months, to track growth of critical consciousness due to program participation. Future research might consider mixed-methods approaches to analyzing growth in critical consciousness in response to the program to add context and explanation for findings. Additionally, in our experience and as others have found (Anyon et al., 2018), adding program alumni to the research team to collaboratively assess program impacts and interpret findings is a valuable endeavor that adds important perspectives and context to research findings.
Findings have implications for youth programming and practitioners. Our findings highlight the value of building short-term intensive opportunities for adolescents to gain knowledge and skills to challenge inequalities and build community with other advocates. The NAI appeared particularly effective in developing youth agency, and other organizations can use these findings to reflect on tangible ways that their programs can foster different aspects of critical agency, such as knowledge and skill-building, community-building, and scaffolded actions. More programs for advocates and activists exist than are found in published literature, and sharing and comparing program models could be worthwhile for scaling up effective practices. Through evaluating short- and long-term program models, we can better pinpoint the ideal duration and touchpoints needed to support growth across dimensions of critical consciousness. Reducing barriers to enrolling in the NAI and other related programs, such as inability to travel to in-person events, pause family obligations, or take time off of paid work to attend, is also important for building equitable opportunity structures to support young people’s advocacy and activism.
Conclusion
Young people are powerful social change agents who are also on a developmental journey to gaining critical awareness of systemic inequalities, agency, and action strategies to challenge inequalities and advocate for a more just world. The ACLU’s National Advocacy Institute is an example of a meaningful opportunity structure that can offer continued education to young social change agents and support their critical consciousness development. We join others in calling for more available spaces that motivate and activate young people. Adolescents are leading voices in challenging society’s inequalities and can benefit from programs designed to further enhance their social justice impact and support their development.
Supplemental Material
sj-docx-1-ero-10.1177_23328584251375052 – Supplemental material for Examining the Role of a Program for Youth Advocates in Critical Consciousness Development
Supplemental material, sj-docx-1-ero-10.1177_23328584251375052 for Examining the Role of a Program for Youth Advocates in Critical Consciousness Development by Laura Wray-Lake, Christopher M. Wegemer, Elan C. Hope, Kristina Cổ-Đoàn, Qin L. Kramer and Emily Greytak in AERA Open
Footnotes
Acknowledgements
Opinions or points of view expressed in this document are those of the authors and do not necessarily reflect the official position of, or a position that is endorsed by, AmeriCorps.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This material is based upon work supported by AmeriCorps under Grant Number 22REACA002.
Note: This manuscript was accepted under the editorial team of Kara S. Finnigan, Editor in Chief.
Authors
LAURA WRAY-LAKE is professor of Social Welfare at UCLA in Los Angeles, California;
CHRISTOPHER M. WEGEMER is a postdoctoral fellow at UCLA in Los Angeles, California,
ELAN C. HOPE is vice president of Research and Evaluation at Policy Research Associates, Inc, in Troy, New York,
KRISTINA CỔ-ĐOÀN is a Youth Advisory Board member for the study and a fourth-year undergraduate student at the University of Toronto in Toronto, Canada,
QIN L. KRAMER is a Youth Advisory Board member for the study and a fourth-year undergraduate student at Northeastern University in Boston, Massachusetts;
EMILY GREYTAK is Director of Research at the American Civil Liberties Union in New York, New York,
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
