Abstract
Services developed to address conduct problems in school contexts show limited efficacy. The current study examined how young adults with childhood histories of conduct problems retrospectively understood service efficacy, inefficacy, and what suggestions they had for service improvement. Participants were 41 young adults from Québec (17–21 years old; 53.7% women; 78% white; 77% below the low-income cut-off for single-person households) who had received services for conduct problems starting in childhood. They completed semi-structured interviews about their service usage experiences. Thematic analysis was used to identify how participants discussed themes relating to efficacy, inefficacy, and service improvement. While considerable overlap was observed in how participants and educational professionals understand efficacy (e.g., reduced symptoms) and inefficacy (e.g., worsening symptoms), participants also noted key differences in terms of how they perceived efficacy (i.e., using services to avoid punishment) that are not generally considered by educational professionals. Including user perspectives when assessing service efficacy and inefficacy can provide crucial insight for improving services for youth with conduct problems, providing a starting point for understanding how users evaluate service success, and how they saw services as influencing their psychosocial outcomes.
Conduct problems, or consistent patterns of antisocial behavior, including aggression and bullying, rule breaking, deceitfulness, destruction of others’ property, as well as comorbid defiance and opposition to authority (American Psychiatric Association, 2013) have substantial financial and societal costs (Rivenbark et al., 2018), including heightened risk for crime, substance use, and mental health difficulties (Bevilacqua et al., 2018). Children with conduct problems are commonly referred for school-based mental health services, and use these services for longer periods than children with other mental health problems (Burnett-Zeigler & Lyons, 2012; Little & McLennan, 2010), reflecting how these children are more aggressive and disruptive in class, and have poorer academic skills compared to children without these difficulties (Bierman et al., 2013; Nocentini et al., 2012). School-based services for youth with conduct problems often either result in no decrease (Rones & Hoagwood, 2000), or exacerbation of conduct problem symptoms (Dempsey et al., 2016; Morgan et al., 2010). Examining how young adults with histories of conduct problems perceived school-based interventions, and how they saw interventions shaping their outcomes can shed light on the varied efficacy of services. Using semi-structured interviews with young adults who had received services for conduct problems starting in childhood, our objective was to understand how young adults themselves evaluated service efficacy and inefficacy as a starting point for improving service outcomes.
School-Based Services and Youth With Conduct Problems
Schools can normalize mental health service usage and increase service access and availability for students (Gronholm et al., 2018), with educators being well-situated to observe and identify students’ mental health needs (Huggins et al., 2016). Efficacy of school-based services, or a service’s ability to improve functioning or reduce symptoms (Rones & Hoagwood, 2000), has important consequences. Indeed, effective interventions can promote positive outcomes, while ineffective interventions place children at higher risk of developing negative attitudes toward learning (K. E. Hoagwood et al., 2007; O’Connor et al., 2005), low academic performance, school failure (Kremer et al., 2016), and dropping out of school (Bradley et al., 2008; Burk et al., 2011). Access to school mental health services is not evenly distributed, with students attending schools in more socioeconomically advantaged areas having better access to mental health services in schools (Sanchez et al., 2018; Verhoog et al., 2022).
Children with conduct problems primarily receive services in schools (Bradshaw et al., 2008; Costello et al., 2014). And, while many specific programs have been developed to address the unique needs of these children such as Incredible Years (Reid et al., 2003) and Coping Power (Lochman et al., 2012; Powell et al., 2011), in real-world situations, youth with conduct problems often receive many different types of services. These services are commonly implemented following a formal assessment of child needs and are based on an individualized service plan. Thus, services in schools are typically highly variable. They aim to help children attain learning-related and socialization goals, and to assist children in a variety of different ways (e.g., behavioral intervention and support, psychological services, and coaching in order to assist teachers with behavioral management in class) (Bradley et al., 2008). While most children with conduct problems receive services within a regular mainstream classroom environment, a small proportion receive services in a restrictive educational setting such as specialized classrooms, or separate educational environments with smaller class sizes, specifically trained teachers, and more structure and attention to address the needs of youth (Déry et al., 2005). Youth within specialized classrooms experience a wide range of prevention and intervention approaches, and while some theorists propose that placement in specialized classroom environments may be justified in cases of poor student-context fit within the broader school environment and when few other options are available to retain the student in any type of school environment (Jull, 2008), placement of youth with conduct problems in specialized classrooms has been linked with greater risk for school non-completion (Powers et al., 2016).
Theory for Program Evaluation of School-Based Service Use
The relevance of empirically-validated school-based interventions for improving individual outcomes has resulted in the development of theoretical frameworks for the effective development, implementation, and evaluation of intervention and prevention programs (Greenberg et al., 2005). More specifically, theoretical evaluation frameworks such as causative theory outline how to develop and evaluate intervention programs for developmental needs based on identifying interventions addressing appropriate mechanisms for change and then evaluating efficacy via a focus on change in the proposed mechanism. Similarly, prescriptive theory (Chen, 1998) provides a framework for evaluating intervention implementation.
Existing theory on evaluating behavioral interventions, however, provides a framework for evaluating specific, targeted interventions. As discussed above, the reality of children who receive services for conduct problems in school environments typically reflects a patchwork of services and approaches, especially for those youth receiving services in classroom settings. The difficulty of existing theory on intervention evaluation in providing insight into the experiences of individuals with complex histories of service use, as is the case for youth with histories of conduct problems, suggests the relevance of employing a qualitative approach to understanding their varied experiences of intensive service users.
Qualitative Approaches to Service Usage Among Adolescents
Much of the existing research on efficacy of services for conduct problems has relied on quantitative evaluations. A critical element of evaluating a service’s efficacy, however, comes from participant feedback (Garmy et al., 2015). Qualitative research on adolescent-perceived service efficacy has identified that adolescents believe school-based mental health programs to be helpful if the services led to positive changes such as increased self-confidence, stress management skills, and positive activities, or because services were reliable, accessible, supportive, and confidential, and because they resulted in improved academic, behavioral, and social outcomes (Bains et al., 2014; Gampetro et al., 2012; Garmy et al., 2015). Qualitative approaches have also highlighted issues such as stigma, doubting confidentiality of services, poor relationships with interventionists, and systemic barriers to services increasing perceptions of service inefficacy (Huggins et al., 2016; Nabors et al., 1999).
While these existing qualitative results align with broader theoretical perspectives on program evaluation (Greenberg et al., 2005), and provide useful insight into youth perceptions of the efficacy of services for mental health problems more broadly, it is unclear if these findings reflect the experiences of youth with conduct problems. Youth with conduct problems are specifically underrepresented in qualitative research (Gampetro et al., 2012; Radez et al., 2021), and what research exists focuses on the perspectives of parents and teachers (Baker-Henningham & Walker, 2009; Drugli et al., 2008). Youth with conduct problems, moreover, may have distinct perspectives on their service experiences, as quantitative work suggests that these youth differ significantly in their perceptions than youth with other behavioral problems, or youth without these problems (Evans et al., 1995). The lack of qualitative research with youth with conduct problems about their perceptions of school-based mental-health services reflects a gap in the literature potentially limiting the ability to improve service efficacy. In particular, understanding how youth with conduct problems define efficacy and inefficacy, and their recommendations for improving services, can help to match proposed programming with the goals of youth themselves, while avoiding mitigating those factors that youth associate with inefficacy. For example, programs may be more effective if they underline goals that for youth are indicative of efficacy (i.e., improvement of interpersonal relationships), compared to if the goals underlined match more strongly with the desired outcomes of adults.
Objectives
Existing theory provides a strong framework for the identification and evaluation of intervention and prevention programs (Greenberg et al., 2005). While existing theoretical frameworks identify how to evaluate specific programs and interventions for efficacy, they are limited in determining where and why heavy service users may ultimately respond poorly to the broad range of services they receive. Defining efficacy, moreover, is central for evaluating this construct. Using a qualitative approach, the objective of this study was to understand how young adults who had been selected for recruitment based on being reflective of children who used school services for conduct problems from Québec perceived their experiences with school-based services. We focused on what participants saw as helpful and effective in the interventions they received, examining the factors they related to efficacy and/or lack of efficacy and identifying how they proposed to improve the services they had extensive experience in receiving.
Method
Participants
All parts of this study were approved by the ethics review board of the second author’s institution. Participants were recruited from an ongoing longitudinal study designed to examine the development of childhood conduct problems that included 434 children (44% girls). For the initial sample, children with conduct problems were initially recruited from eight French-Canadian school boards from four regions of Québec between 2008 and 2010 when they were between 6 and 9 years of age. All girls and one out of every four boys who were receiving services for conduct problems were invited to participate, and an additional 881 students not receiving services were also screened to account for potential biases in school referrals (and to capture youth who were more likely to eventually become service users). Participants were eligible to participate in the study based on having scores on the conduct problems or oppositional defiant problems subscales of the Child Behavior Checklist (CBCL) and the Teacher Report Form (TRF) (Achenbach & Rescorla, 2001) above the borderline clinically significant range (t-score of 65 or 93rd percentile and above) on either scale, with 75.1% of participants consenting to participate. As such, the initial sample reflected youth who were either (a) using services or (b) had behavior profiles indicative of needing services in the focal regions. Among this sample, 92.2% were White, 3% were Black/Afro-Canadian, and the remaining 4.8% were from other ethnicities (Latino, First Nations, Arab, Asian, or multiethnicity). The median annual income of their families was lower than CAN$40,000 which was well below the median annual income of Canadian families (CAN$69,860) according to the data from the 2010 census (Statistics Canada, 2015). All participants and their families were subsequently followed yearly for the next 10 years.
From this initial sample, participants had to meet multiple criteria to be eligible to participate in the current qualitative study (see Figure 1). First, in order for the sample to be relatively homogeneous regarding the initial severity of their conduct problems, participants were eligible to be interviewed for the current project if at study inception, they had a t-score of 70 and above on the DSM-oriented scale for conduct problems on the CBCL or TRF which corresponds to the clinically significant range for this scale, in line with diagnostic criteria in the DSM-5. Second, the participants also had to have participated in the most recent data collection (which occurred approximately 9 years after initial recruitment) and had to have participated at least in three other data collection periods of the longitudinal study. Given our interest in focusing on the transition to adulthood, participants were included if they were aged 17 to 21 years old at the time of the study. Next, participants who had received less than 3 years of services over the course of the 10 years they participated in the study were excluded because of concerns over their potential inability to recall service use, especially given this service usage may have been brief, limited to early childhood, or may have primarily occurred via additional support to their educators of which they may not have been aware. Service use was reported at each data collection wave by parents and teachers using a questionnaire inspired by L’Enquête Québécoise sur la Santé Mentale des Jeunes (Valla et al., 1994). Specifically, parents and teachers reported if the participant had been referred to a specialist during elementary school and/or high school for their conduct problems, with the goal of providing assistance or support to the child within the school context. This could include meeting with psychologists, psychoeducators, social workers, and/or specialized interventionists to address behavioral or other difficulties. Parents and teachers also reported if the participant was placed in a specialized classroom. In specialized classrooms, children experience a higher mandated student-teacher ratio and a slower overall learning pace than in a typical classroom. Some classrooms also have physical spaces designed for students to regulate (i.e., quiet corners). The median duration of service use in our subsample was 7 years. We created equally sized groups based on duration of service usage (more than 7 years, 3–7 years).
Steps for participant selection.
We were also interested in including the experiences of participants identified by their parents as boys (n = 19) or girls (n = 22) at the start of the study, given the potentially gendered nature of the consequences of conduct problems (Loeber & Keenan, 1994). To ensure a sample of participants who varied in both duration of service usage and gender, participants were placed into one of four roughly equal groups: women with 3 to 7 years of service usage, women with 8 years or more of service usage, men with 3 to 7 years of service usage, and men with 8 years or more of service usage. Finally, a further 40 participants were excluded because they lived outside of commuting distance for the interviewers.
Each of the interviewers was randomly assigned three participants from each of the four groups, for a total of 12 interviews. Across the five interviewers, 60 participants were assigned in total to account for refusals to participate, with the aim of obtaining 52 participants. This number was selected based on similar qualitative studies examining service use, and mental health (Moses, 2010). However, the data collection concluded after 41 individuals had been interviewed (see descriptive statistics in Table 1). This decision was initially due to the COVID-19 pandemic, which led to a cessation of in-person interviews. Subsequent analysis of the interviews, however, indicated that little to no new information or themes were being identified by the research team.
Descriptive Information on Participants (N = 41)*.
Note. * One participant had some missing demographic data.
Participants
Participants completed a question derived from the Canadian census regarding race/ethnicity where the options were White, Indigenous, South Asian, Chinese, Black/Afro-Canadian, Filipino, Latino, Arab, South-East Asian, West Asian, Korean, Japanese, or other with a fill in the blank option. Participants could endorse as many choices as they felt were appropriate. In the current sample, one individual identified as Indigenous, three individuals identified as White and Indigenous, three individuals identified as mixed-race (with one of the three specifying that they were White and Black/Afro-Canadian), one individual identified as Black/Afro-Canadian, and the remaining 32 identified as White. The majority of participants (56%) lived with at least one family member at the time of the present study. In terms of family of origin socioeconomic status, at study inception, 61% of the sample were from families with incomes below 40,000$ Canadian dollars, and 42% had parents with less than a high school degree, suggesting that they came from socioeconomically disadvantaged homes. In terms of the participants’ current income, 77% had incomes below $26,426, the one-person low-income cutoff for Canada in 2021 (Statistics Canada, 2023). Furthermore, less than 65% had completed a high school degree (despite only 39% still being high school aged at the time of the study). To summarize, the sample was majority White, came from predominantly disadvantaged socioeconomic environments, and showed indicators of both childhood and current socioeconomic disadvantage.
Interview
The research team consisted of the project director with significant expertise in conduct problems (MD), the project coordinator (APMASKED FOR REVIEW), a graduate student (MT), a postdoctoral fellow (RML), and two experienced researchers in adolescent development and service usage for youth with conduct problems (AMS & CT). All members of the research time were white and cisgender, five were women and one was a man without childhood histories of conduct problems, and four were clinicians/clinicians in training who worked with adolescent and young adult populations in school and clinical settings.
The initial semi-structured interview was developed by the research team. This interview was reviewed and modified by two clinician researchers who were affiliated with the larger quantitative longitudinal research project and who had significant experience in the delivery and evaluation of school-based services for youth with conduct problems. The interview was presented to professionals with specific expertise working with students who have severe conduct problems (such as psychologists, special educators, and psychoeducators working in schools), who provided feedback that was then incorporated into the interview. Finally, the interview was piloted with select participants, and additional modifications were made.
Interviews took approximately 60 minutes. The first questions inquired about participants’ general recollections of school-based services (“What services were available in your elementary and high school to help students?”), and gradually became more specific by addressing their own experiences of the services they received. In line with our overall objective to understand service efficacy, we asked participants direct questions about service efficacy both for themselves and others (“Do you believe these services were helpful?”). To capture a vast range of services, participants were asked about various types of services they received throughout compulsory education, such as individual interventions, group workshops, and specialized classrooms. Additionally, students were asked to give feedback on the services they had received and about how they would improve these services if given the opportunity (“How would you change this service if you could?”). The interviews were recorded, then transcribed for data analysis.
The interviews were conducted by five graduate students who had each received at least 50 hours of training on clinical interviewing over the course of their graduate training. In addition to their clinical training, the interviewers were coached for 20 hours on delivery of the interview used in this study through didactic instruction and role playing. To ensure the maintenance of the interview skills and the progress of data collection, ongoing contact between the research coordinator and the interviewers occurred throughout the interview process. The research coordinator maintained the quality of the interviews by addressing interviewers’ questions about the interview or the participants and ensured the quality of the data by communicating the progress and completion of interviews to the research team, and through reviews of the transcribed interviews.
Procedure
Interviewers contacted participants to assess their interest in an interview about their experiences with school-based services. Participants were informed that they would receive $80 Canadian dollars for their participation and that they could discontinue the interview and still receive compensation. Interviewers and participants met in participants’ homes or private locations suggested by the participant (e.g., reserved rooms in local libraries). Prior to the interviews, signed informed consent was obtained from the participants. Audio recordings were made of each interview. The interviews were in French (the first language of the majority of participants, and the language in which all participants attended school). After the interviews, demographic information was obtained from each participant using a questionnaire.
Data Analysis
Thematic analysis was used to code participant responses (Braun & Clarke, 2006). We chose this approach for several reasons, including its accessibility and theoretical flexibility, and because it fit well with both the exploratory goals of the study and the way in which the data were collected. In particular, we felt that thematic analysis was appropriate given that approaches such as grounded theory, phenomenology, or narrative research require methodological choices taken at the beginning of a study not employed by the current project (i.e., purposeful sampling, simultaneous data collection and analyses), and answer questions (i.e., the nature of phenomena experienced by the individual, the unfolding of individual life narratives, the development of novel theory) that are poorly suited to the goals of the current project (Creswell et al., 2007). Within the context of this thematic analysis, an inductive approach was employed wherein themes were developed using codes based on the interpretation of respondents’ perceptions, and not on a priori theory. We followed the steps outlined by Braun and Clarke (2006). The first step was to familiarize ourselves with the data. Four members of the team (AP, MD, AMS, CT) each read the full transcripts and then met and discussed the identification of the initial codes that reflected the young adults’ perceptions of service efficacy. Next, the first author generated the initial codes, or the smallest units that gave meaning to the data, and sorted these codes to create potential themes. Further coding of the transcripts took place with the help of NVivo to organize the codes (QSR International Pty Ltd, 2020). These codes were discussed in weekly meetings, allowing for the creation of themes and higher order explanations for patterns in the data. Additionally, we continued to review and refine themes until we were able to clearly define and name the themes. Finally, a codebook was created by AP and AMS and was presented to the remainder of the research team (MD, CT, MT, RML). In these discussions, the codebook was reviewed to evaluate how well the proposed codes captured the data. Codes were modified, reworded, added, or reorganized accordingly. Disagreements between coders were discussed until consensus. Feedback of the team was integrated, and further coding was focused on identifying excerpts that added meaning to the concepts and themes identified, in line with previous work in this area (Shenton, 2004). The last step included the final analysis, in which concise, logical, non-repetitive, and interesting themes that clearly illustrated the story emerging from the data were produced. Themes were retained if they were logical, in alignment with the objectives of the study, linked to the overall research question, but distinct from each other in their explanation of the data (Braun & Clarke, 2006; Uhls & Greenfield, 2012).
Results
Efficacy
All interviews were coded for discussion of efficacy, inefficacy, and for recommendations for improving services. To start, a verbatim was coded as relating to service efficacy if participants referenced positive change, a positive experience, or described why a service was helpful (see Table 2). As all participants were asked directly about service efficacy, all participants described services as being helpful in some way. Two broad themes were identified relating to efficacy: how participants defined efficacy, and the mechanisms of efficacy. With regards to how participants defined efficacy, we subsequently identified two subthemes: observed change and the avoidance of negative consequences.
Efficacy of Services.
Defining Efficacy: Observed Change
In line with more traditional definitions of service efficacy (Rones & Hoagwood, 2000) and with existing theory on program evaluation (Greenberg et al., 2005), over half of participants (n = 25; 60% men, 40% women) regarded the positive changes observed in their lives, or the lives of others as reflecting service efficacy. These included changes in the behavior, motivation, emotions, academic or professional path of the participant or someone they know who received services. Respondents’ descriptions suggest that services are perceived as helpful when there was marked improvement in behavior that occurred rapidly, a finding that is consistent with previous work on youth perceptions of service efficacy (Bains et al., 2014; Garmy et al., 2015; Nabors et al., 1999).
Defining Efficacy: Avoidance of Negative Consequences
Eight participants (50% men, 50% women) defined service efficacy as allowing them to avoid negative consequences of their behaviors, such as suspensions, punishment, or legal ramifications associated with their behaviors. This conceptualization of efficacy diverges from those of service providers, other adults, or youth with other mental health struggles. For instance, some participants viewed the services as efficacious in terms of helping them to deceive school officials and avoiding punishment. As expressed by one participant, “She really saved me [. . .] she saved me from suspensions there, maybe a dozen” (age 18, man). Here, the participant stipulates how receiving services (i.e., being sent to the interventionist) allowed him to avoid punishment (i.e., the suspensions) that would have typically resulted from his behavior.
Mechanisms of Efficacy: Tools and Techniques
A total of 34 participants (47% men, 53% women) specifically discussed the tools, techniques, interventions, and methods of intervening that they associated with positive change. Participants discussed the importance of reinforcement and positive experiences, physical aspects of the environment, and more tangible tools to aid with attentional difficulties. For instance, “They called it the cool-down corner, that you could go to if you were upset. If you didn’t understand something, were unable to figure out a math problem, you just asked to go sit there and the teacher would then come to see you and talk. It was really cool.” (age 19, woman). This participant discussed perceived the tools built into the classroom as well as methods of intervening as particularly helpful in addressing her behavioral needs.
Mechanisms of Efficacy: Specialized Educational Environments
Eighteen individuals (44% men, 56% women) identified specialized classrooms or schools as being particularly effective. They described the accommodations present in these educational environments as meeting their needs. For example, “[Teachers] could better answer students’ questions because there were less of us. Also, being fewer students in the class allows us to concentrate better, I think” (age 19, man). The responsiveness of interventionists and teachers, as well as a less busy and overwhelming environment because of smaller class sizes, was identified as an important element of efficacy in the context of a specialized educational environment, in that when well-executed, these classes can be helpful in meeting the needs of individuals with conduct problems. The perspective provided by these youth nuances previous work, which generally characterizes specialized classrooms as being linked with negative behavioral and academic outcomes (Powers et al., 2016).
Mechanisms of Efficacy: Relationship With Interventionist
In line with previous work, the majority of participants (n = 30, 50% men, 50% women) discussed the role of the interventionist in the efficacy of their services (Bains et al., 2014; Garmy et al., 2015). They described the interventionist’s engagement, tenacity, care, advocacy, and availability as a major determinant of service efficacy. One participant explains, “I don’t know, I would say just the fact that [the interventionist] would come see me, would give me attention, it did me a lot of good. You know, he was just there for me, I didn’t have the impression that he was just doing it because it was his job, I had the impression that he wanted to do it too” (age 20, woman).
Mechanisms of Efficacy: Self-Insight
Finally, 16 participants (31% men, 69% women) attributed their own insight and awareness of their difficulties, their need for help, or their desire to change as explaining service efficacy. This theme reflected an acknowledgment of personal characteristics and the impact on their behavior. For example, “You don’t know what you are doing is wrong, well like, [the interventionist] intervenes right away to really, you know, tell you how to fix it then you know.” (age 20, woman). Here, the participant is acknowledging the interventionist’s role in helping her identify problem behaviors and develop self-insight.
Inefficacy
A verbatim was coded as relating to service inefficacy if participants referenced an absence of change, worsening difficulties, poor long-term outcomes, or singled out some aspect to describe why a service was unhelpful (see Table 3). As all participants were asked directly about service inefficacy, almost all participants (n = 39, 46% men, 54% women) discussed inefficacy in some way. Two broad themes were identified in how participants discussed inefficacy: how individuals defined inefficacy, and the mechanisms of inefficacy. With regards to how participants defined inefficacy, we subsequently identified three subthemes: absence of change, increasing short-term emotional and behavioral difficulties, and problematic long-term outcomes. Compared to how participants described efficacy, we identified more themes in young adults’ conceptualizations of inefficacy, and these descriptions were more diverse and detailed than those highlighted in the existing literature (Bains et al., 2014; Nabors et al., 1999).
Inefficacy of Services.
Defining Inefficacy: Absence of Change
Twelve participants (42% men, 58% women) regarded an absence of change or the maintenance of their difficulties, despite receiving services, as related to service inefficacy. Although these participants reported some appreciation into the effort put into their services or recognized their necessity, these participants reported that they were unable to discern any change because of service use.
Defining Inefficacy: Increasing Short-Term Emotional and Behavioral Difficulties
Eleven participants (36% men, 64% women) stated that their difficulties increased, including worsening behavior and greater emotion regulation problems, as a direct result of receiving services. One said, “you know, like, they didn’t even try to listen to me, to try to calm me, decrease the tension. They just fed [my anger], and fed it, and it grew” (age 17, woman). Similarly, participants reported that services left them feeling increasingly overwhelmed and misunderstood. One participant commented, “Well you know, when you’re a teen and you’re already frustrated because your school year isn’t going well, then they ship you to school filled with other people like you [. . .] I learned some bad habits because of that school” (age 19, woman). This participant describes difficulties due to instability associated with service provision, hinting toward the potentially iatrogenic consequences of grouping together youth with conduct problems (Müller, 2010).
Defining Inefficacy: Problematic Long-Term Outcomes
In contrast to concurrent difficulties, we identified an additional theme of negative long-term outcomes as an indicator of inefficacy. Here, participants identified inefficacy in terms of dropout, having limited prospects, worsening symptoms, lacking skills that generalize to other contexts, and a lack of preparation for adult life. Nine participants (67% men, 33% women) reported their services were inefficacious because of negative longer-term outcomes.
Mechanisms of Inefficacy
We identified seven sub-themes in terms of how participants described why services were inefficacious, which highlighted the numerous ways youth with conduct problems perceived interventions as going awry.
Mechanisms of Inefficacy: School System or Service Itself
Sixteen participants (44% men, 56% women) explained service inefficacy as part of a larger ineffective school system. One participant succinctly states, “Apart from the system being (profanity), there’s not much to be said” (age 18, man). Many verbatims in this subtheme had similar comments indicative of a pervasive dislike and distrust of school systems and services.
Mechanisms of Inefficacy: Negative Attitudes of Young Adults About Services
Twenty participants (40% men, 60% women) identified their own attitudes, and more specifically refusal, noncompliance, lack of confidence in services, or the intensity of their difficulties as a major reason why services were ineffective. As stated by one participant, “I didn’t like it, I felt like everyone was trying to control me” (age 19, boy), suggesting that service inefficacy can be the result of a power struggle between youth with conduct problems and service providers. Verbatims in this subtheme reflected unique challenges to providing services to youth with conduct problems who tend to have more negative views about themselves or others, prompting their underutilization of services (Radez et al., 2021).
Mechanisms of Inefficacy: The Use of Inappropriate Interventions
Participants perceived the ways in which interventions were applied, and inappropriate interventions in specialized education environments, inappropriate attitudes of interventionists, and inappropriate tools and techniques in discussing service inefficacy.
Context of Specialized Environments
In line with the larger literature on the iatrogenic impact of peers with conduct problems (Müller, 2010), 20 participants (55% men, 45% women) believed specialized environments were unhelpful due to the social influence of being in close proximity to disruptive peers. Participants believed that educational environments shared with students with more severe conduct problems were detrimental to their learning. This is illustrated by a participant who stated, “you know . . . if one student becomes violent, the other becomes violent, and the other, and the other, it doesn’t stop, you know? If you decided that day you were going to learn, participate, and everyone is losing it all day in class, nothing gets done” (age 20, man). Overall, these classes were identified as ineffective both in terms of the quality of learning as well as the behavioral influences of struggling participants being grouped together.
Inappropriate Attitude of the Interventionist
In line with previous work examining service efficacy, 20 participants (30% men, 70% women) cited interventionists as an explanation for service inefficacy (Nabors et al., 1999). The interventionist’s disengagement, indifference, and lack of confidence or confidentiality contributed to participants’ perceptions of service inefficacy. This included references to interventionists being too intrusive, unkind, aggressive, or denigrating toward youth. For instance, “The judgment I felt sometimes was intense. Pft, it happened often that I overheard them saying I was messed up, that I’m this and that. Look, you leave, you close the door, you overhear them calling you a (profanity) on the phone” (age 18, man). Many participants reported that poor relationships with their interventionists were closely tied to their overall perceptions of service inefficacy.
Inappropriate Tools and Techniques
Eighteen participants (39% men, 61% women) commented on the inefficacy of tools and techniques like suspensions, detentions, overly strict rules, and punishments. One participant remarked, “It was detentions all the time, detentions, detentions, detentions, at one point I was at like 180 detentions. So, I was at like 180 detentions, how did you want me to ever go to them? Want me to come in every morning? Let it go” (age 18, man).
Mechanisms of Inefficacy: Services Do Not Respond to Perceived Needs
Fifteen participants (47% men, 53% women) discussed how the services they were offered did not correspond with their actual needs. A student stated that, “. . . let’s say like, I didn’t get a toolbox” (age 18, man), highlighting participants’ desire for a set of skills and strategies. Another participant spoke to an inability of services to explain underlying processes to youth saying, “instead of like explaining what’s going on in my head, asking if things are not going well in my life. No, they told me, ‘oh take your pills it will get better’ you know” (age 18, woman). These findings on how lack of effective tools is linked to inefficacy mirror previous work suggesting perceptions of efficacy are linked to provision of helpful tools (Bains et al., 2014; Garmy et al., 2015; Nabors et al., 1999).
Service Improvement
Finally, participants were asked how they would improve services (see Table 4), and four subthemes were identified: no suggested changes, better identification of youth needs, better adapted intervention strategies, and systematic changes to services. To start, 10 youth (60% men, 40% women) reported having no suggestions for improving services. For youth in this group, they generally reported finding services appropriate and specified that they did not have anything additional to note. The next subtheme, reported by six participants (100% women), was how services could be improved by better evaluation of youth needs. For example the statement “they (service providers) shouldn’t wait for young people to come to them, as soon as they see someone with a difficulty. . .you know, I would never have gone to them” (age 18, woman) shows how the participant suggested that it is important for schools to be more pro-active in identifying students who need additional support. Participants in this group also discussed the need for service providers to increase awareness of the available services, and for better evaluation to establish the nature and severity of individual difficulties.
Suggestions for Service Improvement.
A third subtheme, identified in the descriptions of 27 participants (27% men, 63% women), was the need for better adapted intervention strategies. For example “They don’t try to understand us, what they want is just solutions solutions, instead of listening to youth” (age 20, man) provides an example of how participants described wanting interventionists who really tried to understand their concerns. They also discussed the importance of increased confidentiality around service usage (i.e., being able to access services privately). Finally, the fourth subtheme was systemic changes to services, endorsed by 14 participants (57% men, 43% women). Participants whose recommendations were coded in this theme suggested changes to the structure of services, including increasing the number of interventionists available, shortening class periods, eliminating specialized classrooms, and creating better continuity for youth moving between services and between schools. For example, “I would include a few more students in specialized classes. To get us used to the large number of kids in regular classes, because, for me, where I was in a class of 7-8 people, then the next year there were 30 people in the class, made it so I didn’t really know what do” (age 18, man). This example shows how it was difficult to go from the specialized classroom to a regular classroom.
Discussion
Children with conduct problems often receive services for their difficulties at school, but existing research has found that these school-based services either do not diminish or exacerbate conduct problem symptoms (Dempsey et al., 2016; Morgan et al., 2010). Research focusing on service efficacy among youth is generally quantitative (Sanchez et al., 2018), and qualitative work on service efficacy has not examined the unique perspectives of youth with conduct problems with regard to their school-based service usage. Findings from this study suggest some but not complete overlap in perceptions of service efficacy among young adults with histories of conduct problems in this study and both previous theory and research on efficacy in school-based services. In particular, participants noted the complexity of the perception of specialized classroom environments of young adults with histories of conduct problems, the significant insight of young adults with histories of conduct problems in linking their own characteristics to service efficacy and service improvement, and how young adults with histories of conduct problems see services as a way of avoiding consequences, each of which are discussed in greater detail below.
Much of participant perceptions of service efficacy and inefficacy were similar to previous qualitative literature on school-based mental health services. The themes “Observed change,” “Relationship with interventionists” as well as the subtheme “Systematic changes to services” where participants indicated the need for more service providers highlighted similar perceptions of efficacy through mechanisms such as positive change observed in the life of participants, positive relationships with and characteristics of interventionist, and accessibility and availability of services (Bains et al., 2014; Garmy et al., 2015; Nabors et al., 1999). The theme “Inappropriate attitudes of interventionists” highlights the importance of relationships with interventionists for youth evaluations of services (Huggins et al., 2016; Nabors et al., 1999), while the theme “Tools and techniques” shows how young adults valued positive reinforcement, positive interactions with their educational staff, an accommodating environment, and tangible tools (e.g., fidget toys) embedded into the services (K. Hoagwood & Erwin, 1997; Nabors et al., 1999). Some of the inefficacy themes also reflect previous work, such as the “Inappropriate tools & techniques” theme, where young adults discussed negative and aversive experiences and described services as unhelpful if they were harsh, punitive, and overly strict or rigid, strategies which may be particularly ineffective for youth with conduct problems (Frederickson et al., 2013; Mayer, 2001).
Interesting complexity, however, was observed regarding attitudes toward specialized classrooms. Previous research illustrates that specialized classrooms are associated with dropout among youth with conduct problems (Powers et al., 2016). These findings were echoed by the participants in this study, half of whom expressed similar dissatisfaction with specialized classes or schools. In particular, when asked how to improve services, students specifically addressed the elimination of specialized classrooms. Less commonly reflected in the literature, the present findings also show that almost half of the participants linked specialized educational environments to efficacy. The majority of youth who linked specialized classrooms with efficacy were long-term service users who reported benefits of specialized classrooms. Their experiences may reflect the proper implementation of a positive learning environment, reduced class size, nurturing relationships with teachers and interventionists, and tangible tools and skills, as reflected by youth’s perceptions of service efficacy. These differing perceptions could also reflect student’s cognitive abilities or learning difficulties. Previous research found the negative effects of specialized educational environments are amplified for students with conduct problems and normative cognitive functioning compared to those with conduct problems and cognitive impairment (Powers et al., 2016).
Youth spoke of how an awareness of their difficulties or their own poor attitudes toward services affected their perceptions of service efficacy. This ability to reflect on their own engagement may suggest the utility of the retrospective approach of the current project, where distance from their experiences may allow youth more perspective in terms of evaluating themselves as service users. These themes “Self-insight” and “Attitudes of young adults about services” reflected significant understanding in how their personal dispositions could influence their perceptions of service efficacy. Similar insights have been briefly discussed by Bains et al. (2014) and Radez (2021), however, the youth in the present study went further by acknowledging the ways in which negative attitudes hindered their progress. Relatedly, when asked how services may be improved, some participants discussed the importance of letting youth seek out services for themselves. Taken together, these findings highlight the importance of increasing service “buy-in,” prior to service reception. Greater “buy-in” may be achieved by including participants in the promotion of school-based mental-health services, thereby allowing the familiarization and de-stigmatization of services, or by having student ambassadors who can highlight potential benefits of services usage.
Most participants whose responses reflected the themes of “Self-insight” and “Attitudes of young adults about services” were women. Specifically, women made up the majority of participants who discussed their desire to change as an active ingredient in service efficacy, as well as the majority of participants who reported that they were partially responsible for service inefficacy because of their own poor attitudes. This may reflect a gender difference in which women have more insight into their own role in influencing service efficacy (Nabors et al., 1999). Nabors et al. (1999) found similar gender differences when examining perceptions of service efficacy, where men were more outcome-oriented, but women were more interested in gaining insights through their services.
Finally, the theme “Avoidance of negative consequences” where participants viewed services as helpful because they perceived them as instrumental in avoiding negative consequences such as detentions, suspensions, or other punishments reflects an experience more likely to be unique to youth with conduct problems. While not observed among the majority of participants, this theme suggests the extent to which youth with conduct problems have negative and punitive interactions in the school system (Mayer, 2001), such that receiving services may serve as a respite from these aversive interactions. This finding may reflect the reduced effectiveness of punishment in youth with conduct problems (Frederickson et al., 2013). A broader literature that underscores the advantages of prioritizing de-escalation practices (i.e., comprehensive behavior management strategies designed to prevent or reduce conflict, approaches that involve modifying the environment or providing sensory tools designed to improve youth self-regulation) over practices that focus on restraint and seclusion (Verret et al., 2019). Indeed, many techniques that participants described as being effective (having a calming area, being provided access to physical manipulation tools) reflect these de-escalation approaches (LeBel et al., 2014). And, while the literature suggests that a focus on de-escalation may reduce the need for seclusion and restraint, it is important to acknowledge that seclusion and restraint may remain necessary options in contexts where the youth is a danger to themselves or others (Verret et al., 2019).
Strengths & Limitations
While the present study offers novel insight into how youth with conduct problems understand efficacy and inefficacy in their service usage, findings should be interpreted considering some limitations. First, while our interviews permit us to understand how youth themselves saw service efficacy, we did not include the perceptions of parents, teachers, or interventionists. Although understanding youths’ perceptions is important, future work may develop these findings by incorporating other perspectives. Furthermore, future work with interventionists may help in identifying pertinent ways in which youth perspectives on treatment planning could be evaluated, and fruitful points where this kind of information could be included in program planning. We also focused on youths’ retrospective perceptions of efficacy and inefficacy. While this approach allowed for insights not possible from concurrent evaluations in terms of how youth linked services to their long-term outcomes, we would expect that a study focusing on concurrent service use would identify some different themes. Second, the majority of the participants in this sample were White and from socioeconomically disadvantaged backgrounds. Indeed, literature consistently suggests that racialized youth are over-represented in services for behavioral problems within the United States, and while similar data does not exist in Canada, research does document the racism, and particularly racism with regards to disciplinary practices within Canadian schools (Codjoe, 2001; Ruck & Wortley, 2002), as well as non-representative referral rates for other types of services among racialized children more broadly in Canadian contexts (Ontario Human Rights Commission, 2018). The makeup of the sample precluded a detailed understanding of how racialized status or socioeconomic disadvantage stratified the way in which youth perceived efficacy of service use. Future work focusing particularly on subgroups of racialized youth may help to further nuance the current findings and inform intervention and prevention approaches. Finally, the current study focused on youth perspectives of efficacy and inefficacy as a qualitative component of a larger project exploring the links between conduct problems and service usage over time. In particular, we focused on individuals with three or more years of service use, who may differ from individuals whose interactions with services are shorter. We did not focus on indicators of program efficacy, although previous research with this sample has generally linked service usage to limited behavior change (Déry et al., 2004, 2023).
Implications & Future Directions
Findings highlighted both similarities and differences in how youth who used services for conduct problems understood efficacy and inefficacy, and the recommendations they had for improving services. By focusing on the recommendations of youth themselves, these findings have potential implications for school-based mental-health services developed for youth with conduct problems, and can inform the planning, implementation, and maintenance of these services for youth. To start, many participants demonstrated keen insight into their experiences with services and were able to provide concrete feedback that could be directly applied to planning and ameliorating services designed to address these pernicious behavioral issues. For instance, given the importance of authentic and high-quality relationships with service providers, these findings suggest the relevance of prioritizing the building of meaningful relationships with young people with conduct problems. Many participants pointed out the value of increasing the number of service providers available to youth. Second, the young adults in this study appreciated tools and techniques such as positive reinforcement in their interventions, and this may have contributed to their perceptions that the services were more helpful. Embedding these tools and techniques within mental health services could benefit youth with conduct problems. Finally, including youth input into treatment goals and objectives may allow youth to be involved in their services, decrease defiance and refusal, and increase mastery and engagement within services. This study revealed participants were more likely to view services as unhelpful if they had negative opinions about them or if they perceived that the services did not meet their needs. They were particularly likely to critique required services as well as the use of specialized classrooms. Avoiding these outcomes could be achieved through discussions between interventionists and youth about each of their treatment goals and could involve collaboration in creating an action plan. Including perspectives of young people who use services when assessing service efficacy and inefficacy can provide vital information in improving services for youth with conduct problems.
Footnotes
Acknowledgements
We would like to acknowledge the team of research assistants who were essential in conducting interviews, as well as Mylène Villeneuve-Cyr who supervised the data collection. We are also grateful to our participants who generously gave their time to participate in this project.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded by a grant from the Social Sciences and Humanities Research Council of Canada (430-2019-00608), as well as by a Canada Research Chair (CRC-2016-00007) to the second author.
