Evaluating the differential effectiveness of social influence and personality-targeted alcohol prevention on mental health outcomes among high-risk youth: A novel cluster randomised controlled factorial design trial

Abstract

Objective:

This study examined the secondary mental health outcomes of two contrasting alcohol prevention approaches, whereby one intervention targets common underlying personality risk for alcohol use and mental health problems (Preventure) and the other targets alcohol- and drug-related behaviours and cognitions (Climate Schools).

Methods:

A 2 × 2 cluster randomised controlled factorial design trial was conducted in 26 Australian schools randomised to the following 4 conditions: Climate Schools (n = 6), Preventure (n = 7), combined Climate Schools and Preventure (CAP; n = 6) or treatment as usual (TAU; n = 7). Participants completed questionnaires at baseline, 6, 12, 24 and 36 months post-baseline including the Brief Symptom Inventory anxiety and depression scales and hyperactivity and conduct scales of the Strengths and Difficulties Questionnaire. Analyses focused on students who were at high-risk based on personality traits (n = 947; M_age = 13.3). The effectiveness of each approach in reducing symptoms of internalising and externalising problems was assessed using multi-level mixed effects analysis.

Results:

Main effects for each intervention relative to not receiving that intervention revealed significant main effects of Preventure in reducing anxiety symptoms (d = −0.27, 95% confidence interval [CI] = [−0.53, −0.01], p < 0.05) and a marginal effect in reducing depressive symptoms (d = −0.24, 95% CI = [−0.49, 0.01], p = 0.06) over 3 years. Interaction effects revealed that when delivered alone, Preventure significantly reduced conduct problems (d = −0.45, 95% CI = [−0.78, −0.11], p < 0.05) and hyperactivity symptoms (d = −0.38, 95% CI = [−0.70,−0.07], p < 0.05) compared to TAU.

Conclusion:

This study is the first to report the effectiveness of personality-targeted alcohol prevention in reducing internalising and externalising symptoms relative to an active control, providing evidence in favour of its specificity in preventing concurrent substance use and mental health problems among high-risk youth.

Keywords

Personality selective prevention internalising externalising alcohol adolescence school mental health factorial randomised controlled trial

Introduction

Substance use and mental disorders are the leading causes of disability among young people, accounting for one quarter of all Years Lived with Disability in those aged up to 24 years old (Erskine et al., 2015). Population surveys across the world consistently report high prevalence rates of these disorders among young people (Polanczyk et al., 2015) with 23.4% of US adolescents expected to experience a mental disorder within a 12-month period (Kessler et al., 2012) and 20.4% reporting that they had been drunk in the past year (Johnston et al., 2018). It is estimated that between 50% and 70% of adults with a mental disorder experienced onset of their disorder in their youth (Kessler et al., 2005). Given that young people aged 0–24 years make up almost half of the world’s population (United Nations, 2011), it is imperative that we focus on improving mental health trajectories of our youth.

Effective prevention has the greatest chance of averting the substantial burden of disease attributed to substance use and mental disorders (Erskine et al., 2015), and schools offer the ideal location to deliver such prevention (Botvin, 2000). The prevalence of many substance use and mental health disorders increases substantially during the period of adolescence, meaning intervention in early adolescence has the potential to prevent the onset of these disorders (McGorry et al., 2011). While many prevention trials document the capacity of interventions to modify risk factors, relatively few studies include adequately controlled designs or demonstrate beneficial effects on substance use behaviours and mental health symptoms (Faggiano et al., 2014; Foxcroft and Tsertsvadze, 2011b; Stockings et al., 2016; Werner-Seidler et al., 2017). Assessing proximal outcomes (e.g. attitudes) is not adequate as they do not always lead to changes in target health behaviours and ramifications of interventions are not always apparent unless evaluated over the longer term.

The most common approach to preventing substance use and mental disorders within schools is universal prevention, delivered to all individuals regardless of risk. Universal programmes that promote knowledge about substance use harm and teach normative education and drug-refusal skills have been shown to yield small to modest effects for the prevention of alcohol and other drug use (Faggiano et al., 2014; Foxcroft and Tsertsvadze, 2011b). Universal programmes that teach mental health literacy and aim to enhance cognitive and behavioural skills for emotion management have also been shown to yield small reductions in anxiety and depression symptoms (Stockings et al., 2016; Werner-Seidler et al., 2017). Furthermore, prevention programmes that have a single focus (e.g. alcohol or depression) have not typically been shown to produce protective effects in other domains (Rohde et al., 2015), requiring that multiple programmes be administered to impact on different behavioural targets. These modest effects, coupled with competing demands in the school curriculum, highlight the need to improve both the impact and efficiency of school-based prevention.

Recognising a large literature on the role of personality and childhood temperament factors as common underlying risk factors for both substance use and mental disorders (Conrod and Nikolaou, 2016), one solution is to target underlying shared risk factors. One intervention that aims to do this is the Preventure programme, which targets high-risk youth with one of the four personality traits linked to substance use and psychopathology: (1) hopelessness (H; low mood, feelings of worthlessness and negative beliefs), (2) anxiety sensitivity (AS; fear of anxiety-related physical sensations), (3) impulsivity (IMP; rapid decision-making and action and poor response inhibition) and (4) sensation seeking (SS; elevated need for stimulation and intolerance to boredom). Based on the principles of cognitive behavioural therapy and motivational interviewing, Preventure teaches young people personality-specific cognitive and behavioural coping skills to promote mental health and prevent the use of maladaptive coping strategies such as alcohol and other drug use over the adolescent years. The effectiveness of Preventure in preventing adolescent drinking has been demonstrated in multiple randomised controlled trials (RCTs) worldwide (Conrod, 2016). Most recently, an Australian evaluation of Preventure demonstrated significant prevention effects for drinking, binge drinking (5+ standard drinks) and alcohol-related harms, up to three years following the interventions for high-risk youth (Newton et al., 2016b). This trial assessed the effectiveness of both Preventure and Climate Schools, a universal, online intervention to prevent alcohol and cannabis use. Preventure and Climate Schools showed equivalent effectiveness in reducing population-level drinking and binge drinking rates (despite Preventure being a selective programme that only delivered intervention to 45% of the population) and Preventure was shown to be superior than the universal intervention in reducing alcohol-related harm, likely due to its focus on factors that lead high-risk youth to misuse alcohol (Newton et al., 2016b; Teesson et al., 2017). In addition, a trial of Preventure in the United Kingdom demonstrated improved mental health outcomes in high-risk youth with significant reductions reported on depressive, anxiety and conduct symptoms up to two years following the interventions (Conrod, 2016).

To date, no study has reported the differential effectiveness of personality-targeted interventions for alcohol misuse relative to an active intervention on mental health outcomes. Furthermore, demonstrating superiority of this intervention on such outcomes relative to universal, alcohol-focused prevention will provide evidence on the validity of the conceptual models guiding these interventions. Finally, understanding their comparative, additive and interacting effects will help guide implementation and resource planning for decision-makers.

The primary aim of this research is to rigorously examine the differential effects of two evidence-based alcohol prevention programmes, Preventure and Climate Schools, on internalising and externalising symptoms over three years. To do this, a cluster RCT with a 2 × 2 factorial design compared the main effects of each intervention relative to the combined effect of the alternative intervention and treatment as usual (TAU), making this a highly conservative ‘placebo’-controlled evaluation of two evidence-based programmes. We hypothesised that the personality-targeted Preventure intervention would be more effective than TAU and universal prevention, in reducing the internalising and externalising symptoms of anxiety and depression (assessed by the Brief Symptom Inventory [BIS]) and hyperactivity and conduct problems (assessed by the Strengths and Difficulties Questionnaire [SDQ]). A secondary aim was to test potential interactions between the interventions over the three-year period.

Methods

Study design

The sample was derived from a four-arm cluster RCT designed to investigate the effectiveness of combined universal and selective school-based prevention for alcohol misuse (Newton et al., 2012). In September 2011, 190 Australian schools (including Government, Independent and Catholic) were randomly selected from a list of schools in New South Wales and Victoria and invited to participate. In February 2012, schools were randomised to one of the following four groups: the selective Preventure group, the universal Climate Schools group, the Climate Schools and Preventure (CAP) group or the Control group (TAU). In total, 2190 Year 8 students (aged 13–14 years old) from 26 schools¹ participated in the trial and behaviours were assessed at an individual level with a structured self-report questionnaire on five occasions at baseline (pre-intervention), immediately post-intervention (6–9 months post-baseline) and 12, 24 and 36 months post-baseline (Newton et al., 2012).

The primary outcomes of the trial in relation to reducing uptake and harmful use of alcohol have been previously published, demonstrating effectiveness of the universal, selective and combined programmes in preventing alcohol use and misuse, however, with no advantage of combining the universal and selective interventions (Newton et al., 2016b; Teesson et al., 2017). Effect sizes for the alcohol use outcomes for the Climate Schools, Preventure and combined interventions compared to controls ranged from d = −0.21 to d = −0.56, and Number Needed to Treat (NNT) values ranged from 6 to 12, comparing favourably to other alcohol prevention programmes (Faggiano et al., 2014; Foxcroft and Tsertsvadze, 2011b).

Participants

Figure 1 provides an overview of participant recruitment, consent, screening and retention. Year 8 students (13–14 years old) attending school in February 2012 were invited to participate in the study. The research protocol (Newton et al., 2012), including informed consent procedures, was approved by the University of New South Wales Human Ethics Committee (HREC 11274), the Sydney Catholic Education Office and the New South Wales Department of Education and Training. This trial is registered with the Australian New Zealand Clinical Trials Registry (ACTRN12612000026820). All data were deidentified. Only students who provided written informed consent and received parental consent were eligible to participate.

Figure 1.

CONSORT figure for participant flow in the CAP Study, at baseline and immediate post-test, 12, 24 and 36 months follow-up.

Randomisation and masking

Blocked randomisation was conducted by an external researcher using Research Randomiser (www.randomiser.org) with schools randomly assigned to one of the four conditions with a 1:1:1:1 ratio: Control, Preventure, Climate Schools or CAP. Participants, investigators and teachers were not blinded to intervention status.

Screening

All students completed the Substance Use Risk Profile Scale (SURPS) at baseline, a 23-item questionnaire assessing four dimensions of personality: sensation seeking (SS), impulsivity (IMP), anxiety sensitivity (AS) and hopelessness (H) (Woicik et al., 2009). The SURPS has been validated in Australia (Newton et al., 2016a). Students scoring one standard deviation above their school mean on any subscale were categorised as high risk and were assigned to the personality group where they deviated most from the mean based on z scores.

Procedures

Personality-targeted selective prevention

Preventure is a personality-targeted, selective prevention programme, which has been modified for Australian youth (Barrett et al., 2015). The Preventure programme comprised two 90-minute group sessions delivered one week apart. Only students categorised as high risk were invited to participate. Invited students were told that the workshops were coping skills workshops where participants would be ‘exploring [their] personality and how [they] can channel [their] strengths to get what [they] want in life’. To minimise potential stigma, there was no mention of substance use or ‘high-risk’ personality styles. Students were called from class to attend the group sessions. Separate sessions were tailored to each of the four personality dimensions assessed by the SURPS. In the first session, psychoeducational strategies were taught to educate students about the target personality style and problematic coping behaviours. Students were encouraged to set goals and introduced to the cognitive behavioural model. In the second session, students were encouraged to identify and challenge personality-specific cognitive thoughts that lead to problematic behaviours. The interventions were provided by a qualified facilitator (registered Clinical Psychologists) and cofacilitators (Bachelor of Psychology Honours degrees), trained according to the Preventure training protocol. Further details on the Preventure programme are described elsewhere (Conrod, 2016; Newton et al., 2016b). Students in schools allocated to receive Preventure continued to attend regular Personal Development Health and Physical Education (PDHPE) lessons when not attending group sessions.

Universal prevention

The universal Climate Schools programme is an Internet-based prevention programme based on a social influence model. It comprises 12 × 40-minute lessons aimed at reducing substance use and related harms and is delivered as part of the PDHPE curriculum. Each lesson comprised a 20-minute online cartoon component completed individually by students, followed by a 20-minute group or class activity delivered by the teacher, which reinforces the information in the cartoons and allows interactive communication. Teachers were provided with a hard-copy manual, online access to the activities, implementation guidelines, links to the education syllabus and teacher and student summaries for each lesson. Lesson contents are described further elsewhere (Newton et al., 2012). Previous research has demonstrated the Climate Schools programme to be effective in reducing alcohol consumption, reducing harmful use of alcohol and cannabis use and improving alcohol- and cannabis-related knowledge up to two years following the interventions (Newton et al., 2010, 2018; Teesson et al., 2017).

Control – TAU

Control schools completed their PDHPE syllabus over the course of the year, which mandates that alcohol, drug and health education is taught to all Year 8 students. All control schools reported implementing some form of alcohol and other drug education, with the number of lessons ranging from two to ten (average length of lesson: 62 minutes) and 57% reported using computers or the Internet to teach these lessons. The main content areas covered were types of drugs, effects of alcohol, laws, decision-making, risk-taking behaviours and the influence of peers and the media.

Measures

A self-report questionnaire was administered to all students in a classroom setting at baseline and at the four subsequent follow-up assessments. Student responses were linked over time using a unique identification code to ensure confidentiality.

Demographic information

Participants were asked to self-report sociodemographic information including age, gender and country of birth.

Internalising symptoms

The depression and anxiety subscales of the Brief Symptom Inventory (BSI) measured anxiety and depression symptoms. Participants were asked to report how much they had experienced symptoms of anxiety (e.g. ‘feeling tense’) and depression (e.g. ‘feeling lonely’) in the past six months on a five-point scale (0 = not at all to 4 = often). Scores were summed separately for the depression and anxiety subscales, yielding total scores between 0 and 24. The BSI has been widely used worldwide and has well-established psychometric properties (Derogatis and Melisaratos, 1983). Cronbach’s alphas in the current sample at baseline were 0.87 for anxiety and 0.93 for depression.

Externalising symptoms

Externalising symptoms were measured through the conduct and hyperactivity subscales of the Strength and Difficulties Questionnaire (SDQ). Participants were asked to respond to items such as ‘I fight a lot, I can make other people do what I want’ on a three-point Likert scale (0 = not true to 2 = certainly true) in relation to ‘how things had been’ in the previous six months. The SDQ is widely used for measuring child and adolescent mental health problems with good psychometric properties and international validation (Goodman et al., 2010). Cronbach’s alpha was 0.68 for conduct problems and 0.77 for hyperactivity at baseline in the current sample. Poorer internal consistency for the conduct problems subscale is consistent with other studies (Dadds et al., 2003; Van Widenfelt et al., 2003).

Sample size calculations

Sample size calculations for the trial were reported in the published trial protocol (Newton et al., 2012) and based on methods for sample size calculation in longitudinal cluster RCTs (Heo and Leon, 2009). The aim was to achieve 80% power to detect a standardised mean difference of 0.3 between groups at the final follow-up, within the high-risk participants. The desired sample size in each group was 192 high-risk participants across six schools. Since the high-risk participants were expected to make up 40% of all participants, 480 total participants were required in each group. While the a priori sample size calculations were based on the primary alcohol outcomes of the trial, as we have similar prevalence across alcohol and other mental health symptoms and the expected effect sizes are similar, the study was considered to be sufficiently powered (80%) to detect effects on all four outcomes at the p < 0.05 level.

Statistical analysis

All analyses were conducted using Stata version 15.0. Preliminary descriptive statistics were obtained and normality assumptions were examined. Positive skew was indicated for the anxiety and depression variables. Data transformation achieved little improvement, thus untransformed data were retained in the interests of interpretability, and given evidence that linear mixed models are robust to violations of normality (Jacqmin-Gadda et al., 2007). Potential differences between participants who completed follow-up assessments and those lost to follow-up were assessed using chi-square tests and t-tests. An alpha level of 0.05 was used for all comparisons.

Primary analyses involved multi-level mixed effects analysis for repeated measures (MMRM), a rigorous method for modelling change over time. The approach has a number of advantages over traditional approaches, including better treatment of missing data, modelling of variance at the individual level and accounting for non-independence of observations in clustered data (Gueorguieva and Krystal, 2004). All models were estimated with gender as a covariate, as gender was expected to be associated with the outcomes and there were differences in the proportion of males and females between groups at baseline. Analysis was based on the intention-to-treat principle, using all available measurements from participants and according to their allocated group. Missing data were accommodated based on all available information using maximum likelihood (ML) estimation. The effectiveness of each intervention was assessed within a 2 × 2 factorial design, with two indicator variables representing implementation of Preventure or not (Preventure and CAP compared to Climate and TAU) and Climate Schools or not (Climate Schools and CAP compared to Preventure and TAU). This approach assumes that the effect of each intervention is unaffected by the other, and thus can be biased when there is an interaction between intervention effects (Dakin and Gray, 2017). Therefore, the interaction combining these two indicator variables and the pure effects of each intervention (Climate Schools only and Preventure only groups compared to TAU) were examined in secondary analyses. Models were first estimated without the interaction term to examine the independent effects of Preventure and Climate Schools (adjusted for the effect of the other); the interaction term was then added. Time was coded as a continuous variable representing six-month blocks from baseline. The effects of the interventions on change in outcomes over time were assessed through Group × Time interaction terms. These models take into account individual differences at baseline, estimating participant-specific starting points and change over time from these baseline levels.

The best-fitting growth function (linear or quadratic) for each variable was determined using likelihood-ratio tests to compare them in unconditional models. An autoregressive covariance structure modelled the relationship between repeated observations for each individual. Given the clustered nature of the data (i.e. students clustered within schools), three-level models were estimated with repeated measurements at the individual level nested within the school level. Random intercepts and slopes at the individual level modelled individual differences in initial symptom level and change over time. Random intercepts at the school level modelled interschool variability in student symptom severity. All models were estimated using the Stata xtmixed command, and the marginal means were estimated using the margins command.

Results

Study recruitment, retention and participant characteristics

The CONSORT diagram (Figure 1) summarises participant flow and retention rates through the study. A total of 947 students were assessed as being ‘high-risk’ according to the SURPS at baseline. This sample is referred to as the ‘high-risk’ sample and used for all analyses. The mean age of students at baseline was 13.3 years (SD = 0.51). In total, 84.4% of the participants were born in Australia, 8.6% were born in another English-speaking country and 7.0% were born in a non-English speaking country. The median school Index of Community Socio-Educational Advantage (ICSEA) score (a measure of socioeducational advantage) was 1129, above the Australian average (Australian Curriculum, Assessment and Reporting Authority [ACARA], 2014). In line with the CONSORT 2010 guidelines, we did not conduct significance tests of baseline covariate differences between intervention groups; however, some large differences were noted. Gender differences were apparent between groups, with a higher proportion of male participants in the Preventure (81.2%) and CAP (77.0%) groups compared to the Control (36.2%) and Climate Schools (36.3%) groups.

The number of participants who completed the follow-up surveys were 711 (75%) at 6 months, 768 (81%) at 12 months, 703 (74.2%) at 24 months and 650 (69%) at 36 months. Attrition occurred when students were absent from school. However, as participants could still complete later surveys after an absence, 894 participants (94.4%) completed at least 1 follow-up survey. Attrition did not differ significantly by the intervention group, χ²(3) = 4.66, p = 0.198, with loss to follow-up rates of 3.0% in the control group, 5.8% in the Climate Schools only group, 6.3% in the CAP group and 7.4% in the Preventure only group. Attrition analyses were conducted to assess baseline differences between participants who were lost to follow-up compared to those who completed one or more follow-up surveys. Participants who completed no follow-up assessments reported more conduct problems, M = 3.4 vs 2.6, t (657) = 2.07, p = 0.039, but did not differ from those who remained on baseline levels of depression, M = 6.2 vs 5.2, t (650) = 0.98, p = 0.326, anxiety, M = 3.2 vs 3.0, t(650) = 0.378, p = 0.706, or hyperactivity, M = 4.8 vs 4.2, t(657) = 1.18, p = 0.237. Implementation fidelity and participant evaluation of the CAP programmes can be found in Supplementary material.

Multilevel modelling

Table 1 presents the raw means for each outcome by time and group. Estimated coefficients from the fitted multilevel models are shown in Table 2. Preliminary model testing indicated the best-fitting models for all outcomes included only a linear change term. Intercept estimates indicated no significant baseline differences between groups that received Preventure or Climate Schools compared to groups that did not on, any of the outcome variables (anxiety, depression, hyperactivity or conduct).

Table 1.

Raw outcome means for each group at each measurement occasion.

Outcome	Time	TAU (n = 236))	Preventure (n = 202)	Climate ( n = 240)	CAP (n = 269)
BSI Depression, M (SD)	Baseline	5.26 (5.42)	6.08 (6.72)	6.39 (5.66)	4.29 (5.49)
	6 months	4.88 (5.64)	5.83 (7.14)	7.14 (7.24)	4.12 (5.02)
	12 months	5.66 (6.22)	6.32 (7.39)	7.55 (7.00)	4.13 (6.46)
	24 months	6.34 (6.47)	4.46 (6.43)	7.44 (6.98)	4.75 (6.00)
	36 months	5.88 (6.00)	4.1 (5.60)	7.49 (6.07)	4.12 (5.15)
BSI Anxiety, M (SD)	Baseline	2.76 (3.27)	3.55 (4.28)	3.78 (3.79)	2.51 (3.36)
	6 months	2.81 (3.73)	3.34 (4.29)	4.21 (5.02)	2.15 (3.33)
	12 months	2.75 (3.66)	3.62 (4.72)	3.67 (4.19)	2.47 (4.06)
	24 months	3.35 (4.20)	2.47 (4.03)	3.86 (4.26)	2.53 (3.71)
	36 months	3.42 (4.15)	2.39 (3.65)	4.4 (4.18)	2.25 (3.15)
SDQ Conduct Problems, M (SD)	Baseline	1.98 (1.70)	3.42 (2.31)	3.07 (2.04)	2.57 (2.15)
	6 months	2.28 (1.88)	3.27 (2.18)	3.49 (2.34)	2.38 (2.02)
	12 months	2.35 (1.94)	3.19 (2.42)	2.62 (1.80)	2.36 (1.96)
	24 months	2.38 (1.87)	2.69 (2.30)	2.9 (1.98)	2.42 (1.99)
	36 months	2.07 (1.87)	2.58 (2.30)	2.57 (1.67)	2.53 (2.20)
SDQ Hyperactivity, M (SD)	Baseline	3.92 (2.32)	4.86 (2.42)	4.67 (2.49)	4.04 (2.55)
	6 months	4.29 (2.35)	4.36 (2.20)	4.76 (1.85)	3.93 (2.42)
	12 months	4.32 (2.24)	4.82 (2.59)	4.57 (2.26)	3.98 (2.38)
	24 months	4.57 (2.36)	4.11 (2.52)	4.72 (2.07)	4.19 (2.46)
	36 months	4.24 (2.38)	4.14 (1.98)	4.86 (1.99)	4.02 (2.40)

TAU: treatment as usual; CAP: climate schools and preventure; BSI: Brief Symptom Inventory; M: mean; SD: standard deviation; SDQ: strengths and difficulties questionnaire.

Table 2.

Coefficients, effect sizes and CIs from multilevel models assessing change in internalising and externalising symptoms over time, to examine independent effects of Preventure, Climate and the combination, compared to health education as usual (no intervention control).

	Linear change: baseline to 36 months			Effect size at 36 months
Outcome	b	95% CI	p	d	95% CI
BSI Depression (n = 693)
Preventure vs no	−0.23	[−0.47, 0.13]	0.064	−0.24	[−0.49, 0.01]
Climate vs no	0.12	[−0.12, 0.36]	0.336	0.12	[−0.13, 0.38]
Interaction Model
Interaction: Climate and Preventure	0.33	[−0.16, 0.82]	0.184	0.34	[−0.16, 0.85]
Preventure only vs no intervention	−0.40	[−0.75, −0.05]	0.024	−0.41	[−0.77, −0.06]
Climate only vs no intervention	−0.05	[−0.40, 0.30]	0.776	−0.05	[−0.42, 0.31]
BSI Anxiety (n = 692)
Climate and Preventure vs no	−0.16	[−0.32, −0.01]	0.041	−0.27	[−0.53, −0.01]
Climate vs no	0.03	[−0.13, 0.19]	0.715	0.05	[−0.21, 0.31]
Interaction Model
Interaction: Climate and Preventure	0.23	[−0.09, 0.54]	0.162	0.37	[−0.15, 0.89]
Preventure only vs no intervention	−0.28	[−0.51, −0.06]	0.015	−0.46	[−0.84, −0.09]
Climate only vs no intervention	−0.09	[−0.31, 0.14]	0.457	−0.14	[−0.52, 0.23]
SDQ Conduct Problems (n = 693)
Preventure vs no	−0.03	[−0.12, 0.05]	0.444	−0.09	[−0.33, 0.14]
Climate vs no	0.02	[−0.06, 0.11]	0.586	0.07	[−0.17, 0.30]
Interaction Model
Interaction: Climate and Preventure	0.24	[0.08, 0.41]	0.004	0.69	[0.22, 1.16]
Preventure only vs no intervention	−0.16	[−0.27, −0.04]	0.009	−0.45	[−0.78, −0.11]
Climate only vs no intervention	−0.10	[−0.22, 0.02]	0.101	−0.28	[−0.62, 0.06]
SDQ Hyperactivity (n = 693)
Preventure vs no	0.22	[−0.26, 0.71]	0.366	−0.18	[−0.40, 0.04]
Climate vs no	0.05	[−0.05, 0.14]	0.323	0.11	[−0.11, 0.33]
Interaction Model
Interaction: Climate and Preventure	0.16	[−0.02, 0.34]	0.077	0.40	[−0.04, 0.84]
Preventure only vs no intervention	−0.16	[−0.29, −0.03]	0.017	−0.38	[−0.70, −0.07]
Climate only vs no intervention	−0.04	[−0.17, 0.09]	0.558	−0.08	[−0.35, 0.18]

CI: confidence interval; BSI: brief symptom inventory; SDQ: strengths and difficulties questionnaire.

The reference category for each main effect was those not receiving that intervention. Main effects were obtained from a model without the interaction; each intervention adjusted for the other. Time was coded in terms of 6-month blocks from baseline, so estimates represent change in symptom scores over 6 months. b = unstandardised coefficient on the scale of the original response. d = effect size (Cohen’s d) at 36 months, calculated by dividing the predicted effect by the baseline standard deviation of the response. n = number of participants included in the analysis. Participants were excluded if all their responses were missing for an outcome.

Internalising symptoms

For anxiety symptoms, the combined effect of receiving Preventure or Climate Schools compared to not receiving these interventions is shown in Table 2 and Figure 2. Students in groups receiving Preventure reported significantly lower growth in anxiety symptoms over time compared to students in groups who did not (d = −0.27 at 36 months). The Climate Schools intervention did not significantly affect students’ rate of change in anxiety over time. A second model including a Preventure by Climate Schools interaction term showed that the effect of each intervention was unaffected by combining with the other intervention (p = 0.162).

Figure 2.

Model predicted internalising symptom scores (BSI anxiety and depression) at each measurement occasion for each intervention group.

For depression symptoms, the combined effect of receiving either Preventure or Climate Schools compared to not receiving these interventions is shown in Table 2 and Figure 2. Students in groups receiving Preventure reported lower depression symptoms over time compared to those who did not (d = −0.24 at 36 months); however, this effect did not reach statistical significance (p = 0.06). Receiving the Climate Schools intervention did not significantly affect students’ rate of change in depressive symptoms. A second model including a Preventure by Climate Schools interaction term indicates that the effect of Preventure was unaffected when combined with Climate Schools (p = 0.184). Although this interaction term was not statistically significant, the interaction model results suggested there was a larger and significant intervention effect on depression when the Preventure only group was compared to the TAU group (d = −0.41 at 36 months; see Figure 2).

Externalising symptoms

As shown in Table 2, there was no evidence that rate of change in conduct problems was affected by receiving Preventure or receiving Climate Schools. A second model including a Preventure by Climate Schools interaction term showed that the combined intervention (CAP group) had a significantly different effect on the rate of change compared to either intervention alone (p = 0.004). The marginal means (Figure 3) show decreasing conduct problems in students who received Preventure only, and this group showed a significantly different rate of change compared to the no intervention group (d = −0.46 at 36 months).

Figure 3.

Model predicted externalising symptom scores (SDQ hyperactivity and conduct problems) at each measurement occasion for each intervention group.

As shown in Table 2, the rate of change in hyperactivity symptoms was not significantly affected by receiving either Preventure or Climate Schools. A second model including a Preventure by Climate Schools interaction term indicated the effect of each intervention was unaffected by combining with the other (p = 0.08). However, inspection of the marginal means (Figure 3) shows decreasing hyperactivity symptoms in students who received Preventure only, and this group showed a significantly different rate of change compared to the no intervention group (d = −0.38 at 36 months).

Discussion

Utilising a novel factorial RCT design, this study was the first to demonstrate the differential effectiveness of personality-targeted alcohol misuse prevention in reducing internalising and externalising problems among high-risk youth, compared to effective universal alcohol prevention and to TAU over a three-year period.

Compared to high-risk students attending schools that did not deliver Preventure, students attending Preventure schools (alone or in combination with Climate Schools) displayed significantly reduced symptoms of anxiety, and a marginal effect on depressive symptoms, over the three-year study period. In addition, when Preventure was implemented alone (Preventure only group), students also demonstrated a decrease in symptoms of depression, conduct problems and hyperactivity compared to students allocated to receive no intervention. Consistent with previous research about mental health in adolescence, the TAU group showed a general increase in symptoms for both internalising and externalising problems (McGorry et al., 2011). The observed effect sizes for the significant reductions were small to medium. To provide context for these effects, the observed effect for anxiety (d = −0.27) would correspond to a participant moving from the 60th percentile of anxiety scores to the 49th percentile of the normal distribution, while the effect for conduct problems (d = −0.45) would correspond to moving from the 60th percentile for conduct problems to the 42rd percentile. These findings add to the knowledge base on the effectiveness of selective prevention programmes, which traditionally show only small, short-term reductions in internalising symptoms (Stockings et al., 2016).

Unlike the Preventure programme, the universal Climate Schools programme was not associated with a reduction in internalising or externalising symptoms, suggesting that it acts more selectively on changing risk factors unique to alcohol use. This is in contrast to the comparable effects the two programmes had on preventing alcohol misuse (Teesson et al., 2017). According to the conceptual models of each intervention, only Preventure was expected to directly impact mental health symptoms among high-risk youth. By contrast, the Climate Schools programme focuses exclusively on alcohol- and drug-related attitudes and behaviours, and, indeed, in a parallel analysis was shown to selectively improve cannabis-related knowledge, whereas the Preventure programme did not (Newton et al., 2018).

In order to explore the specificity of the intervention effects on hypothesised mechanisms, a post hoc analysis was conducted to assess the relationships between mental health and substance use outcomes. Changes in the frequency of drinking, binge drinking and cannabis use between baseline and the 36-month follow-up were calculated for each participant, and the correlations of these changes were calculated, as shown in Figure 4. Changes in drinking outcomes showed significant positive correlations with changes in mental health outcomes in the Preventure group, but not the Climate Schools group, suggesting that while effects of the Climate Schools programme are isolated to the specific substance use outcomes targeted, Preventure may affect substance use via improvements in mental health, or via a shared underlying mechanism. In the Control group, changes in depression and anxiety were positively correlated with changes in drinking outcomes, which may point to their shared risk factors.

Figure 4.

Correlations between changes in mental health outcomes and changes in substance use outcomes.

Demonstrating specificity of each intervention outcome in accordance with hypotheses specific to conceptual models of each intervention, particularly within the context of a randomised design that includes an intervention control is extremely rare in prevention research, where placebo and active control conditions are difficult to emulate. These results further extend the evidence-base for each intervention approach providing critical information on the conceptual validity of personality-targeted interventions and universal alcohol interventions.

Secondary analyses of interaction between the two interventions also highlight an important area for future study. The significant interaction for conduct disorder symptoms suggested that combining Preventure with universal alcohol prevention reduced the positive effect of the Preventure programme. This may be due to incompatible messaging in the interventions or that two interventions are too intensive for adolescents who are prone to conduct symptoms. This aligns with previous research on multi-component programmes for alcohol use, which are not necessarily more effective than single programmes (Foxcroft and Tsertsvadze, 2011a). It should be noted that this study was powered to assess intervention main effects and may be inadequately powered to detect an interaction between interventions, which requires increased sample size (Montgomery et al., 2003). In any case, this is an important finding as it demonstrates that care should be taken when combining interventions due to the potential that additions may dilute or interfere with otherwise effective programmes.

Strengths of this study include the high retention rate over a long-term three-year follow-up, the diverse sample of Government, Independent and Catholic schools and the sophisticated intention-to-treat analysis techniques. In addition, by utilising a novel factorial design, this study allowed for the first time an investigation of the independent effect of Preventure, as well as the effect of combining Preventure with the universal Climate programme.

Limitations of the study include the use of self-report data; however, student self-report is a well-accepted method of collection in prevention trials and shown to be reliable and valid, especially when students are assured confidentiality (Brener et al., 2003). Second, school-level randomisation led to an imbalance in gender across the groups. To account for the potential impact of this imbalance, all analyses adjusted for gender. The multi-level mixed effects models used estimate intervention effects relative to each participant’s starting point, and therefore accounted for any gender differences in initial levels of mental health problems. While the current trial was not powered to assess interactions with gender in a subgroup analysis (Assmann et al., 2000), exploring the mediators and mechanisms of the intervention effects is an important area to explore in future research.

In addition, the Preventure intervention attempts to deliver personality-targeted coping skills education, but the personality subtypes identified may not be fixed, particularly over the period of adolescence, and the SURPS measures used have demonstrated some changing state-like properties (Mackinnon et al., 2014). However, the measures are appropriate for their current use of implementing a preventive intervention, as they have been shown to be reliable over the short term (Woicik et al., 2009), and to predict future substance use and mental health outcomes over a longer period of 18 months (Castellanos-Ryan et al., 2013).

Finally, this study did not address the question of whether the interventions reduce the proportion of adolescents reaching clinical symptom thresholds. Analyses of binary outcomes such as meeting a clinical threshold are generally less sensitive than analyses of continuous symptom scores (Fedorov et al., 2009), and power to detect differences is reduced when prevalence is low. As the average age of onset is 21 years for anxiety disorders (Lijster et al., 2017) and 29 to 43 years for mood disorders (Kessler et al., 2007), prevalence of clinical disorders in the current sample (spanning 13–16 years) is likely to be low. However, reductions in the incidence of disorders may be possible to detect later as incidence of these disorders increases and could be assessed in a longer-term follow-up.

Conclusion

These findings demonstrate for the first time the differential effectiveness of universal and personality-targeted alcohol prevention in reducing internalising and externalising symptoms among high-risk youth over a three-year period. The novel factorial RCT design not only replicated the benefits of Preventure in reducing internalising and externalising problems, for the first time, it also compared the effects of the programme to both universal alcohol prevention and TAU. Importantly, it addressed a number of recommendations on the research evidence hierarchy (Institute of Medicine Committee on Health Behavior: Research Practice Policy, 2001) when evaluating behavioural interventions, including comparison to another reasonable intervention, validation of change model and generalisability to another setting and research context. Together with previous studies supporting the efficacy of Preventure in preventing alcohol misuse and related harms, this study suggests that addressing underlying shared risk factors for both substance use and mental disorders is an efficient and effective way to prevent the substantial burden of disease and social costs associated with substance use and mental disorders. The next step is to scale up this effective programme to equip young people with the skills they need to lead healthy lives and improve their life trajectories.

Supplemental Material

Supplementary_Material – Supplemental material for Evaluating the differential effectiveness of social influence and personality-targeted alcohol prevention on mental health outcomes among high-risk youth: A novel cluster randomised controlled factorial design trial

Supplemental material, Supplementary_Material for Evaluating the differential effectiveness of social influence and personality-targeted alcohol prevention on mental health outcomes among high-risk youth: A novel cluster randomised controlled factorial design trial by Nicola C Newton, Lexine Stapinski, Maree Teesson, Tim Slade, Katrina E Champion, Emma L Barrett, Louise Birrell, Erin Kelly, Marius Mather and Patricia J Conrod in Australian & New Zealand Journal of Psychiatry

Footnotes

Acknowledgements

The authors would like to acknowledge the participating schools, students, teachers and the research assistants and psychologists on the study. The research team also acknowledges the assistance of the NSW Department of Education and Communities for access to its schools (SERAP 2011201).

Author contributions

N.C.N., M.T., P.J.C. and T.S. led the conception and design of the study. L.S. and M.M. led the statistical analyses and interpretation of data, with assistance from T.S. E.L.B., K.E.C., N.C.N. and E.K. coordinated the implementation of the trial within schools. N.C.N. and L.S. initially drafted the article, with input from L.B., and all authors approved the final version to be published.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship and/or publication of this article: P.J.C. is one of the developers of the Preventure programme. N.C.N. and M.T. are the developers of the Climate Schools programme and Directors of Climate Schools Pty Ltd. Preventure is distributed not for profit. The other authors declare that they have no competing interests.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study was funded by the National Health and Medical Research Council (APP1004744). The study sponsors played no role in the design of this study, collection, analysis and interpretation of the data or in writing this manuscript.

Trial registration

This trial is registered with the Australian New Zealand Clinical Trials Registry (ACTRN12612000026820). The protocol is available at .

Supplemental material

Supplemental material for this article is available online.

ORCID iDs

Erin Kelly

Marius Mather

Notes

References

Australian Curriculum, Assessment and Reporting Authority (ACARA) (2014) ICSEA 2013: Technical Report. Sydney, NSW, Australia: ACARA.

Assmann

Pocock

Enos

, et al. (2000) Subgroup analysis and other (mis)uses of baseline data in clinical trials. The Lancet 355: 1064–1069.

Barrett

Newton

Teesson

, et al. (2015) Adapting the personality-targeted Preventure program to prevent substance use and associated harms among high-risk Australian adolescents. Early Intervention in Psychiatry 9: 308–315.

Botvin

(2000) Preventing drug abuse in schools: Social and competence enhancement approaches targeting individual-level etiologic factors. Addictictive Behaviors 25: 887–897.

Brener

Billy

Grady

(2003) Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: Evidence from the scientific literature. Journal of Adolescent Health 33: 436–457.

Castellanos-Ryan

O’Leary-Barrett

Sully

, et al. (2013) Sensitivity and specificity of a brief personality screening instrument in predicting future substance use, emotional, and behavioral problems: 18-month predictive validity of the Substance Use Risk Profile Scale. Alcoholism: Clinical and Experimental Research 37: E281–E290.

Conrod

(2016) Personality-targeted interventions for substance use and misuse. Current Addiction Reports 3: 426–436.

Conrod

Nikolaou

(2016) Annual research review: On the developmental neuropsychology of substance use disorders. Journal of Child Psychology Psychiatry 57: 371–394.

Dadds

Maujean

Fraser

(2003) Parenting and conduct problems in children: Australian data and psychometric properties of the Alabama parenting questionnaire. Australian Psychologist 38: 238–241.

10.

Dakin

Gray

(2017) Economic evaluation of factorial randomised controlled trials: Challenges, methods and recommendations. Statistics in Medicine 36: 2814–2830.

11.

Derogatis

Melisaratos

(1983) The Brief Symptom Inventory: An introductory report. Psychological Medicine 13: 595–605.

12.

Erskine

Moffitt

Copeland

, et al. (2015) A heavy burden on young minds: The global burden of mental and substance use disorders in children and youth. Psychological Medicine 45: 1554–1563.

13.

Faggiano

Minozzi

Versino

, et al. (2014) Universal school-based prevention for illicit drug use. Cochrane Database of Systematic Reviews 12: CD003020.

14.

Fedorov

Mannino

Zhang

(2009) Consequences of dichotomization. Pharmaceutical Statistics 8: 50–61.

15.

Foxcroft

Tsertsvadze

(2011a) Universal multi-component prevention programs for alcohol misuse in young people. Cochrane Database of Systematic Reviews 9: CD009307.

16.

Foxcroft

Tsertsvadze

(2011b) Universal school-based prevention programs for alcohol misuse in young people. Cochrane Database of Systematic Reviews 5: CD009113.

17.

Goodman

Lamping

Ploubidis

(2010) When to use broader internalising and externalising subscales instead of the hypothesised five subscales on the Strengths and Difficulties Questionnaire (SDQ): Data from British parents, teachers and children. Journal of Abnormal Child Psychology 38: 1179–1191.

18.

Gueorguieva

Krystal

(2004) Move over ANOVA: Progress in analyzing repeated-measures data andits reflection in papers published in the archives of general psychiatry. Archives of General Psychiatry 61: 310–317.

19.

Heo

Leon

(2009) Sample size requirements to detect an intervention by time interaction in longitudinal cluster randomized clinical trials. Statistics in Medicine 28: 1017–1027.

20.

Institute of Medicine Committee on Health Behavior: Research Practice Policy (2001) Health and Behavior: The Interplay of Biological, Behavioral, and Societal Influences (The National Academies Collection: Reports funded by National Institutes of Health). Washington, DC: National Academies Press.

21.

Jacqmin-Gadda

Sibillot

Proust

, et al. (2007) Robustness of the linear mixed model to misspecified error distribution. Computational Statistics & Data Analysis 51: 5142–5154.

22.

Johnston

Miech

O’Malley

, et al. (2018) Monitoring the Future National Survey Results on Drug Use, 1975–2017: Overview, Key Findings on Adolescent Drug Use. Ann Arbor, MI: The University of Michigan.

23.

Kessler

Angermeyer

Anthony

, et al. (2007) Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization’s World Mental Health Survey Initiative. World Psychiatry 6: 168–176.

24.

Kessler

Avenevoli

Costello

, et al. (2012) Severity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication Adolescent Supplement. Archives of General Psychiatry 69: 381–389.

25.

Kessler

Berglund

Demler

, et al. (2005) Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry 62: 593–602.

26.

Lijster

Dierckx

Utens

, et al. (2017) The age of onset of anxiety disorders: A meta-analysis. Canadian Journal of Psychiatry 62: 237–246.

27.

McGorry

Purcell

Goldstone

, et al. (2011) Age of onset and timing of treatment for mental and substance use disorders: Implications for preventive intervention strategies and models of care. Current Opinion in Psychiatry 24: 301–306.

28.

Mackinnon

Kehayes

I-L

Clark

, et al. (2014) Testing the four-factor model of personality vulnerability to alcohol misuse: A three-wave, one-year longitudinal study. Psychology of Addictive Behaviors 28: 1000–1012.

29.

Montgomery

Peters

Little

(2003) Design, analysis and presentation of factorial randomised controlled trials. BMC Medical Research Methodology 3: 26.

30.

Newton

Barrett

Castellanos-Ryan

, et al. (2016a) The validity of the Substance Use Risk Profile Scale (SURPS) among Australian adolescents. Addictive Behaviors 53: 23–30.

31.

Newton

Conrod

Slade

, et al. (2016b) The long-term effectiveness of a selective, personality-targeted prevention program in reducing alcohol use and related harms: A cluster randomised controlled trial. Journal of Child Psychology and Psychiatry 57: 1056–1065.

32.

Newton

Teesson

Barrett

, et al. (2012) The CAP study, evaluation of integrated universal and selective prevention strategies for youth alcohol misuse: Study protocol of a cluster randomized controlled trial. BMC Psychiatry 12: 118.

33.

Newton

Teesson

Mather

, et al. (2018) Universal cannabis outcomes from the Climate and Preventure (CAP) study: A cluster randomised controlled trial. Substance Abuse Treatment, Prevention, and Policy 13: 34.

34.

Newton

Teesson

Vogl

, et al. (2010) Internet-based prevention for alcohol and cannabis use: Final results of the Climate Schools course. Addiction 105: 749–759.

35.

Polanczyk

Salum

Sugaya

, et al. (2015) Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. Journal of Child Psychology and Psychiatry 56: 345–365.

36.

Rohde

Stice

Shaw

, et al. (2015) Effectiveness trial of an indicated cognitive-behavioral group adolescent depression prevention program versus bibliotherapy and brochure control at 1- and 2-year follow-up. Journal of Consulting and Clinical Psychology 83: 736–747.

37.

Stockings

Degenhardt

Dobbins

, et al. (2016) Preventing depression and anxiety in young people: A review of the joint efficacy of universal, selective and indicated prevention. Psychological Medicine 46: 11–26.

38.

Teesson

Newton

Slade

, et al. (2017) Combined universal and selective prevention for adolescent alcohol use: A cluster randomized controlled trial. Psychological Medicine 47: 1761–1770.

39.

United Nations (2011) World Population Prospects – The 2010 Revision. New York: United Nations.

40.

Van Widenfelt

Goedhart

Treffers

, et al. (2003) Dutch version of the Strengths and Difficulties Questionnaire (SDQ). European Child & Adolescent Psychiatry 12: 281–289.

41.

Werner-Seidler

Perry

Calear

, et al. (2017) School-based depression and anxiety prevention programs for young people: A systematic review and meta-analysis. Clinical Psychology Review 51: 30–47.

42.

Woicik

Stewart

Pihl

, et al. (2009) The substance use risk profile scale: A scale measuring traits linked to reinforcement-specific substance use profiles. Addictive Behaviors 34: 1042–1055.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB