Abstract
Mental disorders rank third after heart disease and cancers as determinants of the burden of human disease across the lifespan. They are the largest single cause of disability. Unlike physical diseases, mental disorders have an early age of onset. The median age of onset for anxiety disorders is 11 years, alcohol use disorders 20 years, schizophrenia 22 years, and for mood disorders 30 years [1, 2], and prevention must therefore occur earlier rather than later. Prevention can be universal (aimed at the general population), selected (aimed at groups who have risk or vulnerability to developing a disorder), or indicated (aimed at individuals who show early symptoms of a disorder) [3].
In an earlier paper we showed that results with an indicated school-based programme for anxiety were inconclusive [4]. In the aftermath of that study there was some evidence of peer stigmatization of the children taking part and no evidence that schools would see continuation of that type of programme as their core responsibility. In Australia the syllabus directs schools to promote self-confidence and optimism and to teach students about stress management, relaxation techniques and skills for goal setting and problem solving [5, 6]. This teaching agenda aligns with a mental health prevention agenda and a cognitive behavioural approach. The problems with teacher-formulated prevention programmes is that fidelity is difficult to guarantee, scalability is limited, staff are difficult to recruit and costs can be considerable.
Web-based education is different: fidelity is assured, scalability is simple and costs are minimal. We therefore took a model we had developed for the Web-based treatment of common mental disorders in adults and applied it to the high school environment. The adult model used an interactive illustrated story of someone recovering from a specific disorder to engage the patient, which was supplemented by additional written material and a discussion forum in which knowledge could be consolidated [7–10]. In the high school model of Climateschools we again used the illustrated storyline to engage the children, supplemented by additional classroom exercises and discussions. The courses may be viewed at www.climateschools.tv.
We began with programmes designed to influence drinking patterns. The schools, who saw control of drinking alcohol as a priority, were keen to participate and two randomized controlled trials were conducted comparing the results of the Web-based universal prevention programmes with as-usual education on the same topic. The alcohol courses took a harm minimization approach. The courses recommended abstinence as the safest option, but acknowledged that alcohol use is a reality for many young people. The two randomized control trials (in 16 and 10 schools, respectively) found that the courses were, in one or both trials, effective in increasing knowledge of alcohol-related harms; improving attitudes to alcohol consumption; and decreasing average alcohol consumption, excess alcohol consumption and related harms. Both boys and girls agreed that the cartoon story and skills were relevant to current and future experiences in their lives [11, 12].
We could find no accounts of Web-based stress management interventions in the literature and certainly none was contained in the Kraag et al. meta-analysis of school-based stress management programmes [13]. We proceeded to develop our own. The European Network of Health Promoting Schools provides evidence that school-based health prevention programmes are more effective if they are delivered by school staff rather than by health professionals who ‘come into the schools, do their bit and then go away’ [14]. Furthermore, programmes designed by collaborating teams of health and education experts have a higher chance of being effective [15]. St Leger and Nutbeam point out the importance of taking time to ‘map the links’ that exist between education and health so that shared goals can be identified [16].
This research describes the design, development and evaluation of an Internet-based programme for the management of stress in year 8 or middle school students (mean age 13 years). To this end a feasibility study was conducted to gain opinions from students and teaching staff concerning a first draft of the programme. Significant changes were made in light of the results.
Method
Development
The course was designed to develop knowledge about stress and effective coping strategies, increase use of effective coping strategies and decrease less-effective coping strategies, and produce improved mental well-being and improved perceptions of competence to cope with stress.
The course was presented as a cartoon narrative that followed the adventures of two characters called Mia and Ben. Students were exposed to examples of stressful events in the lives of both characters and saw both try to cope using a number of different strategies. Buddy, the tutor, and his assistants, were deliberately drawn to look like aliens to avoid gender or ethnic stereotyping. The narrative unfolded across six lessons that each ran for approximately 30 min. Each lesson included a revision of the previous lesson (pre-testing in lesson 1) and new content activities. Related classroom learning activities were provided to reinforce the knowledge and skills taught on the computer.
A draft course was produced, illustrated and programmed. A series of pilot studies were then undertaken to determine if programme materials were correct, readable and useful. Students (n = 88) were asked if the cartoon format would appeal to adolescents, if the content was relevant and if they had suggestions for making the programme more appealing. School staff (n = 27) were asked if the programme could be implemented and if there were ways to make the programme more effective and efficient. Mental health experts (n = 7) were asked to evaluate the content and delivery. Many aspects of the course were changed in the light of comments from these three groups. The content of the final six lessons was as follows.
The first lesson contained information about how to define stress, the common causes of stress, the negative and positive effects of stress and factors that protect against the negative effects of stress. This lesson delivered the message that it is possible for people to learn how to manage stress more effectively. Lesson 2 introduced the concept of coping. Helpful and unhelpful coping strategies were presented. Avoidance of stressful places, people or things was explored as an example of an unhelpful coping strategy. Helpful coping strategies included talking about problems with a trustworthy person, problem solving, challenging unhelpful thoughts and managing time effectively.
Lessons 3–6 focused on teaching particular skills. Lesson 3 explored how particular styles of thinking influence stress levels and taught the process of thought challenging. Lesson 4 taught the skill of structured problem solving as well as the skills needed to analyse and achieve a goal by breaking it down into smaller, more manageable steps. Lesson 5 presented ways to restore and maintain calm, including progressive muscle relaxation and breathing exercises. Lesson 6 taught the benefits of daily planning and gave information about seeking help. This final lesson also explored the connection between mental well-being and lifestyle factors such as regular exercise, spending time with friends, getting enough rest and making time for fun activities.
Evaluation
Eight independent and Catholic high schools were recruited and were expected to provide a minimum of 500 students, sufficient to provide the power to detect small effect sizes (ESs). Research ethics approval was obtained from University of NSW, and letters explaining the project and containing a consent form were sent to each school principal and to the parents of each student in the targeted classes. All but one parent gave consent. All eight schools consented but two of the schools were subsequently unable to implement the intervention due to timetabling constraints and staffing issues but agreed to have students complete the research questionnaire at the three data collection points. Students from these schools thus became a de facto, if under-powered, comparison group. There were 464 participating students (75% female) in the intervention schools, 189 in the comparison schools.
Measures obtained
The proportion of students completing each lesson and the proportion completing all lessons were measured. Twelve true/false statements about stress and coping (defining stress, helpful ways to cope, thought challenging, structured problem solving, de-arousal strategies, relationship between stress and lifestyle factors) were administered before and after the course as a test of knowledge gained. The Perceived Competence Scale [17] was reworded to measure students’ beliefs in their own competence to cope with everyday stresses. The version of the scale used in this study was trialled with 88 adolescent school students (α = 0.90). Coping behaviours were measured using the Children's Coping Strategies Checklist [18] and the Strengths and Difficulties Questionnaire [19]. Distress was measured at the conclusion of each lesson using a measure of distress (K6) [20] and of well-being (the Delighted–Terrible Scale) [21].
Procedure
Teachers were given instructions for Internet access to the course, lesson outlines, and assessment and reporting information. Teachers were also given class sets of student activity books containing paper-and-pencil tasks for students to complete after the Internet activities to expand or extend student knowledge. Teachers were shown the research questionnaire and were informed of the importance of having students complete the questionnaire methodically and truthfully before lesson 1, after lesson 6 and 3 months later.
A school counsellor at each intervention school was informed that the intervention encouraged students to seek help if they were concerned about how they were coping with stress. Counsellors and students were informed that schools would be contacted if students scored in the very high range on the K6. All participants in the intervention and control schools completed the research questionnaire during regular class time and under the supervision of their regular class teachers. Schools ran the intervention during personal development/health lessons or during lessons designated for student welfare activities.
Analysis
Students completed questionnaires designed to measure knowledge, coping competence and coping behaviour at three time points. Pre-intervention questionnaires were collected in the week prior to the first lesson. Students worked through the six lessons and post-intervention data were then collected. Finally students completed the questionnaire for a third time at a 3 month follow up. Data were analysed using repeated measures multivariate analysis of variance (MANOVA). The results are presented together with the Cohen's d ES improvement from before intervention to 3 months after intervention.
Students in the intervention group completed a questionnaire designed to measure mood (psychological distress and well-being) at the beginning of each of the six lessons. Data were analysed using repeated measures multivariate analysis of variance. These measures prompted feedback to students about their mood at each lesson so that they could monitor their stress levels.
Results
Adherence
Occasionally, and increasingly towards the end of the school term, other school activities pre-empted the stress lessons. Even so, 69% of the 464 students who began the programme completed all components of all six lessons and 79% completed all components of the first five lessons that covered the core knowledge and skills, namely the theory concerning stress and coping, thought challenging, structured problem solving and de-arousal techniques. Five per cent of students scored in the high range of the K6. The students were notified that their scores were high and that they should seek help if they remained high. When scores remained high, schools were notified. Most of these students were known to the schools as being at risk.
Outcomes
A repeated measures MANOVA was conducted for all of the pre-test, post-test and follow-up outcome variables, with time as a within-subjects factor. This analysis indicated a significant effect for time (Pillais F(2,159) = 6.481, p < 0.001, η2=0.38). Post-hoc univariate tests indicated significant effects from before intervention to 3 month follow up for: (i) time×knowledge (F(2,159) = 9.735, p < 0.001, ES = 0.36); (ii) time×support-seeking coping (F(2,159) = 9.921, p < 001, ES = 0.015); (iii) time×avoidant coping (F(2,159) = 3.331, p = 0.037, ES = 0.22); and (iv) time×total difficulties (F(2,159) = 3.298, p = 0.038, ES = 0.16).
There were significant increases in knowledge and support-seeking coping and significant decreases in avoidant coping and total difficulties over time. There were no significant effects for time×perceived competence, time×active coping and time×pro-social coping. School site and gender were entered as covariates to investigate potential moderating effects. There were no significant differences for any of the outcome variables for time×school site or time×gender.
Another repeated-measures MANOVA was conducted for the outcome variables (psychological distress and well-being) that were measured at six time points, corresponding to the six lessons in the intervention, namely, with time as a within-subjects factor. This analysis indicated a significant effect for time (Pillais F(5,315) = 6.481, p < 0.001, η2=0.12). Post-hoc univariate tests indicated significant effects for: (i) time×psychological distress from lesson 1 to lesson 6 (F(2,315), p < 0.001, ES = 0.16); and (ii) time×life satisfaction from lesson 1 to lesson 6 (F(5,315), p = 0.027, ES = 0.1).
There was a significant decrease in psychological distress and a significant increase in well-being over the study period. Again neither school site nor gender accounted for significant differences for either psychological distress or well-being.
Comparison schools
A repeated-measures MANOVA indicated no differences on any of the outcome variables before testing, between the intervention and comparison schools. A repeated-measures MANOVA was conducted for all of the pre-test, post-test and follow-up outcome variables, with time as a within-subjects factor, for the comparison schools. This analysis indicated no significant effect for time for any of the outcome variables.
Discussion
A Web-based course of six lessons to teach Year 8 students how to recognize and manage stress was developed, revised after piloting with students, teachers and health professionals, and implemented with 464 students in six schools. This was an open phase 1 trial but still scored 29/36 possible points on the CCDAN Quality Rating System [22]. Teachers were successful in implementing the computer-based intervention. The 21% drop in completion rate after five lessons was mostly due to timetabling clashes not course difficulty, and compares favourably with completion rates for other mental health interventions delivered by teachers [23].
Students at the intervention schools showed a small but significant increase in their knowledge about stress and coping over the 21 week assessment period. No such increase occurred in the students in the comparison schools but the number of students in the comparison schools was small and a larger sample size might have produced a different result. Students reported increased use of support-seeking coping behaviours. This increase suggests that the programme not only affects knowledge but also changes behaviour. Support-seeking coping may have increased because the intervention addressed help-seeking in all six lessons. Repeated exposure may also explain the significant decrease in avoidant coping behaviours because students were reminded of the negative impact of avoiding problems in all six lessons. Psychological distress decreased and life satisfaction increased, consistent with the intervention having a beneficial effect on mood. The ESs relating to all outcomes were small, as would be expected from a brief universal intervention, and replication is needed.
Other coping behaviours, such as active coping (structured problem solving and thought challenging) were addressed in detail in only one lesson and revised briefly in a follow-up lesson. These coping behaviours remained relatively stable over the study period. Prosocial behaviour did not increase along with support-seeking behaviour because both constructs asked about spending time with friends and family. This discrepancy most probably reflects a difference in the focus of these constructs. Although students were regularly reminded of the benefits of seeking help for themselves, the programme did not explicitly teach about the mental health benefits of offering help to others.
Perceived competence scores did show a non-significant upward trend over the three time points. Dubow et al. noted improvements in self-efficacy 6 months after implementation of a universal anxiety prevention programme [24]. Again we cannot know whether the lack of change in this measure was due to inadequate teaching, or because students had not had enough time to apply many of the coping strategies that they learnt during the course.
Trials of other mental health prevention programmes have found different effects for male and female subjects [25]. Both male and female students benefited from the present course. Evaluation trials of other programmes have reported significant effects for school site [26, 27], but school site was not related to outcomes in the present study. We hypothesized that Internet delivery would ensure fidelity. Six schools are too few to substantiate this.
Some teachers asked for a handout for parents that they could distribute. We were reluctant to provide this given that interested students could use their unique login from home to show their parents the course lessons. A main advantage of Internet delivery of educational material is that although courses remain cheap and simple they will be widely used, whereas satisfying requests from users could eventually restrict usage to the more sophisticated, who are hardly the group in need.
The educational curriculum is becoming crowded as schools are being asked to provide academic and social and emotional learning. Courses targeting one aspect such as emotional regulation or substance abuse need to be as simple as is consistent with being effective, if only to make room for other topics. Courses in subsequent years should be integrated, and reinforce and build on the knowledge and skills taught in previous years. The three Climateschools courses on stress, anxiety and depression all use the same cognitive behaviour therapy constructs and teach much of the same material in increasingly complex situations. The three courses on substance use do likewise.
Conclusion
It appears that a universal, Internet-based approach to stress management is practical and feasible and likely to prove effective. Further research is needed to replicate the present findings and to inform further programme development.