Abstract
Mental health first aid has been defined as ‘the help provided to a person developing a mental health problem or in a mental health crisis. The first aid is given until appropriate professional help is received or the crisis resolves’ [1]. The Mental Health First Aid (MHFA) training course has been developed in Australia to improve the mental health first aid skills of the general public, and this course has spread to many other countries [2]. A number of evaluation studies have been carried out on the course, including uncontrolled trials [3–8], randomized controlled trials with public servants [9], members of the public in a large rural area [10] and high school teachers [11], and a qualitative study examining participants’ stories of providing assistance to others following the course [12]. These evaluation studies have found improvements in mental health knowledge, reduced stigma, increased confidence in providing help, and increased provision of help.
While MHFA courses have been widely disseminated, with over 100,000 people trained in Australia, they require 12–14 hours of face-to-face training. This can be difficult to arrange in some circumstances, for example in certain workplace situations where it is difficult to schedule people to be off duty at the same time, for parents with childcare responsibilities, or for people living in remote areas. One approach which is gaining some evidence of efficacy in these situations is the use of e-learning. E-learning has been applied to both medical education [13,14] and to psychological treatments [15,16] and can produce effects comparable to instructor-delivered training or face-to-face therapy. For this reason, an e-learning version of the course has been developed [17], which allows the participant to do the course in their own time without attending a class. This e-learning version presents information with interactive exercises and includes video clips of people who have mental health problems.
Here we report a randomized controlled trial evaluating the e-learning version. We compared its effects to a no-treatment control condition (people placed on a waiting list) and to studying a printed MHFA manual [18]. The manual was included as an additional control because it is used as an intervention in its own right, with large numbers of people buying or downloading the manual from the MHFA website (www.mhfa.com.au). We hypothesized that both interventions would be superior to a waiting list control, but expected the more interactive and engaging e-learning format to be superior to the printed manual. We also hypothesized that any gains would be sustained, so we assessed participants at 6-months follow up in addition to post test. Because of the limited funding available, we were unable to include a face-to-face training condition in this trial.
Method
Objectives
The aim of the study was to evaluate the effects of MHFA training of the public via an e-learning CD, compared to either receiving the information in a printed MHFA manual or receiving no training. The primary hypotheses were that the e-learning CD would produce a greater improvement in knowledge and recognition of mental disorders (mental health literacy), greater confidence in providing help to someone experiencing a mental health problem, with more first-aid actions and less stigma towards people with mental health problems, than either (i) receiving information via a printed MHFA manual or (ii) being in a waiting list control group, and that these differences would be maintained 6-months following training. A secondary hypothesis was that the e-learning CD would produce greater improvements in first-aider mental health than either (i) receiving information via a printed MHFA manual or (ii) being in a waiting list control group and that these differences would be maintained at 6-months follow up.
Design
The study involved an individually randomized three-group parallel design with an allocation ratio of 1. Participants were randomly assigned to either: (i) receive the MHFA e-learning CD; (ii) receive the MHFA manual, or (iii) be added to a waiting list. The trial has been registered with the Australian and New Zealand Clinical Trials Registry (ACTRN12608000188336).
Participants
Eligible participants were adults from the general community across Australia, who had access to a computer with a CD drive and the Internet (for email contact and web-based administration of questionnaires). Participants were recruited progressively from June 2008 to April 2009, through online advertisements on community and research notice boards (e.g. the Mental Health First Aid website, beyondblue, Black Dog Institute, Mental Health Council of Australia, Mensline and Auseinet); flyers in university libraries, community and mental health centres across Australia (which listed an email contact), and advertisements in local newspapers and newsletters promoting mental health. The advertisements called for participants in an evaluation of MHFA educational materials which were aimed at adult members of the general public.
Interventions
The MHFA e-learning CD [17] teaches members of the general public to recognize the early signs of mental illness and to provide initial help to a person developing a mental disorder or in a mental health crisis situation. It contains detailed action plans which cover: depression, anxiety disorders, psychosis and substance use disorders, and interactive case studies for each. The action plans incorporate five basic actions, as follows: Assess risk of suicide or harm, Listen non-judgementally, Give reassurance and relevant information, Encourage appropriate professional help, Encourage self-help strategies (ALGEE). In addition, the CD contains case studies applying the action plan; a 25 question knowledge quiz (at the start and completion of the CD) to gauge skills learned; videos depicting real stories of anxiety disorder, depression, bipolar disorder and schizophrenia; interactive exercises (e.g. illustrating the various manifestations of anxiety); self-assessment of symptoms (e.g. anxiety symptoms, alcohol use); and links to online resources.
The MHFA manual [18] has the same content as the e-learning CD, but without the video clips and interactive case studies. It contains written action plans for depression, anxiety disorders, psychosis and substance use disorders, and printed forms of an alcohol use questionnaire and an anxiety symptom scale. References and links to printed and online resources are included. A free PDF of the manual can be downloaded from www.mhfa.com.au.
The e-learning CD or manual was sent with the letter advising random assignment. Participants receiving these interventions were sent weekly emails for a month to help pace them through the CD or manual. This gave them 4 weeks to work though the material.
After the trial was completed, participants were sent the intervention materials they had not been randomly assigned to receive. Participants who were assigned to the e-learning CD were sent the manual and those who were assigned to the manual were sent the e-learning CD. Participants assigned to the waiting list received both the CD and manual after completing the 6 month follow up questionnaire.
Randomization: sequence generation
A computerized random number generator (Random Integers option in Random.org) was used to carry out a simple randomization procedure, where a randomly assigned variable (values 1–3) was assigned to each participant's unique ID number to determine which condition they were allocated to.
Randomization: allocation concealment
The principal investigator (A.F.J.) carried out the random number generation using participants’ ID numbers (but no other knowledge of participant details). The allocation sequence was concealed from J.F. until it was time to advise participants of their random assignment and send educational materials as required; that is, after baseline data was collected.
Randomization: implementation
J.F. provided each participant with a unique ID number at enrolment, which was stored in a password-protected Excel spreadsheet. Baseline questionnaire completion was checked daily by J.F. and ID numbers of completers were emailed to A.F.J. The allocation sequence was determined by A.F.J. who in turn advised J.F. of the intervention that participants were assigned to.
Blinding
Blinding of participants was not possible.
Pre-test questionnaire
Prior to random assignment, all participants completed the pre-test questionnaire. It contained socio-demographic questions (gender, age, postcode, highest level of education completed, current studies, Aboriginality, citizenship and language predominantly used at home), reasons for wanting to learn mental health first aid skills (open-ended question), any previous mental health training, and previous personal history of a mental health problem. The responses to the open-ended question on reasons for wanting the training were classified into themes by one of the authors and checked by a second author. The pre-test questionnaire also included the following:
Recognition of mental disorders
Participants were presented with a vignette of a 30 year old portraying major depression (Mary) and one of a 24 year old with schizophrenia (John) and asked ‘From the information given, what, if anything, is wrong with Mary/John?’ Each vignette was written to meet the minimum requirements for DSM-IV-TR criteria for either schizophrenia or depression. Blank lines were provided after each vignette, for an open-ended response. Correct responses were ‘depression’ for the first vignette and ‘schizophrenia’ or ‘psychosis’ for the second. This procedure has been previously used for assessing community mental health literacy and evaluating MHFA training [3,19].
Beliefs about treatment
Participants were asked to rate a list of people, treatments and actions that the person in the vignette might use: ‘If Mary/John was to seek help from any of the following people, is it likely to be “helpful”, “harmful”, “neither” or “don't know” for her/his problem? a typical GP or family doctor; a chemist or pharmacist; a counsellor; a social worker; telephone counselling service, e.g. Lifeline; a psychiatrist; a clinical psychologist; help from her/his close family; help from some close friends; a naturopath or a herbalist; the clergy, a minister or a priest; Mary/John tries to deal with her/his problem on her/his own’. Next was a list of medications: ‘vitamins and minerals; St John's wort; pain relievers such as aspirin, codeine or Panadol; antidepressants; antibiotics; sleeping pills; anti-psychotics; tranquillisers such as Valium’. Finally, participants were given a list of other strategies: ‘becoming more physically active such as playing more sport, or doing a lot more walking or gardening; reading about people with similar problems and how they have dealt with them; getting out and about more; courses on relaxation, stress management, meditation or yoga; cutting out alcohol altogether; counselling; cognitive behaviour therapy; psychotherapy; hypnosis; admission to the psychiatric ward of a hospital; electroconvulsive therapy (ECT); having an occasional alcoholic drink to relax; a special diet or avoiding certain foods’. To score these items, scales were created showing the extent to which participants agreed with health professionals about which interventions would be useful [3]. For depression, a survey of professionals showed a consensus that GPs, psychiatrists, clinical psychologists, antidepressants, counselling and cognitive behaviour therapy are helpful [20]. Therefore, scoring for depression ranged from 0–6, according to the number of these interventions that participants chose as ‘helpful’. For schizophrenia, the survey of professionals showed a consensus that GPs, psychiatrists, clinical psychologists, antipsychotics and admission to a ward are helpful [20]. Therefore scoring for schizophrenia ranged from 0–5, according to the number of these interventions that participants chose as helpful. Because the baseline scores were highly skewed with ceiling effects, these were dichotomized as full score versus less than full score.
Confidence in providing mental health first aid
This was measured by asking participants ‘How confident would you feel in helping Mary/John?’ (not at all, a little bit, moderately, quite a bit, extremely) [3]. Based on the frequency distribution of responses, these were dichotomized at ‘moderately’ or greater.
Actions taken to implement mental health first aid
Actions taken to implement mental health first aid were only assessed at pre-test and 6-month follow up because the participants may not have had time to use any skills learned at 1-month post-test. Participants were asked whether they had talked to a person about their mental health problem over the past 6 months, using a 4-point Likert scale (never, once, a few times, many times). If they had talked with someone, participants were asked to indicate which of the following they had done (multiple responses were allowed): (i) Spent time listening to their problem; (ii) Helped to calm them down; (iii) Talked to them about suicidal thoughts; (iv) Recommended they seek professional help; (v) Recommended self-help strategies; (vi) Gave them information about their problem; (vii) Gave them information about local services; (viii) Made an appointment for them with services; (ix) Referred them to books or websites about their problem; (x) Did you do anything else? (open-ended response). Categories created from open-ended responses included: Went with them to appointment, Enlisted others to help, Took action to help self, Enlisted crisis services to attend, Gave help in a professional role e.g. provided counselling or advice in workplace role, Visited or spent time with them, Offered to be crisis contact, Gave reassurance or practical support, Maintained confidentiality, Shared personal experiences and Did something to raise awareness about mental health in the community. Scoring was based on 1 point per action taken and summing scores.
Stigmatizing attitudes
These were measured by personal and perceived stigma scales [21] and by a social distance scale [3,22]. In personal stigma, the respondent was asked about their personal attitudes towards the person described in each vignette, while in perceived stigma they were asked what they thought other people's attitudes were towards each person in the vignette. A principal component analysis replicated the two-component solution of Griffiths et al. [21]. Because of skew in the distribution, the principal component scores were dichotomized at the median of all the data (pre, post and follow up).
First-aider mental health
This was measured using the Kessler Psychological Distress Scale (K10) [23]. Scores were summed to give a score from 5 to 50 and dichotomized at the 19/20 cutpoint [24].
Post-test and follow up questionnaires
The post-test and follow-up questionnaires repeated the questions above on recognition of mental disorders, beliefs about treatment, confidence in providing mental health first aid, mental health first aid actions taken, stigmatizing attitudes and first-aider mental health.
Participants in the intervention groups were given additional questions at the end of the post-test and follow up surveys: ‘How much of the CD/manual did you read? (none of it, part of it, most of it, all of it); How easy was the MHFA material to understand? (very easy, easy, neither easy nor difficult, difficult, very difficult); Did you learn much from the MHFA material? (a great deal, a fair bit, not very much, almost nothing); How useful was the MHFA material? (very useful, useful, not very useful, not at all useful); Do you think you will use the CD/manual in the future? (yes, no, not sure); What have you done with the CD/manual? (multiple responses were allowed: kept it, lent it to someone, given it away, thrown it away, lost it, don't know); Would you recommend the CD/manual to others? (yes definitely, probably, probably not, definitely not)’.
In the 6 month follow up questionnaire, the intervention groups were given the following open-ended questions: ‘What did you like about these materials?’ and ‘What did you dislike about these materials?’ The open-ended responses were classified into themes by one of the authors and these were checked by a second author.
Sample size estimation
According to SamplePower software, 75 participants per arm (N = 225) was calculated as the minimum number required to detect medium effect sizes (Cohen's f = 0.25 for comparisons of 3 groups) with > 90% power at alpha set at 0.05. This calculation made the conservative assumption of no correlation between pre-test and later scores.
Statistical methods
To take into account the correlation of responses by individual participants between the measurement occasions, mixed models for continuous, dichotomous and count outcome variables, with group by measurement occasion interactions were used to analyse the data. Another advantage of these maximum likelihood-based methods is that they produce unbiased estimates when a proportion of the participants drop out before the completion of the study, provided that they are missing at random [25,26].
All analyses were conducted using Stata Release 10 [27].
Ethics
Ethics approval was granted by the University of Melbourne Health Sciences Human Ethics Sub-Committee.
Results
Participant flow
Figure 1 shows the CONSORT flow diagram of the number of participants at each stage of the trial. All the participants included in the analyses completed the first questionnaire. Nine per cent of participants did not complete the post-test questionnaire and 13% the follow-up questionnaire. An intention-to-treat approach was used, with all participants included in the analyses.
CONSORT Flow Diagram of the number of participants at each stage of the trial.
Participants’ characteristics
The mean age of participants was 40 years (SD = 12), 81% were female, 56% had a bachelor or higher degree, 37% were currently studying, 91% were Australian citizens and 88% had English as their first language. When participants were classified for remoteness according to the Australian Standard Geographical Classification [28], 79% lived in major cities, 18% in regional areas and 3% in remote areas. A history of having a mental health problem was reported by 52%.
When asked an open-ended question about why they were interested in MHFA training, the most common responses were classified as (multiple classifications were possible): professional interest relating to paid or volunteer work (55%), enable the person to assist others (35%), identify mental health problems in family or others (23%), personal interest (32%) and identify or manage mental health problems in self (12%). When asked about previous mental health training, 43% reported that they had received some. The most commonly reported were short courses, seminars or workshops (18%), as part of a bachelor degree (11%) and as part of nurse training (6%).
Changes produced by the interventions
Table 1 shows the data on changes from pre-test to post-test and follow-up for the three groups. Recognition of depression was very high at baseline, leaving little room for improvement, so statistical analysis was not possible. However, recognition of schizophrenia improved with both interventions.
Comparison of outcomes between waiting list, manual and CD groups
∗p < 0.05; ∗∗p < 0.01; ∗∗∗p < 0.001
1Items selected as helpful included: seeing a typical GP, a psychiatrist, clinical psychologist, being prescribed antidepressants, having counselling, and CBT. 2Items selected as helpful included: seeing a typical GP, a psychiatrist, clinical psychologist, being prescribed antipsychotics, and admission to a psychiatric ward. 3First aid actions taken included 20 items: 1) listened to the person's problem; 2) calmed them down; 3) talked to them about suicidal thoughts; 4) recommended they seek professional help; 5) recommended self-help; 6) gave them information about their problem; 7) gave them information about local services; 8) made an appointment for them; 9) referred them to books or websites for information about their problem; 10) Went with them to appointment; 11) enlisted others to help; 12) took action to help self; 13) enlisted crisis services to attend; 14) gave help in a professional role e.g. provided counselling or advice in workplace role; 15) visited or spent time with them; 16) offered to be crisis contact; 17) gave reassurance or practical support: 18) maintained confidentiality; 19) shared personal experiences, and 20) did something to raise awareness about mental health in the community.
Beliefs about treatment of depression became more like those of professionals for both interventions. However, for beliefs about treatment of schizophrenia there was an improvement only for the CD group, but this was not maintained at follow up.
Both interventions reduced personal stigma for depression and schizophrenia. Furthermore, the CD reduced personal stigma of schizophrenia more than did the manual. Perceived stigma was not changed by either intervention. There were trends for reductions in social distance with the depression vignette, but the change was only significant for the manual group at post-test. For social distance with the schizophrenia vignette, the CD produced greater reductions than both waiting list and manual, but the difference from the manual was not maintained at follow-up.
Confidence in providing help was increased by both interventions for depression and schizophrenia. While the CD tended to do better, the difference from the manual was not significant.
There was no significant difference between interventions in whether the participants had spoken with a person about a mental health problem. However, the number of first aid actions taken was significantly greater at follow up in the CD group compared to waiting list.
There was no difference between interventions for change in K10 psychological distress score. However, a secondary analysis of the ‘days of out of role’ questions that are supplementary to the K10, showed that the CD group had significantly greater reduction in days totally unable to work due to negative feelings compared to either the manual group or the waiting list group. The means (SDs) on days out of role were: CD group, pre 1.48 (4.41), post 0.52 (1.76), follow-up 0.68 (3.01); manual group, pre 0.75 (2.74), post 1.54 (5.04), follow-up 0.79 (2.51); waiting list group, pre 1.50 (5.1), post 1.33 (4.13), follow-up 0.96 (2.85). The incident rate ratios for CD versus waiting list were 0.36 (95%CI = 0.13–0.96) at pre-test and 0.37 (95%CI = 0.14–0.96) at follow-up. For CD versus manual they were 0.23 (95%CI = 0.09–0.61) and 0.31 (95%CI = 0.12–0.84) respectively.
Attitudes towards the interventions
Table 2 shows participants’ attitudes towards the CD and manual. It can be seen that the attitudes were very similar for the two interventions.
Participant attitudes towards interventions
When asked open-ended questions about what they liked and disliked about the intervention, responses were received from 55 participants in the CD group and 64 in the manual group. The main likes about the CD were: ease of comprehension, clear, concise (n = 31); the quality of the content, information (n = 24); ease of navigation, easy to use format (n = 14). For the manual the main likes were: ease of comprehension, clear, concise (n = 52); quality of content, information (n = 36); layout of sections (n = 13); and empowering, practical applicability (n = 13). The main dislikes about the CD were: navigation difficult, not convenient for searching (n = 6); content too long or time consuming (n = 5); and oversimplified content, want more depth (n = 5). The main dislikes about the manual were: repetitious (n = 6); oversimplified content, want more depth (n = 5); and learning via manual not preferred learning modality (n = 3).
Adverse events
Given that this was an educational intervention with a non-clinical sample, there was no formal enquiry about adverse events. Informally, no adverse events were reported.
Discussion
Both the e-learning CD and the manual had positive effects compared to waiting list controls on recognition of schizophrenia, beliefs about depression treatment, personal stigma regarding both depression and schizophrenia, and confidence in providing help. The e-learning CD was also superior to the waiting list in producing changes in actual helping behaviour towards people with mental health problems. When the e-learning CD and manual were directly compared, the differences were few, consistent with a meta-analysis finding that Internet-based learning for health professionals shows little difference in outcomes from other educational formats [29]. However, we did find some differences, with the CD superior at reducing the social distance and personal stigma in relation to a person with schizophrenia. Generally speaking, effects that were found at post-test were also present at 6 months follow up.
The reason for having the MHFA manual as a comparison is that it is used as an intervention in its own right. In the 5-year period 2005–2009, there were over 18 000 downloads of a PDF version of the manual from the MHFA website (www.mhfa.com.au), in addition to the many people who bought printed copies of the manual without doing a MHFA course. The present results show that studying the manual alone can produce changes in mental health literacy and reductions in stigma, even though it lacks the interactivity of the e-learning version.
The major advantage of the e-learning CD over the manual appears to be in the reduction of stigma towards a person with schizophrenia. A possible reason for this difference may be that the CD incorporated short video clips of people with mental disorders talking about their experiences and the participants are likely to have had less prior contact with people affected by schizophrenia than depression. Previous research has suggested that personal contact is an effective way to reduce stigma [30,31]. While the contact in the e-learning version was via video rather than in person, it may have had a similar effect. However, contact does not appear to be necessary for stigma reduction, because the manual alone had positive effects. Similarly, a previous trial of an Internet-based educational intervention on depression (BluePages, www.bluepages.anu.edu.au) showed that it reduced stigma, even without any element of contact [32].
The other advantage of the e-learning version was in producing changes in first aid actions taken. While the e-learning CD and manual did not differ significantly from each other, only the CD produced a significant advantage over the waiting list. This difference may be due to the inclusion of interactive case studies in the e-learning version, which required the participant to rehearse what they would do to help in certain circumstances. The manual did not provide these specific exercises.
The psychological distress of participants in all groups tended to improve, with no difference between the intervention group and controls. However, in a secondary analysis the participants in the e-learning condition were found to report significantly fewer days out of role because of negative feelings than either the manual or waiting list groups. In a previous trial of face-to-face MHFA training in a workplace setting, the training produced a significant improvement in mental health compared to the waiting list [9]. In that trial, participants were found to be somewhat worse in mental health than the general population. Similarly, in the present study, 31.7% were found to be above the K10 cut-off at baseline compared to prevalence rates of 10.6% for women and 8.3% for men in the general population [33]. It is possible that face-to-face training has a different impact in this area, but replication of the face-to-face training results is necessary.
When asked about attitudes to the interventions, there was very little difference between the two modes. An overwhelming majority reported that the intervention was easy to understand, informative, useful and would be used in the future. Participant reports of what they liked and disliked about the intervention were also similar for both modes and related more to the content than to the mode of administration.
An unusual observation was that the stigma scores of the waiting list controls tended to increase with repeated assessment. This is a phenomenon we have observed in a previous trial [11] and is opposite to the pattern of decreasing scores typically seen with repeated assessment of psychiatric symptoms [34]. We speculate that it may be due to social desirability effects with stigma questions. Participants may be biased towards giving socially desirable responses at pre-test, but this bias might decrease with each subsequent assessment. Such a re-test effect shows the necessity of having a control group in evaluations of stigma reduction interventions. Studies using pre-test post-test uncontrolled designs may underestimate the effects of the intervention on reducing stigma.
Limitations
Previous trials of MHFA have examined face-to-face training. It would have been desirable to also have a face-to-face condition in the current trial, but this was not possible because of the limited funding available and the wide geographic spread of participants. Face-to-face training involves group interaction, discussion and questions that are not possible by e-learning. Any additional benefits of these features need to be evaluated in future research. Nevertheless, the effects found here are similar to those found in trials of face-to-face training [3–12].
As with other MHFA trials, it has not been possible to access the recipients of first aid actions taken by participants. It is hoped that recipients would experience more helpful behaviours and fewer unhelpful ones and that this would benefit their recovery. While the reports of the participants support these hopes, the evidence is only indirect.
Another limitation is the high level of education and mental health literacy of the participants at baseline. As well as limiting the possible gains from the training, it also means that the generalizability of the findings to other groups is unknown. This bias in participant characteristics might reflect preference for the e-learning mode of instruction or it could relate to selection into a randomized trial.
Finally, there were differences in sample retention across groups, with the waiting list group having the best retention and the CD group the worst. The greater retention in the waiting list group may have been due to the promise that they would receive both the CD and manual at the end of the trial, whereas the CD group already had the intervention that was the focus of the study. Despite the differential retention, the drop-out rate was relatively small and there was no evidence that this biased the findings.
Conclusions
The findings justify the use of both the e-learning version and the manual as methods of imparting mental health first aid knowledge, with the e-learning version superior in reducing stigma and changing helping behaviour. These interventions are most likely to be useful in situations where attendance at face-to-face MHFA training is difficult, such as in workplace situations where it is difficult to roster staff, for people living distant from where courses are available, and for people who have family responsibilities that make it difficult to attend for length of time required. While face-to-face MHFA training is available in many countries, these additional modes of instruction extend the possibilities for dissemination of mental health first aid skills across the population.
Footnotes
Declaration of interest
Betty Kitchener and Anthony Jorm are developers of the Mental Health First Aid training programme. Funding was provided by Australian Rotary Health, a National Health and Medical Research Council Fellowship, and the Colonial Foundation. The development of the e-learning course was supported by funding from the Australian Department of Employment and Workplace Relations. The authors alone are responsible for the content and writing of the paper.