Abstract
Keywords
The present study is part of a research programme exploring the potential utility of computerized cognitive behavioural therapy (CCBT) in enhancing the capacity of existing mental health services (see www.virtualclinic.org.au) to treat high-prevalence internalizing disorders such as anxiety and depression. These disorders affect significant numbers of people and result in considerable disability. The limited availability, however, of skilled mental health professionals and the difficulty of attending treatment during office hour imposes significant barriers to many people with these disorders [1, 2]. CCBT programmes use a computer interface to present structured sessions of cognitive behavioural therapy (CBT) either via the Internet or at a clinic, and may therefore reduce barriers to treatment.
The clinical efficacy or effectiveness of clinician-assisted CCBT programmes has been consistently demonstrated in randomized controlled trials (RCTs) in patients with depression [3, 4], panic disorder [5–9], and social phobia [10–12]. More recent studies have moved from demonstrating ‘proof of concept’ to determining the optimum combination of staff resources, treatment components, and procedures required to result in clinically significant improvements, while maintaining cost-effectiveness [13].
The present study (Shyness 6) is the sixth in a series examining the clinical efficacy and effectiveness of an Internet-based treatment programme for people with social phobia [12, 14–19]. Initial RCTs indicated that 2½ h of clinician time per patient was sufficient to reliably produce short- and longer-term clinical outcomes equivalent to those obtained by good face-to-face treatment [20, 21]. A subsequent RCT explored outcomes in a self-guided version of the Shyness programme without any email or clinician reminders [15]. That condition resulted in a completion rate of 33% and a mean within-group effect size (ES) of 0.66 across two measures of social phobia, indicating potential if a better self-guided programme could be developed and supplemented with reminders.
The self-guided version was subsequently enhanced to include frequent automatic emails and short messaging service (SMS) messages. In a recent RCT, that version resulted in completion rates of 68% and 81%, respectively, for a group who completed the programme independently, as compared to one that received short weekly telephone calls from a technician [17]. The mean within-group ESs across two measures of social phobia for these two groups were 0.86 and 1.15, respectively.
In the present study using the enhanced version of the Shyness programme, participants were randomly allocated to receive a short weekly telephone call from a technician (CCBT + Tel), or access to a clinician-moderated discussion forum (CCBT + Forum). This design tests the reliability of the benefits of a weekly telephone call by a non-clinician reported previously [17], and explores whether another model of guidance is as effective. Replication of the previously obtained results for the CCBT + Tel group would support that model as a potential cost-effective strategy for the dissemination of the Shyness programme. Similar outcomes for the CCBT + Forum group would provide an alternative and equally effective option, but one that could be administered by a clinician.
Method
Participants
Applicants applied online to www.climateclinic.tv (now www.virtualclinic.org.au); details of participant flow are shown in Figure 1. During the 4 weeks of recruitment, 96 individuals applied for this programme and 86 met inclusion criteria, which were the same used in Shyness 4 [17]. Eighty-six individuals were contacted and were given the social phobia section of the Mini International Neuropsychiatric Interview Version 5.0.0 [22] during a telephone interview to determine whether they met DSM-IV [23] criteria for social phobia. A total of 85 applicants met all inclusion criteria and were randomly allocated to CCBT + Tel (n = 43) or CCBT + Forum (n = 42) group.
Shyness 6 study flow chart. CCBT + Tel, computerized cognitive behavioural therapy + weekly telephone group; CCBT + Forum, self-guided + clinician-moderated forum group; MINI, Mini International Neuropsychiatric Interview; PHQ-9, Patient Health Questionnaire–9 Item.
Interventions
Both groups received identical treatments delivered by Internet and computer: six on-line lessons; a summary/homework assignment for each lesson; comments by prior participants about each lesson; access to the supplementary materials; automatic emails and fortnightly SMS messages. Part of the content of each lesson was presented in the form of an illustrated story about a young man with social phobia who, with the help of a clinical psychologist, successfully gains mastery over his symptoms. Lessons 1 and 2 provided education about the symptoms and treatment of social phobia, lesson 3 provided instructions on how to develop an exposure hierarchy and about practising graded exposure, lessons 4 and 5 reinforced principles of graded exposure and demonstrated principles of cognitive restructuring, while lesson 6 included information about relapse prevention. Participants were encouraged to complete the first four lessons within the first 2 weeks, in order to provide more opportunity to practise the graded exposure, cognitive skills and other coping techniques in subsequent weeks. All participants were asked to complete the six lessons within 8 weeks of starting. Telephone group participants (CCBT + Tel) were telephoned each week by a technician, at a time specified by the participant, when they were commended and encouraged to persevere but no clinical advice was offered. Forum group participants (CCBT + Forum) were invited to post messages about progress and questions on a series of online discussion forums moderated by a clinician (NT). CCBT + Forum group participants were instructed that the clinician would read and respond to forum messages each Monday, Wednesday, and Friday. CCBT + Forum group participants also received one introductory telephone call from the clinician at the beginning of the programme.
Hypothesis
It was hypothesized that completion rates and outcomes in the Forum group would be equivalent to the Telephone group.
Outcomes
Applicants completed the Social Interaction Anxiety Scale (SIAS) [24]; the Social Phobia Scale (SPS) [24]; the Patient Health Questionnaire 9 (PHQ-9) [25], the Kessler 10 (K-10) [26]; and the Sheehan Disability Scales (SDS) [27]. Applicants completed a demographic questionnaire and the Credibility/Expectancy Questionnaire (CEQ) [28] to explore whether expectancies or perception of treatment credibility are related to outcomes. Changes in the SIAS and SPS were considered the primary social phobia outcome measures, changes in the PHQ-9, K-10, and SDS were the secondary outcome measures. At post-treatment assessment the participants completed the aforementioned questionnaires and a seven-item treatment satisfaction questionnaire based on the CEQ.
Sample size
Power calculations were based on a non-inferiority trial design comparing parallel-groups. Alpha was set at 0.025, power at 90%, and the mean minimal reliable change index (based on earlier findings) and standard deviations for each group were expected to be equivalent (10 and 12, respectively). Using table VI from Julious [29], the minimal sample size for each group was 35, but more were recruited to increase power.
Randomization
Primary and secondary outcome measures: intention to treat
Statistical methods
Group differences in demographic data, pre-treatment measures, and pre-treatment expectations were analysed on one-way ANOVAs and χ2 tests. Two factor scores (credibility and expectancy) were derived from the CEQ as described [28]. Change scores for each outcome measure were calculated and Pearson product moment correlations (using Bonferroni corrected p) were subsequently calculated between these and the two CEQ factor scores to explore variables that may affect outcomes.
All post-treatment analyses involve an intention-to-treat (ITT) design. Pre–post-treatment changes in participant questionnaire scores were analysed using univariate analyses of covariance (ANCOVAs) [30, 31]. ESs (Cohen's d) were calculated for within-group changes, based on the pooled standard deviation.
The study was approved by the Human Research Ethics Committee (HREC) of St Vincent's Hospital (Sydney, Australia) and the HREC of the University of New South Wales (Sydney, Australia). This trial was registered as ACTRN12609000135213. It is reported as a pragmatic RCT [32] and the results inform decisions about implementation of treatment programmes.
Results
Participant flow
The flow is shown in Figure 1. Recruitment occurred during February 2009 and March 2009. Three CCBT + Forum group participants withdrew before beginning treatment, resulting in 43 CCBT + Tel and 39 CCBT + Forum group participants. Thirty-four (79%) of the CCBT + Tel group and 31 (79%) of the CCBT + Forum group participants completed all six lessons within the required time frame. Consistent with this, CCBT + Tel group participants completed a similar number of lessons (mean = 5.58, SD = 1.03) as participants in the CCBT + Forum group (mean = 5.62, SD = 0.96; p > 0.05).
Baseline data
The mean age of participants was 38.88 years (SD = 12.08), 36 (44%) were men, 35 (43%) were married or in a de facto relationship, 59 (72%) had post-secondary education or training, and 25 (30%) were in full-time employment. Fifty-two (63%) reported having previously talked to a health professional about their symptoms of social phobia, and 23 (28%) reported taking medication for mood and/or anxiety. Thirty-three participants (40%) reported using the Internet more than 10 h per week, and 42 (51%) reported that they were very confident using computers and the Internet. No between-group differences on demographic characteristics, pre-treatment measures or pre-treatment expectations (CEQ factor scores) were observed on one-way ANOVAs (F1,80=0.00–2.30, p > 0.05) or χ2 tests (p>.05).
Numbers analysed
Post-treatment data were collected from 40/43 (93%) CCBT + Tel group participants and 36/39 (92%) CCBT + Forum group participants. In accordance with the ITT paradigm, the pre-treatment scores of the six participants who did not complete the post-treatment questionnaires were replicated as their post-treatment scores.
Primary outcome measures
Univariate ANCOVAs on post-treatment SIAS and SPS scores, controlling for pre-treatment scores (Table 1), did not show a difference between groups in post-treatment SIAS (F1,79=0.79, p < 0.38) or SPS (F1,79=0.80, p < 0.37) scores.
Secondary outcome measures
Univariate ANCOVAs were also conducted on the PHQ-9, K-10, and SDS post-treatment scores, while controlling for pre-treatment scores. These failed to show any significant post-treatment differences between groups for the PHQ-9 (F1,79=0.30, p = 0.59), K-10 (F1,79=0.10, p = 0.76) or SDS (F1,79=0.03, p = 0.86).
Effect sizes
Within-group ESs for the outcome measures are included in Table 1. Large within-group ESs were found for the CCBT + Tel group on the SIAS (1.47), SPS (1.15), K-10 (0.81), and SDS (0.84). Large within-group ESs were also found for the CCBT + Forum group on the SIAS (1.56), SPS (1.51), K-10 (0.95), and SDS (1.08). Smaller within-group ESs were found on the PHQ-9 for the CCBT + Tel group (0.48), and CCBT + Forum group (0.73).
Pre-treatment expectancies and treatment satisfaction
Pre-treatment expectancies, as measured by the expectancy factors from the CEQ, were not significantly correlated with any of the outcome measure change scores. Treatment satisfaction was high and very similar to that obtained in previous studies of the Shyness programme [12, 14–20], with no significant differences between groups in satisfaction levels across a wide range of indices.
Time/contact events spent per participant
The total average time per participant spent by the technician on telephone calls in the CCBT + Tel group and spent by the clinician in the CCBT + Forum group reading and responding to messages on the forum was 38.01 min and 36.92 min, respectively.
Each participant in both groups received three SMS messages; these were sent in bulk, and required approximately 5 min of staff time to send 82 messages. Each participant also received a mean of 13 (SD = 2) automated emails during the programme, including emails sent at the completion of each lesson, reminder emails, and emails providing information about additional resources.
Discussion
The present study tested two models of dissemination of the Shyness programme. Participants were randomly allocated to receive either a short weekly telephone call from a technician (CCBT + Tel), or access to a clinician-moderated discussion forum (CCBT + Forum). It was expected that completion rates and outcomes in the Forum group would be at least equivalent to those in the Telephone group.
Seventy-nine per cent of participants in the CCBT + Tel group completed the six lessons within the required timeframe, <40 min of technician time was required in total per participant during the actual programme (plus an additional 20 min of administrative time including the diagnostic telephone interview), and the mean within-group ES was 1.31. These results closely replicate those reported previously [17], indicating that weekly telephone contact by a non-clinician appears to reliably produce outcomes similar to those reported from good face-to-face treatment programmes for social phobia [20, 21].
Similar results were obtained for the CCBT + Forum group; 79% of these participants completed all six lessons, <40 min of clinician time was required in total during the programme, and the mean within-group ES was 1.54. No differences were found between groups on outcome measures or acceptability to participants.
More than 500 people have now participated in the Shyness programme. The pre-treatment severity of symptoms of social phobia has been non-trivial, and the post-treatment results are reliably similar to face-to-face treatment programmes. Over the course of these six clinical trials the amount of staff time per participant has been reduced from 2½ h to 40 min by providing increasingly sophisticated teaching resources to participants. These resources are systematically released to participants over the duration of each programme, to compensate for the reduced clinician contact. This has occurred without reduction in treatment satisfaction or ESs.
These results raise intriguing questions, in particular, how do we account for these encouraging results in a mental disorder that may be severely debilitating and extremely challenging to treat? Our explanation is that, through experience, we have been able to operationalize basic stages of recovery from social phobia, and have identified skills and knowledge deficits common to each stage. Operationalized in this way, (uncomplicated) social phobia is conceptualized as a skills and knowledge deficit maintained by conditioned fear reactions. Online treatment, therefore, involves guiding patients through stages of acquisition of appropriate knowledge and skills, while de-stigmatizing anxiety and encouraging a perspective of hope and predictability. A simplified analogy is a coaching model, where an expert mentor provides increasing levels of knowledge, skills, and support, based on an individual's level of mastery, personal goals, and levels of motivation. Although we acknowledge that this model is tentative, it raises the question of whether other common mental disorders can be treated in a similar way. Preliminary results will be available at the end of this year for programmes for depression and generalized anxiety disorder that have been constructed using a similar model.
Limitations
These findings need to be replicated by other clinical and research units to further determine their reliability and validity. Manuals for the use of the Shyness programme and software to allow other services to test this programme are being developed.
Conclusions
This study confirms that a self-guided version of the Shyness programme, enhanced with resources, regular reminders, and either weekly telephone calls or access to a clinician-moderated forum, can result in completion rates, clinical outcomes, and acceptability ratings comparable with much face-to-face treatment. Both versions required similar staff time, although it is a pragmatic decision for a practice or service whether to fund either a clinician or a practice nurse. These findings provide further evidence that Internet-based treatment programmes for the common mental disorders may considerably enhance existing mental health services, and indicate that more than one type of contact with patients may be effective and acceptable.
Footnotes
Acknowledgements
The authors gratefully acknowledge the participants for their involvement and helpful comments. We also wish to gratefully acknowledge the contributions to the development of the Virtual Clinic programme by Mr Jason Hando and Mr Daniel Winter.