An RCT Comparing Effect of Two Types of Support on Severity of Symptoms for People Completing Internet-Based Cognitive Behaviour Therapy for Social Phobia

Abstract

Objective: The present study (Shyness 6) compares the benefits and acceptability of two types of guidance on severity of symptoms during Internet treatment for social phobia.

Methods: Non-inferiority randomized controlled trial of Internet treatment, supplemented with weekly telephone calls from a technician (computerized cognitive-behavioural treatment (CCBT + Tel), or regular access to a clinician-moderated online discussion forum (CCBT + Forum), was carried out. An intention-to-treat model was used for data analyses. The participants consisted of 82 volunteers with social phobia. The intervention consisted of six lessons of CCBT for social phobia (the Shyness programme) with complex automated reminders. The main outcome measures were the Social Interaction Anxiety Scale and Social Phobia Scale.

Results: A total of 79% of CCBT + Tel and 79% of CCBT + Forum group participants completed all six lessons. Large mean within-groups effect sizes (Cohen's d) for the two social phobia measures were found for the CCBT + Tel and CCBT + Forum groups (1.31 and 1.54, respectively). Each participant in the CCBT + Tel group received a mean total of 38 min of technician time over the 8 week programme, while participants in the CCBT + Forum group received a mean total of 37 min of clinician time. Quantitative and qualitative data indicate that both the CCBT + Tel and CCBT + Forum procedures were equally and highly acceptable to participants.

Conclusions: The Shyness programme with either telephone support or access to a clinician-moderated online forum resulted in good clinical outcomes with equivalent patient acceptability. These results confirm that people with social phobia may significantly benefit from a highly structured education programme administered by clinical or non-clinical staff.

Keywords

computerized cognitive behavioural therapy Internet randomized controlled trial self-guided social phobia treatment

The present study is part of a research programme exploring the potential utility of computerized cognitive behavioural therapy (CCBT) in enhancing the capacity of existing mental health services (see www.virtualclinic.org.au) to treat high-prevalence internalizing disorders such as anxiety and depression. These disorders affect significant numbers of people and result in considerable disability. The limited availability, however, of skilled mental health professionals and the difficulty of attending treatment during office hour imposes significant barriers to many people with these disorders [1, 2]. CCBT programmes use a computer interface to present structured sessions of cognitive behavioural therapy (CBT) either via the Internet or at a clinic, and may therefore reduce barriers to treatment.

The clinical efficacy or effectiveness of clinician-assisted CCBT programmes has been consistently demonstrated in randomized controlled trials (RCTs) in patients with depression [3, 4], panic disorder [5–9], and social phobia [10–12]. More recent studies have moved from demonstrating ‘proof of concept’ to determining the optimum combination of staff resources, treatment components, and procedures required to result in clinically significant improvements, while maintaining cost-effectiveness [13].

The present study (Shyness 6) is the sixth in a series examining the clinical efficacy and effectiveness of an Internet-based treatment programme for people with social phobia [12, 14–19]. Initial RCTs indicated that 2½ h of clinician time per patient was sufficient to reliably produce short- and longer-term clinical outcomes equivalent to those obtained by good face-to-face treatment [20, 21]. A subsequent RCT explored outcomes in a self-guided version of the Shyness programme without any email or clinician reminders [15]. That condition resulted in a completion rate of 33% and a mean within-group effect size (ES) of 0.66 across two measures of social phobia, indicating potential if a better self-guided programme could be developed and supplemented with reminders.

The self-guided version was subsequently enhanced to include frequent automatic emails and short messaging service (SMS) messages. In a recent RCT, that version resulted in completion rates of 68% and 81%, respectively, for a group who completed the programme independently, as compared to one that received short weekly telephone calls from a technician [17]. The mean within-group ESs across two measures of social phobia for these two groups were 0.86 and 1.15, respectively.

In the present study using the enhanced version of the Shyness programme, participants were randomly allocated to receive a short weekly telephone call from a technician (CCBT + Tel), or access to a clinician-moderated discussion forum (CCBT + Forum). This design tests the reliability of the benefits of a weekly telephone call by a non-clinician reported previously [17], and explores whether another model of guidance is as effective. Replication of the previously obtained results for the CCBT + Tel group would support that model as a potential cost-effective strategy for the dissemination of the Shyness programme. Similar outcomes for the CCBT + Forum group would provide an alternative and equally effective option, but one that could be administered by a clinician.

Method

Participants

Applicants applied online to www.climateclinic.tv (now www.virtualclinic.org.au); details of participant flow are shown in Figure 1. During the 4 weeks of recruitment, 96 individuals applied for this programme and 86 met inclusion criteria, which were the same used in Shyness 4 [17]. Eighty-six individuals were contacted and were given the social phobia section of the Mini International Neuropsychiatric Interview Version 5.0.0 [22] during a telephone interview to determine whether they met DSM-IV [23] criteria for social phobia. A total of 85 applicants met all inclusion criteria and were randomly allocated to CCBT + Tel (n = 43) or CCBT + Forum (n = 42) group.

Figure 1.

Shyness 6 study flow chart. CCBT + Tel, computerized cognitive behavioural therapy + weekly telephone group; CCBT + Forum, self-guided + clinician-moderated forum group; MINI, Mini International Neuropsychiatric Interview; PHQ-9, Patient Health Questionnaire–9 Item.

Interventions

Both groups received identical treatments delivered by Internet and computer: six on-line lessons; a summary/homework assignment for each lesson; comments by prior participants about each lesson; access to the supplementary materials; automatic emails and fortnightly SMS messages. Part of the content of each lesson was presented in the form of an illustrated story about a young man with social phobia who, with the help of a clinical psychologist, successfully gains mastery over his symptoms. Lessons 1 and 2 provided education about the symptoms and treatment of social phobia, lesson 3 provided instructions on how to develop an exposure hierarchy and about practising graded exposure, lessons 4 and 5 reinforced principles of graded exposure and demonstrated principles of cognitive restructuring, while lesson 6 included information about relapse prevention. Participants were encouraged to complete the first four lessons within the first 2 weeks, in order to provide more opportunity to practise the graded exposure, cognitive skills and other coping techniques in subsequent weeks. All participants were asked to complete the six lessons within 8 weeks of starting. Telephone group participants (CCBT + Tel) were telephoned each week by a technician, at a time specified by the participant, when they were commended and encouraged to persevere but no clinical advice was offered. Forum group participants (CCBT + Forum) were invited to post messages about progress and questions on a series of online discussion forums moderated by a clinician (NT). CCBT + Forum group participants were instructed that the clinician would read and respond to forum messages each Monday, Wednesday, and Friday. CCBT + Forum group participants also received one introductory telephone call from the clinician at the beginning of the programme.

Hypothesis

It was hypothesized that completion rates and outcomes in the Forum group would be equivalent to the Telephone group.

Outcomes

Applicants completed the Social Interaction Anxiety Scale (SIAS) [24]; the Social Phobia Scale (SPS) [24]; the Patient Health Questionnaire 9 (PHQ-9) [25], the Kessler 10 (K-10) [26]; and the Sheehan Disability Scales (SDS) [27]. Applicants completed a demographic questionnaire and the Credibility/Expectancy Questionnaire (CEQ) [28] to explore whether expectancies or perception of treatment credibility are related to outcomes. Changes in the SIAS and SPS were considered the primary social phobia outcome measures, changes in the PHQ-9, K-10, and SDS were the secondary outcome measures. At post-treatment assessment the participants completed the aforementioned questionnaires and a seven-item treatment satisfaction questionnaire based on the CEQ.

Sample size

Power calculations were based on a non-inferiority trial design comparing parallel-groups. Alpha was set at 0.025, power at 90%, and the mean minimal reliable change index (based on earlier findings) and standard deviations for each group were expected to be equivalent (10 and 12, respectively). Using table VI from Julious [29], the minimal sample size for each group was 35, but more were recruited to increase power.

Randomization

The 85 people accepted into the programme were randomized by NT via a true randomization process (www.random.org) to either the CCBT + Tel (n = 43) or CCBT + Forum (n = 42) groups (Table 1). Allocation preceded the screening phone call.

Table 1.

Primary and secondary outcome measures: intention to treat

		Pre treatment		Post treatment		Within-group effect size
Outcome measure	n	Mean	SD	Mean	SD	(Cohen's d)
Social Interaction Anxiety Scale
Telephone group	43	54.26	12.21	35.26	13.57	1.47
Forum group	39	54.59	10.17	37.56	11.56	1.56
Social Phobia Scale
Telephone group	43	35.70	13.24	20.88	12.61	1.15
Forum group	39	35.74	10.15	18.82	12.14	1.51
Patient Health Questionnaire–9 Item
Telephone group	43	8.00	5.84	5.40	4.89	0.48
Forum group	39	8.85	4.76	5.38	4.80	0.73
Kessler 10
Telephone group	43	23.40	6.74	18.21	6.10	0.81
Forum group	39	22.56	6.06	17.56	4.35	0.95
Sheehan Disability Scale
Telephone group	43	14.00	6.35	8.26	7.23	0.84
Forum group	39	14.92	6.28	8.38	5.87	1.08

Statistical methods

Group differences in demographic data, pre-treatment measures, and pre-treatment expectations were analysed on one-way ANOVAs and χ² tests. Two factor scores (credibility and expectancy) were derived from the CEQ as described [28]. Change scores for each outcome measure were calculated and Pearson product moment correlations (using Bonferroni corrected p) were subsequently calculated between these and the two CEQ factor scores to explore variables that may affect outcomes.

All post-treatment analyses involve an intention-to-treat (ITT) design. Pre–post-treatment changes in participant questionnaire scores were analysed using univariate analyses of covariance (ANCOVAs) [30, 31]. ESs (Cohen's d) were calculated for within-group changes, based on the pooled standard deviation.

The study was approved by the Human Research Ethics Committee (HREC) of St Vincent's Hospital (Sydney, Australia) and the HREC of the University of New South Wales (Sydney, Australia). This trial was registered as ACTRN12609000135213. It is reported as a pragmatic RCT [32] and the results inform decisions about implementation of treatment programmes.

Results

Participant flow

The flow is shown in Figure 1. Recruitment occurred during February 2009 and March 2009. Three CCBT + Forum group participants withdrew before beginning treatment, resulting in 43 CCBT + Tel and 39 CCBT + Forum group participants. Thirty-four (79%) of the CCBT + Tel group and 31 (79%) of the CCBT + Forum group participants completed all six lessons within the required time frame. Consistent with this, CCBT + Tel group participants completed a similar number of lessons (mean = 5.58, SD = 1.03) as participants in the CCBT + Forum group (mean = 5.62, SD = 0.96; p > 0.05).

Baseline data

The mean age of participants was 38.88 years (SD = 12.08), 36 (44%) were men, 35 (43%) were married or in a de facto relationship, 59 (72%) had post-secondary education or training, and 25 (30%) were in full-time employment. Fifty-two (63%) reported having previously talked to a health professional about their symptoms of social phobia, and 23 (28%) reported taking medication for mood and/or anxiety. Thirty-three participants (40%) reported using the Internet more than 10 h per week, and 42 (51%) reported that they were very confident using computers and the Internet. No between-group differences on demographic characteristics, pre-treatment measures or pre-treatment expectations (CEQ factor scores) were observed on one-way ANOVAs (F_1,80=0.00–2.30, p > 0.05) or χ² tests (p>.05).

Numbers analysed

Post-treatment data were collected from 40/43 (93%) CCBT + Tel group participants and 36/39 (92%) CCBT + Forum group participants. In accordance with the ITT paradigm, the pre-treatment scores of the six participants who did not complete the post-treatment questionnaires were replicated as their post-treatment scores.

Primary outcome measures

Univariate ANCOVAs on post-treatment SIAS and SPS scores, controlling for pre-treatment scores (Table 1), did not show a difference between groups in post-treatment SIAS (F_1,79=0.79, p < 0.38) or SPS (F_1,79=0.80, p < 0.37) scores.

Secondary outcome measures

Univariate ANCOVAs were also conducted on the PHQ-9, K-10, and SDS post-treatment scores, while controlling for pre-treatment scores. These failed to show any significant post-treatment differences between groups for the PHQ-9 (F_1,79=0.30, p = 0.59), K-10 (F_1,79=0.10, p = 0.76) or SDS (F_1,79=0.03, p = 0.86).

Effect sizes

Within-group ESs for the outcome measures are included in Table 1. Large within-group ESs were found for the CCBT + Tel group on the SIAS (1.47), SPS (1.15), K-10 (0.81), and SDS (0.84). Large within-group ESs were also found for the CCBT + Forum group on the SIAS (1.56), SPS (1.51), K-10 (0.95), and SDS (1.08). Smaller within-group ESs were found on the PHQ-9 for the CCBT + Tel group (0.48), and CCBT + Forum group (0.73).

Pre-treatment expectancies and treatment satisfaction

Pre-treatment expectancies, as measured by the expectancy factors from the CEQ, were not significantly correlated with any of the outcome measure change scores. Treatment satisfaction was high and very similar to that obtained in previous studies of the Shyness programme [12, 14–20], with no significant differences between groups in satisfaction levels across a wide range of indices.

Time/contact events spent per participant

The total average time per participant spent by the technician on telephone calls in the CCBT + Tel group and spent by the clinician in the CCBT + Forum group reading and responding to messages on the forum was 38.01 min and 36.92 min, respectively.

Each participant in both groups received three SMS messages; these were sent in bulk, and required approximately 5 min of staff time to send 82 messages. Each participant also received a mean of 13 (SD = 2) automated emails during the programme, including emails sent at the completion of each lesson, reminder emails, and emails providing information about additional resources.

Discussion

The present study tested two models of dissemination of the Shyness programme. Participants were randomly allocated to receive either a short weekly telephone call from a technician (CCBT + Tel), or access to a clinician-moderated discussion forum (CCBT + Forum). It was expected that completion rates and outcomes in the Forum group would be at least equivalent to those in the Telephone group.

Seventy-nine per cent of participants in the CCBT + Tel group completed the six lessons within the required timeframe, <40 min of technician time was required in total per participant during the actual programme (plus an additional 20 min of administrative time including the diagnostic telephone interview), and the mean within-group ES was 1.31. These results closely replicate those reported previously [17], indicating that weekly telephone contact by a non-clinician appears to reliably produce outcomes similar to those reported from good face-to-face treatment programmes for social phobia [20, 21].

Similar results were obtained for the CCBT + Forum group; 79% of these participants completed all six lessons, <40 min of clinician time was required in total during the programme, and the mean within-group ES was 1.54. No differences were found between groups on outcome measures or acceptability to participants.

More than 500 people have now participated in the Shyness programme. The pre-treatment severity of symptoms of social phobia has been non-trivial, and the post-treatment results are reliably similar to face-to-face treatment programmes. Over the course of these six clinical trials the amount of staff time per participant has been reduced from 2½ h to 40 min by providing increasingly sophisticated teaching resources to participants. These resources are systematically released to participants over the duration of each programme, to compensate for the reduced clinician contact. This has occurred without reduction in treatment satisfaction or ESs.

These results raise intriguing questions, in particular, how do we account for these encouraging results in a mental disorder that may be severely debilitating and extremely challenging to treat? Our explanation is that, through experience, we have been able to operationalize basic stages of recovery from social phobia, and have identified skills and knowledge deficits common to each stage. Operationalized in this way, (uncomplicated) social phobia is conceptualized as a skills and knowledge deficit maintained by conditioned fear reactions. Online treatment, therefore, involves guiding patients through stages of acquisition of appropriate knowledge and skills, while de-stigmatizing anxiety and encouraging a perspective of hope and predictability. A simplified analogy is a coaching model, where an expert mentor provides increasing levels of knowledge, skills, and support, based on an individual's level of mastery, personal goals, and levels of motivation. Although we acknowledge that this model is tentative, it raises the question of whether other common mental disorders can be treated in a similar way. Preliminary results will be available at the end of this year for programmes for depression and generalized anxiety disorder that have been constructed using a similar model.

Limitations

These findings need to be replicated by other clinical and research units to further determine their reliability and validity. Manuals for the use of the Shyness programme and software to allow other services to test this programme are being developed.

Conclusions

This study confirms that a self-guided version of the Shyness programme, enhanced with resources, regular reminders, and either weekly telephone calls or access to a clinician-moderated forum, can result in completion rates, clinical outcomes, and acceptability ratings comparable with much face-to-face treatment. Both versions required similar staff time, although it is a pragmatic decision for a practice or service whether to fund either a clinician or a practice nurse. These findings provide further evidence that Internet-based treatment programmes for the common mental disorders may considerably enhance existing mental health services, and indicate that more than one type of contact with patients may be effective and acceptable.

Footnotes

Acknowledgements

The authors gratefully acknowledge the participants for their involvement and helpful comments. We also wish to gratefully acknowledge the contributions to the development of the Virtual Clinic programme by Mr Jason Hando and Mr Daniel Winter.

References

1. Marks

Cavanagh

Gega

. Hands-on help. Computer-aided psychotherapy. Psychology Press Taylor and Francis, New York, NY 2007.

2. Titov

. Status of computerized cognitive behavioural therapy for adults. Aust N Z J Psychiatry 2007; 41: 95–114.

3. Andersson

Bergstrom

Hollandare

Carlbring

Kaldo

Ekselius

. Internet–based self-help for depression: randomised controlled trial. Br J Psychiatry 2005; 187: 456–461.

4. Perini

Titov

Andrews

. Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial. Aust N Z J Psychiatry 2009; 43: 571–578.

5. Carlbring

Nilsson-Ihrfelt

Waara

. Treatment of panic disorder: live therapy vs. self-help via Internet. Behav Res Ther 2005; 43: 1321–1333.

6. Carlbring

Bohman

Brunt

. Remote treatment of panic disorder: a randomized trial of Internet-based cognitive behavior therapy with telephone calls. Am J Psychiatry 2006; 163: 2119–2125.

7. Kiropoulos

Klein

Austin

. Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT?. J Anxiety Disord 2008; 22: 1273–1284.

8. Shandley

Austin

Klein

. Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists?. J Med Internet Res 2008; 10(2)14.

9. Klein

Richards

Austin

. Efficacy of Internet therapy for panic disorder. J Behav Ther Exp Psychiatry 2006; 37: 213–238.

10.

10. Andersson

Carlbring

Sparthan

. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomised controlled trial. J Consult Clin Psychol 2006; 74: 677–686.

11.

11. Carlbring

Gunnarsdóttir

Hedensjö

Andersson

Ekselius

. Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br J Psychiatry 2007; 190: 123–128.

12.

12. Titov

Andrews

Schwencke

Drobny

Einstein

. Shyness 1: distance treatment of social phobia over the Internet. Aust N Z J Psychiatry 2008; 42: 585–594.

13.

13. Marks

Cavanagh

. Computer-aided psychological treatments: evolving issues. Annu Rev Clin Psychol 2009; 5: 121–141.

14.

14. Titov

Andrews

Schwencke

. Shyness 2: treating social phobia online: replication and extension. Aust N Z J Psychiatry 2008; 42: 595–605.

15.

15. Titov

Andrews

Choi

Schwencke

Mahoney

. Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aust N Z J Psychiatry 2008; 42: 1030–1040.

16.

16. Titov

Andrews

Johnston

Schwencke

Choi

. Shyness programme: longer term benefits, cost-effectiveness, and acceptability. Aust N Z J Psychiatry 2009; 43: 36–44.

17.

17. Titov

Andrews

Choi

Schwencke

Johnston

. Randomized controlled trial of web-based treatment of social phobia without clinician guidance. Aust N Z J Psychiatry 2009; 43: 913–919.

18.

18. Aydos

, Titov

, Andrews

. Shyness 5: clinical effectiveness of Internet-based clinician-assisted treatment for social phobia. Australas Psychiatry in press.

19.

19. Titov

Gibson

Andrews

McEvoy

. Internet treatment for social phobia reduces comorbidity. Aust N Z J Psychiatry 2009; 43: 754–759.

20.

20. McEvoy

. Effectiveness of a cognitive behavioural group therapy for social phobia in a community clinic: a benchmarking study. Behav Res Ther 2007; 45: 3030–3040.

21.

21. Gaston

Abbott

Rapee

Neary

. Do empirically supported treatments generalize to private practice? A benchmark study of a cognitive-behavioural group treatment programme for social phobia. Br J Clin Psychol 2006; 45: 33–48.

22.

22. Sheehan

Lecrubier

Sheehan

. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998; 59(S20)22–33.

23.

23. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, fourth edition, text revision. Washington, DC: American Psychiatric Association, 2000.

24.

24. Mattick

Clarke

. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther 1998; 36: 455–470.

25.

25. Kroenke

Spitzer

Williams

JBW

. The PHQ-9. Validity of a brief depression severity measure. J Gen Intern Med 2001; 16: 606–613.

26.

26. Kessler

Andrews

Colpe

. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32: 959–976.

27.

27. Sheehan

. The anxiety disease. Scribner, New York 1983.

28.

28. Devilly

Borkovec

. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry 2000; 31: 73–86.

29.

29. Julious

. Tutorial in biostatistics: samples sizes for clinical trials with normal data. Stat Med 2004; 23: 1921–1986.

30.

30. Vickers

. Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Med Res Methodol 2005; 5: 35.

31.

31. Vickers

. Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosom Med 2005; 67: 652–655.

32.

32. Zwarenstein

Treweek

Gagnier

. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ 2008; 337: a2390.