Shyness 3: Randomized Controlled Trial of Guided Versus Unguided Internet-Based CBT for Social Phobia

Abstract

Objective: In two previous randomized controlled trials Titov et al. demonstrated significant benefit from an Internet- and email-based treatment programme for social phobia. The present study (Shyness 3) explores whether participants are able to complete this programme independently.

Method: A total of 98 individuals with social phobia were randomly assigned to a clinician-assisted computerized cognitive behavioural treatment (CaCCBT) group, a self-guided computerized CBT (CCBT) group, or to a waitlist control group. CaCCBT group participants completed the usual Shyness programme consisting of six online lessons, cognitive behavioural homework assignments, email contact with a therapist, and participation in an online discussion forum. CCBT group participants accessed the same resources except for therapist emails. An intention-to-treat model was used for data analyses.

Results: A total of 77% of CaCCBT and 33% of CCBT group participants completed all lessons. Significant differences were found after treatment between CaCCBT and control groups (mean between-groups effect size (ES) for the social phobia measures = 1.04), and between the CaCCBT and CCBT groups (mean between-groups ES for the social phobia measures = 0.66). No significant differences were found after treatment between the CCBT and control groups (mean between-groups ES for the social phobia measures = 0.38). CCBT participants, however, who completed the six lessons made good progress (mean within-group ES for the social phobia measures = 0.62). Quantitative and qualitative data indicate that both the CaCCBT and CCBT procedures were acceptable to participants.

Conclusions: The reliability of this Internet-based treatment programme for social phobia has been confirmed. The therapist-guided condition was superior to the self-guided condition, but a subgroup of participants still benefited considerably from the latter. These data confirm that self-guided education or treatment programmes for common anxiety disorders can result in significant improvements.

Keywords

clinician-assisted computerized cognitive behavioural therapy Internet randomized controlled trial self-guided social phobia treatment

High prevalence disorders such as anxiety and depression result in considerable disability [1], [2] but can be treated effectively with cognitive behavioural therapy (CBT) [3–5]. The present study is part of a research programme exploring the potential of computerized CBT (CCBT) to enhance the capacity of mental health services [2]. CCBT presents structured sessions of CBT via computer interface with varying levels of therapist support. Results from recent randomized controlled trials (RCTs) indicate that clinician-assisted CCBT (CaCCBT) with email or telephone contact can result in clinically significant improvements in patients with high prevalence disorders including social phobia [6–9], depression [10] and panic disorder [11–16]. This research indicates that CaCCBT may result in improvements similar to those obtained from face-to-face therapy, but require less therapist contact time. CCBT programmes without therapist guidance providing Internet-based education about anxiety or depression have produced equivocal outcomes [17], [18], and are generally associated with lower completion rates and, therefore, lower mean effect sizes. But, supplementing existing models of service delivery with self-guided programmes may be helpful in educating patients about their disorder and its treatment, and for increasing motivation to participate in future treatment [19]. Self-guided CCBT programmes can be presented to large numbers of people and may also result in significant improvements for some patients because they present content in a consistent manner. Internet-based CCBT programmes reduce barriers to treatment resulting from geographical distance, stigma or from the usual requirement to attend therapy during standard business hours [6], [19], [20].

The present study extends our recent work on the clinical efficacy of an Internet-based treatment programme for people with social phobia (the Shyness programme). In two RCTs (described as Shyness 1 and Shyness 2) Titov et al. randomly allocated a total of 180 participants with social phobia to treatment or waitlist control groups [6], [7]. Treatment consisted of four components: six online lessons (www.climateclinic.tv); cognitive behavioural homework assignments; participation in an online discussion forum; and regular email contact with a therapist. The Shyness 1 programme resulted in large within- and between-group effect sizes across the two primary social phobia outcome measures of 1.15 and 0.95, respectively. These clinically significant effect sizes were obtained with only 2½ h of therapy contact time per participant. That methodology and treatment programme was replicated in Shyness 2, producing similar levels of clinical efficacy for the same amount of therapist time, indicating that the Shyness treatment programme produced reliable results.

The present study, Shyness 3, extends this body of research by removing the therapist from the programme. As indicated here, research with self-guided CCBT programmes for education about anxiety and depression has resulted in equivocal findings, with some RCTs finding no clinical gains [21], [22], while others have reported modest but statistically significant improvements in symptoms [17], [23], [24]. Consistent with these latter studies, some participants from Shyness 1 and 2 appeared capable of making clinically significant improvements in symptoms without reliance on the therapist. Unfortunately, little is known about the characteristics of people who benefit from self-guided CCBT. If the potential of CCBT programmes is to increase the capacity of existing services, then identification of who will benefit from self-guided CCBT, therapist assisted CCBT, or face-to-face therapy, is necessary. Such data will help inform the development of clinical pathways to assist clinicians to utilize scarce treatment resources in the most clinically and cost-effective manner.

The present study used a three-group design. Participants were randomly assigned to either one of two treatment groups; therapist assisted (CaCCBT) or self-guided (CCBT) or to a waitlist control group. It was expected that the CaCCBT group would achieve treatment gains comparable to those observed in the two previous Shyness RCTs, while the CCBT group would achieve treatment gains between those of the CaCCBT and control group. Secondary analyses were concerned with identifying characteristics of participants to determine who may or may not benefit from which CCBT programme, identifying levels of treatment satisfaction in each group, and identifying the impact of each group on subsequent motivation to participate in future treatment.

Method

Recruitment and outcome measures

No significant changes were made either to the inclusion criteria or materials used in the previous Shyness programmes [6], [7], except for the replacement of the World Health Organization Disability Assessment Schedule II (WHODAS-II) [25] with the Sheehan Disability Scale (SDS) [26]. Applicants applied online at a website designed for this programme (www.shyness.tv, now www.climateclinic.tv) and those who met the inclusion criteria completed the following questionnaires prior to treatment: Social Interaction Anxiety Scale (SIAS) [27]; Social Phobia Scale (SPS) [27], the Patient Health Questionnaire Nine-Item (PHQ-9) [28], the Kessler 10 (K-10) [29]; and the SDS [26]. Changes in the SIAS and SPS were considered the primary social phobia outcome measures, changes in the PHQ-9, K-10, and SDS were the secondary outcome measures. To provide data about behavioural change, participants were also asked to estimate the number of activities they had avoided and the number of events they had attended in the previous 7 days (the tertiary outcome measures). Applicants then completed a questionnaire enquiring about demographic characteristics.

Applicants also completed the Anxiety Change Expectancy Questionnaire (ACES) [30] and the Credibility/Expectancy Questionnaire (CEQ) [31] to explore whether expectancies or perception of treatment credibility are related to outcomes.

Applicants with severe symptoms of depression (≥20 on the PHQ-9), or those indicating suicidal ideation were excluded and encouraged to discuss their symptoms with their general practitioner (GP) or with another health professional. A follow-up email was also sent to applicants to reinforce this message, and they were invited to telephone one of the authors (GA) for further information.

One hundred and sixty-four people completed the application (Figure 1), 115 met the inclusion criteria and were invited to return consent forms prior to a telephone interview. Twelve applicants did not return consent forms or could not be contacted by telephone. The remaining applicants were administered the social phobia section of the Mini International Neuropsychiatric Interview Version 5.0.0 (MINI) [32] during a telephone interview to determine whether they met DSM-IV [33] diagnostic criteria for social phobia. In two applicants another disorder was the prime source of difficulty and they were excluded from the study and advised about more appropriate treatment options. Three eligible applicants chose not to participate prior to randomization.

Figure 1.

Study flow chart. DSM-IV, Diagnostic and statistical manual of mental disorders 4th edn; CaCCBT, clinician-assisted computerized cognitive behavioural therapy group; CCBT, self-guided computerized cognitive behavioural therapy group; MINI, Mini International Neuropsychiatric Interview; PHQ-9, Patient Health Questionnaire–Nine-Item.

After treatment, participants completed the aforementioned questionnaires and a seven-item treatment satisfaction questionnaire based on the Credibility/Expectancy Questionnaire [31], [34]. The study was approved by the Human Research Ethics Committee (HREC) of St Vincent's Hospital (Sydney, Australia) and the HREC of the University of New South Wales (Sydney, Australia). This trial was registered as ACTRN12608000192381.

Procedure

Participants and attrition

The 98 people accepted into the programme were randomized via a true randomization process (www.random.org) to either the CaCCBT group (n = 32), CCBT group (n = 31) or waitlist control group (n = 35; Table 1). One CaCCBT, one CCBT and one control group participant withdrew before beginning treatment. The mean age of participants was 37.97 years (SD = 11.29), and 39% were men. Approximately 66% of participants reported having previously talked to a health professional about their symptoms of social phobia, and approximately 22% reported currently taking medication for mood and/or anxiety. Approximately 41% of participants reported using the Internet more than 11 h per week, and approximately 47% reported that they were very confident using computers and the Internet. No between-group differences were observed (p > 0.05) on any of these variables using an analysis of variance (ANOVA) or χ² tests.

Table 1.

Participant details

	CaCCBT (n = 30–31)		CCBT (n = 30)		Control (n = 32–34)		Total (n = 93–95)
Variable	n	%	n	%	n	%	n	%
Gender
Female	17	54.84	23	76.67	18	52.94	58	61.05
Male	14	45.16	7	23.33	16	47.06	37	38.95
Age (years)
Mean age (SD)	39.71	(9.50)	36.86	(10.78)	37.32	(13.22)	37.97	(11.29)
Range	19–9		18–64		18–64		18–64
Marital status
Single or never married	10	32.26	10	33.33	14	43.75	34	36.56
Married or de facto	19	61.29	15	50.00	14	43.75	48	51.61
Separated or divorced	2	6.45	5	16.67	4	12.50	11	11.83
Education
High school	12	40.00	11	36.67	9	28.13	32	34.78
Tertiary	18	60.00	19	63.33	23	71.88	60	65.22
Employment status
Part-time or student	10	32.26	10	33.33	10	31.25	30	32.26
Full-time	20	64.52	17	56.67	17	53.13	54	58.06
Unemployed, retired, or disabled	1	3.23	3	10.00	5	15.63	9	9.68
Previously discussed symptoms with health professional	19	61.29	18	66.67	24	70.59	61	66.30
Taking medication	11	35.48	3	11.11	6	17.65	20	21.74
Hours per week of internet use
≥ 11	12	38.71	12	44.44	14	41.18	38	41.30
0–10	19	61.29	15	55.56	20	58.82	54	58.70
Confidence using computers and Internet
Very confident	15	48.39	14	51.85	14	42.42	43	47.25
Confident	7	22.58	7	25.93	8	24.24	22	24.18
Average	6	19.35	4	14.81	9	27.27	19	20.88
Mildly confident	3	9.68	2	7.41	2	6.06	7	7.69
Not confident	0	0.00	0	0.00	0	0.00	0	0.00

CaCCBT, clinician-assisted computerized cognitive behavioural therapy; CCBT, self-guided computerized cognitive behavioural therapy.

Treatment conditions

The treatment conditions and procedure were carefully described to each applicant before randomization. For ethical reasons, applicants were also informed that those in the CCBT group could, after the conclusion of the programme, choose to subsequently join a CaCCBT group. Surprisingly, approximately half of the applicants who were randomized to the CCBT group immediately reported that they were relieved not to be working with a therapist, reporting that because of their symptoms, they preferred to work through the programme independently.

CaCCBT group

This group received the usual Shyness treatment programme consisting of six online lessons; homework assignments; participation in an online discussion forum; and regular email contact with a therapist. Part of the content of each lesson was presented in the form of an illustrated story about a young man with social phobia who, with the help of a clinical psychologist, successfully gains mastery over his symptoms. Lessons 1 and 2 provide education about the symptoms and treatment of social phobia, lesson 3 provides instructions on how to develop an exposure hierarchy and about practising graded exposure, lessons 4 and 5 reinforce principles of graded exposure and demonstrate principles of cognitive restructuring, while lesson 6 includes information about relapse prevention. Each lesson begins with a summary of the material described in the previous lesson.

Each lesson also includes a printable summary and homework assignment. Participants were expected to complete each homework assignment and post messages about their homework on a secure online discussion forum, using an alias. The forum software was phpBB 2.0.22. The therapist moderated the forum and aimed to respond to postings and to emails within 24 h. After completing each lesson the participants were emailed by the therapist. The themes of the therapist's emails varied from encouragement to practise the relevant treatment skills, answering questions, and encouragement to continue participation and to complete lessons and homework assignments.

CCBT group

This group received the Shyness treatment programme as described in the previous section, but without regular emails or forum responses from the therapist. That is, CCBT group participants had access to the six lessons, homework assignments, and forum. The therapist continued to monitor the forum but did not post messages to individuals. For the purposes of data collection, participants received two emails from the researchers during this programme: one 3 weeks before the end of the termination date of the programme; and a subsequent reminder at 1 week prior to the end of the programme.

Waitlist control group

This group completed questionnaires at the same times as the treatment groups. The control group began CaCCBT treatment shortly after the treatment groups completed.

Treatment procedure

Participants in both treatment groups were directed by email to the forum to read posts providing instructions, guidelines, and advice for completing the programme. Participants were encouraged to complete the first four lessons within the first 2 weeks, in order to provide more opportunity to practise the graded exposure, cognitive skills and other coping techniques. All participants were asked to complete the six lessons within 10 weeks of starting and both treatment groups received an email at week 7 reminding them of the completion date and follow-up procedures. CaCCBT group participants generally received feedback from emails within 24 h. They also received weekly reminder emails and, if necessary, were telephoned by the therapist.

Therapist

One clinical psychologist (NT) provided all clinical contact with participants in the two treatment groups, while another clinical psychologist (AM) subsequently provided treatment for the waitlist group. Two research assistants (IC and GS) provided administrative support to collate data, follow progress and assist in the telephone interviews. The duration of total daily contact with participants was recorded.

Statistical analyses

Group differences in demographic data, pretreatment measures, and pretreatment expectations were analysed with one-way ANOVAs and χ² tests. Two factors scores (credibility and expectancy) were derived from the CEQ by subjecting the scores to a linear transformation with a minimum of 1 and maximum of 9, to create factor scores with a minimum of 3 and maximum of 27 as described in [31]. Change scores for each outcome measure were calculated for each treatment group participant, and Pearson product moment correlations (using Bonferroni corrected p) were subsequently calculated between these and between the two CEQ factors (expectancy and credibility), and the ACES total score to explore variables that may affect outcomes.

Changes in participants’ questionnaire scores were analysed using repeated measures analyses of covariance (ANCOVAs), followed by t-tests with Bonferroni corrected p. This approach is recommended as a robust and reliable statistical strategy for analysing the results of RCTs [35], [36]. Effect sizes (Cohen's d) were calculated both within and between groups, based on the pooled standard deviation.

All post-treatment analyses involve an intention-to-treat (ITT) design. Following these initial analyses, participants in each treatment group were further divided into completers and non-completers. Independent group t-tests were then conducted on pretreatment and post-treatment variables. Effect sizes (Cohen's d) were also calculated for the two groups derived from the CCBT group.

Results

The distributions of all dependent variables were examined, and in order to satisfy assumptions of homogeneity of variance the SIAS and SPS measures were transformed using a square root and logarithmic transformation, respectively. Post-treatment data were collected from 91 participants (30/31 CaCCBT group participants, 27/30 CCBT group participants, and from 34/34 control group participants). In accordance with the ITT paradigm, the pretreatment scores of the four participants who did not complete the post-treatment questionnaires were replicated as their post-treatment scores.

Pretreatment measures

No between-group differences on the pretreatment measures were observed using one-way ANOVAs (F_2,92=0.10–2.28, p > 0.05). No between-group differences in pretreatment expectations (ACES, TEQ, TCQ) were observed using one-way ANOVAs (F_2,92=0.17–1.46, p > 0.05).

Completion rates

Twenty-four members of the CaCCBT group (77%) and 10 members of the CCBT group (33%) completed all six lessons within the required time frame. Of the non-completers from the CaCCBT group four participants (13%) formally withdrew during the programme (two citing work commitments, one citing relationship breakdown, one citing death in the family), one did not complete, citing exam pressures, one did not complete, citing work commitments, and one could not be contacted. Of the non-completers from the CCBT group one participant formally withdrew during the programme (citing work commitments), three did not complete, citing work commitments, 11 reported that they were unwilling to complete the exposure component of treatment independently, while five either could not be contacted or did not provide a reason for not completing.

The mean number of lessons completed was 5.39 (SD = 1.31) and 3.97 (SD = 1.87), for the CaCCBT and CCBT groups, respectively (t₅₉=3.43, p < 0.001).

CaCCBT versus CCBT versus Control

Primary outcome measures

A univariate analysis of covariance on post-treatment SIAS scores, controlling for pretreatment SIAS scores, indicated a significant difference for group (F_2,91=16.17, p < 0.001). Post-hoc pairwise comparisons of groups (Table 2) showed that the CaCCBT group was significantly more improved on the SIAS than the control (p < 0.001), and CCBT groups (p < 0.001)∗∗∗, and no difference between the control and CCBT group was observed.

Table 2.

Results of primary outcome measures:

		Before treatment		After treatment		Effect Sizes
Assessment	n	Mean	SD	Mean	SD	Within	Between
							Control	CaCCBT
Social Interaction Anxiety Scale
CaCCBT group	31	54.71	10.59	40.87^a,b		8.00	1.47	0.99
Control group	34	52.09	13.60	53.06^b		15.53	−0.07	0.99
CCBT group: Total	30	52.50	9.30	48.03^a	13.59	0.38	0.34	0.64
Completers	10	48.70	11.82	40.60	18.40	0.52	0.73	0.02
Non-completers	20	54.40	7.37	51.75	8.84	0.36	0.10	1.29
Social Phobia Scale
CaCCBT group	31	34.71	15.04	18.65^a, ^b		12.20	1.17	1.08
Control group	34	34.38	18.77	35.44^b		18.42	−0.06	1.08
CCBT group: Total	30	32.87	17.02	28.27^a	16.27	0.28	0.41	0.67
Completers	10	26.20	14.20	16.40	13.22	0.71	1.19	0.18
Non-completers	20	36.20	17.64	34.20	14.49	0.12	0.07	1.16

CaCCBT, clinician-assisted computerized cognitive behavioural therapy; CCBT, self-guided computerized cognitive behavioural therapy. Similar superscripts indicate statistically significant differences between groups.

A univariate analysis of covariance on post-treatment SPS scores, controlling for pretreatment SPS scores, also indicated a significant difference for group (F_2,91=17.63, p < 0.001). Post-hoc pairwise comparisons of groups showed that the CaCCBT group was significantly more improved on the SPS than the control (p < 0.001) and CCBT groups (p < 0.002), and no difference was observed between the control and CCBT groups.

Secondary outcome measures

Univariate analysis of covariances were also conducted on the PHQ-9, K-10, and SDS post-treatment scores, while controlling for pretreatment scores. These indicated a significant difference for group for the SDS (F_2,87=3.48, p < 0.001)∗∗∗, but not for the PHQ-9 (F_2,91=2.12, p > 0.05) or K-10 (F_2,91=2.33, p > 0.05). Post-hoc pairwise comparisons of groups (Table 3) indicated a significant difference between the CaCCBT and control group on the SDS (p < 0.003), but not between the CaCCBT and CCBT, or between CCBT and control groups.

Table 3.

Results of secondary and tertiary outcome measures

		Before treatment		After treatment		Effect sizes
Assessment	n	Mean	SD	Mean	SD	Within	Between†
							Control	CaCCBT
Patient Health Questionnaire–Nine-Item
CaCCBT group	31	7.65	4.72	6.10	4.38	0.34	0.30
Control group	34	7.03	5.28	7.56	5.28	−0.10		0.30
CCBT group	30	7.00	5.27	7.10	5.79	−0.02	0.08	0.20
Completers	10	4.40 ^a	2.41	4.70	3.50	−0.10	0.64	0.35
Non-completers	20	8.30^a	5.86	8.30	6.38	0.00	0.13	0.40
Kessler 10
CaCCBT group	31	21.10	6.20	18.94	6.49	0.34	0.28
Control group	34	20.35	6.13	20.74	6.26	−0.06	0.28
CCBT group	30	20.47	5.30	19.00	5.70	0.27	0.29	0.01
Completers	10	18.20	3.85	15.80	4.37	0.58	0.91	0.57
Non-completers	20	21.60	5.63	20.60	5.70	0.18	0.02	0.27
Sheehan Disability Scale
CaCCBT group	31	14.06	8.10	8.84	6.63	0.71	0.37
Control group	34	12.13	7.18	11.50	7.48	0.07	0.37
CCBT group	30	16.48	8.42	12.87	8.54	0.41	0.17	0.52
Completers	10	11.20 ^a	6.71	5.70	5.93	0.87	0.84	0.50
Non-completers	20	19.26 ^a	8.01	16.45	7.35	0.37	0.66	1.09
No. events avoided (in past 7 days)
CaCCBT group	31	2.68	2.76	1.26^a, ^b	1.69	0.62	0.59
Control group	34	2.65	2.44	2.50^b	2.43	0.06	0.59
CCBT group	30	3.20	2.12	2.43^a	2.49	0.33	0.03	0.55
Completers	10	3.10	1.91	1.30	1.42	1.07	0.60	0.03
Non-completers	20	3.25	2.27	3.00	2.73	0.10	0.19	0.77
No. events attended (in past 7 days)
CaCCBT group	31	1.52	1.43	2.10	1.51	0.39	0.10
Control group	34	2.00	2.06	2.29	2.30	0.13	0.10
CCBT group	30	1.60	1.48	1.63	1.56	0.02	0.34	0.30
Completers	10	1.30	0.82	1.70	1.06	0.42	0.33	0.50
Non-completers	20	1.75	1.71	1.60	1.79	−0.09	0.34	1.09

CaCCBT, clinician-assisted computerized cognitive behavioural therapy; CCBT, self-guided computerized cognitive behavioural therapy.; †Between-groups effect sizes are absolute values.

Tertiary outcome measures

A univariate analysis of covariance on the estimate of avoidance, while controlling for pretreatment avoidance, indicated a marginally significant difference for group (F_2,91=3.63, p < 0.003). Post-hoc pairwise comparisons of groups (Table 3) indicated a marginally significant difference between the CaCCBT and control groups on the estimate of avoidance (p < 0.003), but not between the CaCCBT or CCBT, or CCBT and control groups (p > 0.05). Finally, a univariate analysis of covariance on the estimate of attendance, while controlling for pretreatment attendance, failed to show a significant difference for group (F_2,91=0.88, p > 0.05).

Effect sizes

Within groups

Effect sizes for the primary outcome measures are included in Table 2. Large within-group effect sizes were found for the CaCCBT group on the SIAS (1.47) and SPS (1.17). Small and negative within-group effect sizes were found for the control group on both measures, and small but positive effect sizes were found for the CCBT group on the SIAS (0.38) and SPS (0.28).

Between groups

Large effect sizes were observed between the CaCCBT and control groups on the SIAS (0.99) and SPS (1.08). Moderate effect sizes were found between the CaCCBT and CCBT groups on the SIAS (0.64) and SPS (0.67), and small effect sizes were observed between the CCBT and control groups on the SIAS (0.34) and SPS (0.41).

Effect of pretreatment expectancies and perceptions of credibility

Pearson's correlation coefficients were calculated between the ACES total score, the expectancy factors from the CEQ, and the change scores for each group. For the CaCCBT group, the expectancy factor from the CEQ was positively correlated with the SIAS change score (r = 0.51, p < 0.001), and the SDS change score (r = 0.47, p < 0.001) but no other correlations reached the standard of significance. No correlations were found between pretreatment expectations and change in outcome measures for the CCBT group.

CaCCBT group: completers and non-completers

CaCCBT group participants were divided into completers (n = 24) and non-completers (n = 7). No differences between these groups were found before treatment on demographic characteristics, expectations, usage of the Internet, confidence in using computers, or outcome measures (p > 0.05). At post-treatment assessment, independent group t-tests on the change scores for the outcome measures showed that the CaCCBT group completers reported significantly greater improvements than non-completers on the following measures: self-rated avoidance (t₂₉=2.35, p < 0.003), SPS scores (t₂₉=2.06, p < 0.005), K-10 scores (t₂₉=2.84, p < 0.001), and SDS scores (t₂₉=2.21, p < 0.004).

CCBT group: completers and non-completers

CCBT group participants were divided into completers (n = 10) and non-completers (n = 20; Tables 2 and 3). No differences were found between these groups before treatment on demographic characteristics, expectations, usage of the Internet, or confidence in using computers (p > 0.05). Independent groups t-tests on pretreatment outcome measures did not indicate statistically significant differences between completers and non-completers on the SIAS or SPS, but did show that the non-completers had significantly higher pretreatment PHQ-9 scores (t₂₈=2.57, p < 0.002) and SDS scores (t₂₈=2.72, p < 0.001). No other differences were significant. After treatment, independent groups t-tests on the change scores for the outcome measures showed that the completers reported significantly greater improvements than non-completers on the following measures: self-rated avoidance (t₂₈=2.96, p < 0.001), SPS scores (t₂₈=3.09, p < 0.001), and SDS scores (t₂₈=2.12, p < 0.05). The mean between-group effect size (Table 2) on the primary outcome measures between the CCBT group completers and control group was 0.96. Pearson's correlation coefficients were calculated to determine the relationship between the number of lessons completed and change scores, indicating statistically significant relationships between number of lessons completed and changes in SPS scores (r = 0.47, p < 0.008), lessons completed and number of events avoided (r = 0.51, p < 0.004), and lessons completed and number of events attended (r = 0.52, p < 0.003).

Treatment satisfaction

Ninety-seven per cent of participants in the CaCCBT group indicated a high level of satisfaction with the overall treatment (either very satisfied or mostly satisfied) compared with 62% of the CCBT group. The Pearson χ² statistic for this difference was significant (χ²=11.30, p < 0.01). Ninety-seven per cent of CaCCBT group participants rated the quality of Internet correspondence with the therapist as excellent or good; 3% rated it as satisfactory.

When asked to provide a rating from 1 to 10, where 10 indicates a high level of agreement, on average, CaCCBT group participants rated the treatment as very logical (9/10); they reported feeling very confident that the treatment would be successful at teaching them techniques for managing their symptoms (8/10); and they reported a high level of confidence in recommending this treatment to a friend with social phobia (9/10). CCBT group participants did not differ in the ratings they provided to the same questions (p > 0.05).

When asked how participation had affected their confidence that they could learn to manage their symptoms, 93% of CaCCBT and 84% of CCBT group participants reported that it had increased or significantly increased their confidence, while 7% of CaCCBT and 16% of CCBT group participants reported no change (p > 0.05). When asked whether participation had affected their motivation to keep practising the techniques they had learned or to seek ongoing treatment, 93% of CaCCBT and 68% of CCBT group participants reported that it had increased or significantly increased their confidence, while 7% of CaCCBT and 32% of CCBT group participants reported that it had resulted in no change (p > 0.05).

Time/contact events spent per participant (CaCCBT group)

The average therapist time per CaCCBT group participant was 168 min (SD = 40 min) including monitoring of the discussion forum and feedback. An additional average 25 min per patient was required for administrative purposes, including the diagnostic telephone interview. A review of email contact showed that a total of 568 emails were sent to treatment group participants (mean = 18.32, SD = 4.84). Additional contacts included telephone calls to participants who were inactive or had not contacted or responded to the therapist for more than 2 weeks, and regular responses to forum postings (time taken to do this is included in the time noted here). These data do not include time spent on project development.

Discussion

The present study compared the clinical efficacy of a clinician-assisted versus self-guided Internet-based CCBT treatment programme for people with social phobia. Consistent with the previous evaluations of the Shyness programme, the CaCCBT group achieved treatment gains comparable to those previously reported [6], [7]. For example, the CaCCBT group had significantly reduced symptoms of social phobia (SIAS and SPS) after treatment compared to the control group. In addition, and generally consistent with the previous studies, participation in the Shyness programme resulted in a significant reduction in symptoms of disability (SDS) relative to controls. The CaCCBT group also reported a significantly greater reduction in the frequency of avoidance, indicating that the change in questionnaire scores was associated with behavioural change (although a statistically significant increase in the number of events attended was not found).

The pattern and magnitude of effect sizes across the primary outcome measures for CaCCBT group participants was very similar to those observed in the previous Shyness studies. That is, large within-group effect sizes were observed for the SIAS and SPS (1.47 and 1.17, respectively), in addition to large between-group (CaCCBT vs control) effect sizes on these measures (0.99 and 1.08, respectively). The average therapist time per participant (<3 h) was also consistent with previous findings. Furthermore, both the pretreatment severity of symptoms of social phobia and the magnitude of effect sizes observed in the present study are consistent with those seen in face-to-face clinics and from other Internet-based programmes for social phobia [8], [9], [37], [38]. These findings provide further evidence for the reliability of the results produced by the Shyness Programme.

CCBT group

As expected, the CCBT group had lower completion rates than the CaCCBT group (33% and 77%, respectively). The CCBT group generally obtained post-treatment outcome scores between those of the CaCCBT and control groups (Tables 2 and 3), although no statistically significant differences were observed between the CCBT and control groups. After dividing the CCBT group, however, into those who completed the six lessons (n = 10) and those who did not (n = 20), several patterns emerged. First, the completers in the CCBT group had significantly lower symptoms of depression and disability than non-completers, although no differences were observed across demographic variables. This is consistent with clinical expectations, that patients with less severe symptoms are more equipped to complete self-guided programmes. These results are also consistent with those observed in the earlier Shyness programmes [6], [7], but other studies have not consistently reported characteristics of CCBT completers, indicating that the findings reported here need to be replicated with a larger sample and with different diagnostic groups.

Second, the CCBT group completers obtained a mean within-group effect size across the two social phobia measures of 0.62, and a mean between-group effect size (CCBT vs control) of 0.96, indicating that considerable benefit may accrue for an appropriately targeted population. This is consistent with previous research indicating that people who complete CCBT programmes can gain considerable benefit [17], [23], [24].

Finally, a moderate dose effect was observed; that is, a positive correlation was observed between the number of lessons completed and improvements on some of the outcome measures. This is consistent with clinical expectations that participants are more likely to improve with increased exposure to therapeutic content, and raises questions about the barriers to completion. In the present study 37% of the CCBT group reported that they were unable or unwilling to complete the exposure component of the programme. This issue of difficulty level of programme content has not yet received due attention in this literature and the challenges to researchers of determining relevant metrics for difficulty level across different disorders are considerable. It is likely, however, that, as materials typically used in face-to-face treatment are modified (and simplified) for use in therapist-assisted CCBT, those materials might need to be modified (and simplified further) for self-guided CCBT. This model is consistent with a stepped care model of service provision, where CCBT can be used to provide sensible treatment education, CaCCBT to provide guided self-help, while face-to-face treatment is used for people with complex presentations.

Although not measured in the present study, another likely and critical barrier to independent completion of CCBT programmes is motivation. The CaCCBT group received regular emails and forum postings from their therapist that may have served as reminders and encouragement to complete the programme. This contact was absent from the CCBT group. The limited available evidence suggests that reminders are required to help participants complete self-guided CCBT [24], [39], and the form and frequency of these needs to be further explored.

Treatment satisfaction

The completion rate (77%) for the CaCCBT group is consistent with those observed in the previous Shyness RCTs. Furthermore, the levels of treatment satisfaction were consistent between this study and the previous ones. As expected, a smaller proportion of CCBT participants completed the programme (33%) and, in comparison with the CCBT group, overall ratings of satisfaction with treatment were significantly higher in the CaCCBT group. No significant difference, however, was found between the groups’ confidence that they could learn to manage their symptoms, or between the groups’ motivation to keep practising, or to seek ongoing treatment. These questions need to be replicated in future comparisons of CCBT and CaCCBT, because it is important to learn how participants’ experience of CCBT or CaCCBT affect their subsequent motivation for future treatment.

Limitations

An important potential threat to the validity of these findings is the possibility that differences in outcomes between the CaCCBT and CCBT groups were due to the promise that CCBT group participants could subsequently join a therapist-guided programme. This could have resulted in reduced motivation, thus accounting for the low completion rates in the CCBT group. We doubt, however, that this explanation is sufficient, for at least two reasons. First, approximately 50% of participants who were randomized to the CCBT group spontaneously reported that they were pleased to be randomized to that group because they did not wish to work with a therapist. Second, no differences were observed in pretreatment expectations between groups or between the completers and non-completers in the CCBT group.

Other threats to validity include the absence of significant between-group differences across treatment on measures of depression and psychological distress. These patterns have also been observed in the previous Shyness programmes, and we suspect that these are partly due to floor effects (relatively low pretreatment depression scores) and due to the fact that the Shyness Programme does not specifically address these areas. The finding that changes are still observed in symptoms of disability and in levels of self-reported avoidance indicate that change is occurring in behavior.

A critical outstanding issue involves identifying strategies for increasing the completion rates of the CCBT condition, because this is associated with improved outcomes. An RCT with a simplified version of the Shyness programme comparing different types of reminders is being developed to answer this question. The outcomes of that study will inform the development of a CCBT programme for social phobia, and will also inform the development of clearer guidelines about the characteristics of people who are likely to benefit and those who should seek other more complex forms of treatment.

In conclusion, the present results confirm the reliability of this Internet-based treatment programme for social phobia. Approximately 250 Australian adults have now been treated. These results also confirmed expectations that the therapist-guided condition would be superior to the self-guided condition, but that some participants are clearly able to successfully complete such programmes independently and that they may still gain considerable benefit.

Footnotes

Acknowledgements

The authors gratefully acknowledge the participants for their involvement and helpful comments.

References

1. Andrews

Henderson

Hall

. Prevalence, comorbidity, disability and service utilisation. Br J Psychiatry 2001; 178: 145–153.

2. Andrews

Titov

. Changing the face of mental health care through needs-based planning. Aust Health Rev 2007; 31: S122–S128.

3. Barlow

Ellard

Hainsworth

Jones

Fisher

. A review of self-management interventions for panic disorders, phobias, and obsessive-compulsive disorders. Acta Psychiatr Scand 2005; 111: 272–285.

4. Butler

Chapman

Forman

Beck

. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev 2006; 26: 17–31.

5. Otto

Smits

JAJ

Reese

. Cognitive-behavioral therapy for the treatment of anxiety disorders. J Clin Psychiatry 2004; 65(Suppl 5)34–41.

6. Titov

Andrews

Schwencke

Drobny

Einstein

. Shyness 1: a randomized controlled trial of an internet-based treatment for social phobia. Aust N Z J Psychiatry 2008; 42: 585–594.

7. Titov

Andrews

Schwencke

. Shyness 2: a replication of Shyness 1. Are the results reliable, and who benefits?. Aust N Z J Psychiatry 2008; 42: 595–605.

8. Andersson

Carlbring

Sparthan

. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomised controlled trial. J Consult Clin Psychol 2006; 74: 677–686.

9. Carlbring

Gunnarsdóttir

Hedensjö

Andersson

Ekselius

. Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br J Psychiatry 2007; 190: 123–128.

10.

10. Andersson

Bergstrom

Hollandare

Carlbring

Kaldo

Ekselius

. Internet-based self-help for depression: randomized controlled trial. Br J Psychiatry 2005; 187: 456–461.

11.

11. Carlbring

Ekselius

Andersson

. Treatment of panic disorder via the Internet: a randomized trial of CBT vs. applied relaxation. J Behav Ther Exp Psychiatry 2003; 34: 129–140.

12.

12. Carlbring

Nilsson-Ihrfelt

Waara

. Treatment of panic disorder: live therapy vs. self-help via Internet. Behav Res Ther 2005; 43: 1321–1333.

13.

13. Richards

Klein

Austin

. Internet cognitive behavioural therapy for panic disorder: does the inclusion of stress management information improve end-state functioning?. Clin Psychologist 2006; 10: 2–15.

14.

14. Klein

Richards

Austin

. Efficacy of Internet therapy for panic disorder. J Behav Ther Exp Psychiatry 2006; 37: 213–238.

15.

15. Kiropoulos

, Klein

, Austin

Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT?

J Anxiety Disord 2008; 22:1273–1284 [Epub ahead of print].

16.

16. Shandley

Austin

Klein

. Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists?. J Med Internet Res 2008; 10(2)14.

17.

17. Farvolden

Denisoff

Selby

Bagby

Rudy

. Usage and longitudinal effectiveness of a web-based self-help cognitive behavioral therapy program for panic disorder. J Med Internet Res 2005; 7: e7.

18.

18. Spek

Cuijpers

Nyklícek

Riper

Keyzer

Pop

. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007; 37: 319–328.

19.

19. Titov

. Status of computerized cognitive behavioural therapy for adults. Aust N Z J Psychiatry 2007; 41: 95–114.

20.

20. Marks

Cavanagh

Gega

. Hands-on help. Computer-aided psychotherapy. Psychology Press, New York, NY 2007.

21.

21. Clarke

Reid

Eubanks

. Overcoming Depression on the Internet (ODIN): a randomized controlled trial of an internet depression skills intervention program. J Med Internet Res 2002; 4(3)14.

22.

22. Patten

. Prevention of depressive symptoms through the use of distance technologies. Psychiatr Serv 2003; 54: 396–398.

23.

23. Christensen

Griffiths

Jorm

. Delivering interventions for depression by using the internet: randomised controlled trial. BMJ 2004; 328: 265–269.

24.

24. Clarke

Eubanks

Reid

. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005; 7(2)16.

25.

25. World Health Organization. World Health Organization Disability Assessment Schedule, 2nd edn. Available from URL: http://www.who.int/icidh/whodas/ (accessed 20 January 2007).

26.

26. Sheehan

. The anxiety disease. Scribner, New York 1983.

27.

27. Mattick

Clarke

. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther 1998; 36: 455–470.

28.

28. Kroenke

Spitzer

Williams

JBW

. The PHQ-9. Validity of a brief depression severity measure. J Gen Intern Med 2001; 16: 606–613.

29.

29. Kessler

Andrews

Colpe

. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32: 959–976.

30.

30. Dozois

DJA

Westra

. Development of the Anxiety Change Expectancy Scale (ACES) and validation in college, community, and clinical samples. Behav Res Ther 2005; 43: 1655–1672.

31.

31. Devilly

Borkovec

. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry 2000; 31: 73–86.

32.

32. Sheehan

Lecrubier

Sheehan

. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998; 59(Suppl 20)22–33.

33.

33. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th edn, text revision. Washington, DC: American Psychiatric Association, 2000.

34.

34. Borkovec

Nau

. Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry 1972; 3: 257–260.

35.

35. Vickers

. Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Med Res Methodol 2005; 5: 35–46.

36.

36. Vickers

. Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosom Med 2005; 67: 652–655.

37.

37. McEvoy

. Effectiveness of a cognitive behavioural group therapy for social phobia in a community clinic: a benchmarking study. Behav Res Therapy 2007; 45: 3030–3040.

38.

38. Gaston

Abbott

Rapee

Neary

. Do empirically supported treatments generalize to private practice? A benchmark study of a cognitive-behavioural group treatment programme for social phobia. Br J Clin Psychol 2006; 45: 33–48.

39.

39. Mackinnon

Griffiths

Christensen

. Comparative randomised trial of online cognitive-behavioural therapy and an information website for depression: 12-month outcomes. Br J Psychiatry 2008; 192: 130–134.