Abstract
Keywords
Social phobia is a common anxiety disorder with a 12 month prevalence in Australia and New Zealand of approximately 2.3% [1] and 5.1% [2], respectively, and of 7.9% in the USA [3]. While the specific prevalence rates appear to vary with the survey interview and with cultural differences, it is recognized that social phobia contributes to substantial disability [4].
Although social phobia can be treated effectively using cognitive behavioural therapy (CBT), only approximately 20% of people report seeking treatment. The median delay from onset to seeking treatment is 28 years [5]. Reasons for the low treatment rates and delay in treatment include the avoidance and fear of negative evaluation that characterizes social phobia, as well as reasons that are common to low treatment seeking for other common mental disorders including the lack of awareness about treatment options and effectiveness, the potential expense of treatment, barriers associated with geographical location, and the lack of trained therapists.
One potential solution to some of these barriers is the use of computer technology to assist therapists and clinicians to provide psychological therapies via stand-alone computers or via the Internet [6], [7]. Reviews of this field indicate the plethora and diversity of computerized cognitive behavioural therapy (CCBT) programmes, which vary by level of therapist contact, and the resourcing and infrastructure required [7], [8]. For these reasons it is useful to differentiate between programmes that include therapist contact versus those that rely on software and technology alone, without therapist involvement. We have dubbed the former ‘clinician-assisted computerized cognitive behavioural therapy’ (or CaCCBT), while using the term ‘computerized cognitive behaviour therapy’ for systems without therapist contact.
Another reason for differentiating between CaCCBT and CCBT programmes is the clinical utility of such systems. A growing body of literature indicates that although CCBT systems provide an important educational resource, and although clinically helpful to some people, they are associated with lower completion rates and lower effect sizes than CaCCBT systems [6], [7], [9]. In contrast, several CaCCBT systems are associated with completion rates and outcomes comparable to face-to-face therapy. For example, two such CaCCBT systems, Beating the blues [10] and Fearfighter [11], have recently been approved for use in the UK National Health Service [12].
While these two systems rely on face-to-face contact and stand-alone computers, Internet-based programmes supplemented with email or telephone contact have also been demonstrated to be useful in treating common mental disorders. Recent results from randomized controlled trials (RCTs) indicate that Internet programmes supplemented with ≤3 h of email or telephone contact can result in clinically significant improvements in patients with depression [13], social phobia [14], [15], and panic disorder [16–20]. Furthermore, Internet-based programmes offer the additional advantage of overcoming the tyranny of distance as well as allowing patients to access treatment resources at times when clinics are closed. All of these advantages indicate that such programmes offer potential for increasing the capacity of existing mental health services. But before Internet-based treatment programmes for common mental disorders can be considered a regular component of services, numerous practical and clinical questions must be answered [7].
The present study explored the effectiveness of a CaCCBT Internet-based programme for the treatment of social phobia, using a CONSORT (Consolidated Standards of Reporting Trials)-Revised compliant RCT design. Two RCTs and one open study of Internet-based treatment for social phobia have been conducted in Sweden with excellent clinical results [14], [15], [21], but no published reports were found of Internet-based treatment programmes for Australian adults with social phobia. Based on participant feedback, the programme developed for the present study has been named the Shyness programme.
Method
Recruitment and outcome measures
We provided material about the programme for newspaper articles and radio interviews, and interested people were directed to a website created specifically for the study (www.shyness.tv, now www.shynessclinic.tv). This website provided information about social phobia, a description of the study, and a link to apply online to join the programme. The first stage of the recruitment process involved participants completing questionnaires online to determine that they (i) were a resident of Australia; (ii) were at least 18 years of age, (iii) had access to a computer, the Internet, and use of a printer; (iv) were not currently participating in CBT; (v) were not using illicit drugs or consuming more than three standard drinks/day; (vi) were not currently experiencing a psychotic mental illness or severe symptoms of depression (defined as >19, or responding >0 to Question 9 (suicidal ideation) on the Patient Health Questionnaire–Nine Item; PHQ-9 [22]); and (vii) had been taking the same dose for at least 1 month and did not intend to change that dose during the course of the programme (if taking medication for social phobia). Applicants who failed one or more of these criteria were immediately informed via an on-screen message and were sent an email thanking them for their application, regretting that they did not meet the inclusion criteria and encouraging them to discuss their symptoms with their primary care physician. The participant flow, point of randomization, and dropouts at each stage of treatment are shown in Figure 1.
Shyness 1 study flow chart. CBT, cognitive behavioural therapy; CIDI 3.0, Composite International Diagnostic Interview Version 3.0; PHQ-9, Patient Health Questionnaire–Nine Item. †Depression total score >19 on the PHQ-9, or score on item 9 > 0. ‡Three reported unable to begin due to new employment, three were unable to be contacted. §Reasons for not completing or withdrawing included: death or illness in family (n = 2); unexpected holiday (n = 1); pregnancy (n = 1); work or study commitments (n = 3); exposure too stressful (n = 1); did not think programme was effective (n = 1); no reason given (n = 2).
Participants who met the inclusion criteria then completed the following questionnaires online: The Social Interaction Anxiety Scale [23] (SIAS); the PHQ-9 [22], the Social Phobia Scale [23] (SPS), the Kessler-10 [24] (K-10); and the World Health Organization Disability Assessment Schedule (2nd edn) [25] (WHODAS-II). All of these measures are considered reliable, valid, and appropriate for both clinical and research purposes, with recent research indicating that online administration of questionnaires results in responses equivalent to those obtained in paper and pencil administration [26]. The SIAS and SPS are widely used measures of interaction and performance anxiety, respectively, the PHQ-9 measures symptoms of depression, the K-10 measures psychological distress and the WHODAS-II measures disability. Changes in the SIAS and SPS were considered the primary social phobia outcome measures, while the PHQ-9, K-10, and WHODAS-II were the secondary outcome measures.
Applicants then completed a 10-item questionnaire about age, gender, educational attainment, marital status, and employment status, and provided a telephone number and a convenient time for follow-up contact by the researchers.
Applicants who were excluded due to severe levels of depressive symptoms or if they indicated suicidal ideation immediately saw an on-screen message explaining that their symptoms of low mood exceeded the clinical capacity of this programme to provide appropriate support and encouraged them to seek further information about depression and its treatment from two Australian websites (www.beyondblue.org.au and www.crufad.org). The message also encouraged the applicant to discuss their symptoms with their general practitioner or with another health professional. A follow-up email was also sent to applicants to reinforce this message, and they were invited to telephone one of the authors (GA) for further information.
Three hundred and thirty-three people applied, 130 met the inclusion criteria and were telephoned by the researchers to confirm the diagnosis and supply more details about the programme. We administered the social phobia section of the Composite International Diagnostic Interview version 3.0 [27] with additional questions relating to other psychiatric disorders to assist with differential diagnosis derived from the DSM-IV-TR [28]. Twenty-one applicants could not be contacted, resulting in 109 people completing the telephone interviews. In three applicants another disorder was the prime source of difficulty and they were excluded from the study and advised about more appropriate treatment options. One eligible applicant withdrew during the telephone interview. The 105 eligible participants were then sent an informed consent form that they were required to print, complete, and return, before being accepted into the study. All did so.
At post-treatment assessment the participants completed the aforementioned questionnaires, and a seven-item treatment satisfaction questionnaire based on the Credibility/Expectancy Questionnaire [29], [30]. The study was approved by the Human Research Ethics Committee (HREC) of the University of New South Wales (Sydney, Australia), and by the HREC of St Vincent's Hospital (Sydney, Australia).
Procedure
Participants and attrition
The 105 people accepted into the Shyness programme were randomized via a true randomization process (www.random.org) to either the treatment (n = 50) or waitlist control (n = 55) groups. More waitlist control group participants were recruited in anticipation of attrition and, consistent with this expectation, six control group participants withdrew before beginning treatment. The mean age of participants was 38.13 years (SD = 12.24), and 41% of the sample were men. The demographic characteristics of the sample are included in Table 1. Approximately 30% of participants reported having previous treatment, approximately 30% reported currently taking medication for mood and/or anxiety, and the reported mean age of onset was approximately 14.53 years (SD = 9.03). No between-group differences were observed (p > 0.05) on any of these variables using t-tests or χ2 tests, except for employment/vocational status (Pearson's χ2=8.51, p < 0.01), where the treatment group appeared to have more part-time or student participants than the control group. The trial was registered as: ACTRN12607000235404.
Participant details
†Medication and/or psychological treatment.
Treatment programme
The Shyness programme consists of four components: six online lessons; homework assignments; participation in an online discussion forum; and regular email contact with a therapist. The six online lessons were based on the online social phobia lessons from the CLIMATEGP programme (www.climategp.tv) written by JD and ED and modified for the Shyness programme by GA, NT, and GS. The material in each lesson consists of best practice principles and techniques typically used in CBT programmes for social phobia. Part of the content of each lesson is presented in the form of an illustrated story about a young man with social phobia who, with the help of his aunt (a clinical psychologist), learns about his symptoms, learns about treatment, and successfully gains mastery over his symptoms. Lessons 1 and 2 provide education about the symptoms and treatment of social phobia; lesson 3 provides instructions on how to develop an exposure hierarchy; lessons 4 and 5 demonstrate principles of cognitive restructuring; while lesson 6 includes information about relapse prevention. Each lesson begins with a summary of the material described in the previous lesson.
Each lesson also includes a printable summary and homework assignment. Participants were expected to complete the homework tasks prior to completing the next lesson. Participants were also expected to regularly post messages and homework assignments on a secure and confidential online discussion forum, using an alias. The forum software was phpBB 2.0.22. Separate forums were created for the treatment and waitlist control groups, but participants were not able to access the other group's forum. The treatment group was instructed to provide specific messages on the forum, pertaining to the topic covered in the most recent lesson. The waitlist control group was instructed to post only one message in total in their forum, which was a greeting to the other group members. The purpose of this instruction was to try to reduce attrition while waitlist control group participants waited for the treatment group to complete the Shyness programme, at which time the waitlist control group were able to begin the programme. The therapist moderated the treatment group forum and aimed to respond to postings within 24 h. After completing each lesson participants were emailed by the therapist. The themes of the therapist's emails varied from reinforcement for continued participation and efforts, encouragement to practise the relevant treatment skills including graded exposure, encouragement to complete lessons and homework assignments, and enquiries about progress.
Treatment procedure
Treatment group participants began the programme shortly after randomization, and were advised to complete one lesson every 7–10 days and to complete the six lessons within 10 weeks of starting. Feedback on questions was generally provided within 24 h, and reminder emails were sent if necessary. Participants who had not logged into the programme for 2 weeks and had not returned emails were telephoned by the therapist.
Waitlist control group participants completed questionnaires at the same times as the treatment group. The waitlist control group began treatment immediately after the treatment group completed.
Therapist
One clinical psychologist (NT) provided all clinical contact with participants. A research assistant (GS) provided administrative support to collate data, follow progress and assist in the telephone interviews. The duration of each contact with participants was recorded.
Statistical analyses
Group differences in demographic data and pre-treatment measures were analysed using t tests and χ2 tests. Changes in participants’ questionnaire scores were analysed using repeated measures analyses of variance, followed by t-tests with Bonferroni-corrected ps. Effect sizes were calculated as Cohen's d, both within and between groups, based on the pooled standard deviation. All post-treatment analyses involved an intention-to-treat design.
Results
Pre-treatment differences
No between-group differences in pre-treatment questionnaire scores were observed using t-tests (t97=–1.91–0.24, p = 0.06–0.98).
Completion rates
Eleven members of the treatment group (22%) failed to complete all six lessons within the required time frame. Of these non-completers, two formally withdrew citing lack of time and motivation after experiencing a death or illness in the family; one reported that the exposure exercises were too anxiety provoking; one reported that he did not find the programme helpful; one reported taking an overseas holiday; three cited a change in work or study commitments affecting their ability to complete the programme requirements; one reported complications due to her pregnancy and two did not give a reason. The mean number of lessons completed was 5.2, and 39 participants completed all six lessons. The overall completion rate was 78%.
Post-treatment data were collected from 93 participants (44/50 treatment group participants and 49/49 waitlist control group participants). In accordance with the intention-to-treat paradigm, the pre-treatment scores of these six participants who did not complete the post-treatment questionnaires were replicated as their post-treatment scores.
Primary outcome measures
Repeated measures analyses of variance were performed on the two primary outcome measures; the SIAS and SPS. These analyses indicated significant main effects for time, group, and significant main effects for group×time interactions for both the SIAS and SPS (Table 2). Planned post-hoc t-tests with Bonferroni-corrected ps indicated that the treatment group had significantly lower SIAS scores (t97=4.28, p < 0.001) and significantly lower SPS scores (t97=5.19, p < 0.001) than the waitlist control group at the post-treatment assessment.
Primary outcome measures
∗p < 0.05, ∗∗p < 0.01, ∗∗∗p < 0.001.
Secondary outcome measures
Repeated measures analyses of variance were also performed on the three secondary outcome measures: the WHODAS-II, the K-10, and the PHQ-9 (Table 3). These analyses indicated significant main effects for time and group for the WHODAS-II but the time×group interaction was not significant.
Secondary outcome measures
∗p < 0.05, ∗∗p < 0.01, ∗∗∗p < 0.001.
Main effects for time and for the group×time interaction (but not for group) were shown for the Kessler-10. A planned post-hoc t-test with Bonferroni-corrected ps indicated that the treatment group had significantly lower K-10 scores (t97=2.97, p < 0.004) than the waitlist control group at the post-treatment assessment. But for the PHQ-9, only a main effect for time was observed, indicating that both groups experienced reductions in severity of depressive symptoms over the treatment period, but that the treatment group did not report more improvements that the waitlist control group (p > 0.05).
Effect sizes
Effect sizes for the primary and secondary outcome measures are included in Tables 2 and 3, respectively. There were large (>0.8) within- and between-group effect sizes for the primary outcome measures. Effect sizes for the secondary outcome measures varied, with medium within-groups effect sizes for the WHODAS-II (0.63, and 0.67, respectively), large to medium within- and between-group effect sizes for the Kessler-10 (0.82, and 0.60, respectively), and medium and small within- and between-group effect sizes for the PHQ-9 (0.57, and 0.20, respectively).
Treatment satisfaction
The results from the treatment satisfaction questionnaire (Table 4) indicated a high level of satisfaction with the overall treatment, with all participants reporting being either ‘very satisfied’ or ‘mostly satisfied’. More than 97% of participants rated the quality of the treatment modules as ‘excellent’ or ‘good’, and 93% rated the quality of Internet correspondence with the therapist as ‘excellent’ or ‘good’. With respect to the pace or tempo of treatment, 88% reported that there was too little time, 7% indicated that the amount of time was exactly correct, while 5% reported that the tempo was too slow.
Treatment satisfaction
†No. participants providing data.
When asked to provide a rating from 1 to 10, where 10 indicates a high level of agreement, on average, participants rated the treatment as very logical (9/10), they reported feeling very confident that the treatment would be successful at teaching them techniques for managing their symptoms (8/10), and they reported a high level of confidence in recommending this treatment to a friend with social phobia (9/10).
Time spent per participant
It was estimated that the mean amount of therapist time per participant was 125 min (SD = 25 min), including monitoring of the discussion forum and providing email and forum-based feedback, but not including the project development time. An additional average 25 min per patient was required for administrative purposes, including the diagnostic telephone interview. This is comparable to the amount of treatment time reported by two other RCTs of Internet treatment for social phobia [14], [15] with estimates of approximately 2.5–3 h.
Discussion
The Shyness programme resulted in significantly reduced symptoms of social phobia (SIAS and SPS) in the treatment group. While change was also observed in the waitlist control group across time, the reduction in symptoms was much greater in the treatment group. A similar trend was observed on level of disability (WHODAS-II) and on a measure of psychological distress (K-10), providing evidence that the change in symptoms of social anxiety was associated with a reduction in reported disability and psychological distress.
Large within- and between-group effect sizes were obtained, particularly for the SIAS and SPS (within-group effect sizes of 1.24 and 1.06, respectively), with encouraging effect sizes for the K-10 and WHODAS-II (within-group effect sizes 0.82 and 0.63, respectively). The mean within-group effect size across the five outcome measures was 0.86.
The magnitudes of the effect sizes for the symptoms of social phobia (SIAS and SPS results) found in the present study are comparable to those reported by a benchmarking study of anxiety disorders clinics for face-to-face group and individual CBT programmes for social phobia [31], and comparable to two other RCTs of Internet-based treatment programmes for social phobia [14], [15]. Importantly, the pre-treatment severity of the symptoms of social phobia of the participants in the present study was also comparable to these other studies [14], [15], [31].
The Shyness programme was acceptable to participants, in that high levels of satisfaction with the programme were reported (Table 4). The majority (93%) of treatment group participants reported that the quality of Internet correspondence with the therapist was either excellent or good. A review of the postings on the forum indicated a high level of self-disclosure and insightful personal observations early in the treatment programme, which may not be observed until later in face-to-face treatments for social phobia. Participants did report that they generally wanted more time to complete the lessons, although our clinical observations during this programme suggested that increasing the time frame might be counterproductive, in that problems with avoidance and increased attrition were more likely to occur.
There are two threats to the validity of the findings. Improvements were observed across all measures in the waitlist control group, which is likely to have attenuated group differences. We suspect that in addition to the effects of regression to the mean, that this was also due to the availability of the online forum for that group. This forum was provided to try to reduce attrition while the waitlist control group waited for treatment. But during that time that forum was accessed by members of the waitlist control group on more than 500 occasions, indicating that it may have acted as a therapeutic intervention that reduced symptoms. Similar results were observed for the waitlist control group in a previous RCT on Internet treatment for social phobia [14], which included a forum for the control group. A subsequent RCT by the same research group [15], which did not provide waitlist control participants with a forum, did not report a change in waitlist group scores.
The second threat relates to the absence of significant differences between groups after treatment on the measure of depressive symptoms (PHQ-9). A review of the pre-treatment scores suggests that excluding applicants with severe levels of depression resulted in a floor effect (the pre-treatment mean of 8 on the PHQ-9 for each group corresponded to a rating of mild depression). Loosening the inclusion criteria to allow participants with higher levels of depressive symptoms would help determine whether this finding was in fact a methodological artefact.
There are limitations that need to be acknowledged and explored further. Replication is essential and is in progress. The outcomes in the longer term should be determined, and 6 month follow ups with the treatment group participants have been scheduled. This programme excluded children and adolescents and adults with high depression scores. A relaxation of these two inclusion criteria will enable us to determine the characteristics of participants most likely to benefit from such Internet treatment programmes (and thus, information about who is unlikely to benefit). This is particularly important in the case of adolescents [32], [33]. Estimates of the clinical significance of these results also need to be determined, and this will occur when larger samples have completed this programme. Finally, in order to provide results that would be useful in cost-effectiveness analyses, the contribution of the therapist contact element needs to be determined [34].
In conclusion, although considerable research still remains to be conducted to determine the reliability of these findings, to determine the cost-effectiveness of such programmes, and to determine the characteristics of patients who may best benefit, these encouraging but preliminary results provide positive data about the potential utility of Internet-based guided self-help programmes for people with common mental disorders in general and with social phobia in particular.
Footnotes
Acknowledgements
The authors gratefully acknowledge the participants for their involvement and helpful comments. Dr Peter McEvoy and Dr Alison Mahoney kindly provided supervision and encouragement. This project was partially funded by a research grant from the Faculty of Medicine, UNSW.
