Abstract
Keywords
Panic disorder and agoraphobia are common mental disorders with 12-month prevalence in Australia of approximately 2.6 and 2.8%, respectively [1]. These conditions can be treated effectively with cognitive behavioural therapy (CBT) [2–4], but fewer than 50% of those with panic disorder report seeking treatment in the preceding 12 months [5].
The present study aimed to explore the efficacy of a new clinician-assisted Internet-based CBT (ICBT) treatment program for panic disorder, the Panic program, using a randomized controlled trial (RCT) design. This program was recently evaluated in a small, open design trial with 10 participants with a diagnosis of panic disorder with agoraphobia [6]. That trial revealed medium to large pre- to post-treatment effect sizes (Cohen's d) on questionnaires measuring agoraphobic avoidance, physical sensations associated with anxiety, catastrophic and anticipatory thoughts, and depression. Seventy percent of participants did not meet criteria for panic disorder with agoraphobia at post-treatment, and the mean therapist time spent per participant over the duration of that program was 206 minutes.
Internet-based treatment programs have potential to overcome barriers to treatment such as the limited availability of clinicians, the stigma of attending a mental health professional, and the inability of many patients to attend treatment during working hours [7]. The Australian Federal Government has recognized the potential of ICBT in enhancing the capacity of existing mental health services and is now funding several clinician and self-guided ICBT programs. It is essential that the efficacy, acceptability, and reliability, of such programs be carefully evaluated [7]. Panic disorder has been the focus of a number of studies of internet delivered computerized CBT. The nine studies listed by Cuijpers et al. demonstrate an average effect size superiority over the control group of ES = 1.00 (range 0.33–1.52) [8].
This study is part of a larger research program developing and evaluating Internet-based structured CBT programs for common mental disorders (see http://www.vitualclinic.org.au). Using a similar methodology, we have recently reported the results of randomized controlled trials (RCTs) evaluating the clinical efficacy and acceptability of clinician-guided ICBT programs for depression [9,10], generalized anxiety disorder [11,12], and social phobia [13–16]. It was hypothesised that participants in the present study would show significant improvements on measures of symptoms of panic disorder and agoraphobia, and that the frequency of panic attacks would significantly decrease following treatment.
Method
Design and power
A CONSORT-Revised compliant [17] RCT of treatment vs. delayed treatment (waitlist control) groups was conducted. Power calculations based on a between-group ES of 0.8, with power of 80%, and α of 0.05 indicated that a total sample size of 42 participants were required, but more were recruited to hedge for attrition. Because of the limited availability of the clinician (EW), recruitment and treatment occurred in two cohorts: 29 people were recruited into the first cohort between September and October 2008, and treated between October 2008 and April 2009; and 25 people were recruited into the second cohort in April 2009, and treated between April 2009 and October 2009. No differences were observed between each treatment and each control group on any pre-treatment variables or demographics, so the groups were combined.
Recruitment
Participants were recruited via a website (http://www.virtualclinic.org.au), providing information about common mental disorders, including panic disorder, and a link to apply online to join a treatment program. The first stage of the recruitment process involved participants completing questionnaires online to determine that they: (i) were a resident of Australia; (ii) were at least 18 years of age; (iii) had access to a computer, the Internet, and use of a printer; (iv) were not currently participating in CBT; (v) were not using illicit drugs or consuming more than three standard drinks/day; (vi) were not currently experiencing a psychotic mental illness or severe symptoms of depression (defined as a total score > 23 or responding > 2 to Question 9 (suicidal ideation) on the Patient Health Questionnaire – 9 Item (PHQ-9) [18]; (vii) if taking medication, had been taking the same dose for at least 1 month and did not intend to change that dose during the course of the program; (viii) and were not taking benzodiazepines. Applicants who did not meet any of these criteria were informed via an on-screen message and email thanking them for their application, and encouraging them to discuss their symptoms with their physician. Participants who met the inclusion criteria then completed a 25-item questionnaire enquiring about demographic details and treatment history.
Details about participant flow are in Figure 1. The first cohort was recruited between 02/9/2008 and 10/10/2008, the second between 3/4/2009 and 17/4/2009, coinciding with the availability of the clinician (EW). During the total of 7.5 weeks of recruitment, 102 individuals applied for the Panic program. Ten applicants failed the exclusion criteria, 11 of these applicants could not be contacted but the remaining 81 were telephoned by the researchers who administered the Mini International Neuropsychiatric Interview Version 5.0.0 (MINI) [19] to determine whether they met DSM-IV [20] criteria for panic disorder with or without agoraphobia and to screen for comorbidity. Fifty-nine applicants met all inclusion criteria. The unsuccessful applicants were advised about more appropriate treatment options. The study was approved by the Human Research Ethics Committee (HREC) of St Vincent's Hospital (Sydney, Australia), and the trial was registered as ACTRN12608000548336.
Study flowchart. PHQ-9: Patient Health Questionnaire – 9 Item, MINI: Mini-International Neuropsychiatric Interview 5.0.0.
Participants
The 59 people accepted into the program were randomized via a true randomization process (http://www.random.org) to either the Panic program (n = 32) or to a waitlist control group (n = 27) (Table 1). Three treatment group and two control group participants withdrew before beginning treatment.
Demographic description of participants
The mean age of participants was 42.08 years (SD = 12.29). Seventy-six percent were female, 57% were married or in a de facto relationship, 28% were single, and 15% were separated or divorced. Fifty-four percent were either in full-time employment or full-time parents, 30% were either in part-time employment or part-time students, and 16% were unemployed, retired, or receiving a disability pension. Most (67%) of participants reported having previously talked to a health professional about their symptoms of depression, and 31% were currently taking medication for panic. Participants used the Internet for an average of 14.60 hours per week, with a range of 1–70 hours. Ninety-one percent of participants said they were either confident or very confident using computers and the Internet. No differences were observed between groups on these variables.
Outcome measures
One week prior to beginning the trial participants completed the following questionnaires online: The Mobility Inventory for Agoraphobia (MI) [21], the Agoraphobic Cognitions Questionnaire (ACQ) [22], the Body Sensations Questionnaire (BSQ) [22], the Patient Health Questionnaire – 9 Item (PHQ-9) [18], and the Sheehan Disability Scale (SDS) [23]. Participants were also telephoned by the clinician who administered the Panic Disorder Severity Scale [24], which also provided estimates of the frequency of panic attacks. The BSQ is a 17-item questionnaire that measures physical sensations, while the MI is a 26-item questionnaire with one optional question, measuring severity of symptoms of agoraphobia in two parts: Firstly participants rate avoidance when alone (MI-Alone) and, secondly, they rate avoidance when accompanied by a trusted person (MI-Accompanied). The ACQ measures cognitions frequently reported in agoraphobia. The PHQ-9 is a valid and reliable measure of symptoms and severity of depression, and the PDSS is a composite measure of all aspects of panic disorder, measuring the frequency of attacks, the severity of anticipatory anxiety, the extent of situational and interoceptive avoidance and also measures occupational and social disability resulting from panic disorder. All of these measures are considered valid and reliable to assess people with panic disorder with agoraphobia, with recent research indicating that online administration of questionnaires results in acceptable reliability of responses [25]. The administration of treatment satisfaction questionnaires was also planned at post-treatment, but these were not administered due to technical problems.
Changes in the PDSS, MI, ACQ, BSQ and frequency of panic attacks were considered the primary outcome measures, while changes in the PHQ-9 and SDS were secondary measures. Outcome questionnaires were re-administered to all participants again 1 week post-treatment and the clinician telephoned all participants at the same time to re-administer the PDSS. Treatment group participants also completed outcome questionnaires and the telephone administered PDSS 1 month post-treatment; however, the control group had begun treatment by that time, and were not administered the measures at that time point.
Treatment program
The Panic program consists of four components: six online lessons, homework assignments, participation in an online discussion forum, and regular email contact with a mental health clinician. The six online lessons represent best practice principles used in CBT programs for panic disorder with or without agoraphobia. Part of the content of each lesson is presented in the form of an illustrated story about a woman with panic and agoraphobia who, with the help of a therapist, learns how to gain mastery over her symptoms. Principles and techniques of CBT described in the Panic program include psychoeducation, graded exposure, cognitive restructuring, physiological de-arousal, and relapse prevention. Each lesson includes a printable summary and homework assignment. Participants were expected to complete the homework tasks prior to completing the next lesson. Participants were also expected to regularly post messages and homework assignments on a secure and confidential online discussion forum, using an alias. The forum software was phpBB 3.0.1. The therapist moderated the forum and responded to postings within 24 hours. After completing each lesson participants were emailed by the therapist. The themes of the therapist's emails varied from reinforcement for continued participation and efforts, encouragement to practice the relevant treatment skills, encouragement to complete lessons and homework assignments, enquiries about progress, and responses to questions.
Treatment procedure
Treatment group participants in each cohort began the 8-week treatment program at the same time. Participants were advised to complete the first three lessons within a week as they consisted of psychoeducation, de-arousal techniques and the principles of cognitive restructuring. Thereafter they were given access to the following lessons at fortnightly intervals and required to complete the six lessons within 8 weeks of starting. Feedback on questions was generally provided within 24 hours, and reminder emails were sent if necessary. Participants who had not logged into the program for 2 weeks and had not returned emails were telephoned by the therapist. Control group participants in each cohort began treatment 1 week after the treatment group completed.
Therapist
One psychiatry registrar (EW) provided all clinical contact with participants. A research assistant provided administrative support to collate data.
Statistical analyses
Group differences in demographic data, pre-treatment measures, and pre-treatment expectations were analyzed with one-way ANOVAs and chi-square tests. Changes in participants’ questionnaire scores were analyzed using repeated measures analyses of covariance (ANCOVAs). This approach is recommended as a robust and reliable statistical strategy for analyzing the results of RCTs [26,27]. Effect sizes (Cohen's d) were calculated both within- and between-groups, based on the pooled standard deviation. All post-treatment analyses adopt an intention-to-treat (ITT) design.
Results
Pre-treatment differences
No between group differences in pre-treatment scores were observed using one way ANOVAs (F 1, 42–43 = 0.01–3.59, p > 0.05).
Completion rates
Twenty-three members of the treatment group (79%) completed all six lessons within the required time frame. Two participants found the program required more time than they were able to set aside, one person dropped out due to increased severity of their anxiety which required inpatient admission, another became ill, another found the course too difficult and the final participant moved house during the program and no longer had internet access.
Post-treatment data was collected from 44 participants (22/29 treatment group and 22/25 waitlist control group). In accordance with the intention-to-treat paradigm, the pre-treatment scores of the participants who did not complete the post-treatment questionnaires were replicated as their post-treatment scores. Post-treatment telephone interviews to administer the PDSS were conducted with 45/54 participants (23/29 treatment group and 22/25 control group), with pre-treatment scores from the nine participants who were not able to be interviewed at post-treatment carried forward.
Primary outcome measures
Separate univariate ANCOVAs on post-treatment PDSS, BSQ, ACQ scores, and frequency of panic attacks (in the last month), controlling for pre-treatment scores (see Table 2), revealed that the Treatment group had significantly lower post-treatment scores than the Control group (F 1, 51 = 13.10, p < 0.01) (F 1, 51 = 8.28, p < 0.01), (F 1, 51 = 4.75, p < 0.03), (F 1, 51 = 5.06, p < 0.03), respectively. Univariate ANCOVAs failed to reveal post-treatment differences in MI-Alone and MI-Accompanied scores after controlling for pre-treatment scores (F 1, 51 = 3.44, p < 0.07) (F 1, 51 = 0.85, p < 0.36), respectively.
Results of outcome measures: Means, standard deviations and pooled effect sizes (Cohen's d) for each group (intention to treat)
Secondary outcome measures
Univariate ANCOVAs were also conducted on the PHQ-9 and SDS post-treatment scores, while controlling for pre-treatment scores. These revealed that the Treatment group had significantly lower PHQ-9 post-treatment scores than the Control group (F 1, 51 = 6.29, p < 0.02) but not post-treatment SDS scores (F 1, 51 = 0.79, p < 0.38).
Effect sizes: Pre- to post-treatment
A large within-groups’ effect size (ES, see Table 2) was found for the Treatment group for the PDSS (0.93). Moderate within-groups’ ESs were found for the Treatment group for the BSQ (0.54), ACQ (0.68), PHQ-9 (0.57), and SDS (0.65). Moderate between-group effect sizes were found for the PDSS (0.59) and ACQ (0.51).
Effect sizes: Pre-treatment to 1-month follow-up
Between pre-treatment and 1-month follow-up large within-groups’ effect size were found for the Treatment group for the PDSS (1.05) and SDS (0.85). Moderate within-groups’ ESs were found for the Treatment group for the BSQ (0.51), PHQ-9 (0.57), and in the number of panic attacks (0.56).
Clinical outcomes: Qualitative data
Of the 23 participants who completed treatment and were interviewed, 21 reported that they had improved as a result of the program. This was evidenced by day to day issues which had resolved for them and had a particular significance for the individual, but which was not readily detected on objective measures. For example, one individual reported going to the local supermarket for the first time in 20 years. Similarly another reported no longer needing to do their shopping over the internet while a third reported going for coffee with a friend for the first time in over 20 years. All individuals reported personally significant, idiosyncratic changes as a result of the program, and while not major, these changes meant a lot for the participants and their families.
Time/contact events per participant
The mean therapist time per participant was 75 minutes including monitoring of the discussion forum and feedback. An additional average 60 minutes per patient was required for administrative purposes, including the diagnostic telephone interview. A review of email and forum contact showed that a total of 218 emails were sent to treatment group participants (mean = 7.5 emails per participant), with 41 forum postings made to the entire group by the therapist.
Discussion
The present study tested the efficacy of a clinician-assisted ICBT treatment program vs waitlist control for panic disorder using a CONSORT-revised compliant RCT. All participants met DSM-IV criteria for panic with or without agoraphobia, and 79% completed the six lessons in the required time, a completion rate comparable to those associated with face to face treatment for panic disorder in the literature [28] and based on our experience in our face to face clinic.
The first hypothesis, that participants would show significant improvements on measures of panic, was supported. Mean within- and between-group effect sizes (Cohen's d) for the PDSS were 0.93 and 0.59, respectively, comparable to those observed in other ICBT programs for panic [29,30] and to face to face treatment of panic [31]. Remission rates (PDSS total scores below 8 is the cut-off for a likely diagnosis of panic [32]) at post-treatment were 31 and 8% for the treatment and control group participants respectively, providing further evidence of clinical efficacy. This effect was strengthened at one month follow up with an increase to 41% in the treatment group. If we consider remission in terms of DSM IV criteria with fewer than two panic attacks in the last month, remission rates increase to 45 and 12% in treatment and control groups, respectively, at post treatment, and 59% in the treatment group at 1 month follow up. Clinically, of the 23 participants in the treatment group who were interviewed post treatment, 21 said they had improved based on qualitative changes and as evidenced by improvement on the Sheehan Disability Scale. This included one individual who experienced an increase in panic attacks which they attributed to being more active, and avoiding less post treatment. All those who completed had reductions in their PDSS (pre – 1 month follow up) in the range of 15 – 1 points (mean 6). Similarly 18 out of 23 completers had reductions in the number of panic attacks they experienced.
The second hypothesis, that treatment group participants would also show reductions on frequency of panic attacks relative to controls was supported, providing further evidence of efficacy. At pre-treatment, treatment group participants reported a mean of 8.03 panic attacks in the previous 4 weeks, 3.72 at post-treatment, and 2.76 at 1-month post-treatment, compared to 5.84, 8.20, and 8.60 for the control group at the same time points. Further evidence of efficacy was obtained from the significant reductions in disability reported by the treatment group compared to the control group.
While encouraging, the magnitude of improvements in the treatment group were below those expected from this program based on the open trial [6] and the results from the same research team exploring the efficacy of ICBT for other common mental disorders, including depression [9,10,33], generalized anxiety disorder [11,12], and social phobia [13–16,34–36]. Several factors are likely to account for this. Firstly, even though the same therapist conducted treatment in both groups the amount of therapist time per participant (75 minutes) in the present trial was considerably less than those in the open trial [6]. This may have adversely affected outcome contrary to the findings of Klein et al. [37]. Despite the reduced time the number of emails per participant did not reduce, although there was significantly reduced number of postings to the forum. Secondly, no additional resources were provided to participants to offset against the reduction in therapist time. For example, other ICBT programs from this research team have added additional resources, automatic reminder emails, and stories from previous participants, which were not included in this study.
Limitations
The absence of treatment satisfaction questionnaires due to technical problems was one limitation of this study. Such measures provide an index of acceptability, a key criteria that determines the ability to implement a new program. A second limitation was the relatively small sample size, and this issue will be addressed in a forthcoming RCT that will aim to replicate the design and methods with a larger sample. A third limitation was the absence of automatic reminders and email prompts, which have been demonstrated to considerably improve clinical outcomes [15,34]. Finally the lack of blinding in the administration of the PDSS is a source of bias, which may account for the larger effect sizes in this domain. This issue will be addressed in future RCTs by the administration of the self rating version.
In summary, the results of the present study indicate that the treatment procedure employed in the Panic program has encouraging clinical efficacy, completion rates comparable to those associated with face to face treatment and a procedure that appears acceptable to participants. These results are also consistent with the findings of the pilot study, confirming the reliability of the Panic program at reducing symptoms of panic. In conclusion, ICBT would appear to be as effective as face to face treatment with a marked reduction in clinician time required (based on our face to face group program with average clinician time of 320 minutes per patient). Clearly there is room for further improvements in these individuals, but perhaps the benefit of ICBT is a freeing up of clinician time to focus on those who require more input, or to prime participants before receiving face to face therapy. The impact of ICBT prior to face to face treatment requires investigation.
Footnotes
Acknowledgements
The authors gratefully acknowledge the participants for their involvement and helpful comments. We also wish to gratefully acknowledge the contributions to the development of the VirtualClinic program by Mr Jason Hando, and Mr Daniel Winter. Finally we would like to thank Karen Solley and Luke Johnson for administrative support and the entire CRUfAD team for ongoing motivation and encouragement.