Clinician-Assisted Internet-Based Treatment is Effective for Depression: Randomized Controlled Trial

Abstract

Objective: The aim of the present study was to determine the efficacy of an Internet-based clinician-assisted computerized cognitive behavioural treatment (CaCCBT) programme for depression.

Method: Forty-five individuals meeting diagnostic criteria for depression were randomly assigned to the Sadness programme or to a waitlist control group. In the clinician-assisted Sadness programme, participants complete six online lessons, weekly homework assignments, receive weekly email contact from a clinical psychologist, and contribute to a moderated online discussion forum with other participants. An intention-to-treat model was used for data analyses.

Results: A total of 20 (74%) treatment group participants completed all lessons within the 8 week programme, and post-treatment data were collected from 18/27 treatment group and 17/18 waitlist group participants. Treatment group participants reported significantly reduced symptoms of depression as measured by the Beck Depression Inventory–second edition and the Patient Health Questionnaire–Nine Item. Treatment group participants each received an average of eight email contacts (111 min of therapist time]. Mean within- and between-group effect sizes (Cohen's d) across the two measures of depressive symptoms were 0.98 and 0.75, respectively. Participants found the treatment programme acceptable and satisfactory.

Conclusions: These results replicate those from the pilot trial reported by Perini et al. and are consistent with literature indicating that Internet-based programmes for depression and other mental disorders combined with clinical guidance can result in clinically significant improvements. These data provide further support for the development of Internet-based treatment for common mental disorders.

Keywords

clinician-assisted computerized cognitive behavioural therapy depression Internet treatment

Depression is both common and disabling [1–3]. Although it can be treated effectively using cognitive behavioural treatment (CBT) and/or antidepressants, the percentage of people who seek treatment in a 12 month period is low, and not all who seek treatment receive appropriate treatment [4].

Barriers to treatment include the limited availability of clinicians to provide face-to-face CBT, the stigma of attending a mental health professional, and the inability of many patients to attend treatment during working hours. One strategy for reducing these barriers is the development of computerized cognitive behaviour therapy (CCBT) programmes, which present structured sessions of CBT via a computer interface [5, 6].

Attempts at translating features of psychological therapy into computer-delivered programmes have occurred for at least four decades [7], with a significant growth in the number of randomized controlled trials (RCTs) of CCBT programmes in recent years. Such programmes may enhance the capacity of existing services by automatically providing detailed psychoeducation, administering questionnaires, and providing feedback to users. Delegating such tasks to a computer system that then presents evidence-based information in an engaging manner with absolute fidelity can free clinicians to spend time on more sophisticated and specialized components of therapy [5, 6, 8, 9].

In a recent pilot study Perini et al. treated 13 participants who met DSM-IV criteria for major depression with an Internet-based clinician-assisted computerized cognitive behavioural treatment (CaCCBT) programme for depression (the Sadness programme) [10, 11]. The Sadness programme provides structured sessions of CBT for depression, and participants complete six online lessons, weekly homework assignments, receive weekly email contact from a mental health professional, and contribute to a moderated online discussion forum with other participants, over a period of 8–9 weeks.

In that open trial 77% of participants completed the programme within 9 weeks. Using an intention-to-treat (ITT) design, the mean within-group effect size (Cohen's d) on the Patient Health Questionnaire–Nine Item (PHQ-9) [12] and Depression Anxiety Stress Scales (DASS) [13] was 1.0, comparable to that observed in face-to-face treatment [14]. Mean therapist time spent per patient during the programme was 208 min, and participants reported that the programme was both acceptable and helpful.

The Sadness programme is part of a larger research programme exploring the potential utility of Internet-based CCBT programmes (www.climateclinic.tv). Research indicates that Internet-based CaCCBT programmes with email or telephone contact can result in clinically significant improvements in patients with depression [10, 15], social phobia [Titov N et al: unpublished data, 2009; [16–21] and panic disorder [22–26], at a level similar to that provided by face-to-face clinicians, and often requiring less therapist contact time.

The present study contributes to this body of research by extending the materials and methods used in the open trial of the Sadness programme using a CONSORT-Revised compliant RCT design [27]. We hypothesized that treatment group participants would show significant improvement on measures of depression and disability relative to controls; and that participants would rate the procedure as acceptable.

Method

Recruitment

Participants were recruited via a website (www.climateclinic.tv). This website provides information about common mental disorders, including depression, and a link to apply online to join a treatment programme. The first stage of the recruitment process involved participants completing questionnaires online to determine that they: (i) were a resident of Australia; (ii) were at least 18 years of age; (iii) had access to a computer, the Internet, and use of a printer; (iv) were not currently participating in CBT; (v) were not using illicit drugs or consuming more than three standard drinks/day; (vi) were not currently experiencing a psychotic mental illness or severe symptoms of depression (defined as a total score >23 or responding >2 to Question 9 (suicidal ideation) on the PHQ-9 [12]; (vii) had a total score >5 (indicating mild or subthreshold depression) on the PHQ-9; and (viii) if taking medication, had been taking the same dose for at least 1 month and did not intend to change that dose during the course of the programme. Applicants who did not meet all of these criteria were informed via an on-screen message and email thanking them for their application, and encouraging them to discuss their symptoms with their physician. Participants who met the inclusion criteria then completed a 25-item questionnaire enquiring about demographic details and treatment history.

Details about participant flow are given Figure 1. The first stage of the application process is generic to a broader research programme that includes Internet-based treatment programmes for other common mental disorders. During the 6 weeks of recruitment, 420 individuals applied for one of the three treatment programmes at the www.climateclinic.tv website. A total of 103 applied for the Sadness programme. Eleven of these applicants could not be contacted but the remaining 92 were telephoned by the researchers who administered the depression section of the Mini International Neuropsychiatric Interview Version 5.0.0 [28] to determine whether they met DSM-IV criteria for depression. Forty-eight applicants met all inclusion criteria. The unsuccessful applicants were advised about more appropriate treatment options. The study was approved by the Human Research Ethics Committee (HREC) of St Vincent's Hospital (Sydney, Australia) and the HREC of the University of New South Wales (Sydney, Australia), and the trial was registered as ACTRN12609000009213.

Figure 1.

Study flowchart. MINI, Mini-International Neuropsychiatric Interview; PHQ-9: Patient Health Questionnaire–Nine Item.

Participants

The 48 people accepted into the programme were randomized via a true randomization process (www.random.org) to either the Sadness programme (n = 29) or to a waitlist control group (n = 19; Table 1). Two treatment group and one control group participant withdrew before beginning treatment. The mean age of participants was 49.29 years (SD = 12.06). Fifty-three per cent were married or in a de facto relationship, 16% were single, and 31% were separated or divorced. Fifty-six per cent were either in full-time employment or full-time parents, 27% were either in part-time employment or part-time students, and 18% were unemployed, retired, or receiving a disability pension. Most (71%) of participants reported having previously talked to a health professional about their symptoms of depression, and 51% were currently taking medication for depression. Participants used the Internet for an average of 13.73 h per week, with a range of 1–50 h. Eighty per cent of participants said they were either confident or very confident using computers and the Internet. No differences were observed between groups on these variables.

Table 1.

Demographic description of participants

	Treatment group (n = 27)		Waitlist control group (n = 18)		Total (n = 45)
Variable	n	%	n	%	n	%
Gender
Female	23	85.19	12	66.67	35	77.78
Male	4	14.81	6	33.33	10	22.22

Age (years)
Mean age (SD)	49.28	(9.38)	49.30	(15.56)	49.29	(12.06)
Range	24–68		19–85		19–85

Marital status
Single or never married	3	11.11	4	22.22	7	15.56
Married or de facto	16	59.26	8	44.44	24	53.33
Separated or divorced	8	29.63	6	33.33	14	31.11

Education
High School	6	22.22	5	27.78	11	24.44
Tertiary	21	77.78	13	72.22	34	75.56

Employment status
Part-time or student	9	33.33	3	16.67	12	26.67
Full-time work or full-time parent	14	51.85	11	61.11	25	55.56
Unemployed, retired, or disabled	4	14.81	4	22.22	8	17.78

Previously discussed symptoms with health professional
	19	70.37	13	72.22	32	71.11
Taking medication	11	40.74	12	66.67	23	51.11

Mean hours per week of Internet use (±SD)
	13.52	±12.34	14.06	±10.88	13.73	±11.65

Confidence using computers and internet
Very confident	11	40.74	10	55.56	21	46.67
Confident	11	40.74	4	22.22	15	33.33
Average	4	14.81	3	16.67	7	15.56
Mildly confident	1	3.70	1	5.56	2	4.44
Not confident	0	0.00	0	0.00	0	0.00

Outcome measures

One week prior to beginning the trial participants completed the following questionnaires: The PHQ-9, the Beck Depression Inventory–II (BDI-II) [29], the Positive and Negative Affect Scales (PANAS) [30], the Kessler 10 (K-10) [31], and the Sheehan Disability Scales (SDS) [32]. The PHQ-9 and BDI-II are widely used measure of symptoms of depression. A score of 10 on the PHQ-9 has been identified as providing an important threshold for identifying DSM-IV congruent depression [33, 34]. All questionnaires were re-administered 1 week after treatment, while the PHQ-9 and BDI-II were also administered mid-treatment (at week 4). All of these measures are considered reliable, valid, and appropriate for clinical and research purposes, with recent research indicating that online administration of questionnaires results in acceptable reliability of responses [35]. Changes in the PHQ-9 and BDI-II were considered the primary outcome measures, while changes in the PANAS, K10, and SDS were the secondary outcome measures.

Treatment programme

The Sadness programme consists of four components: six online lessons, homework assignments, participation in an online discussion forum, and regular email contact with a mental health clinician. The six online lessons represent best-practice principles used in CBT programmes for depression. Part of the content of each lesson in the Sadness programme is presented in the form of an illustrated story about a woman with depression who, with the help of a clinical psychologist, learns how to gain mastery over her symptoms. Principles and techniques of CBT described in the Sadness programme include behavioural activation, cognitive restructuring, problem solving, and assertiveness skills. Each lesson includes a printable summary and homework assignment. Participants were expected to complete the homework tasks prior to completing the next lesson. Participants were also expected to regularly post messages and homework assignments on a secure and confidential online discussion forum, using an alias. The forum software was phpBB 3.0.1. The therapist moderated the forum and responded to postings within 24 h. After completing each lesson the participants were emailed by the therapist. The themes of the therapist's emails varied from reinforcement for continued participation and efforts, encouragement to practise the relevant treatment skills, encouragement to complete lessons and homework assignments, enquiries about progress, and responses to questions.

Treatment procedure

All participants began the 8 week treatment programme at the same time. Participants were advised to complete one lesson every 7–10 days and to complete the six lessons within 8 weeks of starting. Feedback on questions was generally provided within 24 h, and reminder emails were sent if necessary. Participants who had not logged into the programme for 2 weeks and had not returned emails were telephoned by the therapist.

Therapist

One clinical psychologist (SP) provided all clinical contact with participants. A research assistant provided administrative support to collate data.

Statistical analysis

Group differences in demographic data, pre-treatment measures, and pre-treatment expectations were analysed with one-way ANOVAs and χ² tests. Changes in participants’ questionnaire scores were analysed using repeated measures analyses of covariance (ANCOVAs). This approach is recommended as a robust and reliable statistical strategy for analysing the results of RCTs [36, 37]. Effect sizes (Cohen's d) were calculated both within and between groups, based on the pooled standard deviation. All post-treatment analyses adopted an ITT design.

Pre-treatment and post-treatment PHQ-9 scores were compared with optimum cut-offs for a probable diagnosis of depression using evidence from the literature [12, 34] to provide an index of remission. An estimate of recovery was made by identifying the proportion of participants in each group who demonstrated a significant reduction in their symptoms (defined here as a reduction of 50% of pre-treatment PHQ-9 scores).

Results

Pre-treatment differences

No between-group differences in pre-treatment scores were observed using one-way ANOVAs (F_1,42–43=0.00–0.94, p > 0.05).

Completion rates

Twenty members of the treatment group (74%) completed all six lessons within the required time frame. Reasons why participants did not complete the Sadness programme were not available.

Post-treatment data were collected from 35 participants (18/27 treatment group and 17/18 waitlist control group). The collection of post-treatment data coincided with the end of the year, and this is thought to have affected motivation of treatment group participants who had completed treatment, but not the motivation of control group participants, who were waiting to start their treatment. In accordance with the ITT paradigm, the pre-treatment scores of the participants who did not complete the post-treatment questionnaires were replicated as their post-treatment scores.

Primary outcome measures

Univariate ANCOVAs on post-treatment PHQ-9 and BDI-II scores, controlling for pre-treatment scores (Table 2), showed that the treatment group had significantly lower post-treatment scores than the control group (F_1, ₄₂=8.97, p < 0.01; F_1, ₄₂=6.01, p < 0.02, respectively).

Table 2.

Results of outcome measures (intention to treat)

	Before treatment			After treatment		Effect sizes† (Cohen's d)
Assessment	n	Mean	SD	Mean	SD	Within-group	Between-group

Patient Health Questionnaire–9 Item
Treatment group	27	13.78	4.53	9.59^∗∗	5.82	0.80	0.89
Control group	18	14.72	3.91	14.11^∗∗	4.21	0.15

Beck Depression Inventory II
Treatment group	27	27.30	7.30	17.30^∗	9.86	1.15	0.63
Control group	17	27.24	6.18	23.33^∗	9.29	0.50

Positive and Negative Affect Scales–Positive Affect Scale
Treatment group	27	21.07	5.60	23.93	6.92	0.45	0.41
Control group	17	19.59	3.62	21.47	5.00	0.43

Positive and Negative Affect Scales–Negative Affect Scale
Treatment group	27	25.78	8.25	22.19	8.25	0.44	0.22
Control group	17	26.53	5.19	23.94	7.39	0.41

Kessler 10
Treatment group	27	28.15	4.70	23.70	7.29	0.73	0.23
Control group	17	28.18	4.77	25.24	5.91	0.55

Sheehan Disability Schedule
Treatment group	27	19.00	5.66	14.37	8.39	0.65	0.16
Control group	17	18.29	5.35	15.53	5.35	0.52

†All effect sizes are absolute values.; ∗Post-treatment means differ at p < 0.05, ∗∗post-treatment means differ at p < 0.01.

Secondary outcome measures

Univariate ANCOVAs were also conducted on the PANAS-Positive, PANAS-Negative, K-10, and SDS post-treatment scores, while controlling for pre-treatment scores. These failed to show any significant difference between treatment and control groups for the PANAS-Positive (F_1, ₄₁=0.73, p = 0.40), PANAS-Negative (F_1, ₄₁=0.43, p = 0.52), K-10 (F_1, ₄₁=0.87, p = 0.36) or SDS (F_1, ₄₁=1.03, p = 0.32).

Effect sizes

Within-group effect sizes for the primary outcome measures are included in Table 2. Large within-group effect sizes were found for the treatment group for the PHQ-9 (0.80) and BDI-II (1.15). Large to moderate between-group effect sizes were found for the PHQ-9 (0.89) and BDI-II (0.63).

Clinical outcomes: remission and recovery

Before treatment 22/27 (81%) of the treatment group and 17/18 (94%) of the control group participants had a PHQ-9 score >10, indicating a diagnosis of depression. After treatment (using the ITT design), 15/27 (44%) of the treatment group participants and 16/18 (89%) of the control group participants continued to have a PHQ-9 score >10. Based on the criteria for recovery (a reduction of pre-treatment PHQ-9 scores of at least 50%) at post-treatment assessment, 41% of treatment group participants were classified as recovered, compared with 6% of control group participants.

Treatment satisfaction

Treatment group participants who completed the post-treatment questionnaires reported an acceptable level of satisfaction with the overall programme, with 14/17 (82%) reporting being either very satisfied or mostly satisfied, and 3/17 neutral/somewhat satisfied. Most responding participants (94%) rated the quality of the treatment modules as excellent or good; 71% rated the quality of Internet correspondence with the therapist as excellent or good, while 29% rated it as satisfactory.

When asked to provide a rating from 1 to 10, where 10 indicates a high level of agreement, the average participant rated the treatment as logical (8/10); they reported feeling confident that the treatment would be successful at teaching them techniques for managing their symptoms (7/10); and they reported a high level of confidence in recommending this treatment to a friend with depression (8/10).

Time/contact events per participant

The mean therapist time per participant was 111 min including monitoring of the discussion forum and feedback. An additional average 30 min per patient was required for administrative purposes, including the diagnostic telephone interview. A review of email and forum contact showed that a total of 216 emails were sent to treatment group participants (mean = 8.33 emails per participant), with nine forum postings made to the entire group by the therapist.

Discussion

The aim of the present study was to test the efficacy of an Internet-based CaCCBT treatment programme versus waitlist control for depression using an RCT. All participants met DSM-IV criteria for depression, and 74% completed the six lessons in the required time, a completion rate comparable to those associated with face-to-face treatment for depression [38].

The first hypothesis, that participants would show significant improvements on measures of depression, was supported. Mean within- and between-group effect sizes (Cohen's d) for the PHQ-9 and BDI-II were 0.98 and 0.76, respectively, comparable to those observed in face-to-face treatment of depression [14]. These results are also similar to those obtained in the pilot study of the Sadness programme [10] and to another Internet-based CaCCBT programme for depression [15], but considerably larger than those obtained from CCBT programmes without therapist support [39–42]. Remission rates (PHQ-9 total scores below the cut-off for a likely diagnosis of depression) at post-treatment assessment were 56% and 11% for the treatment and control group participants, respectively, providing further evidence of clinical efficacy. A similar pattern emerged in rates of recovery (i.e. reductions of >50% of pre-treatment PHQ-9 scores), with 41% of treatment group participants and 6% of controls meeting this criterion after treatment.

The second hypothesis, that participants would also show reductions on measures of disability, was not supported: There were no differences in disability levels between groups after treatment. Finally, the third hypothesis, that participants would find the treatment programme acceptable, was supported. That is, 74% of participants completed the programme, 82% of treatment group participants who completed the post-treatment questionnaires reported being either very satisfied or mostly satisfied with the programme, 18% evaluated the programme as neutral, with no participants reporting that they were dissatisfied with the programme.

The results reported here were obtained with a mean 111 min of therapist time per patient. While this duration of therapist time is probably regarded by most as efficient given the magnitude of the effect sizes, it is important to note that several procedures currently conducted by the therapist, such as sending emails to remind participants to complete homework tasks, could be sent automatically by the software, providing additional time for a clinician to address the more sophisticated aspects of treatment. Although CCBT programmes with clinician guidance have previously been associated with higher levels of compliance and better clinical outcomes than self-guided programmes [43], recent evidence from two CaCCBT programmes [Titov N et al: unpublished data, 2009; [44] indicates that considerable savings of therapist time in CaCCBT can be made, with no significant reduction in either clinical outcomes or patient satisfaction. If these results are replicated such programmes may be of significant benefit in enhancing existing mental health services.

Limitations

The relatively small sample size is one limitation of this study, and this issue will be addressed in a forthcoming RCT that will aim to replicate the design and methods with a larger sample. A second limitation was the relatively low completion rates of the post-treatment questionnaires by the treatment group. As indicated, the collection of post-treatment data coincided with the end of the year and this is thought to have affected motivation of treatment group participants who had completed treatment, but not the motivation of control group participants, who were waiting to start their treatment and for whom completion of the questionnaires was a condition of treatment. Low completion rates of the post-treatment questionnaires limits the conclusions about acceptability, and different strategies for collecting data will be used in the next RCT to try to increase completion rates. But because an ITT paradigm was used, the results reported here are likely to underestimate the efficacy of the treatment procedure.

In summary, the results of the present study indicate that the treatment procedure used in the Sadness programme has encouraging clinical efficacy, completion rates comparable to those associated with face-to-face treatment and a procedure that appears acceptable to participants. These results are also consistent with the findings of the pilot study, confirming the reliability of the Sadness programme at reducing symptoms of depression. In conclusion, these encouraging results provide further support for larger scale trials to determine the clinical efficacy and effectiveness of CCBT programmes for the common mental disorders.

Footnotes

Acknowledgements

The authors gratefully acknowledge the participants for their involvement and helpful comments. We also wish to gratefully acknowledge the contributions to the development of the Climate Clinic programme by Mr Jason Hando, and Mr Daniel Winter.

References

1. Andrews

Henderson

Hall

. Prevalence, comorbidity, disability and service utilisation. Br J Psychiatry 2001; 178: 145–153.

2. Wells

Oakley Browne

Scott

McGee

Baxter

Kokaua

. Prevalence, interference with life and severity of 12 month DSM-IV disorders in Te Rau Hinengaro: the New Zealand Mental Survey. Aust N Z J Psychiatry 2006; 40: 845–854.

3. Andrews

Titov

. Depression is very disabling. Lancet 2007; 370: 808–809.

4. Sanderson

Andrews

Corry

Lapsley

. Reducing the burden of affective disorders: is evidence-based health care affordable?. J Affect Disord 2003; 77: 109–125.

5. Marks

Cavanagh

Gega

. Hands-on help. Computer-aided psychotherapy. Psychology Press, New York, NY 2007.

6. Titov

. Status of computerized cognitive behavioural therapy for adults. Aust N Z J Psychiatry 2007; 41: 95–114.

7. Cavanagh

Shapiro

. Computer treatment of common mental health problems. J Clin Psychol 2004; 60: 239–251.

8. Proudfoot

Goldberg

Mann

Everitt

Marks

Gray

. Computerized, interactive, multimedia cognitive behavioural therapy reduces anxiety and depression in general practice: a RCT. Psychol Med 2003; 33: 217–222.

9. Proudfoot

. Computer-based treatment for anxiety and depression: is it feasible? Is it effective?. Neurosci Biobehav Rev 2004; 28: 353–363.

10.

10. Perini

Titov

Andrews

. The Sadness program: an open trial of Internet-based treatment for panic disorder. E-J Appl Psychol [Online] 2008; 4: 2.

11.

11. American Psychiatric Association. Diagnostic and statistical manual of mental disorders, fourth edition, text revision. Washington, DC: American Psychiatric Association, 2000.

12.

12. Kroenke

Spitzer

Williams

JBW

. The PHQ-9. Validity of a brief depression severity measure. J Gen Intern Med 2001; 16: 606–613.

13.

13. Lovibond

Lovibond

. Manual for the Depression Anxiety Stress Scales (DASS). Psychology Foundation of Australia, Sydney 1995.

14.

14. Haby

Donnelly

Corry

Vos

. Cognitive behavioural therapy for depression, panic disorder, and generalized anxiety disorder: a meta-regression of factors that may predict outcome. Aust N Z J Psychiatry 2005; 40: 9–19.

15.

15. Andersson

Bergstrom

Hollandare

Carlbring

Kaldo

Ekselius

. Internet-based self-help for depression: randomized controlled trial. Br J Psychiatry 2005; 187: 456–461.

16.

16. Titov

Andrews

Schwencke

Drobny

Einstein

. Shyness 1: distance treatment of social phobia over the Internet. Aust N Z J Psychiatry 2008; 42: 585–594.

17.

17. Titov

Andrews

Schwencke

. Shyness 2: treating social phobia online: replication and extension. Aust N Z J Psychiatry 2008; 42: 595–605.

18.

18. Titov

Andrews

Choi

Schwencke

Mahoney

. Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aust N Z J Psychiatry 2008; 42: 1030–1040.

19.

19. Titov

Andrews

Johnston

Schwencke

Choi

. Shyness programme: longer term benefits, cost-effectiveness, and acceptability. Aust N Z J Psychiatry 2009; 43: 36–44.

20.

20. Andersson

Carlbring

Holmstrom

. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: a randomised controlled trial. J Consult Clin Psychol 2006; 74: 677–686.

21.

21. Carlbring

Gunnarsdóttir

Hedensjö

Andersson

Ekselius

. Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br J Psychiatry 2007; 190: 123–128.

22.

22. Klein

Richards

Austin

. Efficacy of Internet therapy for panic disorder. J Behav Ther Exp Psychiatry 2006; 37: 213–238.

23.

23. Kiropoulos

Klein

Austin

. Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT?. J Anxiety Disord 2008; 22: 1273–1284.

24.

24. Shandley

Austin

Klein

. Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists?. J Med Internet Res 2008; 10(2)14.

25.

25. Carlbring

Nilsson-Ihrfelt

Waara

. Live therapy vs self-help via internet. Behav Res Ther 2005; 43: 1321–1333.

26.

26. Wims

Titov

Andrews

. The Panic program: an open trial of Internet-based treatment for panic disorder. E-J Appl Psychol [Online] 2008; 4: 2.

27.

27. Moher

, Schulz

, Altman DG for the CONSORT group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials JAMA 2001; 285:1987–1991.

28.

28. Sheehan

Lecrubier

Sheehan

. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry 1998; 59(Suppl 20)22–33.

29.

29. Beck

Steer

Brown

. Beck Depression Inventory II manual2nd edn. Harcourt Brace, San Antonio, TX 1996.

30.

30. Watson

Clark

Tellegen

. Development and validation of brief measures of positive and negative affect: the PANAS Scales. J Pers Soc Psychol 1988; 54: 1063–1070.

31.

31. Kessler

Andrews

Colpe

. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32: 959–976.

32.

32. Sheehan

. The anxiety disease. Scribner, New York 1983.

33.

33. Gilbody

Richards

Brealey

Hewitt

. Screening for depression in medical settings with the Patient Health Questionnaire (PHQ): a diagnostic meta-analysis. J Gen Intern Med 2007; 22: 1596–1602.

34.

34. Löwe

Unützer

Callahan

Perkins

Kroenke

. Monitoring depression treatment outcomes with the Patient Health Questionnaire-9. Med Care. 2004; 42: 1194–1201.

35.

35. Garb

. Computer-administered interviews and rating scales. Psychol Assess 2007; 19: 4–13.

36.

36. Vickers

. Parametric versus non-parametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Med Res Methodol 2005; 5: 35.

37.

37. Vickers

. Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosom Med 2005; 67: 652–655.

38.

38. Kaltenthaler

Sutcliffe

Parry

Beverley

Rees

Ferriter

. The acceptability to patients of computerized cognitive behavioural therapy for depression: a systematic review. Psychol Med 2008; 21: 1–10.

39.

39. Christensen

Griffiths

Jorm

. Delivering interventions for depression by using the internet: randomised controlled trial. Br Med J 2004; 328: 265–269.

40.

40. Mackinnon

Griffiths

Christensen

. Comparative randomised trial of an online cognitive-behavioural therapy and an information website for depression: 12-month outcomes. Br J Psychiatry 2008; 192: 130–134.

41.

41. Clarke

Reid

Eubanks

. Overcoming Depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention program. J Med Internet Res 2002; 4(3)14.

42.

42. Clarke

Eubanks

Reid

. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005; 7(2)16.

43.

43. Spek

Cuijpers

NyklíČek

Riper

Keyzer

Pop

. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007; 37: 319–328.

44.

44. Klien

Shandley

Austin

Nordin

. A pilot trial of ‘Panic Online’ as a self-guided treatment for panic disorder. E-J Appl Psychol [Online] 2008; 4: 2.