Sage Journals: Discover world-class research

Abstract

Objective: In two randomized controlled trials Titov et al. demonstrated significant benefit from an Internet- and email-based treatment programme for social phobia: the Shyness programme. Data are presented about the longer term outcomes (6 months after treatment), cost-effectiveness relative to face-to-face treatment, and the acceptability of the programme to participants.

Method: Participants completed outcome and acceptability questionnaires at 6 months after treatment. Repeated measures analyses of variance were calculated using an intention-to-treat design. Cost-effectiveness in years lived with disability averted were calculated based on between-group effect sizes.

Results: A total of 59% of treatment group participants completed the 6 month follow-up questionnaires. Between post-treatment and 6 month follow up participants continued to make improvements in symptoms of social phobia, while maintaining improvements in mood, psychological distress, and disability. At 6 month follow up the mean within-group effect size (Cohen's d) for the two social phobia measures increased from 1.2 to 1.4. Cost-effectiveness in years lived with disability (YLD) averted was calculated as one-quarter that of face-to-face group treatment, or $AUD1495 for one YLD gained, compared to $AUD5686/YLD gained. Participants rated the Internet treatment to be as effective and helpful as face-to-face treatment.

Conclusions: The present results confirm the reliability of the short-term findings reported in the first two Shyness programmes. The procedure appears to be very cost-effective, and acceptable to participants. These data provide further support for the development of Internet-based virtual clinics for common mental disorders.

Keywords

clinician-assisted computerized cognitive behavioural therapy Internet randomized controlled trial social phobia treatment virtual clinic

Three recent Australian randomized controlled trials (RCTs) demonstrated that computerized cognitive behavioural treatment (CCBT) programmes administered via the Internet can significantly reduce symptoms of social phobia [1–3]. Across these studies the clinician-assisted computerized CBT (CaCCBT) programme (the Shyness programme) resulted in a mean within-group effect size (Cohen's d) across two measures of social phobia of 1.2. This effect size is comparable to that observed in face-to-face treatment [4], and by other Internet-based treatment programmes for social phobia [5, 6], but required <50% of the comparable face-to-face clinician time. Similar results have been reported for the clinical efficacy and saving in therapist time with CaCCBT programmes developed to treat panic disorder [7–11], and depression [12]. These results are consistent with a growing body of evidence supporting the clinical efficacy and effectiveness of CaCCBT programmes for anxiety and depression [13, 14].

The present study contributes to this body of knowledge by exploring the longer term results of the Shyness programme, its cost-effectiveness, and acceptability to participants. Such data are necessary for determining how CaCCBT programmes can best contribute to enhancing existing mental health services. A number of strategies exist for evaluating the different aspects of cost-effectiveness and acceptability [15, 16]. The approaches reported here were selected to provide conservative, but tangible comparisons thought likely to be relevant to clinicians.

The present study utilized the 6 month follow-up data from the treatment group participants from the first two Shyness RCTs [1, 2], described as Shyness 1 and Shyness 2, respectively. The present aims were (i) to determine whether the benefits of the Shyness programme were maintained at 6 month follow up; (ii) to provide estimates of the cost-effectiveness of the CaCCBT Shyness programme compared to face-to-face treatment; and (iii) to explore the acceptability of the Shyness programme to participants, including their perceptions of advantages and disadvantages of Internet treatment.

Method

No significant differences were made to the inclusion criteria, recruitment procedure, treatment procedure, or materials used in the first two Shyness RCTs, which are described in previous reports [1, 2] and summarized here.

Recruitment and retention

Participants were recruited via a website designed for this programme (www.shyness.tv, now www.climateclinic.tv). Inclusion and exclusion criteria were (i) Australian residence; (ii) age ≥18 years; (iii) regular access to the Internet; (iv) not currently participating in CBT; (v) no use of illicit drugs or consumption of >3 standard drinks/day; (vi) no history of psychosis or current severe symptoms of depression (defined as >19 on the Patient Health Questionnaire 9 Item; PHQ-9) [17]; (vii) ≥1 month stability of social phobia medication dose (if taking medication) and lack of intention to change that dose during the course of the programme; and (viii) meeting of DSM-IV criteria for social phobia.

A total of 180 participants began Shyness 1 or 2. In Shyness 1, 50 participants were randomized to the treatment group and 49 participants to the waitlist control group. Post-treatment data were provided by 44/50 treatment group participants, and 25/50 treatment group participants provided 6 month data. In Shyness 2, 40 participants were randomized to the treatment group and 41 participants to the waitlist control group. Post-treatment data were provided by 38/41 treatment group participants, and 29/41 treatment group participants provided 6 month data. Recruitment and retention details are summarized in Figure 1. At the 6 month follow up the treatment group participants were emailed or telephoned on up to three occasions to complete questionnaires.

Figure 1.

Shyness 1 and 2 (combined) study flow chart. CBT, cognitive behavioural therapy; CIDI 3.0, Composite International Diagnostic Interview Version 3.0; DSM-IV-TR, Diagnostic and statistical manual of the American Psychiatric Association–4th edn (Text Revision); PHQ-9, Patient Health Questionnaire–9 Item. †Depression total score >19 on the PHQ-9, or score on item 9 >0.

Measures

The following questionnaires were administered before treatment, 1 week after treatment, and at 6 month follow up: Social Interaction Anxiety Scale [18] (SIAS); Social Phobia Scale [18] (SPS), PHQ-9 [17], Kessler 10 [19] (K-10); and the World Health Organization Disability Assessment Schedule II (WHODAS-II) [20]. Changes in the SIAS and SPS were considered the primary social phobia outcome measures. Applicants also completed a questionnaire enquiring about demographic characteristics. Waitlist control group participants also completed these questionnaires at before and after treatment, but not at 6 month follow up because they began treatment immediately after their respective treatment group. At 6 month follow up the treatment groups were invited to complete questionnaires enquiring about the acceptability of the Shyness programme. Because of the absence of standardized questionnaires in this field, a questionnaire was developed specifically for this purpose and consisted of questions based on the creditability/expectancy questionnaires [21, 22] and included questions about participants’ preferences for Internet-based or face-to-face treatment for social phobia. Participants were also asked to describe advantages and disadvantages of Internet-based treatment.

Procedure

The treatment programme operated over 8–10 weeks and consisted of four components: six online lessons (see www.climateclinic.tv); cognitive behavioural homework assignments; participation in an online discussion forum; and regular email contact with a therapist. The content of the programme consists of education about symptoms and treatment of social phobia, instructions and guidelines for developing an exposure hierarchy and beginning exposure tests, information and guidelines about cognitive restructuring and information about relapse prevention. Downloadable homework assignments and summary sheets are provided for each of the six online lessons. Treatment group participants generally received feedback from emails within 24 h, they also received weekly reminder emails and, if necessary were telephoned by the therapist. These RCTs were registered as ACTRN12607000235404.

Statistical analysis

All analyses include treatment group participants from Shyness 1 and 2 only; waitlist control participants did not complete the 6 month questionnaires because they began treatment before the 6 month follow up. Independent group t-tests and χ² tests were used to determine whether the Shyness 1 and 2 treatment groups were equivalent. No differences were seen (p > 0.05) on any pre- or post-treatment assessments, or 6 month follow-up outcome measures, or across any demographic variables (age, gender, marital status, employment status, history of previous treatment, or current use of medications for social phobia). Unless otherwise specified, subsequent analyses report the results of these combined groups.

Differences in outcome scores at before and after treatment, and at 6 month follow up were calculated using one-way repeated measures analysis of variance (ANOVAs) with Bonferroni-corrected post-hoc t-tests. Effect sizes (Cohen's d) were calculated within groups, based on the pooled standard deviation. Unless otherwise specified, all post-treatment analyses using parametric statistics used an intention-to-treat design. The χ² statistics were calculated to identify differences in acceptability of the Shyness programme between participants who had previously experienced face-to-face treatment and those who had not.

Cost-effectiveness estimates for the Shyness programme were based on direct costs of treatment (staff time to provide the treatment), estimated as 3 h of clinician time per patient [1, 2]. These estimates excluded developmental and set-up costs, standing costs, and the potential effects of the intervention on other services, because we had no evidence on which to base cost offsets. The health gain was expressed as years lived with disability (YLD) averted, calculated by adjusting the mean effect size superiority over the control group from the two primary outcome measures (SIAS and SPS) using the conversion factor method described in Haby et al. (2000) [16]. This transfer factor represents the average change in disability-adjusted life year disability weight for the equivalent of a standard deviation change in severity, and these have been calculated for different mental disorders including social phobia [23]. Effect sizes used in the cost-effectiveness analyses were based on between-groups post-treatment data, because effect sizes for the control group were not available at 6 month follow up. All calculations are based on 2008 $AUD and are based on an estimate of $AUD100 per hr clinician time (including on-costs).

Results

The distributions of all dependent variables were examined and no violations of assumptions required for parametric or non-parametric tests were identified.

Completion rates

Post-treatment data were provided by 82/91 (90.1%) treatment group participants, and 54/91 (59.3%) treatment group participants provided 6 month data. Acceptability questionnaires, which were appended to the 6 month questionnaires, were completed by 48/91 participants (53%).

Six month follow-up completers versus non-completers

Independent group t-tests and χ² tests were calculated to explore differences between participants who completed the 6 month follow-up questionnaires and non-completers. No differences were found between completers and non-completers in either RCT, or for the combined groups (p > 0.05) on any pre-, post-treatment, treatment satisfaction, or 6 month follow-up outcome measure, or on any demographic variables (age, gender, marital status, employment status, history of previous treatment, or current use of medications for social phobia).

Pre-treatment versus post-treatment versus 6 month follow-up differences

Repeated measures one-way ANOVAs were calculated to examine differences between outcome scores before and after treatment, and at 6 month follow up. Significant differences were found for time for the SIAS (F_2,180=143.74, p < 0.001), SPS (F_2,180=91.79, p < 0.001), PHQ-9 (F_2,180=37.24, p < 0.001), K-10 (F_2,180=39.76, p < 0.001), and WHODAS-II (F_2,180=45.39, p < 0.001). Post-hoc pairwise comparisons with Bonferroni corrections (Table 1) indicated statistically significant differences between SIAS and SPS means before and after treatment, and at 6 months (p < 0.001). For the PHQ-9, K-10, and WHODAS-II, statistically significant differences were found between means at pre- and post-treatment assessment (p < 0.001), and between means at pre-treatment assessment and 6 month follow up (p < 0.001), but not between means at post-treatment assessment and 6 month follow up.

Table 1.

Primary and secondary outcome measures†

	Pre-treatment (n = 91)		Post-treatment (n = 91)		6 month follow up (n = 91)		Effect Sizes (n = 91)
Assessment	M	SD	M	SD	M	SD	Pre-Post-treatment	Pre- 6 month follow up
Social Interaction Anxiety Scale	55.30^a	12.53	39.54^a	13.19	35.21^a	14.17	1.24	1.50
Social Phobia Scale	34.08^a	14.86	19.51^a	11.41	16.84^a	11.19	1.10	1.31
Patient Health Questionnaire – 9 Item	8.20^a,b	5.28	4.86^a	4.57	4.30^b	4.28	0.68	0.81
Kessler 10	21.10^a,b	6.07	16.84^a	5.72	16.25^b	5.78	0.72	0.82
WHODAS-II	23.57^a,b	13.04	14.53^a	12.86	13.34^b	13.48	0.70	0.77

WHODAS-II, World Health Organization Disability Assessment Schedule II.; Similar superscripts indicate significantly different means (p < 0.001).; †Within-group pooled effect sizes (Cohen's d) (intention to treat).

Effect sizes

All effect sizes for the outcome measures increased from post-treatment to 6 month follow up (Table 1). From post-treatment assessment to 6 month follow up within-group effect sizes increased from 1.24 to 1.50 for the SIAS, from 1.10 to 1.31 for the SPS (the primary outcome measures), and from 0.68 to 0.81 for the PHQ-9, from 0.72 to 0.82 for the K-10, and from 0.70 to 0.77 for the WHODAS-II, the secondary outcome measures.

Cost-effectiveness

Cost-effectiveness of the Shyness programme was calculated by multiplying the mean effect size superiority for the SIAS and SPS over the control group (Cohen's d = 1.07) by the transfer factor calculated for social phobia (0.1876) [23]. Direct therapist costs per participant ($AUD300) were then divided by the adjusted transfer factor (0.2007) to produce the costs per YLD gained ($AUD1495).

The within-group effect size and estimates of staff time were then derived from published results of a benchmarking study of face-to-face group treatment for social phobia [4]. Assuming that the costs per hour of therapist time between that group programme and the Shyness programme were the same, that seven participants were included in each group, and that two therapists were present for the duration of each of the seven sessions of 4 h of group treatment, the equivalent estimate of direct therapist cost per participant was $AUD800 and the cost-effectiveness for face-to-face group treatment was calculated as $AUD5686 per YLD gained.

Acceptability

At post-treatment assessment and at 6 month follow up the participants were asked to rate specific characteristics of the Shyness programme on a scale from 1 to 10, where 10 indicates a high level of agreement. On average, after treatment the participants rated the treatment as logical (9/10); they reported feeling confident that the treatment would be successful at teaching them techniques for managing their symptoms (8/10); and they reported a high level of confidence in recommending this treatment to a friend with social phobia (9/10). At 6 month follow up the respective ratings given were 8, 8, and 8.

At the 6 month follow up participants were also asked whether they would prefer face-to-face, internet, or either treatment for social phobia, and which type of treatment they thought would be more effective. Responses to these questions (Table 2) were divided based on whether participants had previously received face-to-face treatment for social phobia (n = 24; that is, CBT or counselling for social phobia), or whether they had not previously received face-to-face treatment (n = 24). Overall, χ² tests did not indicate differences in participant preferences of face-to-face or Internet-based treatment for social phobia, or in their estimates of the relative effectiveness of these treatments (p > 0.05). χ² tests also failed to indicate differences in preferences or expectations of effectiveness between participants who had reported previous face-to-face treatment compared to those who did not (p > 0.05).

Table 2.

Participant preferences for face-to-face versus Internet-based treatment for social phobia (n = 48)

		Previously had face-to-face treatment?†
		Yes (n = 24) n (%)	No (n = 24) n (%)	Total (n = 48) n (%)
Based on your experience, which type of treatment would you prefer to have to treat your social phobia?
	Face to face	8 (16.7)	3 (6.3)	11 (22.9)
	Internet	10 (20.8)	11 (22.9)	21 (43.8)
	Both about same	6 (12.5)	10 (20.8)	16 (33.3)

Based on your experience, which type of treatment do you think would be more effective for treating social phobia?
	Face to face	7 (14.6)	6 (12.5)	13 (27.1)
	Internet	10 (20.8)	9 (18.8)	19 (39.6)
	Both about same	7 (14.6)	9 (18.8)	16 (33.3)

†Face-to-face cognitive behavioural therapy or counselling for social phobia.; No differences significant based on χ² tests (p > 0.05).

Participants were also asked to describe advantages and disadvantages of Internet treatment for social phobia (Table 3) and responses were grouped according to themes. Participants reported the main advantages of Internet-based treatment for social phobia as (percentage reporting this): (i) anonymity of the Internet approach (56%); (ii) convenience of not having to travel to see a therapist, to take time off work or other duties, or that it was a free service (54%); and (iii) advantages of the technology including access 24 h per day and how lessons could be easily accessed and repeated (17%). The main disadvantages described included: (i) reduced sense of motivation by not having to physically report to a therapist (31%); (ii) inability to modify the programme content specifically for the needs of each participant or to modify the timeframes of the trials (32%); and (iii) not liking the user interface (6%). Examples of each of these comments are included in Table 3.

Table 3.

Participant responses to questions about advantages and disadvantages of Internet-based treatment for social phobia (n = 48)

Advantages (percent reporting)		Disadvantages (percent reporting)
Theme	Examples	Theme	Examples
Anonymity of the Internet approach (56%)	“You feel free to express yourself, instead of restricting your comments in front of other people.”	Not as motivating as face-to-face treatment (31%)	“One could be more motivated to complete some of the tasks (like the exposure tasks) if they had to report back to a person how it went.”
	“There isn't the same stigma attached to having treatment over the internet as actually going to see a shrink.”		“It's up to the individual to take responsibility for their learning and completing the lessons.”
	“It's so much easier to disclose information over the Internet.”	Programme time frames not flexible or content not personalized (32%)	“Lacks the face-to-face treatment which may help motivate the patient.”
	“Flexibility – you can log on at any time of the day or night.”		“I found it hard to keep up and too fast for me.”
Convenience (54%)	“The material is there to go back to and go over a number of times.”		“The treatment had a tight timeline to complete the tasks in.”
	“Don't have to drive miles to get there.”	Didn't like interface (e.g. Forum) (6%)	“Also, some people probably need someone to explain their individual symptoms, that can be done online though, but I'm sure it can't be easy to diagnose people from what they write.”
			“The therapist would have to be able to assess you better in a face-to-face meeting, and with better assessment would come more targeted treatment.”
Advantages of technology (17%)	“It is much easier to get started with the treatment because the internet format is easy to follow.”		“Some people may not feel comfortable using the Forum environment.”
	“It allows you to feel like you are in control of how much progress you make.”		“All though most people do, not everyone will have access to the internet all the time.”
	“The system was relatively easy to do.”

Discussion

The present study explored the longer term results of the Shyness programme, its cost-effectiveness, and acceptability to participants. The results of the 6 month follow-up questionnaires showed that participants continued to make improvements in symptoms of social phobia, while at least maintaining improvements in mood, psychological distress, and disability. Consistent with this, within-group effect sizes increased for all the outcome measures from post-treatment assessment to 6 month follow up. These results are consistent with those reported by other Internet-based treatment programmes for social phobia [5, 6] and indicate that the benefits of an Internet-based treatment programme for social phobia are maintained at least in the medium to longer term.

Cost-effectiveness of the Shyness programme was calculated to determine the cost per YLD gained. YLD is a standard metric that allows comparison of health gain and cost, across treatments and disorders. The effect size used was derived from the pre–post-treatment between-groups effect size, because this was not available at 6 month follow up. These calculations indicated that the Shyness programme was able to produce the same gain in health status at fourfold the efficiency of face-to-face group treatment [4], or $AUD1495 for one YLD gained, compared to $AUD5686.

To reduce demand characteristics, questions about the acceptability of the Shyness programme were administered at both after treatment and at 6 month follow up. At both post-treatment assessment and at 6 month follow up the participants were asked to rate the Shyness programme with respect to three criteria based on the Credibility/Expectancy Questionnaire [21, 22]: (i) how logical they thought it was; (ii) how confident they were that treatment would be successful at teaching them techniques for managing their symptoms; and (iii) their confidence in recommending this treatment to a friend with social phobia. The mean ratings at post-treatment assessment on a scale from 1 to 10 (where 10 indicates a high level of agreement) were 9, 8, and 9, respectively. These scores were similar to those that participants provided at 6 month follow up (8, 8, and 8, respectively), indicating that participants consistently perceived the programme to have been helpful.

At the 6 month follow up the participants were also asked to indicate whether they would prefer face-to-face, Internet treatment for social phobia, or both, and to indicate which they thought would be more effective (Table 2). No significant differences in preferences were found for type of treatment, even after controlling for whether participants had previously experienced face-to-face treatment for social phobia. This is an important comparison, and although the sample sizes are limited (total n = 48), these data indicate that participants rated the Shyness programme at least as effective as face-to-face treatment.

Finally, participants were asked to describe advantages and disadvantages of the Internet approach for treatment of their social phobia. As expected, given the nature of social phobia, >50% of participants noted the anonymity of the Internet approach as an important reason for why they sought this treatment. But importantly, a similar percentage noted the convenience of being able to log on from home, not having to travel long distances, not having to take time off work, or the fact that the service was free. Clearly, Internet-based treatment reduces important barriers to help-seeking by patients, and the potential convenience of Internet-based treatment is equally relevant to clinicians working in rural areas, but also to clinicians who, for practical reasons, may wish to work remotely with their patients.

A potential disadvantage of the Shyness programme, reported by 31% of participants, relates to the reduced sense of obligation or motivation to complete lessons or assignments compared to face-to-face treatment because participants did not see their therapist. A second aspect of the programme described as disadvantageous by 32% of participants was the relative inability to modify the lesson content for individual cases or to modify the time frames for completing the programme. The latter disadvantage can be ameliorated outside formal clinical trials that, by their nature, require standardization of content and time frames. The perception, however, that the ‘social contract’ or obligations within the milieu of the Shyness programme is less than that experienced in face-to-face treatment, is an important difference between the two treatment modalities. Consistent with this concept, CCBT (self-guided) programmes typically have much lower completion rates than CaCCBT (clinician-guided) programmes [14, 24], but generally the literature indicates that the completion rates for CaCCBT programmes is equivalent to face-to-face treatment, and the completion rates for the Shyness programme (79%) are consistent with this. Regular emails or telephone calls that reinforce progress and encourage further completion of lessons increases completion rates, which improve clinical outcomes [25, 26], and the use of multimedia interactive functionality in websites could improve compliance further.

Limitations

An important potential threat to the validity of these findings relates to the completion rates of the 6 month outcome and acceptability questionnaires (59% and 53% of the original treatment group, respectively). It should be noted, however, that those recorded as not completing the questionnaires included eight participants who formally withdrew during treatment and were not sent the questionnaires at 6 months. Additionally, at least six email accounts and contact phone numbers used by participants in the study were no longer operational, so questionnaires could not be sent to these participants. These 14 participants would account for 15% of the original sample. Importantly, no differences were found between follow-up completers and non-completers on demographic variables, on any pre- or post-treatment outcome measures, or treatment satisfaction ratings at post-treatment assessment. Furthermore, variables such as change scores over treatment, number of email contacts during treatment, or qualitative evaluations based on apparent motivation during treatment did not predict who completed the 6 month follow up questionnaires. For these reasons, we suspect that the participants who completed the 6 month questionnaires are generally representative of the original sample.

A potential limitation of the cost-effectiveness analyses relate to the limited statistical analyses conducted. While it is acknowledged that a comprehensive analysis would consider numerous direct and indirect costs, including transfer costs to other health services, for the purposes of comparing the resources required by the Shyness programme with those required by a face-to-face group programme, a straightforward comparison of therapist cost per unit of health gain appeared most useful. Of more importance is the assumption that these two groups of patients were equivalent. This can be properly addressed only by randomizing participants from the same population. The pre-treatment means, however, on the SIAS and SPS for each study are equivalent, indicating that at the least the severity of groups is comparable.

A potential limitation of the acceptability analyses is that responses were not available from the eight participants who withdrew from the programme, or from those we could not contact at the 6 month follow up, potentially creating a positive bias. We suspect that the reported results do reflect the views of the original sample because most of those who withdrew reported that it was for reasons other than dissatisfaction with the programme, and some of those who completed the acceptability questionnaires reported that they preferred face-to-face treatment, indicating that participants felt sufficiently confident to criticize the programme.

In conclusion, these results confirm that the benefits of the virtual clinic Shyness programme are maintained at least in the medium to longer term, the treatment is cost-effective, and that after 6 months participants continue to rate the programme as effective and at least as acceptable as face-to-face treatment.

Footnotes

Acknowledgements

The authors gratefully acknowledge the participants for their involvement and helpful comments. We also wish to gratefully acknowledge the statistical advice provided by Mr Matthew Sunderland, and the contributions to the development of the Shyness programme by Mr Bruce Sim, Mr Jason Hando, and Mr Daniel Winter.

References

1. Titov

Andrews

Schwencke

Drobny

Einstein

. Shyness 1: distance treatment of social phobia over the Internet. Aust N Z J Psychiatry 2008; 42: 585–594.

2. Titov

Andrews

Schwencke

. Shyness 2: treating social phobia online: replication and extension. Aust N Z J Psychiatry 2008; 42: 595–605.

Titov

Andrews

Choi

Schwencke

(In press). Shyness 3: randomized controlled trial of guided versus unguided Internet-based CBT for social phobia. Aust N Z J Psychiatry 2008; 42:1022–1032.

4. McEvoy

. Effectiveness of a cognitive behavioural group therapy for social phobia in a community clinic: A benchmarking study. Behav Res Ther 2007; 45: 3030–3040.

5. Andersson

Carlbring

Sparthan

. Internet-based self-help with therapist feedback and in vivo group exposure for social phobia: A randomised controlled trial. J Consult Clin Psychol 2006; 74: 677–686.

6. Carlbring

Gunnarsdóttir

Hedensjö

Andersson

Ekselius

. Treatment of social phobia: randomised trial of internet-delivered cognitive-behavioural therapy with telephone support. Br J Psychiatry 2007; 190: 123–128.

7. Carlbring

Nilsson-Ihrfelt

Waara

. Treatment of panic disorder: live therapy vs. self-help via Internet. Behav Res Ther 2005; 43: 1321–1333.

8. Klein

Richards

Austin

. Efficacy of Internet therapy for panic disorder. J Behav Ther Exp Psychiatry 2006; 37: 213–238.

9. Kiropoulos

Klein

Austin

. Is internet-based CBT for panic disorder and agoraphobia as effective as face-to-face CBT?. J Anxiety Disord 2008; 22: 1273–1284.

10.

10. Shandley

Austin

Klein

. Therapist-assisted, Internet-based treatment for panic disorder: can general practitioners achieve comparable patient outcomes to psychologists?. J Med Internet Res 2008; 10(2)14.

11.

11. Klein

, Austin

, Pier

Internet-based treatment for panic disorder: does frequency of therapist contact make a difference?

Cog Behav Res; in press.

12.

12. Andersson

Bergstrom

Hollandare

Carlbring

Kaldo

Ekselius

. Internet-based self-help for depression: randomized controlled trial. Br J Psychiatry 2005; 187: 456–461.

13.

13. Marks

Cavanagh

Gega

. Hands-on help. Computer-aided psychotherapy. Psychology Press Taylor and Francis, New York, NY 2007.

14.

14. Titov

. Status of computerized cognitive behavioural therapy for adults. Aust N Z J Psychiatry 2007; 41: 95–114.

15.

15. Miller

Magruder

. Introduction: services and cost-effectiveness research for psychotherapy. Cost-effectiveness of psychotherapy: a guide for practitioners, researchers, and policymakers, Miller

Magruder

. Oxford University Press, New York 1999; XV–XXX.

16.

16. Haby

Carter

Mihalopoulos

. Assessing cost-effectiveness – mental health: introduction to the study and methods. Aust N Z J Psychiatry 2004; 38: 569–578.

17.

17. Kroenke

Spitzer

Williams

JBW

. The PHQ-9. Validity of a brief depression severity measure. J Gen Intern Med 2001; 16: 606–613.

18.

18. Mattick

Clarke

. Development and validation of measures of social phobia scrutiny fear and social interaction anxiety. Behav Res Ther 1998; 36: 455–470.

19.

19. Kessler

Andrews

Colpe

. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med 2002; 32: 959–976.

20.

20. World Health Organization. World Health Organisation Disability Assessment Schedule 2nd edn. Geneva: World Health Organization, Available online at: http://www.who.int/icidh/whodas/ .

21.

21. Devilly

Borkovec

. Psychometric properties of the credibility/expectancy questionnaire. J Behav Ther Exp Psychiatry 2000; 31: 73–86.

22.

22. Borkovec

Nau

. Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry 1972; 3: 257–260.

23.

23. Issakidis

Sanderson

Corry

Andrews

Lapsley

. Modelling the population cost-effectiveness of current and evidence-based optimal treatment for anxiety disorders. Psychol Med 2004; 34: 19–35.

24.

24. Spek

Cuijpers

Nyklí(ek

Riper

Keyzer

Pop

. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 2007; 37: 319–328.

25.

25. Clarke

Eubanks

Reid

. Overcoming Depression on the Internet (ODIN) (2): a randomized trial of a self-help depression skills program with reminders. J Med Internet Res 2005; 7(2)16.

26.

26. Mackinnon

Griffiths

Christensen

. Comparative randomised trial of online cognitive-behavioural therapy and an information website for depression: 12-month outcomes. Br J Psychiatry 2008; 192: 130–134.

Shyness Programme: Longer Term Benefits,Cost-Effectiveness,and Acceptability

Abstract

Keywords

Method

Recruitment and retention

Measures

Procedure

Statistical analysis

Results

Completion rates

Six month follow-up completers versus non-completers

Pre-treatment versus post-treatment versus 6 month follow-up differences

Effect sizes

Cost-effectiveness

Acceptability

Discussion

Limitations

Footnotes

Acknowledgements

References