EPPIC mirage: Cost-effectiveness of early psychosis intervention

Two recent pieces in this journal (Amos, 2013; Jorm, 2013) have questioned the cost-effectiveness of the Early Psychosis Prevention and Intervention Centre (EPPIC) programme and criticized the evidence base for claims that it can save money (e.g. McGorry, 2010a). These offerings provide a much-needed contrast to the general uncritical acceptance and promotion of the purported merits of the programme.

Some of the issues raised by these critics are similar to issues I have previously discussed in some detail (Raven, 2011) in my analysis of Patrick McGorry and colleagues’ longitudinal study of the effectiveness (McGorry et al., 1996) and cost-effectiveness (Mihalopoulos et al., 1999, 2009) of EPPIC treatment for first-episode psychosis (“the EPPIC study”). This study, which compared EPPIC patients recruited in 1993 with pre-EPPIC patients recruited in 1989–1992 (i.e. historical controls), is the main source of evidence used to support claims of effectiveness and cost-effectiveness.

Jorm (2013) has argued that the EPPIC study was inadequate to justify the Federal Government’s expenditure of more than AU$200 million to expand EPPIC services, rolling out a total of 16 additional sites across Australia (Roxon et al., 2011). He noted the small sample (102 patients), the use of historical controls, the lack of randomization, and the use of cost data two decades old, simply adjusted to 2009 figures.

Amos (2012) highlighted the methodological flaws of the EPPIC study, with its claims of dramatically reduced expenditure, compared with more rigorous research with much less positive findings. An accompanying commentary by Carr (2012) explained that the case for cost-effectiveness of early intervention has not been convincingly made and that claims of cost savings are not based on credible evidence.

Crucially, however, none of these critics has pointed out an over-riding flaw in the evidence about EPPIC: the fact that the EPPIC study did not compare early intervention (EI) with standard mainstream intervention, and therefore does not demonstrate the economic merits of EI relative to mainstream treatment. In fact, EPPIC treatment has never been systematically compared with mainstream treatment. However, the EPPIC study is routinely claimed to provide such a comparison, and providing conclusive evidence of the relative effectiveness and cost-effectiveness of EPPIC.

The EPPIC study compared an innovative outpatient EI service (EPPIC) with an innovative inpatient EI service (“pre-EPPIC”) run by McGorry immediately before the establishment of EPPIC in October 1992. McGorry et al. (1996) described pre-EPPIC as “the first-generation model” (p. 321) and EPPIC as “the second-generation-model” (p. 305). Furthermore, they acknowledged that “earlier provision of treatment … does not seem to have occurred to a widespread extent” (p. 322) in the EPPIC sample; in fact the pre-EPPIC historical control group actually had a shorter median duration of untreated psychosis (DUP) (30 days) than the EPPIC group (52 days), but a higher mean DUP (702.7 days vs. 483.6) because of some outliers with extremely long DUPs.

Copolov et al. (1989) outlined the structure of the pre-EPPIC service. After the acute phase of illness, pre-EPPIC patients remained in hospital to engage in modules involving family assessment and intervention, psychoeducation, social and living skills, and community survival skills. In effect, they received a prototype EI service while they were hospitalized. Apart from the absence of assertive community treatment, the pre-EPPIC service provided the key components of current EI services (e.g. Bertelsen et al., 2008).

Consequently, the cost differences reported by Mihalopoulos et al. (1999) show only that it was more expensive over 12 months to provide EI during an extended hospital admission (the default treatment, regardless of clinical need) rather than on an outpatient basis (with hospitalization only if clinically required). The findings shed no light on the relative costs of treating first episode psychosis in a specialist EI service versus a standard mental health service.

The methodological problems intensified in the very small follow-up study of Mihalopoulos et al. (2009), which assessed patients approximately 8 years later. It had high attrition rates (only 32 of the original 51 EPPIC patients and 33/51 pre-EPPIC patients) were included. Again the median DUP was shorter for pre-EPPIC (31 days) than for EPPIC (47.5 days), but the mean was longer (720.7 vs. 600 days) because of outliers. In such small samples, outliers can seriously bias results. Furthermore, men – who tend to have worse outcomes than women (Grossman et al., 2008) – were under-represented in the EPPIC follow-up group: 53.1% compared with 69.7% in the pre-EPPIC follow-up group (there were 65% in each group at baseline). Ignoring this disparity, Mihalopoulos et al. stated “we are confident that this smaller cohort is representative of the original cohort” (p. 916), and misleadingly characterized the pre-EPPIC follow-up sample as a “matched” (p. 909) cohort of 33 participants.

More importantly, Mihalopoulos et al. (2009) inaccurately described the nature and significance of the EPPIC study, reframing it as a comparison of EI and mainstream treatment, rather than a comparison of two models of EI. This would not be apparent without reading the earlier papers, particularly McGorry et al. (1996), which accurately described the study as a comparison of two models of EI: “The present study is a naturalistic longitudinal study with multidimensional outcome measures, aiming to evaluate the effectiveness of the EPPIC program on 12-month outcome in first-episode psychosis, in contrast to the previous model of care – which, it should be remembered, also differed to a significant extent from ‘standard care’.” [emphasis added] (p. 315).

However, Mihalopoulos et al.’s (2009) report, co-authored by McGorry, inaccurately described the study, stating in the abstract that “32 participants initially treated for up to 2 years at EPPIC were compared with a matched cohort of 33 participants initially treated by generic mental health services” [emphasis added] (p. 909) and “Specialized early psychosis programs can deliver a higher recovery rate at one-third the cost of standard public mental health services [emphasis added] (p. 909).

Mihalopoulos et al.’s (2009) use of the words “standard” and “generic” directly contradicts McGorry et al.’s (1996 p. 315) emphasis that pre-EPPIC “also differed to a significant extent from ‘standard care’” and Mihalopoulos et al.’s (1999 p. 48) acknowledgement that “the comparator or control condition also provided enhanced or specialist interventions for first-episode patients”. More recently and more dramatically, McGorry referred to the pre-EPPIC control group as patients who had “become very severely ill and disabled and treated in traditional mental health care” (Sullivan, 2012).

In fact, McGorry and colleagues have repeatedly referred to pre-EPPIC treatment as mainstream treatment in recent years. Consequently the Australian Government has been systematically misled about the nature and implications of the EPPIC study, not only in the academic literature and the media, but also in formal submissions. In a statement to the Australian Senate, McGorry referred to pre-EPPIC treatment as both “standard psychiatric care” and “normal late intervention”:

if you compare patients that are treated in standard psychiatric care with patients that go through these streamed early psychosis programs, the costs are three times as much over an eight-year period if patients just go through the normal late intervention system. So it is actually irresponsible of Australian governments, state and federal, to not roll this out. (McGorry, 2010a p. 12)

The inaccuracy was incorporated into the Senate report, which quoted McGorry almost verbatim: “Professor McGorry stated that the costs are three times as much over an eight-year period if a person goes through the normal late intervention system” (Senate Finance and Public Administration References Committee, 2010 pp. 86–87).

Similarly, Hickie (2008) referred to pre-EPPIC treatment as “conventional services” (p. 7), in a discussion paper commissioned by the National Health and Hospitals Reform Commission (NHHRC), which subsequently recommended a national rollout of EPPIC (NHHRC, 2009 p. 272). The national rollout of EPPIC was also promoted in the “Early psychosis feasibility study report” prepared by Orygen Youth Health Research Centre (2011) for the National Advisory Council on Mental Health (NACMH), which repeatedly referred to pre-EPPIC treatment as “standard care” (pp. 22, 29, 30). Unfortunately, these inaccuracies have been uncritically and enthusiastically accepted by the Australian Government, along with the media and many mental health professionals. With increasing awareness of young people’s mental health problems, there is a well-motivated appetite for good news stories about effective solutions, which has been skilfully harnessed.

Furthermore, it is troubling that McGorry and colleagues were commissioned by the NACMH to prepare the misleading feasibility report, for which they were paid AU$99,550 (McGorry, 2010b p. 55). Similarly, Hickie was commissioned to prepare the discussion paper for the NHHRC. This is symptomatic of a highly problematic recursive policy loop in which a small number of high-profile policy advocates are repeatedly invited to advise on mental health reform, and their flawed and misleading advice is accepted uncritically and used as the basis for large-scale expenditure.

In summary, the EPPIC study is a methodologically weak study that has repeatedly been inaccurately characterized. A few critics have identified some major flaws, but not the crucial fact that the study does not compare EI with mainstream treatment as claimed. Earlier reports (McGorry et al., 1996; Mihalopoulos et al., 1999) accurately stated that it compared two models of EI, but the more recent follow-up report (Mihalopoulos et al., 2009) and numerous public statements have inaccurately described the control intervention as standard public mental health services. Unfortunately, this has been uncritically and enthusiastically accepted as conclusive evidence that EPPIC treatment is much more effective and cost-effective than mainstream treatment, resulting in a major mental health system reform that is not evidence-based. This populist response will undoubtedly bias resource allocation for years to come, with inevitable opportunity costs, and will impede the formulation of effective strategies to improve the wellbeing of Australians.