The costs of early intervention in psychosis: Restoring the balance

Early intervention in psychosis (EIP) has become the subject of a rich literature built up over two decades. This has been an international collaborative endeavour involving academic clinicians, neuroscientists, epidemiologists, health services researchers and economists conducting a range of studies producing a wealth of new evidence. Recently, Australian and New Zealand Journal of Psychiatry published papers calling into question aspects of this evidence base, reprising and extending the debate (Amos, 2012; Castle, 2012). The most recent is a review of studies evaluating economic aspects of EIP services (Amos, 2012). We welcome the open debate of this evidence, but we are concerned that this particular review has fallen short in terms of fairness, balance and, above all, factual accuracy.

Clarifying the Australian policy context

Before discussing Amos’s review of economic studies of EIP it is important to correct some assertions regarding mental healthcare resource allocation in the 2011 Australian budget. Amos begins his review by claiming that expansion of EIP services in Australia is responsible for the contraction of other services, specifically general practitioner-managed psychological interventions. No evidence for this assertion, which has been made previously by others, is provided. The National Health and Hospitals Reform Commission, chaired by Dr Christine Bennett (Bennett, 2009), listed 14 recommendations for reform in mental health based on a review of best available international evidence. The top two recommendations involved expansion of the headspace system of community-based youth mental health-enhanced primary care and the comprehensive rollout of the Early Psychosis Prevention and Intervention Centre (EPPIC) model of early psychosis care pioneered in Australia but more successfully scaled up elsewhere. These two funding commitments were first on the list for support in the 2011 Federal budget and the EPPIC allocation constituted around 10% of the package. Other commitments such as the child psychiatry investment and the ‘Partners in Recovery’ scheme for severe and enduring mental illness, the largest single allocation, were not formulated until late in the budget process. The cuts (now at least temporarily restored by the government) to Better Access and related general practice programs were a government decision based, it appeared, on data and advice from the Department of Health and Aging and an independent evaluation (Pirkis et al., 2011). There is absolutely no basis to link the reduction in this funding to any particular element of the 2011 budget package and specifically not to the EPPIC commitment.

Second, because some critics in response to the reforms underway in Australia have questioned the evidence showing that better functional outcomes are achieved by specialised EIP services, Amos asserts that early psychosis researchers have tacitly accepted these critiques and sought to buttress the case with cost-effectiveness data. In fact, the record shows that studies examining cost-effectiveness were conducted early in the field’s development and in parallel with outcomes research. Furthermore, many, if not most, economic studies have been conducted by independent economic evaluators neither employed nor funded by EIP services (e.g. the economic analyses in Cullberg et al. (2006) were conducted by researchers from the Institution of the National Economy at the University of Växjö). Very few health service reforms in mental health have been so heavily evaluated in these complementary ways.

Clarifying basic economic principles

One repeated concern running through Amos’s paper is his impression that every study included in his review has incorrectly costed outpatient care associated with EIP services. His logic is based on the lower caseloads presumably associated with outpatient care in EIP services. It is important to clarify how unit costs are derived: they are calculated by dividing the total costs of delivering a service by the numbers of units of use. With high caseloads the pragmatic reality is that clinicians may not see clients very often. With lower care loads, clinicians can see clients more frequently. Therefore, the unit cost per hour of contact (for example) may be similar in different caseload services but the overall costs per patient will differ depending on how frequently patients are seen. In the first economic evaluation of the EPPIC, which was not included in the Amos review, unit costs were based on the actual EPPIC budget (Mihalopoulos et al., 1999) and were not substantially different to unit costs associated with care in non-specialised services. Unit costs may differ between EIP and standard care services but this would be due to differences in the ratio of direct patient time to other time. Indeed, this means that a team with a low caseload could easily have lower costs per hour than a team with a high caseload if they spent more time with their (albeit smaller) number of patients. We do not claim this is the case but it is no less feasible than the scenario suggested by Amos.

In a related argument, Amos argues that McCrone et al. (2010) did not accurately assess the costs associated with EIP services because they used patient recall of service use and did not include out-of-hours services associated with EIP services. He is wrong on both counts. First, while accuracy of patient recall is an important issue, previous research has demonstrated that patients (including those in their first episode of psychosis) demonstrate an acceptable level of service use recall (Calsyn et al., 1993; Goldberg et al., 2002; Mihalopoulos et al., 1999), and there is no evidence of differential accuracy of recall between EIP and other patients. Second, McCrone et al. (2010) included all services associated with the EIP program and its comparator.

Amos argues that reductions in hospitalisations rarely translate into financial cost savings which can then be used elsewhere in the system. Amos’s discussion of this issue confuses the concepts of opportunity and financial costs, and contradicts the recommendations by Drummond et al. (2005) described in his paper. Opportunity costs are defined as the benefit forgone by not using resources in their next best alternative use, and their use in economic evaluations has good theoretical foundations (Drummond et al., 2005). With respect to inpatient service use, it is true that short-term financial cost savings will be lower than average cost, but a bed not used for an EIP patient will be available for another patient or purpose, with benefits associated with such utilisation. If the bed is not used, the variable costs associated with that bed are released as financial cost savings. In the long-term, all resources are variable and there can be planned contraction of service use. In our experience, decision-makers are generally aware of such differences. More importantly, given Amos’s employment of the Drummond recommendations to gauge the quality of the studies reviewed, it is not appropriate to pick and choose which methodological recommendations to accept and which to discard.

There are other occasions when Amos’s comments might mislead the unwary reader. For example, it is not the case that a societal perspective implies the need for quality-adjusted life years (QALYs), nor is it necessarily a failing of previous studies not to have used these measures given the well-known difficulties in the field of psychosis studies (Brazier, 2010).

Problems with interpretation of studies

Of particular concern is Amos’s interpretation and reporting of the papers included in his review. For example, he criticises the Mihalopoulos et al. (2009) study as being ‘biased’. This conclusion is based on his interpretation that the sample was ‘opportunistically’ selected, something which he states is true for many of the case–control studies, and because of differential follow-up time periods between the two comparison groups. In fact, sample selection was not opportunistic but comprised 51 consecutive EPPIC patient admissions, who were then carefully matched to a comparison control. Similarly, in Cullberg et al. (2006), the sample comprised all first-episode patients during the 2 years of the study, not an opportunistic selection. Second, while follow-up periods did differ slightly between groups, costs for the first 5 years for both groups were separately reported by Mihalopoulos et al. (2009). The study duration also minimised the potential influence of the historical nature of the controls. The issue of selection bias was formally addressed in the study, with no evidence found of selection bias between the original study sample and the long-term follow-up sample.

Amos also describes the retrieval of patient files in the Mihalopoulos et al. (2009) study as ‘ad hoc’. This is an unfairly disparaging description and one designed to seed doubt in readers’ minds about the rigour of this study. Far from being ‘ad hoc’, the study involved meticulous, time-consuming and systematic scrutiny of occasions of service use along with records of pharmacological therapy.

Similarly, incorrect interpretations are made by Amos about other studies in his review. For example, he states that Goldberg et al. (2006) noted that reduced hospitalisation rates in their study did not change bed numbers or occupancy, and therefore EIP did not ‘reduce hospital costs even where individual hospitalization is reduced’ (Amos, 2012: 13). In fact, what the authors state was that despite no changes in bed numbers or occupancy rates during the study period, reductions in inpatient service use was still observed. The authors were in fact making the same argument as that noted above, that beds freed by EIP patients are available to be alternatively used. Amos’s claim that Goldberg et al. (2006) did not include behavioural measures is also incorrect; the paper reports data describing diagnoses, suicide attempts, violent behaviour and injuries.

More generally, while comparisons between studies conducted in different settings is helpful, it must be appreciated that cross-country comparisons can be compromised by differences in the structure of healthcare financing, organisation and delivery (e.g. see Knapp et al. (2002) in relation to the treatment costs of schizophrenia across five European countries, and Knapp et al. (2004) on international comparisons of schizophrenia costs). Amos asserts that the Mihalopoulos et al. (2009) study must be ‘biased’ since its results are not consistent with Bertelsen et al. (2008) (even though the latter also observed reduced hospitalisation at 5-year follow-up), yet fails to recognise that a community intervention such as EIP will necessarily have impacts that are influenced by the characteristics of the system within which it is located.

Likewise, Amos discusses the modelling approaches to assessing economic impact and argues that they are ‘invalidated’ because subsequent studies have generated different parameters, without recognition of differences across countries or over time. The term ‘invalidated’ is somewhat loaded and offers a clue to the approach adopted in his review. Models make assumptions about their parameters but most also test these assumptions using sensitivity analyses. The fact that the estimate of transition to psychosis used by Valmaggia et al. (2009) was different from that produced by different authors using a different method in a different country at a different time does not ‘invalidate’ those earlier results.

Amos further raises an interesting point contrasting the longer-term findings of Gafoor et al. (2010) with those of McCrone et al. (2010). Gafoor and colleagues suggested that, after patients had been discharged from EIP services to general community services, the initial gains experienced under EIP were lost. If this were true this does suggest that the intensity of care provided by EIP might need to be maintained even after discharge and not, as implied by Amos, that EIP services do not work over the longer term. Also important to such conclusions is whether each of the studies was appropriately powered to detect significant differences. This is a more general issue (Drummond et al., 2005) and many of the studies acknowledge that they may have been underpowered to reach statistical significance (e.g. Cullberg et al., 2006; Goldberg et al., 2006).

The use of a commonly accepted set of quality criteria for economic evaluations (Drummond et al., 2005) is a strength of Amos’s review, but he does not apply the same rigour in his review, failing to adhere to internationally accepted quality criteria for systematic reviews, such as in the Cochrane Handbook (Higgins and Green, 2011). For example, it is common convention to include two reviewers in such a review, and to apply consistent inclusion criteria. However, Amos’s review included a paper by Phillips et al. (2009) which he described as a cost-effectiveness analysis when (as the authors stated) it was not as it did not explicitly include intervention outcomes as well as costs, and which was not actually about EIP but rather the costs associated with individuals who are at ultra-high risk of psychosis. Second, while the model-based evaluation of Valmaggia et al. (2009) was included in Amos’s review, another model-based evaluation by McCrone et al. (2009) was not included, which is surprising since the latter was published in an EIP-specific journal. Amos also argues that if two studies include information regarding the same sample, only the more recent study is included, but if different data are reported by the different studies, it would be more helpful to readers of a review to see the full span of evidence, since there is no danger of over-weighting the study if there is no meta-analysis.

Conclusion

While individual weaknesses can be found in all of the studies reviewed by Amos, and these certainly need to be highlighted, it is important that reviews are conducted in an informed, fair and consistent manner, and (in this case) in cognisance of the basic principles of economic evaluation. So, when Amos writes that ‘It should be clear that each set of authors selected a positive cost-effectiveness measure from among multiple possible comparisons’, he is making a strong accusation of what amounts to professional misconduct by the researchers concerned. In our view, Amos has failed to live up to the standards he expects of others.

We are certainly not arguing that the evidence in support of an economic case for EIP services is proven beyond shadow of doubt, but that the accumulation of evidence in some contexts (such as the UK, Norway and Australia) suggests a different conclusion from that drawn by Amos.