Abstract
Objectives:
This study investigates two approaches to estimate the potential impact of a population-level intervention on Australian suicide, to highlight the importance of selecting appropriate analytic approaches for informing evidence-based strategies for suicide prevention.
Methods:
The potential impact of a psychosocial therapy intervention on the incidence of suicide in Australia over the next 10 years was used as a case study to compare the potential impact on suicides averted using: (1) a traditional epidemiological measure of population attributable risk and (2) a dynamic measure of population impact based on a systems science model of suicide that incorporates changes over time.
Results:
Based on the population preventive fraction, findings suggest that the psychosocial therapy intervention if implemented among all eligible individuals in the Australian population would prevent 5.4% of suicides (or 1936 suicides) over the next 10 years. In comparison, estimates from the dynamic simulation model which accounts for changes in the effect size of the intervention over time, the time taken for the intervention to have an impact in the population, and likely barriers to the uptake and availability of services suggest that the intervention would avert a lower proportion of suicides (between 0.4% and 0.5%) over the same follow-up period.
Conclusion:
Traditional epidemiological measures used to estimate population health burden have several limitations that are often understated and can lead to unrealistic expectations of the potential impact of evidence-based interventions in real-world settings. This study highlights these limitations and proposes an alternative analytic approach to guide policy and practice decisions to achieve reductions in Australian suicide.
Introduction
Suicide remains a significant public health problem in Australia and is associated with significant social, economic and health system costs (Australian Bureau of Statistics, 2014; Mendoza and Rosenberg, 2010). Despite national and state investment, alongside resources contributed by business, community and philanthropic sources, suicide prevention programmes appear to have had limited impact on population-level suicide rates in Australia to date (Atkinson et al., 2015a). Consequently, prevention of suicide and suicide attempts has re-emerged at the centre of Australia’s national discourse on mental health. Following the recommendations of the 2014 review by the National Mental Health Commission (2014), there has been an allocation of increased funds and an appropriate focus on regional implementation of suicide prevention programmes, utilisation of new technologies, research and systematic national evaluation.
Decision makers, however, face challenges in the design and implementation of effective policies and programmes. These challenges have resulted in a move towards implementation of comprehensive or multilevel strategies that apply a system-wide approach targeting whole populations (Black Dog Institute, 2016; Hegerl et al., 2008; Van der Feltz-Cornelis et al., 2011). The rationale is that if more risk factors are targeted and service intervention initiatives are included, then the overall impact on a given health outcome will be greater than strategies targeting a single risk factor or intervention (Atkinson et al., 2015a). However, such ‘comprehensive’ strategies may lack focus on locally relevant or evidence-based interventions, or sufficient actual investment in time, resources and capacity for implementation in specific geographic and socio-economic contexts. Consequently, ‘comprehensive’ approaches may undermine potential effectiveness by spreading finite resources over too broad a range of poorly targeted programmes and services that do not account for local factors (Atkinson et al., 2015a). It is also difficult to estimate the potential impacts of these comprehensive, multi-factorial interventions using traditional epidemiological measures of impact (such as measures of attributable risk and their variants).
Accordingly, this study introduces an alternative approach to assess the potential impact of suicide prevention strategies in populations, using systems science modelling methods. These models seek to account for the complex and potentially interacting effects of comprehensive intervention strategies implemented in populations as described above. This study compares the potential impact of a population-level intervention on the subsequent incidence of Australian suicide using two approaches: (1) a traditional epidemiological measure of population impact and (2) a dynamic measure of population impact based on a systems science model of suicide (see Box 1, and as described previously (Page et al., 2017), and available at: http://saxinstitute.scem.uws.edu.au/netsims/ja/suicide_v405/index.html; accessing and using this simulation requires Adobe Flash player–enabled web browser).
A system dynamics model of Australian suicide (Page et al., 2017).
Limitations of traditional analytic tools to support decision making in suicide prevention
Traditional analytic tools for prioritising risk factors or interventions have important limitations when applied to complex problems such as suicidal behaviour. One example is the attributable fraction (and related variants) that estimates the comparative burden each risk factor contributes in a given population and the proportion of that condition that could be averted by targeting specific prioritised risk factors (Rockhill et al., 1998). The assumptions underpinning the attributable fraction are that exposure variables are independent, and relationships between exposures and outcomes are unidirectional, linear and constant through time.
Time is largely ignored in traditional epidemiological approaches for calculating attributable fractions, despite analytic methods available for longitudinal data (Eide, 2008). Time from a putative exposure to disease, and time from a given intervention to its impact on disease, are not considered in standard attributable fraction estimates used for policy planning. The attributable fraction is a static measure that considers a fixed counterfactual scenario, for a specific (often undefined) point of time, assuming a risk distribution that does not change over time (Eide, 2008). However, complex problems are characterised by interaction and feedback (i.e. risk factors influence each other), thresholds (or breaking points) and changing behaviour over time, all of which violate the assumptions of traditional metrics for estimating attributable risk (Sterman, 2006).
Additionally, traditional decision analytic tools, which seek to prioritise interventions on the basis of their comparative costs, benefits or return on investment, also have important limitations for informing policy and practice decision for complex problems. These tools do not account for population dynamics, behavioural dynamics, the interaction of individuals and the influence of social networks and spatial factors (such as the built environment), variation in the impact of interventions over time and the non-additive effects of interventions (Marshall et al., 2015). This limits the applicability of these approaches to complex public health problems such as suicide prevention; to estimate the potential impact of comprehensive, multi-factorial prevention strategies; and also to inform time to effect and scale-up of interventions, financial and human resource sustainability, and translation to other contexts.
An alternative approach to guide investment decisions in suicide prevention
Systems science modelling methods (system dynamics, agent-based modelling and discrete event simulation – collectively known as dynamic simulation modelling) allow researchers and decision makers to move beyond simple additive and reductionist assumptions on which traditional analytic approaches rely. Dynamic simulation models account for real-world sources of inertia and delay, changes over time, feedback loops (i.e. vicious and virtuous cycles, as well as regulatory processes), unintended consequences, contextual interactions, implementation challenges in resource-constrained environments and the non-additive effects of interventions (Sterman, 2006). These methods have been applied extensively in engineering, defence, economics, ecology and business to simulate and help solve complex problems, and optimise the use of limited resources (Homer and Hirsch, 2006). Their value in helping to solve complex problems in the health sector is being increasingly recognised (Galea et al., 2009), with local and international applications for the prevention of infectious and chronic diseases, reduction in alcohol-related harms, prevention of childhood obesity, and for informing operational aspects of healthcare capacity and delivery, including patient flows through emergency departments and workforce planning (Atkinson et al., 2015b, 2017; Freebairn et al., 2016). More recently, this modelling approach has also been used to evaluate suicide prevention initiatives in Australia (Page et al., 2017).
Dynamic simulation models bring together a variety of sources of evidence including (1) research evidence, (2) expert and local knowledge, (3) practice experience and (4) primary and secondary data in order to map and quantify complex problems. In the population health context, such sources may include systematic reviews and key studies; policy guidelines and recommendations, workforce priorities and resource capacity for a given population catchment or context; and data from health registers, hospital admissions data or general population health surveys (Atkinson et al., 2015a; Page et al., 2017). Model outputs can be compared against real-world historic data patterns across a range of indicators to establish plausibility of the model to ensure it is a valid representation of the real-world system. The resulting validated model can then be used as a ‘what-if’ tool to test the likely impacts of counterfactual situations and different combinations of interventions over the short and long term (including their often interacting costs and benefits) before they are implemented in the real world (Marshall et al., 2015). Consequently, these tools can help local and national decision makers determine where best to target investments for suicide prevention and with what intensity.
In particular, system dynamics models offer value in informing strategic planning in time frames that are congruent with the timelines and priorities of service providers and policy makers. It is an approach that represents aggregate patterns of change and the effects of interventions in shifting the mean. As such, these models do not provide the full texture of the variability of the behaviours of individuals, and they assume perfect mixing of individuals and random social network effects. However, system dynamics is well suited to the analysis of non-linear dynamics, the non-additive effects of intervention combinations, the identification of leverage points where interventions can deliver greater than anticipated returns, and can guide broad strategic directions for achieving population-level reductions in suicidal behaviour. Using such models does not preclude the prospective use of traditional epidemiological and programme evaluation methods, and they can be employed as an adjunct to such approaches over time. Information from monitoring and evaluation of implemented strategies, as well as new research findings, can be used to iteratively update and refine the model and guide decision making on an ongoing basis.
A comparison of static versus dynamic approaches to assess the potential impact of psychosocial programmes to prevent suicide
A common initial step in an evidence-based approach to programme planning for suicide prevention is to undertake a systematic review of current evidence, and an assessment of population needs and available services in order to estimate service gaps (Erlangsen et al., 2015). A recent systematic review of the best strategies for suicide prevention suggested a range of evidence-based interventions of relevance to the Australian context (Krysinska et al., 2016). This systematic review also estimated population preventive fractions (PPFs) for each of the identified strategies, which have since been summed to reach a conclusion that 20% of suicide and 30% of attempted suicide can be prevented over the next 5 years (Black Dog Institute, 2016).
This study focusses on one of the identified strategies as a case study, that of a psychosocial therapy intervention (Erlangsen et al., 2015). This intervention involved a combination of therapies including cognitive, problem solving, crisis, dialectical behaviour and integrated care, offered to individuals following a primary episode of self-harm and delivered nationally in Denmark in an outpatient context. This intervention showed a lower risk of suicide after 10 years of follow-up compared to those receiving usual care (odds ratio [OR] = 0.71, 95% confidence interval [95% CI] = [0.56, 0.91]) with an absolute risk reduction of 0.6% (95% CI = [0.1, 0.9]) between intervention and control groups. When combined with the estimated proportion of increased access to this intervention of 18.5% in Australia (based on the assumption of an increase from 31.6% to 50% (Krysinska et al., 2016)), the PPF can be calculated as 0.185 × (1 – 0.71) = 5.4% (as similarly calculated by Krysinska et al. (2016)). That is, assuming that the psychosocial therapy intervention is implemented immediately and sustained over the next 10 years with a participant uptake of 7.5% (Erlangsen et al., 2015) among all eligible individuals in the Australian population, then 5.4% of suicides (or 1936 suicides) potentially would be averted (Table 1).
Summary of estimated reductions in suicide from 2015 to 2025 for psychosocial programmes (Erlangsen et al., 2015) among those with a previous history of self-harm, using attributable fraction and dynamic simulation approaches.
Aside from the usual caveats associated with attributable fraction measures, relating to the internal validity of a study (such as sources of bias and confounding; Eide, 2008; Rockhill et al., 1998), this PPF is a snapshot of a particular period and does not account for change over time. It does not account for likely changes in the effect size over time (as evident in survival curves at different points in follow-up; Erlangsen et al., 2015), the time taken for the intervention to begin having an impact in the population, nor does it account for likely barriers to the uptake and availability of services for this intervention and differing social contexts. Each of the parameters used in the calculation of the preventive fraction is likely to change, as a function of demography, behavioural factors, contextual factors and health service availability.
Such limitations are routinely acknowledged in epidemiological papers using attributable fractions, however, it is often not made explicit in the communication of results, and in managing expectations of intervention effectiveness, among stakeholders and policy planners. This is important from a policy planning and resourcing perspective, as researchers need to be able to better articulate over what period the intervention might have an effect, how long it might take for an intervention to take effect, and the impact of behavioural aspects of uptake and participation in the intervention.
In contrast, dynamic simulation models do account for time in estimating the likely population-level impacts of interventions over time. Applying the same follow-up period of 10 years as Erlangsen et al. (2015), the system dynamics model of Australian suicide (Box 1) was used to estimate the likely number of cases of suicide prevented over time, assuming that the same effect size and proportion of the population exposed as above (PPF = 5.4%), but incorporating underlying population dynamics. A number of scenarios were run for comparison: (1) models that assumed different levels of participation in the intervention among those with a previous history of self-harm – 7.5% (as reported by Erlangsen et al. (2015)), 50% or 100% uptake; and (2) models that assumed different time periods to scale up to full implementation of the intervention, 2, 3 or 5 years (Table 1). Findings suggest that implementing the psychosocial therapy intervention with the same participant uptake as Erlangsen et al. (2015) would avert a lower proportion of suicides (between 0.4% and 0.5%) compared to the estimates based on the PPF alone (5.4%), and that in this case, the time to scale up to full implementation of the intervention does not have much impact on suicides averted (Table 1). Model estimates suggest that participant uptake would need to be 100%, and implementation achieved relatively quickly (within 3 years), to achieve a similar proportion of averted suicides (5.0–6.0%) as suggested by the PPF alone (Table 1).
Explanation for the difference in estimates of effect
The difference in the estimates of effects of psychosocial programmes on suicide produced using the static versus dynamic approaches relates to how the different models account for the ‘stocks’ and the ‘flows’ of people in the Australian population. Stocks and flows are the terms employed in system dynamics models and are analogous to ‘prevalence’ and ‘incidence’ in epidemiological parlance. Thus ‘stocks’ represent a quantity of people in a particular state (e.g. prevalence) and ‘flows’ relate to the rate of movement of people into, or out of, a stock (e.g. incidence and mortality or recovery rates). The level of the stock changes over time and is a function of (1) the rate at which people move into the given state and (2) the rate at which people move out of the given state. Figure 1 highlights a single stock from the conceptualised pathway of suicidal behaviour in the Australian population, where a proportion of the population move between no suicidal thoughts or behaviours, to suicidal ideation, to the planning of suicidal acts, to attempted suicide and completed suicide or survival.
An illustration of the architecture of system dynamics modelling.
In calculating an attributable fraction, it is assumed that the stocks and the flows in the population do not change: that is, the proportion of people accessing the intervention, the number of suicides to prevent and the underlying population remain constant. In contrast, Figure 2 shows that when incorporating the flows of movement of people through each of these states – particularly the feedback loops of people moving back into previous states – the proportion of the population exposed, the number of suicides to prevent and the underlying populations at risk over time
Three snapshots of the system at time (A)
For example, at the time depicted in Figure 2(A), the underlying population ‘considering suicide’ at time
Figure 2 highlights the change in values over time along the pathway of suicidal behaviour and illustrates where along that pathway the psychosocial therapy intervention acts in this model of Australian suicide. The effect of psychosocial therapy (as described above) is applied to modify the flows between those who have survived a suicide attempt and those considering suicide; and between those who were considering suicide and those in the ex-ideator population. The values in red show the subsequent impact of the intervention on the estimated number of people in each of the stocks in the model.
The psychosocial therapy intervention reduces the estimated number of suicide attempts and suicides compared to baseline. However, over time, the number of suicide attempts and suicides continue to increase: from 3135 (Figure 2(A)) to 4188 (Figure 2(C)) for suicide attempts; and from 280 (Figure 2(A)) to 282 (Figure 2(C)) for suicides. While the intervention appears to be effective, it is not sufficiently effective to reduce the ongoing accumulation of the estimated number of those considering suicide or attempting suicide, as the ‘inflow’ still exceeds the ‘outflow’.
This is illustrated more clearly in Figure 3, which shows the model estimated number of suicides in Australia over the period 2016–2026 with or without the impact of the psychosocial therapy intervention. Considering the trend in estimated suicides following the implementation of the intervention, the plateau in the estimated number of suicides from 2019 to 2021, followed by a subsequent continued increase, implies that the intervention was not successful. However, in comparison with the baseline scenario of no intervention, the model suggests that the psychosocial programme was associated with the prevention of a substantial number of suicides (approximately 1052 suicides under the scenario presented). While this is a simplified illustration, it demonstrates the need to develop analytical models that capture the dynamic and complex nature of the populations being targeted for suicide prevention, factors such as fluctuations in population size, socio-demographic composition, psychological distress, and differential uptake of interventions and services. It highlights the value of dynamic simulation modelling in supporting evaluation of suicide prevention programmes and decision making regarding what combination of interventions are likely to ‘bend the curve’ as shown in Figure 3 to achieve more significant and sustained reductions in suicide in Australia.
Forecasts of suicide deaths per annum in Australia 2016–2026; baseline versus psychosocial therapy intervention effect.*
Conclusion
Traditional epidemiological measures such as the attributable risk and its variants provide intuitive approximations of likely impacts of risk factors and interventions on a given population health burden and are computationally straightforward and can be quickly estimated. However, the underlying assumptions of these metrics are often understated, especially in the communication of findings to policy makers and in managing expectations of the potential impact of evidence-based interventions in real-world settings. It is important to acknowledge these limitations and to note that there are more sophisticated analytic tools that relax the strictures of many of the assumptions of traditional techniques to capture the complexity of the real world and which can potentially provide more appropriate information to guide decision making and resource allocation in suicide prevention.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.