Suicide risk assessments: Why are we still relying on these a decade after the evidence showed they perform poorly?

Abstract

Suicide deaths have a profound impact on whānau and community and are a tragic loss. However, from a statistical point of view, suicide is a relatively rare event. Predicting rare events is difficult, and the implications for suicide prevention were highlighted in an important editorial in this journal more than a decade ago, yet little seems to have changed. Risk assessment that focuses on accurate prediction of suicide in real-world contexts is given a great deal of attention in mental health services, despite the fact that current scientific knowledge and best practice guidelines in this area highlight that it is unlikely to be a good basis on which to provide access to treatment. It is our view that we have a good enough understanding of the common conditions people who struggle with suicidal distress experience and energy is better directed at acting to reduce exposure to these conditions and providing treatment for those who seek it. Blueprints for successful suicide prevention exist. If we lessen the focus on prediction, we will have greater resources to focus on treatment and prevention.

Keywords

Risk assessment suicide prevention access to treatment

Suicide deaths have a profound impact on whānau and community (Berman, 2011; Cerel et al., 2019) and are a tragic loss. However, from a statistical point of view, suicide is a relatively rare event with a total population rate of 11.3 per 100,000 in New Zealand (Ministry of Health, 2019b) and 12.1 per 100,000 in Australia each year (Australian Institute of Health and Welfare, 2020). This is about the same rate as deaths due to diabetes (10.6 per 100,000) and much lower than the rate of deaths due to all types of cancers in New Zealand (114.9 per 100,000) (Ministry of Health, 2019a). Predicting rare events is difficult, and the implications for suicide prevention were highlighted in an important editorial more than a decade ago (Mulder, 2011), yet little seems to have changed in clinical practice.

The difficulty in predicting suicide was highlighted in an article by Rosen in 1954 (Rosen, 1954) and again four decades ago by Pokorny in 1983 (Pokorny, 1983). Recent reviews have shown that, despite advances in the understanding of risk factors for self-harm and suicide, and the development of various scales and checklists for risk assessment of those who are suicidal, there has been no appreciable advance in our ability to predict who will go on to engage in self-harm or suicide (Carter et al., 2017; Chan et al., 2016; Franklin et al., 2017; Large et al., 2016; Quinlivan et al., 2016; Ribeiro et al., 2016; Runeson et al., 2017; Woodford et al., 2019). This includes unassisted clinician prediction (Woodford et al., 2019), approaches that focus solely on the presence of suicidal ideation (McHugh et al., 2019) and the use of algorithms including via machine learning approaches (Belsher et al., 2019; McHugh and Large, 2020). One of these recent reviews offers an elegant demonstration of the findings across these reviews, showing that among mental health service users, the positive predictive value of risk assessment measures are low, ranging from 5.5% (95% confidence interval, CI = [3.9, 7.9]) for suicide to 26.3% (95% CI = [21.8, 31.3]) for self-harm and up to 35.9% (95% CI = [25.8, 47.4]) for self-harm plus suicide (Carter et al., 2017). This means the chances of a suicide risk assessment, having classified a patient as ‘high risk’ or ‘test positive’, accurately predicting that person will later die by suicide is around 5%, much lower than the accuracy of flipping a coin (50%).

Suicide risk stratification, or classifying a person as being at low, medium or high risk of suicide based on these risk assessment tools or measures, continues nevertheless to be used to decide who should, or should not, get access to treatment, despite recommendations to the contrary from professional bodies in Australia, New Zealand and the United Kingdom (Carter et al., 2016; National Institute for Health and Clinical Excellence (NICE), 2011). Given the poor performance of risk assessment approaches outlined above, many people are misclassified and may either receive treatments they don’t need or alternatively be denied access to treatments that would potentially help them (Carter et al., 2017; Hetrick et al., 2021; Large et al., 2017). The latter is particularly concerning (Sutherland, 2021), given those categorised as being at low risk of suicide do not get timely access to treatments that would reduce exposure to those factors that increase their risk of self-harm¹ and suicide such as treatment of depression, anxiety, post-traumatic stress disorder (PTSD) and substance misuse. There is also an iatrogenic effect of a person being labelled ‘low risk’ having reached out for help; this is extremely invalidating and potentially prevents future help-seeking (Bellairs-Walsh et al., 2020). The reliance on risk assessments to rationing access to treatment creates a toxic dynamic where service users, particularly those with prior experience of risk assessments, temper their responses to be suicidal enough to get access to treatment, but not to appear so distressed that they will be declined by a service (Anonymous, 2020) or be ‘sectioned’ under the Mental Health Act. This dynamic amplifies the extent to which certain types of knowledge are privileged over other types of knowledge, such as whānau perspectives, without consideration of how this knowledge was derived, or how valid it is considered to be by the service user. This is a significant challenge in the context of obligations of Te Tiriti o Waitangi and the principals of Māori data sovereignty (Kukutai and Cormack, 2021).

Behavioural science has borrowed important ideas from biological science in the domain of risk assessment, but one reason it does not work well is that suicidal behaviour is not a discrete condition like whether or not a person is infected with a disease. It seems much more likely that suicidal distress is a dynamic process, which ebbs and flows across time (Runeson et al., 1996). Studies using a life course approach identify fluctuations in suicidal distress that may emerge and then disappear only to re-emerge many years later (Fortune et al., 2007). Individuals and systems also behave as though there is a linear progression between ideation, plans and attempts. Theoretical models in the field of suicide prevention potentially reinforce this idea (O’Connor and Kirtley, 2018; van Orden et al., 2010) by describing phenomena that may be involved in the transition from suicidal thinking to suicidal behaviour. However, in clinical practice, this creates a preoccupation with quantifying a person’s intent to die and categorisations of behaviour such as non-suicidal self-injury and suicide attempt. While it is good practice to enquire directly and compassionately about suicidal ideation, planning and intent, clinicians need to understand that a number of studies have shown that those who deny suicidal ideation are very similar to those who endorse suicidal ideation. For example, Berman (2018) found that two-thirds of those who denied suicidal ideation died by suicide within 2 days of this denial (Berman, 2018), confirming findings of an earlier study that showed more than three-quarters of those who died by suicide denied suicidal thinking when they had been last asked (Busch et al., 2003). A more recent study found that severity of ideation did not differentiate between those who did and did not make a subsequent suicide attempt (Bryan et al., 2019). In addition, a number of studies over a number of years, using various methodologies including ecological momentary assessments (EMAs), have demonstrated that suicidal ideation varies significantly over very short periods of time such as a few hours (e.g. Kleiman and Nock, 2018) or one day to the next (Czyz et al., 2019). Time follow-back studies have also suggested significant variability across individuals in affective behaviour (Bagge et al., 2017) and intensity prior to suicide attempts (Bagge et al., 2014), which means ‘warning signs’ may be idiosyncratic to a person rather than universal.

The presence or absence of a plan, as an indicator of a person’s intent to die, is often given a great deal of weight in risk assessment. However, this is another area where the literature suggests a great deal of variability in the extent to which this can be relied upon to make predictions. Sometimes an apparent lack of plan will lead to an act being described as impulsive and somehow less serious than an event in which the person is able to describe a clear plan. Wyder and DeLeo (2007) conducted a study of community-based individuals who had made a suicide attempt that was categorised as either impulsive or non-impulsive. They found there were few differences between those categorised as impulsive and non-impulsive including on scores of impulsivity, where impulsive attempts were defined as occurring in the absence of a suicide plan, during a fluctuating suicidal process and experienced by the participant as impulsive. Similarly, Millner et al. (2015) found similar planning behaviours by those who reported a plan and those who denied a suicide plan. This study also showed that the initial onset of suicidal ideation and selection of method occurred many years before the index suicide attempt, but proximal steps, including onset of ideation, were within a week and most within 12 hours of the attempt. The order of steps was highly heterogeneous across people with some having no suicidal ideation until just prior to making an attempt. Overall, the rapid fluctuation (Kleiman and Nock, 2018) and non-linear nature of ideation, and onset of planning very proximal to an attempt can make it difficult to ‘catch’ the transition from suicidal ideation to plan or the existence of severe ideation or a plan in cross-sectional assessments. Clinicians should rather put energy into focusing on interventions a person can use when they experience escalating suicidal distress (e.g. Pisani et al., 2016).

Given the challenges outlined above, clinicians may sometimes rely on the method of self-harm used as a proxy for intent. However, findings in New Zealand (Fortune, 2006) and overseas (Owens et al., 2015) have shown that help-seeking young people, for example, switch between methods of self-harm, have poor knowledge of the lethality of methods and often underestimate risks (Daly et al., 2020; Harris and Myers, 1997; Wagner et al., 2002), making subgroups defined by method indistinguishable (Fortune, 2006). This is compounded by the fact that New Zealand has a higher prevalence of suicide deaths by hanging than many comparable countries. Therefore, most people have knowledge of hanging as a method, which is both highly available and very dangerous compared with some other methods.

Research and clinical experience suggest that front-line staff, those with lived experience and their whānau all have suggestions for how things could be improved (Fortune et al., 2021; Scarth et al., 2021). The context in which a risk assessment is conducted and the process of conducting a risk assessment are both likely to have a significant impact on both the person being assessed and the assessor, for example, the type of information a person may choose to share. The terminology of risk assessment is often experienced by those with lived or living experience of suicidal distress as judgmental and labelling. Current risk assessment approaches are binary rather than holistic and service users are clear in their desire for collaborative dialogue and involvement in treatment decision making, which often does not happen (Bellairs-Walsh et al., 2020). For these and other reasons, those being assessed may feel ambivalent about the process, engaging in retrospective rationalisation about the nature of, and reasons for, self-harm depending on responses they have received previously and what they believe about the attitudes to self-harm of those around them (Anonymous, 2020; Fortune, 2006; Rodham et al., 2004; Sheehan et al., 2017). For example, a young person who has experienced punitive measures after disclosing self-harm is less likely to do so again, or someone with a complex package of care due to their suicidal distress may be paradoxically reluctant to report less distress because it will lead to reductions in psychosocial support. There are often tensions between front-line workers’ aspirations for meaningful engagement with clients and the competing demands created by large workloads and compliance requirements from their organisation. Meaningful engagement has also been said to be compromised by the use of risk assessment tools and templates that are used by clinicians when interacting face-to-face (Graney et al., 2020).

Ideally, a discussion about suicide risk is based on a co-appraisal of the person’s circumstances and the things in their lives (exposures) thought to be associated with increased risk of engaging in self-harm or dying by suicide (Kessler et al., 2020). This discussion might include the use of paper-and-pencil and digital approaches to self-report, completed independent of the clinician, which appear acceptable, can result in greater disclosure and can be incorporated into a thorough clinical assessment of treatment needs (Bradford and Rickwood, 2015a, 2015b; Gnambs and Kaspar, 2015; Hetrick et al., 2017). There are also examples of tested approaches to a co-appraisal approach to assessment such as the Collaborative Assessment and Management of Suicidality (CAMS) (Swift et al., 2021) and SafeSide: Prevention Oriented Risk Formulation (Pisani et al., 2016).

The second important step, often neglected, is a robust formulation of interventions that could reduce those exposures identified as problematic for the person, from their perspective (Carter et al., 2017). In contrast to this ideal approach, a number of commentators have noted that organisations often lean on risk assessments to demonstrate competent care in the face of adverse events investigations (Graney et al., 2020). Front-line workers are aware of the trauma that a death by suicide creates and are highly motivated to prevent this outcome. Frequent repetition of risk assessment in the face of escalating concerns about a person with suicidal distress (Suicide Mortality Review Committee, 2016) can alleviate this anxiety, but at the same time distracts from progressing with interventions that will reduce exposure to factors that are amenable to intervention. However, risk assessments should not be frequently repeated across clinicians and services merely documenting static risk factors. Rather, the impetus must be to understand the modifiable factors associated with suicidal ideation and self-harm (why is this thinking/behaviour happening now and what might happen in the future) and provide treatment to address these modifiable factors. The key question is ‘what is the most helpful thing that I can do to meet the needs of this person right now?’.

As part of this discussion, it is important to reflect on the evidence for effective treatments to prevent self-harm and suicide. Our best evidence summary to date includes 76 trials focused on psychological (talking therapy) approaches to treating adults with self-harm (Witt et al., 2021b) that showed positive effects of cognitive behavioural therapy (CBT) approaches on repetition of self-harm at longer follow-up assessments (6 and 12 months), and for mentalisation-based therapy (MBT), and emotion-regulation psychotherapy at post-intervention. There was evidence that dialectical behaviour therapy (DBT) reduces the frequency of self-harm (Witt et al., 2021b). Evidence for the very large number of other approaches that have been tested was not very certain. It is notable that the approaches tested in this review tend to have a narrow focus on individual psychological processes underlying distress, rather than taking a holistic account of the circumstances and issues a person might be facing. Compassion-based and clinically accepted common sense interventions to address individual and whānau/family support, social connection, cultural identity, poverty and housing needs, restriction of access to means where the treatment is unlikely to cause harm and is relatively low cost are important (Carter et al., 2017; Hetrick et al., 2016). A brief psychodynamic interpersonal-focused intervention targeting the precipitants to self-harm illustrates this type of approach (Guthrie et al., 2001).

For other modifiable risk factors such as particular mental or personality disorders, there are some evidence-based interventions. For example, for borderline personality disorder, there is evidence for the efficacy of DBT; for depression there is evidence of the efficacy of CBT and Interpersonal psychotherapy (IPT) for depression. Antidepressant medication is also often prescribed, although evidence in children and adolescents shows that the effects of antidepressants in children are, on average, so small that there may be no meaningful impact for the child/adolescent and their whānau and these medications themselves carry increased risk of suicidal thinking and self-harm (Hetrick et al., 2021). It may be clinically important to try them in an individual child/adolescent, but close monitoring of the impact on symptoms is required. There is emerging evidence to suggest focusing on suicidal ideation and self-harm may be more effective than focusing on depression alone (Comtois and Linehan, 2006; Tarrier et al., 2008; Witt et al., 2021a). This focused approach nevertheless has good impact on depression and other symptoms (Hawton et al., 2015). Again, it is critical to monitor the effect of treatment (Witt et al., 2021a) with regular consideration of the positive effects and potential for adverse outcomes (Hetrick et al., 2021).

Overall, there is some way to go in developing and testing robust treatments for those with suicidal distress. On one hand, we know far too little about our treatments and their mechanisms of action to be able to engage in ‘precision medicine’ (Kessler et al., 2020). On the other hand, offering evidence-based interventions in the context of monitoring their impact, and ensuring that they address identified modifiable factors, is likely to be worthwhile given the large range of other negative outcomes those with suicidal distress are at risk of experiencing. Ultimately, the approach is to offer treatment to all those who choose to present to a service of the basis of assessment of modifiable risk factors and needs, rather than using risk assessment as a way to triage people away from services.

Innovative approaches to risk prediction

There is enthusiasm around various innovative approaches to enhance the practice of risk assessment, such as the promise of big data with passively collected information from electronic medical records and linked data from administrative data sets such as police, prison, housing and welfare contacts (Kessler et al., 2020). In addition, there are advances in terms of EMA approaches, which involve repeated random sampling in the natural environment in real time. People might be asked, for example, to rate their mood several times a day. This is combined with objectively collected data via personal electronic wearable devices (e.g. smart watches) such as levels of activity (Sedano-Capdevila et al., 2021). Researchers are also exploring the use of implicit assessments, with investigations of the associations people make subconsciously rather than pursuing explicit reporting of suicide intent given noted limitations of people’s ability or willingness to report this (Harrison et al., 2018; Nock et al., 2010). There are also advances in the use of natural language processing on data extracted from clinical records and social media (Berman and Carter, 2020; Kessler et al., 2020). However, there are ethical and practical constraints across these approaches, including the oversight and guardianship of the data (Kukutai and Cormack, 2021). It is unclear if there is social licence to use data collected for one purpose, for other reasons like monitoring suicide risk. There are concerns about the storage of data (often off shore to where they are collected), the security, privacy and access and opportunity for data to be corrected. Of concern, for example, is the tendency for agencies to assign diagnostic categories, which are stigmatising or inaccurate, and for these to cloned or copied forward in every future contact across agencies and systems (Berman and Carter, 2020). Furthermore, across all these approaches, the same issues regarding the predictive ability of these more recent approaches remain, with no evidence to date that suggests the Positive Predictive Value (PPV) is in any way improved (Berman and Carter, 2020).

Conclusion

In sum, risk assessment that focuses on accurate prediction of suicide in real-world contexts is given a great deal of attention in mental health services, but the current state of scientific knowledge and best practice guidelines in this area highlight that it is unlikely to be a good basis on which to Control access to treatment. However, the implementation of evidence-informed best practice into routine clinical practice is often limited; it takes an average of 17 years for evidence-based practice to be incorporated into routine health care (Morris et al., 2011). It is our view that we have a good enough understanding of the common conditions people who struggle with suicidal distress experience and energy is better directed at acting to reduce exposure to these conditions and providing treatment for those who seek it. This will reduce pain, increase hopefulness, enhance connection and reduce capability for suicide. Blueprints for successful suicide prevention exist. If we end our preoccupation with prediction, we have greater resources to focus on treatment and prevention.

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship and/or publication of this article: A/Prof Hetrick is Principal Clinical Advisor, Suicide Prevention Office. Dr Fortune is Chairperson of the Suicide Mortality Review Committee.

Funding

The author(s) received no financial support for the research, authorship and/or publication of this article.

ORCID iDs

Sarah Fortune

Sarah Hetrick

Notes

References

Anonymous (2020) The reality of risk assessment. Lancet Psychiatry 7: P1003–P1004.

Australian Institute of Health and Welfare (2020) Suicide and self-harm monitoring. Available at: www.aihw.gov.au/suicide-self-harm-monitoring/data/deaths-by-suicide-in-australia/suicide-deaths-over-time (accessed 22 February).

Bagge

Littlefield

Glenn

(2017) Trajectories of affective response as warning signs for suicide attempts: An examination of the 48 hours prior to a recent suicide attempt. Clinical Psychological Science 5: 259–271.

Bagge

Littlefield

Connor

, et al (2014) Near-term predictors of the intensity of suicidal ideation: An examination of the 24h prior to a recent suicide attempt. Journal of Affective Disorders 165: 53–58.

Bellairs-Walsh

Perry

Krysinska

, et al (2020) Best practice when working with suicidal behaviour and self-harm in primary care: A qualitative exploration of young people’s perspectives. BMJ Open 10: e038855.

Belsher

Smolenski

Pruitt

, et al (2019) Prediction models for suicide attempts and deaths: A systematic review and simulation. JAMA Psychiatry 76: 642–651.

Berman

(2011) Estimating the population of survivors of suicide: Seeking an evidence base. Suicide & Life-Threatening Behavior 41: 110–116.

Berman

(2018) Risk factors proximate to suicide and suicide risk assessment in the context of denied suicide ideation. Suicide & Life-Threatening Behavior 48: 340–352.

Berman

Carter

(2020) Technological advances and the future of suicide prevention: Ethical, legal, and empirical challenges. Suicide & Life-Threatening Behavior 50: 643–651.

10.

Bradford

Rickwood

(2015a) Acceptability and utility of an electronic psychosocial assessment (myAssessment) to increase self-disclosure in youth mental healthcare: A quasi-experimental study. BMC Psychiatry 15: 305.

11.

Bradford

Rickwood

(2015b) Young people’s views on electronic mental health assessment: Prefer to type than talk? Journal of Child and Family Studies 24: 1213–1221.

12.

Bryan

Rozek

Butner

, et al (2019) Patterns of change in suicide ideation signal the recurrence of suicide attempts among high-risk psychiatric outpatients. Behaviour Research and Therapy 120: 103392.

13.

Busch

Fawcett

Jacobs

(2003) Clinical correlates of inpatient suicide. The Journal of Clinical Psychiatry 64: 14–19.

14.

Carter

Milner

McGill

, et al (2017) Predicting suicidal behaviours using clinical instruments: Systematic review and meta-analysis of positive predictive values for risk scales. The British Journal of Psychiatry: The Journal of Mental Science 210: 387–395.

15.

Carter

Page

Large

, et al (2016) Royal Australian and New Zealand College of Psychiatrists clinical practice guideline for the management of deliberate self-harm. Australian and New Zealand Journal of Psychiatry 50: 939–1000.

16.

Cerel

Brown

Maple

, et al (2019) How many people are exposed to suicide? Not six. Suicide and Life Threatening Behaviour 49: 529–534.

17.

Chan

Bhatti

Meader

, et al (2016) Predicting suicide following self-harm: Systematic review of risk factors and risk scales. The British Journal of Psychiatry: The Journal of Mental Science 209: 277–283.

18.

Comtois

Linehan

(2006) Psychosocial treatments of suicidal behaviors: A practice-friendly review. Journal of Clinical Psychology 62: 161–170.

19.

Czyz

Horwitz

Arango

, et al (2019) Short-term change and prediction of suicidal ideation among adolescents: A daily diary study following psychiatric hospitalization. Journal of Child Psychology and Psychiatry, and Allied Disciplines 60: 732–741.

20.

Daly

Griffin

McMahon

, et al (2020) Repeat self-harm following hospital-presenting intentional drug overdose among young people—A national registry study. International Journal of Environmental Research and Public Health 17: 6159.

21.

Fortune

(2006) An examination of cutting and other methods of DSH among children and adolescents presenting to an outpatient psychiatric clinic in New Zealand. Clinical Child Psychology and Psychiatry 11: 407–416.

22.

Fortune

Pavlova

Hetrick

, et al (2021) First responder, clinician, and non-clinical support staff knowledge, attitudes, and behaviours towards people presenting for emergency care following self-harm: A mixed evidence synthesis. Cochrane Database of Systematic Reviews 12: 14939.

23.

Fortune

Stewart

Yadav

, et al (2007) Suicide in adolescents: Using life charts to understand the suicidal process. Journal of Affective Disorders 100: 199–210.

24.

Franklin

Ribeiro

Fox

, et al (2017) Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin 143: 187–232.

25.

Gnambs

Kaspar

(2015) Disclosure of sensitive behaviors across self-administered survey modes: A meta-analysis. Behavior Research Methods 47: 1237–1259.

26.

Graney

Hunt

Quinlivan

, et al (2020) Suicide risk assessment in UK mental health services: A national mixed-methods study. The Lancet Psychiatry 7: 1046–1053.

27.

Guthrie

Kapur

Mackway-Jones

, et al (2001) Randomised controlled trial of brief psychological intervention after deliberate self poisoning. British Medical Journal 323: 135–138.

28.

Harris

Myers

(1997) Adolescents’ misperceptions of the dangerousness of acetaminophen overdose. Suicide and Lifethreatening Behavior 27: 274–277.

29.

Harrison

Stritzke

WGK

Fay

, et al (2018) Suicide risk assessment: Trust an implicit probe or listen to the patient? Psychological Assessment 30: 1317–1329.

30.

Hawton

Witt

Taylor Salisbury

, et al (2015) Interventions for self-harm in children and adolescents. Cochrane Database of Systematic Reviews 12: CD012013.

31.

Hetrick

Goodall

Yuen

, et al (2017) Comprehensive online self-monitoring to support clinicians to manage risk of suicide in youth depression: A pilot study. Crisis 38: 147–157.

32.

Hetrick

McKenzie

Bailey

, et al (2021) New generation antidepressants for depression in children and adolescents: A network meta-analysis. Cochrane Database of Systematic Reviews Issue 5: CD013674.

33.

Hetrick

Robinson

Spittal

, et al (2016) Effective approaches to reduce repetition of deliberate self-harm: A systematic review and meta-regression. BMJ Open 6: e011024.

34.

Kessler

Bossarte

Luedtke

, et al (2020) Suicide prediction models: A critical review of recent research with recommendations for the way forward. Molecular Psychiatry 25: 168–179.

35.

Kleiman

Nock

(2018) Real-time assessment of suicidal thoughts and behaviors. Current Opinion in Psychology 22: 33–37.

36.

Kukutai

Cormack

(2021) ‘Pushing the space’: Data sovereignty and self-determination in Aotearoa NZ. In: Walter

Kukutai

Carroll

, et al (eds) Indigenous Data Sovereignty and Policy. London: Routledge, pp. 21–35.

37.

Large

Kaneson

Myles

, et al (2016) Meta-analysis of longitudinal cohort studies of suicide risk assessment among psychiatric patients: Heterogeneity in results and lack of improvement over time. PLOS ONE 11: e0156322.

38.

Large

Chung

Davidson

, et al (2017) In-patient suicide: Selection of people at risk, failure of protection and the possibility of causation. BJPsych Open 3: 102–105.

39.

McHugh

Corderoy

Ryan

, et al (2019) Association between suicidal ideation and suicide: Meta-analyses of odds ratios, sensitivity, specificity and positive predictive value. BJPsych Open 5: e18.

40.

McHugh

Large

(2020) Can machine-learning methods really help predict suicide? Current Opinion in Psychiatry 33: 369–374.

41.

Millner

Lee

Nock

(2015) Single-item measurement of suicidal behaviors: Validity and consequences of misclassification. PLOS ONE 10: e0141606.

42.

Ministry of Health (2019a) Mortality 2017 data tables. Available at: www.health.govt.nz/publication/mortality-2017-data-tables (accessed 26 June 2021).

43.

Ministry of Health (2019b) Suicide facts: Data tables 1996−2016. Available at: www.health.govt.nz/publication/suicide-facts-data-tables-19962016#reporting (accessed 12 July 2021).

44.

Morris

Wooding

Grant

(2011) The answer is 17 years, what is the question: Understanding time lags in translational research. Journal of the Royal Society of Medicine 104: 510–520.

45.

Mulder

(2011) Problems with suicide risk assessment. Australian and New Zealand Journal of Psychiatry 45: 605–607.

46.

National Institute for Health and Clinical Excellence (NICE) (2011) Self-harm in over 8’s: Longer-term management. NICE clinical guideline133 guidance. Available at: nice.org.uk/cg133 (issued November 2011).

47.

Nock

Park

Finn

, et al (2010) Measuring the suicidal mind: Implicit cognition predicts suicidal behavior. Psychological Science 21: 511–517.

48.

O’Connor

Kirtley

(2018) The integrated motivational-volitional model of suicidal behaviour. Philosophical Transactions of the Royal Society of London, Series B: Biological Sciences 373: 20170268.

49.

Owens

Kelley

Munyombwe

, et al (2015) Switching methods of self-harm at repeat episodes: Findings from a multicentre cohort study. Journal of Affective Disorders 180: 44–51.

50.

Pisani

Murrie

Silverman

(2016) Reformulating suicide risk formulation: From prediction to prevention. Academic Psychiatry: The Journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry 40: 623–629.

51.

Pokorny

(1983) Prediction of suicide in psychiatric patients: Report of a prospective study. Archives of General Psychiatry 40: 249–257.

52.

Quinlivan

Cooper

Davies

, et al (2016) Which are the most useful scales for predicting repeat self-harm? A systematic review evaluating risk scales using measures of diagnostic accuracy. BMJ Open 6: e009297.

53.

Ribeiro

Franklin

Fox

, et al (2016) Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: A meta-analysis of longitudinal studies. Psychological Medicine 46: 225–236.

54.

Rodham

Hawton

Evans

(2004) Reasons for deliberate self-harm: Comparison of self-poisoners and self-cutters in a community sample of adolescents. Journal of the American Academy of Child and Adolescent Psychiatry 43: 80–87.

55.

Rosen

(1954) Detection of suicidal patients: An example of some limitations in the prediction of infrequent events. Journal of Consulting Psychology 18: 397–403.

56.

Runeson

Beskow

Waern

(1996) The suicidal process in suicides among young people. Acta Psychiatrica Scandinavica 93: 35–42.

57.

Runeson

Odeberg

Pettersson

, et al (2017) Instruments for the assessment of suicide risk: A systematic review evaluating the certainty of the evidence. PLOS ONE 12: e0180292.

58.

Scarth

Pavlova

Hetrick

, et al (2021) Service users’ experiences of emergency care following an episode of self-harm: A mixed evidence synthesis. Cochrane Database of Systematic Reviews 6: 11.

59.

Sedano-Capdevila

Porras-Segovia

Bello

, et al (2021) Use of ecological momentary assessment to study suicidal thoughts and behavior: A systematic review. Current Psychiatry Reports Volume 23: 41.

60.

Sheehan

Corrigan

Al-Khouja

, et al (2017) Stakeholder perspectives on the stigma of suicide attempt survivors. Crisis 38: 73–81.

61.

Suicide Mortality Review Committee (2016) Ngā Rāhui Hau Kura (Suicide Mortality Review Committee. Feasibility Study 2014-15).

62.

Sutherland

(2021) The road to nowhere: New Zealanders struggle to get the help they need, 2 years on from a funding boose for mental health services. The Conversation, 16 April. Available at: https://theconversation.com/road-to-nowhere-new-zealanders-struggle-to-get-the-help-they-need-2-years-on-from-a-funding-boost-for-mental-health-services-158868 (accessed 16 April 2021).

63.

Swift

Trusty

Penix

(2021) The effectiveness of the Collaborative Assessment and Management of Suicidality (CAMS) compared to alternative treatment conditions: A meta-analysis. Suicide & Life-Threatening Behavior 51: 882–896.

64.

Tarrier

Taylor

Gooding

(2008) Cognitive-behavioral interventions to reduce suicide behavior: A systematic review and meta-analysis. Behavior Modification 32: 77–108.

65.

van Orden

Witte

Cukrowicz

, et al (2010) The interpersonal theory of suicide. Psychological Review 117: 575–600.

66.

Wagner

Wong

Jobes

(2002) Mental health professionals’ determinations of adolescent suicide attempts. Suicide & Life-Threatening Behavior 32: 284–300.

67.

Witt

Madsen

Berk

, et al (2021a) Trajectories of change in depression symptoms and suicidal ideation over the course of evidence-based treatment for depression: Secondary analysis of a randomised controlled trial of cognitive behavioural therapy plus fluoxetine in young people. Australian and New Zealand Journal of Psychiatry 55: 506–516.

68.

Witt

Hetrick

Rajaram

, et al (2021b) Interventions for self-harm in children and adolescents. Cochrane Database of Systematic Reviews Issue 3: CD013667.

69.

Woodford

Spittal

Milner

, et al (2019) Accuracy of clinician predictions of future self-harm: A systematic review and meta-analysis of predictive studies. Suicide & Life-Threatening Behavior 49: 23–40.

70.

Wyder

DeLeo

(2007) Behind impulsive suicide attempts: Indications from a community study. Journal of Affective Disorders 104: 167–173.