For decades, our ability to predict suicidal thoughts and behaviors (STBs) has been at near-chance levels. The objective of this study was to advance prediction by addressing two major methodological constraints pervasive in past research: (a) the reliance on long follow-ups and (b) the application of simple conceptualizations of risk. Participants were 1,021 high-risk suicidal and/or self-injuring individuals recruited worldwide. Assessments occurred at baseline and 3, 14, and 28 days after baseline using a range of implicit and self-report measures. Retention was high across all time points (> 90%). Risk algorithms were derived and compared with univariate analyses at each follow-up. Results indicated that short-term prediction alone did not improve prediction for attempts, even using commonly cited “warning signs”; however, a small set of factors did provide fair-to-good short-term prediction of ideation. Machine learning produced considerable improvements for both outcomes across follow-ups. Results underscore the importance of complexity in the conceptualization of STBs.
Barak-CorrenY.CastroV. M.JavittS.HoffnagleA. G.DaiY.PerlisR. H., . . . ReisB. Y. (2016). Predicting suicidal behavior from longitudinal electronic health records. American Journal of Psychiatry, 174, 154–162.
2.
BarrettL. F.SimmonsW. K. (2015). Interoceptive predictions in the brain. Nature Reviews Neuroscience, 16, 419–429.
3.
BastienC. H.VallièresA.MorinC. M. (2001). Validation of the Insomnia Severity Index as an outcome measure for insomnia research. Sleep Medicine, 2, 297–307.
4.
BeckA. T.SteerR. A. (1988). BHS, Beck Hopelessness Scale: Manual. San Antonio, TX: Psychological Corporation.
5.
BeckA. T.SteerR. A. (1991). BSI, Beck scale for suicide ideation: Manual. San Antonio, TX: Psychological Corporation.
6.
BentleyK. H.FranklinJ. C.RibeiroJ. D.KleimanE. M.FoxK. R.NockM. K. (2016). Anxiety and its disorders as risk factors for suicidal thoughts and behaviors: A meta-analytic review. Clinical Psychology Review, 43, 30–46.
7.
BirchwoodM.SmithJ. O.CochraneR.WettonS.CopestakeS. (1990). The Social Functioning Scale. The development and validation of a new scale of social adjustment for use in family intervention programmes with schizophrenic patients. The British Journal of Psychiatry, 157, 853–859.
8.
BrittonP. C.ConnerK. R. (2010). Suicide attempts within 12 months of treatment for substance use disorders. Suicide and Life-Threatening Behavior, 40, 14–21.
9.
BullS. S.VallejosD.LevineD.OrtizC. (2008). Improving recruitment and retention for an online randomized controlled trial: Experience from the Youthnet study. AIDS care, 20, 887–893.
10.
BuschK.FawcettJ.JacobsD. (2003). Clinical correlates of inpatient suicide. Journal of Clinical Psychiatry, 64, 14–19.
11.
CarterG.MilnerA.McGillK.PirkisJ.KapurN.SpittalM. J. (2017). Predicting suicidal behaviours using clinical instruments: Systematic review and meta-analysis of positive predictive values for risk scales. The British Journal of Psychiatry, 210, 387–395.
12.
CaslerK.BickelL.HackettE. (2013). Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29, 2156–2160.
ChangB. P.FranklinJ. C.RibeiroJ. D.FoxK. R.BentleyK. H.KleimanE. M.NockM. K. (2016). Biological risk factors for suicidal behaviors: A meta-analysis. Translational Psychiatry, 6, e887. doi:10.1038/tp.2016.165
15.
ChuC.BodellL. P.RibeiroJ. D.JoinerT. E. (2015). Eating disorder symptoms and suicidal ideation: The moderating role of disgust. European Eating Disorders Review, 23, 545–552.
16.
CrosbyA.GfroererJ.HanB.OrtegaL.ParksS. E. (2011). Suicidal thoughts and behaviors among adults aged ≥18 years — United States, 2008-2009. Atlanta, GA: Centers for Disease Control and Prevention.
EdelmanG. M.GallyJ. A. (2001). Degeneracy and complexity in biological systems. Proceedings of the National Academy of Sciences, 98, 13763–13768.
19.
FranklinJ. C.FoxK. R.FranklinC. R.KleimanE. M.RibeiroJ. D.JaroszewskiA. C.. . . NockM. K. (2016). A brief mobile app reduces nonsuicidal and suicidal self-injury: Evidence from three randomized controlled trials. Journal of Consulting and Clinical Psychology, 84, 544–557.
20.
FranklinJ. C.PuziaM. E.LeeK. M.LeeG. E.HannaE. K.SpringV. L.PrinsteinM. J. (2013). The nature of pain offset relief in nonsuicidal self-injury: a laboratory study. Clinical Psychological Science, 1, 110–119.
21.
FranklinJ. C.RibeiroJ. D.FoxK. R.BentleyK. H.KleimanE. M.HuangX.. . . NockM. K. (2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143, 187–232.
22.
HastieT.TibshiraniR.FriedmanJ. H. (2001). The elements of statistical learning: Data mining, inference, and predictions. New York, NY: Springer.
23.
HauserD. J.SchwarzN. (2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48, 400–407.
24.
HendinH.MaltsbergerJ. T.SzantoK. (2007). The role of intense affective states in signaling a suicide crisis. The Journal of Nervous and Mental Disease, 195, 363–368.
25.
HuangX.FoxK. R.RibeiroJ. D.FranklinJ. C. (2018). Psychosis as a risk factor for suicidal thoughts and behaviors: A meta-analysis of longitudinal studies. Psychological Medicine, 48, 765–776.
26.
HuangX.RibeiroJ. D.MusacchioK. M.FranklinJ. C. (2017). Demographics as predictors of suicidal thoughts and behaviors: A meta-analysis. PLOS ONE, 12(7), Article e0180793. doi:10.1371/journal.pone.0180793
27.
IpE. J.BarnettM. J.TenerowiczM. J.PerryP. J. (2010). The touro 12-step: A systematic guide to optimizing survey research with online discussion boards. Journal of Medical Internet Research, 12(2), Article e16. doi:10.2196/jmir.1314
28.
JamesG.WittenD.HastieT.TibshiraniR. (2013). An introduction to statistical learning. New York, NY: Springer.
29.
JoinerT. E.RuddM. D.RajabM. H. (1997). The Modified Scale for Suicidal Ideation: Factors of suicidality and their relation to clinical and diagnostic variables. Journal of Consulting and Clinical Psychology, 106, 260–265.
30.
KesslerR. C.van LooH. M.WardenaarK. J.BossarteR. M.BrennerL. A.CaiT.. . . NierenbergA. A. (2016). Testing a machine-learning algorithm to predict the persistence and severity of major depressive disorder from baseline self-reports. Molecular Psychiatry, 21, 1366–1371.
31.
KesslerR. C.WarnerC. H.IvanyC.PetukhovaM. V.RoseS.BrometE. J.. . . FullertonC. S. (2015). Predicting suicides after psychiatric hospitalization in US Army soldiers: The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). JAMA Psychiatry, 72, 49–57.
32.
KraemerH. C.KazdinA. E.OffordD. R.KesslerR. C.JensenP. S.KupferD. J. (1997). Coming to terms with the terms of risk. Archives of General Psychiatry, 54, 337–343.
NockM. K.HolmbergE. B.PhotosV. I.MichelB. D. (2007). Self-Injurious Thoughts and Behaviors Interview: Development, reliability, and validity in an adolescent sample. Psychological Assessment, 19, 309–317.
35.
PayneB. K.ChengC. M.GovorunO.StewartB. D. (2005). An inkblot for attitudes: Affect misattribution as implicit measurement. Journal of Personality and Social Psychology, 89, 277–293.
36.
R Core Team. (2014). R: A language and environment for statistical computing (Version 3.1.2) [Computer software]. Retrieved from https://www.r-project.org/index.html
37.
RibeiroJ. D.BenderT. W.SelbyE. A.HamesJ. L.JoinerT. E. (2011). Development and validation of a brief self-report measure of agitation: The Brief Agitation Measure. Journal of Personality Assessment, 93, 597–604.
38.
RibeiroJ. D.BodellL.JoinerT. E. (2012, November). Disgust with self, others, and world in suicidality. Poster Presented at the 46th Annual Meeting of the Association for Behavioral and Cognitive Therapies, National Harbor, MD.
39.
RibeiroJ. D.FranklinJ. C.FoxK. R.BentleyK. H.KleimanE. M.ChangB. P.NockM. K. (2016a). Self-injurious thoughts and behaviors as risk factors for future suicide ideation, attempts, and death: A meta-analysis of longitudinal studies. Psychological Medicine, 46, 225–236.
40.
RibeiroJ. D.FranklinJ. C.FoxK. R.BentleyK. H.KleimanE. M.ChangB. P.NockM. K. (2016b). Letter to the editor: Suicide as a complex classification problem: Machine learning and related techniques can advance suicide prediction—a reply to Roaldset (2016). Psychological Medicine, 46, 2009–2010.
41.
RibeiroJ. D.HuangX.FoxK. R.FranklinJ. C. (2018). Depression and hopelessness as risk factors for suicide ideation, attempts and death: Meta-analysis of longitudinal studies. The British Journal of Psychiatry, 212, 279–286.
42.
RibeiroJ. D.WitteT. K.Van OrdenK. A.SelbyE. A.GordonK. H.BenderT. W.JoinerT. E.Jr. (2014). Fearlessness about death: The psychometric properties and construct validity of the revision to the Acquired Capability for Suicide Scale. Psychological Assessment, 26, 115–126.
43.
RobinX.TurckN.HainardA.TibertiN.LisacekF.SanchezJ.-C.. . . MüllerM. (2018). pROC: Display and analyze ROC curves (Version 1.12.1) [Software]. Retrieved from https://cran.r-project.org/web/packages/pROC
44.
RuddM. D.BermanA. L.JoinerT. E.Jr.NockM. K.SilvermanM. M.MandrusiakM.. . . WitteT. (2006). Warning signs for suicide: Theory, research, and clinical applications. Suicide and Life-Threatening Behavior, 36, 255–262.
45.
SmithA. R.VelkoffE. A.RibeiroJ. D.FranklinJ. (2018). Are eating disorders and related symptoms risk factors for suicidal thoughts and behaviors? A meta-analysis. Suicide and Life-Threatening Behavior, 49, 221–239.
46.
SmithG. C.SeamanS. R.WoodA. M.RoystonP.WhiteI. R. (2014). Correcting for optimistic prediction in small data sets. American Journal of Epidemiology, 180, 318–324.
47.
Substance Abuse and Mental Health Services Administration. (2017). Key substance use and mental health indicators in the United States: Results from the 2016 National Survey on Drug Use and Health (HHS Publication No. SMA 17-5044, NSDUH Series H-52). Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. Retrieved from https://www.samhsa.gov/data/
48.
WalshC. G.RibeiroJ. D.FranklinJ. C. (2017). Predicting risk of suicide attempts over time through machine learning. Clinical Psychological Science, 5, 457–469.
49.
WalshC. G.RibeiroJ. D.FranklinJ. C. (2018). Predicting suicide attempts in adolescents with longitudinal clinical data and machine learning. Journal of Child Psychology and Psychiatry, 59, 1261–1270.
50.
WitteT. K.GauthierJ. M.HuangX.RibeiroJ. D.FranklinJ. C. (2018). Is externalizing psychopathology a robust risk factor for suicidal thoughts and behaviors? A meta-analysis of longitudinal studies. Journal of Clinical Psychology, 74, 1607–1625.
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
