Restricted accessResearch articleFirst published online 2021-12
Using Intensive Longitudinal Data to Identify Early Predictors of Suicide-Related Outcomes in High-Risk Adolescents: Practical and Conceptual Considerations
Mobile technology offers new possibilities for assessing suicidal ideation and behavior in real- or near-real-time. It remains unclear how intensive longitudinal data can be used to identify proximal risk and inform clinical decision making. In this study of adolescent psychiatric inpatients (N = 32, aged 13-17 years, 75% female), we illustrate the application of a three-step process to identify early signs of suicide-related crises using daily diaries. Using receiver operating characteristic (ROC) curve analyses, we considered the utility of 12 features—constructed using means and variances of daily ratings for six risk factors over the first 2 weeks postdischarge (observations = 360)—in identifying a suicidal crisis 2 weeks later. Models derived from single risk factors had modest predictive accuracy (area under the ROC curve [AUC] 0.46-0.80) while nearly all models derived from combinations of risk factors produced higher accuracy (AUCs 0.80-0.91). Based on this illustration, we discuss implications for clinical decision making and future research.
AlmirallD.ComptonS. N.Gunlicks-StoesselM.DuanN.MurphyS. A. (2012). Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy. Statistics in Medicine, 31(17), 1887-1902. https://doi.org/10.1002/sim.4512
2.
BaggeC. L.GlennC. R.LeeH. J. (2013). Quantifying the impact of recent negative life events on suicide attempts. Journal of Abnormal Psychology, 122(2), 359-368. https://doi.org/10.1037/a0030371
Ben-ZeevD.YoungM. A.DeppC. A. (2012). Real-time predictors of suicidal ideation: Mmobile assessment of hospitalized depressed patients. Psychiatry Research, 197(1-2), 55-59. https://doi.org/10.1016/j.psychres.2011.11.025
5.
BollandJ.McCallumD.LianB.BaileyC.RowanP. (2001). Hopelessness and violence among inner-city youths. Maternal and Child Health Journal, 5(4), 237-244. https://doi.org/10.1023/A:1013028805470
CollinsL. M.TrailJ. B.KuglerK. C.BakerT. B.PiperM. E.MermelsteinR. J. (2013). Evaluating individual intervention components: Making decisions based on the results of a factorial screening experiment. Translational Behavioral Medicine, 4(3), 238-251. https://doi.org/10.1007/s13142-013-0239-7
8.
CzyzE. K.BohnertA. S.KingC. A.PriceA. M.KleinbergF.IlgenM. A. (2014). Self-efficacy to avoid suicidal action: Factor structure and convergent validity among adults in substance use disorder treatment. Suicide and Life-Threatening Behavior, 44(6), 698-709. https://doi.org/10.1111/sltb.12101
9.
CzyzE. K.HorwitzA. G.ArangoA.KingC. A. (2018). Short-term change and prediction of suicidal ideation among adolescents: A daily diary study following psychiatric hospitalization. Journal of Child Psychology and Psychiatry, 60(7), 732-741. https://doi.org/10.1111/jcpp.12974
10.
CzyzE. K.HorwitzA. G.KingC. A. (2016). Self-rated expectations of suicidal behavior predict future suicide attempts among adolescent and young adult psychiatric emergency patients. Depression & Anxiety, 33(6), 512-519. https://doi.org/10.1002/da.22514
11.
CzyzE. K.KingC. A.Nahum-ShaniI. (2018). Ecological assessment of daily suicidal thoughts and attempts among suicidal teens after psychiatric hospitalization: Lessons about feasibility and acceptability. Psychiatry Research, 267, 566-574. https://doi.org/10.1016/j.psychres.2018.06.031
12.
CzyzE. K.KingC. A.BiermannB. J. (2019). Motivational interviewing-enhanced safety planning for adolescents at high suicide risk: A pilot randomized controlled trial. Journal of Clinical Child & Adolescent Psychology, 48(2), 250-262.
13.
DavidsonC. L.AnestisM. D.GutierrezP. M. (2017). Ecological momentary assessment is a neglected methodology in suicidology. Archives of Suicide Research, 21(1), 1-11. https://doi.org/10.1080/13811118.2015.1004482
14.
EbesutaniC.ReganJ.SmithA.ReiseS.Higa-McMillanC.ChorpitaB. F. (2012). The 10-item positive and negative affect schedule for children, child and parent shortened versions: Application of item response theory for more efficient assessment. Journal of Psychopathology and Behavioral Assessment, 34(2), 191-203. https://doi.org/10.1007/s10862-011-9273-2
15.
FranklinJ. C.RibeiroJ. D.FoxK. R.BentleyK. H.KleimanE. M.HuangX.MusacchioK. M.JaroszewskiA. C.ChangB. P.NockM. K. (2017). Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychological Bulletin, 143(2), 187-232. https://doi.org/10.1037/bul0000084
16.
GlennC. R.NockM. K. (2014). Improving the short-term prediction of suicidal behavior. American Journal of Preventive Medicine, 47(3, Suppl. 2), S176-S180. https://doi.org/10.1016/j.amepre.2014.06.004
17.
HallenslebenN.SpangenbergL.ForkmannT.RathD.HegerlU.KerstingA.KallertT. W.GlaesmerH. (2018). Investigating the dynamics of suicidal ideation. Crisis, 39(1), 65-69. https://doi.org/10.1027/0227-5910/a000464
18.
HuskyM.OliéE.GuillaumeS.GentyC.SwendsenJ.CourtetP. (2014). Feasibility and validity of ecological momentary assessment in the investigation of suicide risk. Psychiatry Research, 220(1-2), 564-570. https://doi.org/10.1016/j.psychres.2014.08.019
19.
Huth-BocksA. C.KerrD. C.IveyA. Z.KramerA. C.KingC. A. (2007). Assessment of psychiatrically hospitalized suicidal adolescents: Self-report instruments as predictors of suicidal thoughts and behavior. Journal of the American Academy of Child & Adolescent Psychiatry, 46(3), 387-395. https://doi.org/10.1097/chi.0b013e31802b9535
20.
JoinerT. (2005). Why people die by suicide. Harvard University Press.
21.
KennardB. D.GoldsteinT.FoxwellA. A.McMakinD. L.WolfeK.BiernesserC.MooreheadA.DouaihyA.ZulloL.WentrobleE.OwenV.ZelaznyJ.IyengarS.PortaG.BrentD. (2018). As Safe as Possible (ASAP): A brief app-supported inpatient intervention to prevent postdischarge suicidal behavior in hospitalized, suicidal adolescents. American Journal of Psychiatry, 175(9), 864-872. https://doi.org/10.1176/appi.ajp.2018.17101151
22.
KleimanE. M.TurnerB. J.FedorS.BealeE. E.HuffmanJ. C.NockM. K. (2017). Examination of real-time fluctuations in suicidal ideation and its risk factors: Results from two ecological momentary assessment studies. Journal of Abnormal Psychology, 126(6), 726-738. https://doi.org/10.1037/abn0000273
23.
KlonskyE. D.MayA. M. (2015). The Three-Step Theory (3ST): A new theory of suicide rooted in the “ideation-to-action” framework. International Journal of Cognitive Therapy, 8(2), 114-129. https://doi.org/10.1521/ijct.2015.8.2.114
24.
LeiH.Nahum-ShaniI.LynchK.OslinD.MurphyS. A. (2012). A “SMART” design for building individualized treatment sequences. Annual Review of Clinical Psychology, 8, 21-48. https://doi.org/10.1146/annurev-clinpsy-032511-143152
25.
MillnerA. J.LeeM. D.NockM. K. (2017). Describing and measuring the pathway to suicide attempts: A preliminary study. Suicide and Life-Threatening Behavior, 47(3), 353-369. https://doi.org/10.1111/sltb.12284
26.
MohrD. C.ZhangM.SchuellerS. M. (2017). Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annual Review of Clinical Psychology, 13, 23-47. https://doi.org/10.1146/annurev-clinpsy-032816-044949
27.
Nahum-ShaniI.QianM.AlmirallD.PelhamW. E.GnagyB.FabianoG. A.WaxmonskyJ. G.YuJ.MurphyS. A. (2012). Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological Methods, 17(4), 457-477. https://doi.org/10.1037/a0029372
28.
Nahum-ShaniI.SmithS. N.SpringB. J.CollinsL. M.WitkiewitzK.TewariA.MurphyS. A. (2017). Just-in-time adaptive interventions (JITAIs) in mobile health: Key components and design principles for ongoing health behavior support. Annals of Behavioral Medicine, 52(6), 446-462. https://doi.org/10.1007/s12160-016-9830-8
29.
National Action Alliance for Suicide Prevention. (2014). Research prioritization task force: A prioritized research agenda for suicide prevention: an action plan to save lives. National Institute of Mental Health and Research Prioritization Task Force.
30.
PencinaM. J.D’AgostinoR. B.Sr.D’AgostinoR. B.Jr.VasanR. S. (2008). Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond. Statistics in Medicine, 27(2), 157-172. https://doi.org/10.1002/sim.2929
31.
PencinaM. J.D’agostinoR. B.PencinaK. M.JanssensA. C. J.GreenlandP. (2012). Interpreting incremental value of markers added to risk prediction models. American Journal of Epidemiology, 176(6), 473-481. https://doi.org/10.1093/aje/kws207
32.
PettitJ. W.SilvermanW. K.ReyY.MarinC.JaccardJ. (2016). Moving to second-stage treatments faster: Identifying midtreatment tailoring variables for youth with anxiety disorders. Journal of Clinical Child & Adolescent Psychology, 45(4), 457-468. https://doi.org/10.1080/15374416.2015.1038824
33.
PosnerK.BrownG. K.StanleyB.BrentD. A.YershovaK. V.OquendoM. A.… MannJ. J. (2011). The Columbia–Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. American journal of psychiatry, 168(12), 1266-1277.
34.
ReinertsenE.CliffordG. D. (2018). A review of physiological and behavioral monitoring with digital sensors for neuropsychiatric illnesses. Physiological Measurement, 39(5), Article 05TR01. https://doi.org/10.1088/1361-6579/aabf64
35.
RiceM. E.HarrisG. T. (2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29(5), 615-620. https://doi.org/10.1007/s10979-005-6832-7
36.
RobinX.TurckN.HainardA.TibertiN.LisacekF.SanchezJ. C.MüllerM. (2011). pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12, Article 77. https://doi.org/10.1186/1471-2105-12-77
RunesonB.OdebergJ.PetterssonA.EdbomT.AdamssonI. J.WaernM. (2017). Instruments for the assessment of suicide risk: A systematic review evaluating the certainty of the evidence. PLOS ONE, 12(7), Article e0180292. https://doi.org/10.1371/journal.pone.0180292
39.
SchneiderS.StoneA. A. (2016). Ambulatory and diary methods can facilitate the measurement of patient-reported outcomes. Quality of Life Research, 25(3), 497-506. https://doi.org/10.1007/s11136-015-1054-z
40.
SchuellerS. M.AguileraA.MohrD. C. (2017). Ecological momentary interventions for depression and anxiety. Depression & Anxiety, 34(6), 540-545. https://doi.org/10.1002/da.22649
ShneidmanE. S. (1993). Suicide as psychache: A clinical approach to self-destructive behavior. Jason Aronson.
43.
SteidtmannD.ManberR.BlaseyC.MarkowitzJ. C.KleinD. N.RothbaumB. O.ThaseM. E.KocsisJ. H.ArnowB. A. (2013). Detecting critical decision points in psychotherapy and psychotherapy+ medication for chronic depression. Journal of Consulting and Clinical psychology, 81(5), 783-792. https://doi.org/10.1037/a0033250
44.
StoneA. A.BroderickJ. E.SchneiderS.SchwartzJ. E. (2012). Expanding options for developing outcome measures from momentary assessment data. Psychosomatic Medicine, 74(4), 387-397. https://doi.org/10.1097/PSY.0b013e3182571faa
45.
TibshiraniR. J. (2014). In praise of sparsity and convexity. In LinX.GenestC.BanksD. L.MolenberghsG.ScottD. W.WangJ.-L. (Eds.), Past, present, and future of statistical science (pp. 497-505). CRC. https://doi.org/10.1201/b16720
46.
TorousJ.LarsenM. E.DeppC.CoscoT. D.BarnettI.NockM. K.FirthJ. (2018). Smartphones, sensors, and machine learning to advance real-time prediction and interventions for suicide prevention: A review of current progress and next steps. Current Psychiatry Reports, 20(7), Article 51. https://doi.org/10.1007/s11920-018-0914-y
47.
Van OrdenK. A.WitteT. K.CukrowiczK. C.BraithwaiteS. R.SelbyE. A.JoinerT. E.Jr. (2010). The interpersonal theory of suicide. Psychological Review, 117(2), 575-600. https://doi.org/10.1037/a0018697
48.
WenzelA.BeckA. T. (2008). A cognitive model of suicidal behavior: Theory and treatment. Applied and Preventive Psychology, 12(4), 189-201. https://doi.org/10.1016/j.appsy.2008.05.001
49.
WuC. F. J.HamadaM. S. (2009). Experiments: Planning, analysis, and parameter design optimization (2nd ed.). Wiley.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
