Evaluating Machine Learning for Predicting Youth Suicidal Behavior up to 1 Year After Contact With Mental-Health Specialty Care

Abstract

In this article, we assessed the performance of several predictive modeling algorithms of suicide attempt resulting in inpatient hospitalization or suicide among youths ages 9 to 18 (N = 34,528) after contact (6–12 months) with a mental-health specialist in Stockholm, Sweden, from 2006 to 2012. Using 209 predictors across domains (e.g., clinical, demographic, family, neighborhood, social) identified from national registers, we applied standard logistic regression, regularized logistic regression, and machine-learning algorithms (i.e., random forests, gradient boosting, support vector machines). Standard logistic regression (area under the receiver operating characteristic curve [AUC] = 0.77, 95% confidence interval [CI] = [0.72, 0.82]) and random-forest models (AUC = 0.80, 95% CI = [0.74, 0.86]) demonstrated the highest AUCs. Sensitivities ranged from 0.33 (support vector machines) to 0.91 (standard logistic regression). Although the study was underpowered to detect a difference between logistic regression and machine-learning algorithms (outcome prevalence = 0.7%), performance metrics were similar across models. Logistic regression is not clearly worse than machine-learning approaches. Ongoing research is needed to examine how prediction models can augment clinical decision-making.