Detecting Cry in Daylong Audio Recordings Using Machine Learning: The Development and Evaluation of Binary Classifiers

Abstract

Atypical cry in infants/toddlers may serve as early, ecologically valid, and scalable indicators of irritability, a transdiagnostic mental health risk marker. Machine learning may identify cry in daylong audio recordings toward predicting outcomes. We developed a novel cry detection algorithm and evaluated performance against our reimplementation of an existing algorithm. In PyTorch, we reimplemented a support vector machine classifier that uses acoustic and deep spectral features from a modified AlexNet. We developed a novel classifier combining wav2vec 2.0 with conventional audio features and gradient boosting machines. Both classifiers were trained and evaluated using a previously annotated open-source data set (N = 21). In a new data set (N = 100), we annotated cry and examined the performance of both classifiers in identifying this ground truth. The existing and novel algorithms performed well in identifying ground truth cry in both the data set in which they were developed (AUCs = 0.897, 0.936) and the new data set (AUCs = 0.841, 0.902), underscoring generalization to unseen data. Bayesian comparison demonstrated that the novel algorithm outperformed the existing algorithm, which can be attributed to the novel algorithm’s feature space and use of gradient boosting machines. This research provides a foundation for efficient detection of atypical cry patterns, with implications for earlier identification of dysregulated irritability presaging psychopathology.