Machine learning methods to discriminate posttraumatic stress disorder: A protocol of systematic review and meta-analysis

Abstract

Introduction

Recent years have witnessed a persistent threat to public mental health, especially during and after the COVID-19 pandemic. Posttraumatic stress disorder (PTSD) has emerged as a pivotal concern amidst this backdrop. Concurrently, machine learning (ML) techniques have progressively applied in the realm of mental health. Therefore, our present undertaking seeks to provide a comprehensive assessment of studies employing ML methods that use diverse data modalities on the classification of people with PTSD.

Methods and analysis

In pursuit of pertinent studies, we will search both English and Chinese databases from January 2000 to May 2022. Two researchers will independently conduct screening, extract data and assess study quality. We intend to employ the assessment framework introduced by Luis Francisco Ramos-Lima in 2020 for quality evaluation. Rate, standard error and 95% CIs will be utilized for effect size measurement. A Cochran's Q test will be applied to assess heterogeneity. Subgroup and sensitivity analysis will further elucidate the source of heterogeneity and funnel plots and Egger's test will detect publication bias.

Ethics and dissemination

This systematic review and meta-analysis does not encompass patient interactions or engagements with healthcare providers. The outcomes of this research will be disseminated through scholarly channels, including presentations at scientific conferences and publications in peer-reviewed journals.

PROSPERO registration number CRD42023342042.

Keywords

Machine learning posttraumatic stress disorder protocol systematic review meta-analysis

Introduction

Posttraumatic stress disorder (PTSD) is a severe and significant psychiatric disorder that may occur in people who have experienced or witnessed a traumatic event or a series of circumstances.¹ According to the American Psychiatric Association (APA), PTSD affects approximately 3.5% of U.S. adults each year. The lifetime prevalence of PTSD in adolescents aged 13−18 is 8%. It is estimated that 1 in 11 people will be diagnosed with PTSD in their lifetime and women are twice as likely to have PTSD as men.² The World Mental Health Survey Consortium revealed that over 70% of the general population worldwide has experienced at least one traumatic event in their lifetime around the world.³ PTSD is most commonly associated with combat veterans. It may also occur in people who have experienced natural disasters, serious accidents, sexual assault, historical trauma, intimate partner violence and bullying.⁴

As the Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5) noted, PTSD is characterized by intrusive experience, persistent avoidance of stimuli, negative alterations in cognition and mood and marked alterations in arousal and reactivity related to the traumatic events.⁵ The symptoms of PTSD can vary in severity and be long-lasting.⁶ Some research studies have found that a significant proportion of individuals with PTSD continue to experience symptoms years after the traumatic events.⁷ Therefore, the early detection of PTSD and early intervention are of great importance and essential for proper treatment.

Machine learning (ML) is a powerful analytic technique that can process multiple types of data. ML approaches are gradually being applied to classify, identify and predict individuals with various psychiatric disorders.¹ Recently, ML methods have been utilized in classifying PTSD individuals,⁸ identifying the onset of their disorders,⁹ regulating their emotion,¹⁰ selecting treatment for them¹¹ and analyzing their multivariate predictors¹² and ranking predicting features.¹³ The commonly used ML algorithms are logistic regression (LR),¹⁴ random forest (RF)¹⁵ and support vector machine (SVM).¹⁶ We aim to review the literature on the use of ML techniques in the assessment of PTSD subjects to distinguish individuals with PTSD from other psychiatric disorders or from trauma-exposed and healthy controls or to optimize the predictors of PTSD.

Review question

What accuracy can ML techniques achieve in the classification of people with PTSD by analyzing different types of data?

Methods

The present protocol was formatted in accordance with the guidelines outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) statement.¹⁷ The completed PRISMA-P checklist is shown in Supplemental Table S1.

Eligibility criteria

As this is a retrospective study and does not involve direct patient participation, ethical approval and informed consent are not needed. The selection criteria of studies are defined according to the PICOTS framework as shown in Table 1.

Table 1.

Selection criteria of studies in PICOTS format.

PICOTS	Inclusion criteria	Exclusion criteria
Population (P)	Individuals with PTSD.	Patients with non-PTSD related brain injury or other psychiatric disorders.
Interventions (I)	A clear description of the ML model type: supervised, unsupervised, semi-supervised or ensemble; A clear description of the data type; A clear description of the data amount used by the models not only the sample size.	Studies without ML approaches; No clear description of ML models or data types; No data amount.
Comparator (C)	Not applicable.
Outcomes (O)	Primary: best accuracy of ML models. Secondary: sensitivity, specificity, AUC and positive and negative predictive values.	No ML models’ accuracy metrics.
Timing frame (T)	Since January 2000 to May 2022
Settings (S)	PTSD is the dependent variable.	Correlation studies; Qualitative studies; Case studies; Protocols; Meta-analysis; Reviews; Editorials, comments, letters, notes.
Other limits	Language = English or Chinese

Abbreviations: ML: machine learning; PTSD: posttraumatic stress disorder; AUC: area under the curve.

Population (P)

This study will consider individuals affected by PTSD after experiencing trauma, without imposing any demographic or health-related exclusions based on age, gender, ethnicity or other relevant factors. Trauma events include military missions, traffic accidents, abuse, etc.

Interventions (I)

Supervised, unsupervised, semi-supervised or ensemble ML models are all eligible interventions to classify PTSD individuals. For the sake of computational convenience, we will also verify the following three aspects. (a) A clear description of the ML model, (b) a clear description of the data type model used and (c) a clear description of the data amount processed by the model which determined performance metrics. Studies will be excluded if they did not apply ML approaches or report data type and amount clearly.

Outcomes (O)

The performance of ML models in discriminating PTSD will be measured by the best accuracy of all the models reported in one article. Other performance metrics, such as sensitivity, specificity, area under the curve (AUC) and positive and negative predictive values, if reported, will also be recorded. Studies reported no accuracy metrics will be excluded.

Timing frame (T)

Given the evolution of ML technology and its surge in the field of mental health in recent years, literature retrieval will be conducted from the year 2000 to the present cutoff date of the search (May 2022). Researches conducted outside this timeframe will be ineligible.

Settings (S)

Studies with PTSD as a dependent variable in both clinical and non-clinical settings will be included. Correlation studies, qualitative studies, case studies, protocols, meta-analysis, reviews, editorials, comments, letters and notes will be excluded.

Due to our language limitations, only studies of English and Chinese will be included.

Data sources

We will search both English and Chinese databases for relevant studies: PubMed, Embase, Scopus, PsycINFO and Cochrane Library for publications in English and China National Knowledge Infrastructure Database (CNKI), the Wanfang database and China Science and Technology Journal Database for publications in Chinese.

Search strategy

The search strategy will follow the PICOTS frame including terms relating to ML and PTSD and being strict to the time span. A systematic literature search will be carried out based on the following terms: ((“Machine Learning”[Mesh]) OR (“reinforcement learning”) OR (“Natural Language Processing”[Mesh]) OR ((“Deep Learning”[Mesh]) OR (“Hierarchical Learning”)) OR (“Unsupervised Machine Learning”[Mesh]) OR ((“Supervised Machine Learning”[Mesh]) OR (“semi-supervised learning”) OR (“semi-supervised machine learning”)) OR (“Big Data”[Mesh]) OR (“Artificial Intelligence”[Mesh])) AND ((“Stress Disorders, Post-Traumatic”[Mesh]) OR (“post traumatic stress disorder*”) OR (“posttraumatic stress disorder*”) OR (“post-traumatic stress disorder*”)). A comprehensive search for relevant literature will also be conducted, which will entail tracing the references of the included studies to identify potential eligible sources.

Study selection

The screening process for the present study will involve independent assessments of each article retrieved during the search phase by two researchers to determine whether they satisfy the predefined inclusion criteria and to exclude any irrelevant articles. Reasons for study exclusion will be documented. In case of disagreements between the two researchers, a third researcher will be consulted to adjudicate. All articles identified during the initial search will undergo a title and abstract screening, with subsequent full-text review of the remaining articles. The selection process will be recorded and explained in detail using a flow chart as suggested by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020. A summarized PRISMA flow diagram is shown in Figure 1.

Figure 1.

Flow diagram outlines of review process and study selection.

Data extraction and management

Two independent reviewers will be responsible for extracting relevant information from the included studies. They will draw piloting forms independently and check with each other when complete. Any discrepancies between reviewers will be resolved through discussion or, if necessary, a third reviewer will be consulted.

Data extraction will encompass four key domains: (a) study characteristics (authors, year of publication, data type, diagnostic PTSD tool); (b) participant information (sample size, diagnosis) and (c) models (machine learning model, model type, accuracy, other measures). (d) Additional information deemed pertinent to the study, including event details and accompanying commentary, will also be extracted. To ensure the completeness of data, we will contact the authors for any missing or unclear information by e-mail with a response time limit of 10 days. The explicit information of extracted data is shown in Table 2.

Table 2.

Proposed data extraction form and description.

Data heading	Description
First author, year	The last name of first author and year of publication
Data type	Data type processed using ML model
Diagnostic PTSD tools	The tools discriminate PTSD and control group
Sample size	Total number of study subjects
Diagnosis	The respective numbers of PTSD and control group
ML model	Names of all reported ML models
Type of ML model	Supervised or unsupervised ML models
Accuracy	The accuracy of the best-performing ML model
Event	Traumatic events
Commentary	Additional notes to the above data

Abbreviations: ML: machine learning; PTSD: posttraumatic stress disorder.

Types of outcomes

The primary outcomes are the accuracies of the ML models in the included studies applied in the field of PTSD. We also chose other metrics, such as the AUC, sensitivity and specificity, and positive and negative predictive values for model performance as secondary outcomes.

Risk of bias assessment

Quality assessment (Supplemental Table S2) of the included studies will be conducted using the quality assessment tool proposed by Luis Francisco Ramos-Lima proposed in 2020.¹⁸ It is divided into nine domains: representativeness of the sample, confounding variables, outcome assessment, ML approach, performance/accuracy, how the authors handled missing data, whether the test dataset was “unseen,” how the authors handled class imbalance, feature selection and hyperparameter tuning. Two investigators will conduct an independent assessment of the risk of bias for the studies included in this review. The evaluation outcomes will be documented in a concise tabular format with “Y” (yes) or “N” (no) for each item. In the event of discrepancies between the two reviewers, a third researcher will be consulted to resolve any disagreements.

Data synthesis and meta-analysis

The following data will be extracted from each selected study: (a) Qualitative synthesis: detailed characteristics of the study: medical profile and number of participants, type of ML algorithms, type of data, sample sizes, representativeness of the sample, confounding variables, outcome assessment, how the authors handled missing data, whether the test dataset was “unseen,” how the authors handled class imbalance, feature selection and hyperparameter tuning. (b) Quantitative synthesis: ML model metrics: accuracy, sensitivity, specificity, AUC and positive and negative predictive values.

Meta-analysis will be conducted on studies where data availability allows summary estimation with 95% confidence intervals (CI) for accuracy. The accuracy will be declared by point estimation and 95% CI. The significant level of two-tailed t-test will be set as 0.05. Forest plots will be used for the presentation of results. All statistics will be calculated in STATA MP 17 (Stata Corp LLC, 4905 Lakeway Drive, College Station, Texas 77845, USA).

Examining heterogeneity

The Cochrane's Q test will be chosen to qualitatively check for heterogeneity, with p less than 0.05 suggesting a significant heterogeneity across studies. I² will be quantitively checked for the degree of heterogeneity. If I²< 50%, heterogeneity is considered negligible and a fixed-effect model is appropriate for meta-analysis. When I²≥ 50%, heterogeneity is considered to be present and a random-effects model is applicable. If I²> 75%, heterogeneity is considered to be significant among studies. A random-effects DerSimonian–Laird model will be applied to estimate the overall accuracy of all the included studies.

Subgroup analysis and sensitivity analysis

Studies of different data type, ML methods, type of ML models, PTSD diagnostic tools and traumatic events will be considered to classify into one group, when there are >2 studies in the group, to see the combined effects and heterogeneity for subgroup analysis. Sensitivity analysis will be applied by excluding one study each time to explore whether the results will change. If the results are driven by one study, we will exclude the study and perform descriptive analysis.

Publication bias

Funnel plots and Egger's test¹⁹ would be adopted to detect publication bias only when there are at least 10 studies reporting the primary outcomes.²⁰ In case the available data are not in the required format or are highly heterogeneous, narrative analysis will follow.

Current status and plan

The review is undergoing a preliminary search and is expected to be completed in December 2023.

Ethics and dissemination

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241239238 - Supplemental material for Machine learning methods to discriminate posttraumatic stress disorder: A protocol of systematic review and meta-analysis

Supplemental material, sj-docx-1-dhj-10.1177_20552076241239238 for Machine learning methods to discriminate posttraumatic stress disorder: A protocol of systematic review and meta-analysis by Jing Wang, Hui Ouyang, Runda Jiao, Haiyan Zhang, Suhui Cheng, Zhilei Shang, Yanpu Jia, Wenjie Yan, Lili Wu and Weizhi Liu in DIGITAL HEALTH

Footnotes

Acknowledgements

The authors would like to acknowledge the volunteers who participated in the study.

Contributorship

JW, HO, RJ and HZ contributed to the writing of this article and are the co-first authors. SC, ZS, YJ, WY and JW contributed to the article review and revised the article. WL and LW led the whole study, including putting this study forward and carrying out the study; they are the co-corresponding authors.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Innovative Research Team Project (20200106); Science and Technology Supply Project (2020JY17); Military Postgraduate Funding Projects (202346-295).

Guarantor

ORCID iD

Weizhi Liu

Supplemental material

Supplemental material for this article is available online.

References

Saba

Rehman

Shahzad

, et al. Machine learning for post-traumatic stress disorder identification utilizing resting-state functional magnetic resonance imaging. Microsc Res Techniq 2022;85(6): 2083–2094.

APA. What is Posttraumatic Stress Disorder (PTSD)? 2022 [cited 2023 February 27th]. Available from: https://www.psychiatry.org/patients-families/ptsd/what-is-ptsd.

Benjet

Bromet

Karam

, et al. The epidemiology of traumatic event exposure worldwide: results from the world mental health survey consortium. Psychol Med 2016; 46: 327–343.

Sun

Huang

, et al. Military-related posttraumatic stress disorder and mindfulness meditation: A systematic review and meta-analysis. Chin J Traumatol 2021; 24: 221–230.

APA. Diagnostic and statistical manual of mental disorders: DSM-5. 5th ed. Arlington, VA: American Psychiatric Publishing, 2013, 991 p.

Shalev

Liberzon

Marmar

. Post-traumatic stress disorder. N Engl J Med 2017; 376: 2459–2469.

Mota

Bolton

Enns

, et al. Course and predictors of posttraumatic stress disorder in the Canadian armed forces: A nationally representative, 16-year follow-up study: cours et prédicteurs du trouble de stress post-traumatique dans les forces armées canadiennes: une étude de suivi de 16 ans nationalement représentative. Can J Psychiatry 2021; 66: 982–995.

Sawalha

Yousefnezhad

Shah

, et al. Detecting presence of PTSD using sentiment analysis from text data. Front Psychiatry 2022; 12: 1–15.

Deng

Yang

, et al. Using machine learning algorithm to predict the risk of post-traumatic stress disorder among firefighters in Changsha. Zhong Nan Da Xue Xue Bao Yi Xue Ban 2023; 48: 84–91.

10.

Christ

Elhai

Forbes

, et al. A machine learning approach to modeling PTSD and difficulties in emotion regulation. Psychiatry Res 2021; 297: 113712.

11.

Held

Schubert

Pridgen

, et al. Who will respond to intensive PTSD treatment? A machine learning approach to predicting response prior to starting treatment. J Psychiatr Res 2022; 151: 78–85.

12.

Schultebraucks

Sijbrandij

Galatzer-Levy

, et al. Forecasting individual risk for long-term posttraumatic stress disorder in emergency medical settings using biomedical data: A machine learning multicenter cohort study. Neurobiol Stress 2021; 14: 1–10.

13.

Jiang

Dutra

Lee

, et al. Toward reduced burden in evidence-based assessment of PTSD: A machine learning study. Assessment 2021; 28: 1971–1982.

14.

Wani

Aiello

Kim

, et al. The impact of psychopathology, social adversity and stress-relevant DNA methylation on prospective risk for post-traumatic stress: A machine learning approach. J Affect Disord 2021; 282: 894–905.

15.

Gokten

Uyulan

. Prediction of the development of depression and post-traumatic stress disorder in sexually abused children using a random forest classifier. J Affect Disord 2021; 279: 256–265.

16.

Schultebraucks

Qian

Abu-Amara

, et al. Pre-deployment risk factors for PTSD in active-duty personnel deployed to Afghanistan: A machine-learning approach for analyzing multivariate predictors. Mol Psychiatry 2021; 26: 5011–5022.

17.

Moher

Shamseer

Clarke

, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015; 4: 1.

18.

Ramos-Lima

Waikamp

Antonelli-Salgado

, et al. The use of machine learning techniques in trauma-related disorders: A systematic review. J Psychiatr Res 2020; 121: 159–172.

19.

Irwig

Macaskill

Berry

, et al. Bias in meta-analysis detected by a simple, graphical test. Graphical test is itself biased. Br Med J 1998; 316: 470.

20.

Egger

Smith

Schneider

, et al. Bias in meta-analysis detected by a simple, graphical test. Br Med J 1997; 315: 629–634.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB