Abstract
Gleichgerrcht E, Munsell BC, Alhusaini S, et al. Neuroimage Clin. 2021;31:102765. doi: 10.1016/j.nicl.2021.102765. Online ahead of print. Artificial intelligence has recently gained popularity across different medical fields to aid in the detection of diseases based on pathology samples or medical imaging findings. Brain magnetic resonance imaging (MRI) is a key assessment tool for patients with temporal lobe epilepsy (TLE). The role of machine learning and artificial intelligence to increase detection of brain abnormalities in TLE remains inconclusive. We used support vector machine (SV) and deep learning (DL) models based on region of interest (ROI-based) structural (n = 336) and diffusion (n = 863) brain MRI data from patients with TLE with (“lesional”) and without (“non-lesional”) radiographic features suggestive of underlying hippocampal sclerosis from the multinational (multi-center) ENIGMA-Epilepsy consortium. Our data showed that models to identify TLE performed better or similar (68-75%) compared to models to lateralize the side of TLE (56-73%, except structural-based) based on diffusion data with the opposite pattern seen for structural data (67-75% to diagnose vs 83% to lateralize). In other aspects, structural and diffusion-based models showed similar classification accuracies. Our classification models for patients with hippocampal sclerosis were more accurate (68-76%) than models that stratified non-lesional patients (53-62%). Overall, SV and DL models performed similarly with several instances in which SV mildly outperformed DL. We discuss the relative performance of these models with ROI-level data and the implications for future applications of machine learning and artificial intelligence in epilepsy care.
Commentary
The ENIGMA (Enhancing Neuroimaging Genetics through Meta Analysis) Consortium is an international neuroimaging collaboration of over 1400 scientists across 43 countries studying brain function and structure in health and disease. 1 Over 50 working groups in ENIGMA leverage shared datasets to address large-scale questions in neurological disorders that may benefit from a “big data” approach. The ENIGMA-Epilepsy working group was created to share research ideas and increase sample size in epilepsy neuroimaging studies, and data collection has included structural magnetic resonance imaging (MRI), diffusion weighted imaging, resting-state functional MRI (fMRI), and clinical variables across multiple centers. 2 Given marked heterogeneity across epilepsy patients, single-center neuroimaging studies risk being under-powered to address clinically important questions, such as those related to epilepsy subtype classification and prediction of response to therapy. Recent ENIGMA-Epilepsy studies have begun to characterize white matter abnormalities across various epilepsy syndromes, identifying reduced fractional anisotropy and increased mean diffusivity in ipsilateral limbic pathways in temporal lobe epilepsy (TLE) patients that are most pronounced in individuals with mesial temporal sclerosis (MTS). 3 Can subtle neuroimaging abnormalities such as these observed in TLE patients aid with diagnosis or lateralization?
In the presently highlighted study, Gleichgerrecht and colleagues leveraged ENIGMA-Epilepsy structural and diffusion-weighted MRI data using a machine learning approach to evaluate epilepsy patient and control subgroups. 4 For structural T1-weighted MRI analysis, data were collected from 16 centers and included 336 unilateral adult TLE patients and 631 matched control subjects, while diffusion data originated from 21 sites and included 863 individuals with TLE and 976 controls. Approximately 56% of TLE patients were diagnosed with left sided seizure onset, and structural T1 data were only available for patients with MTS, although diffusion data were obtained for patients with MTS and non-lesional TLE. The authors utilized support vector machine and deep learning models to identify region-based MRI differences to help stratify patients vs controls and left vs right-sided patients. This pipeline was evaluated with a 10-fold cross-validation approach that included a grid search technique, with 80% of data used for training and validation of each model, and 20% saved to test the model performance. Using structural data (cortical thickness and surface area), models demonstrated 73-75% accuracy in distinguishing all patients vs controls, and 77-83% accuracy in identifying left vs right TLE patients with MTS. Key regions of influence in the structural models included ipsilateral hippocampus, contralateral amygdala, ipsilateral thalamus, and various frontotemporal cortical areas. Diffusion-based models in TLE patients predicted laterality in individuals with MTS using both fractional anisotropy (74-76% accuracy) and radial diffusivity (55-67%) measurements, while model accuracy for lateralization in non-lesional patients was low (51-56%). Bilateral cingulum and external capsule bundles contributed strongly to diffusion models. Deep learning and support vector machine models performed relatively similarly in most analyses. Overall, the authors argue that their models may allow classification and lateralization of TLE patients using region-level MRI data with moderate accuracy.
A notable strength of this study by Gleichgerrecht and colleagues is its use of a large, multicenter dataset to achieve a sample size rarely seen in neuroimaging studies of epilepsy. Machine learning approaches are often limited by undersampling of data and overfitting models, and they are sometimes applied inappropriately. Sizable, shared datasets such as ENIGMA-Epilepsy are well-suited for investigations using artificial intelligence to detect subtle abnormalities in MRI data. The authors also appropriately used multiple cross validations to generate models using a subset of data, which were then applied to remaining data. Clinically, improved strategies for lateralizing TLE remains an important goal, as rapid contralateral spread of mesial temporal lobe seizure activity is common in this disorder, 5 making confident electrographic lateralization challenging in some cases. However, models in the highlighted study will need to be improved in patients with non-lesional TLE, where lateralization is more difficult than in individuals with MTS visible on MRI. Along the same lines, expanding these analyses to distinguish patients who harbor bilateral vs unilateral temporal lobe seizure onset will be another important goal in the future, as contralateral seizures in TLE may not be captured during inpatient video-EEG studies that only span days to weeks. In one study of ambulatory bilateral TLE patients implanted with a neurostimulator, nearly one-third of individuals required more than four weeks to manifest bilateral seizures. 6 Also, abnormal connectivity patterns of the contralateral hippocampus may predict seizure recurrence after TLE surgery, 7 further suggesting that bilateral hippocampal pathology is sometimes missed in this disorder.
A notable limitation of the highlighted study is that analyses were performed on region-level structural and diffusion MRI measurements, and raw data were unavailable. It is likely that models based on these high-level features will perform less favorably than those using sophisticated feature extraction from raw imaging data, and the investigators cite this as an important future goal. Also, surgical outcome prediction was not performed in this investigation, and such an analysis may benefit from the large dataset available in ENIGMA-Epilepsy. However, the lead authors did recently report studies in which machine learning approaches were used to predict surgical outcome with favorable accuracy using the MRI structural connectome derived from raw diffusion and T1 data.8,9 In those investigations and studies by other groups, models based on structural MRI often outperform prediction accuracy based on clinical variables alone.
While the growth of machine learning approaches in surgical epilepsy is welcome, we must also remember not to neglect traditional regression methods, which remain a powerful gold standard in relating neuroimaging features to clinically meaningful variables. For example, a recent T1 volumetric analysis utilizing a software package approved by the US Food and Drug Administration identified the presence or absence of MTS in TLE patients, with similar or improved accuracy compared to machine learning literature. 10 Finally, in machine learning studies, we should not allow the “machine” to turn into a pure black box. It is worthwhile to simultaneously consider paradigms that create directly interpretable predictive features without the need for reverse interrogation of machine learning latent space or occlusion mapping. Directly interpretable predictive features can serve as hypothesis generators for future studies focused on understanding the fundamental biophysical reality of epilepsy. Thus, the future of machine learning applications in neuroimaging should balance model accuracy and hypothesis generation – this balance is needed to satisfy both the pragmatic translation to the clinic to improve diagnosis and prognosis, and the need for deeper fundamental understanding of the disease to develop novel treatments. Overall, our field is extremely fortunate to have a growing number of neuroimaging datasets in which both “big data” and traditional regression methods can be utilized by ENIGMA-Epilepsy and other collaborative groups.
