Neurological disorders, such as multiple sclerosis (MS), can significantly impair bodily functions due to the immune system mistakenly attacking the body. Assessing cognitive workload in MS patients is crucial for understanding their condition, and one effective approach is the use of an n-back working memory task combined with magnetoencephalography (MEG) recordings. MEG provides valuable insights into brain activity through its magnetic signals.
Methods:
This study proposes an automated framework to detect mental workload levels in MS patients using MEG data. The EEGNet model is employed to assess mental workload, with transfer learning techniques used for fine-tuning to enhance model performance. Additionally, traditional machine learning models are evaluated to compare their performance with the deep learning-based approach.
Results:
Experimental results indicate that the proposed model achieves an accuracy of 51.68 ± 11.92% for healthy subjects and 51.77 ± 13.29% for MS patients across various workload levels, significantly outperforming baseline methods.
Conclusions:
Deep learning-based end-to-end models can effectively assess mental workload in MS patients, achieving competitive performance without requiring explicit feature extraction or dimensionality reduction steps typically used in conventional classification pipelines.
MillerGA. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol Rev, 1994; 101(2):343–352.
2.
CowanN. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behav Brain Sci, 2001; 24(1):87–114.
3.
ParasuramanR. Neuroergonomics: Research and practice. Theor Issues Ergon Sci, 2003; 4(1–2):5–20.
4.
PattynN, NeytX, HenderickxD, et al.Psychophysiological investigation of vigilance decrement: Boredom or cognitive fatigue? Physiol Behav, 2008; 93(1–2):369–378.
5.
ParasuramanR, BahriT, DeatonJE, et al.Theory and design of adaptive automation in aviation systems (progress report). Hum Factors, 1992; 34:305–320.
6.
JohnsonAW, OmanCM, SheridanTB, et al.Dynamic task allocation in operational systems: Issues, gaps, and recommendations. In: Proceedings of the 2014 IEEE Aerospace Conference. IEEE; 2014; pp. 1–15.
7.
YuK, PrasadI, MirH, et al.Cognitive workload modulation through degraded visual stimuli: A single-trial EEG study. J Neural Eng, 2015; 12(4):e046020.
8.
Fernandez RojasR, BrownN, WaddingtonG, et al.A systematic review of neurophysiological sensing for the assessment of acute pain. NPJ Digit Med, 2023; 6(1):76.
9.
AyazH, ÇakirMP, İzzetoğluK, et al.Monitoring expertise development during simulated UAV piloting tasks using optical brain imaging. In: 2012 IEEE aerospace conference. IEEE; 2012, pp. 1–11.
10.
SafariM, ShalbafR, BagherzadehS, et al.Classification of mental workload using brain connectivity and machine learning on electroencephalogram data. Sci Rep, 2024; 14(1):9153.
11.
ParkJ-H. Mental workload classification using convolutional neural networks based on fNIRS-derived prefrontal activity. BMC Neurol, 2023; 23(1):442.
12.
GuanK, ChaiX, ZhangZ, et al.Evaluation of mental workload in working memory tasks with different information types based on EEG. In: 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE; 2021, pp. 5682–5685.
13.
MijaniA, CherlooMN, TangH, et al.Spectrum-enhanced TRCA (SE-TRCA): A novel approach for direction detection in SSVEP-based BCI. Comput Biol Med, 2023; 166:107488.
14.
CherlooMN, MijaniAM, ZhanL, et al.A novel multiclass-based framework for P300 detection in BCI matrix speller: Temporal EEG patterns of non-target trials vary based on their position to previous target stimuli. Eng Appl Artif Intell, 2023; 123:106381.
15.
SeedatZA, PierKS, HolmesN, et al.Simultaneous whole-head electrophysiological recordings using EEG and OPM-MEG. Imaging Neurosci (Camb), 2024; 2:1–15.
16.
Marín–GarcíaA, Arzate-MenaJD, Corsi-CabreraM, et al.Stationary correlation pattern in highly non-stationary MEG recordings of healthy subjects and its relation to former EEG studies. PLoS One, 2024; 19:e0307378.
17.
KirchnerWK. Age differences in short-term retention of rapidly changing information. J Exp Psychol, 1958; 55(4):352–358.
18.
ConfavreuxC, CompstonA. The natural history of multiple sclerosis. McAlpine’s multiple sclerosis; 2009: pp. 183.
KaracaBK, AkşahinMF, ÖcalR. Detection of multiple sclerosis from photic stimulation EEG signals. Biomed Signal Process Control, 2021; 67:102571.
21.
PourmohammadiA, MotahharyniaA, ShaygannejadV, et al.Working memory dysfunction differs between secondary progressive and relapsing multiple sclerosis: Effects of clinical phenotype, age, disease duration, and disability. Mult Scler Relat Disord, 2023; 69:104411.
22.
HämäläinenM, HariR, IlmoniemiRJ, et al.Magnetoencephalography—theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev Mod Phys, 1993; 65(2):413–497.
23.
BailletS. Magnetoencephalography for brain electrophysiology and imaging. Nat Neurosci, 2017; 20(3):327–339.
24.
BishopCM. Pattern Recognition and Machine Learning. Springer, 2006.
25.
LotteF, CongedoM, LecuyerA, et al.A review of classification algorithms for EEG-based brain–computer interfaces. J Neural Eng, 2007; 4(2):R1–R13.
26.
BlankertzB, TangermannM, VidaurreC, et al.The Berlin brain–computer interface: Non-medical uses of BCI technology. Front Neurosci, 2010; 4:198.
27.
LawhernVJ, SolonAJ, WaytowichNR, et al.EEGNet: A compact convolutional neural network for EEG-based brain–computer interfaces. J Neural Eng, 2018; 15(5):e056013.
28.
CraikA, HeY, Contreras-VidalJL. Deep learning for electroencephalogram (EEG) classification tasks: A review. J Neural Eng, 2019; 16(3):e031001.
29.
RoyY, BanvilleH, AlbuquerqueI, et al.Deep learning-based electroencephalography analysis: A systematic review. J Neural Eng, 2019; 16(5):e051001.
SyrjäläJ, BastiA, GuidottiR, et al.Decoding working memory task condition using magnetoencephalography source level long-range phase coupling patterns. J Neural Eng, 2021; 18(1):e016027.
32.
BhaleraoSV, PachoriRB. Automated classification of cognitive visual objects using multivariate swarm sparse decomposition from multichannel EEG-MEG signals. IEEE Trans Human-Mach Syst, 2024; 54(4):455–464.
33.
RossiC, VidaurreD, CostersL, et al.A data-driven network decomposition of the temporal, spatial, and spectral dynamics underpinning visual-verbal working memory processes. Commun Biol, 2023; 6(1):1079.
34.
RossiC, VidaurreD, CostersL, et al.Disrupted working memory event-related network dynamics in multiple sclerosis. Commun Biol, 2024; 7(1):1592.
35.
CîmpanuC, UngureanuF, MantaVI, et al.A comparative study on classification of working memory tasks using EEG signals. In: 2017 21st International Conference on Control Systems and Computer Science (CSCS); IEEE; 2017, pp. 245–251.
36.
KuanarS, AthitsosV, PradhanN, et al.Cognitive analysis of working memory load from EEG, by a deep recurrent neural network. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2018, pp.2576–2580.
37.
MijaniA, EinizadeA, ShamsollahiM, et al.Cross-subject and crossparadigmlearning using convolutional neural network for P300 event-related potential detection. J Neurol Neurosci, 2020; 11:329.
38.
XuF, MiaoY, SunY, et al.A transfer learning framework based on motor imagery rehabilitation for stroke. Sci Rep, 2021; 11(1):19783.
39.
PolmanCH, ReingoldSC, BanwellB, et al.Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann Neurol, 2011; 69(2):292–302.
40.
KurtzkeJF. Rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS). Neurology, 1983; 33(11):1444–1452.
41.
CostersL, Van SchependomJ, LatonJ, et al.Spatiotemporal and spectral dynamics of multi‐item working memory as revealed by the n‐back task using MEG. Hum Brain Mapp, 2020; 41(9):2431–2446.
42.
HearstMA, DumaisST, OsunaE, et al.Support vector machines. IEEE Intell Syst Their Appl, 1998; 13(4):18–28.
43.
BreimanL. Random forests. Mach Learn, 2001; 45(1):5–32.
TauluS, SimolaJ, KajolaM. Applications of the signal space separation method. IEEE Trans Signal Process, 2005; 53(9):3359–3372.
46.
OostenveldR, FriesP, MarisE, et al.FieldTrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell Neurosci, 2011; 2011:156869.
JenkinsonM. BET2: MR‐based estimation of brain, skull and scalp surfaces. In: Eleventh Annual Meeting of the Organization for Human Brain Mapping, 2005. ScienceDirect; 2005; pp. 167.
49.
JcM. A probabilistic atlas of the human brain: Theory and rationale for its development: The international consortium for brain mapping (ICBM). Neuroimage, 1995; 2:89–101.
50.
WoolrichM, HuntL, GrovesA, et al.MEG beamforming using Bayesian PCA for adaptive data covariance matrix regularization. Neuroimage, 2011; 57(4):1466–1479.
51.
QuinnAJ, VidaurreD, AbeysuriyaR, et al.Task-evoked dynamic network analysis through hidden Markov modeling. Front Neurosci, 2018; 12:603; doi: 10.3389/fnins.2018.00603
52.
VidaurreD, HuntLT, QuinnAJ, et al.Spontaneous cortical activity transiently organises into frequency specific phase-coupling networks. Nat Commun, 2018; 9(1):2987.
53.
IoffeS. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv Preprint, 2015; doi: arXiv:1502.03167
54.
ClevertD-A, UnterthinerT, HochreiterS. Fast and accurate deep network learning by exponential linear units (ELUs). arXiv Preprint, 2015; doi: arXiv:1511.07289
55.
SrivastavaN, HintonG, KrizhevskyA, et al.Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res, 2014; 15:1929–1958.
56.
AbadiM, BarhamP, ChenJ, et al.TensorFlow: A system for large-scale machine learning. In: Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI). USENIX; 2016; pp. 265–283.
57.
CostersL, Van SchependomJ, LatonJ, et al.The role of hippocampal theta oscillations in working memory impairment in multiple sclerosis. Hum Brain Mapp, 2021; 42(5):1376–1390.
58.
Van der MeerM, TewarieP, SchoonheimM, et al.Cognition in MS correlates with resting-state oscillatory brain activity: An explorative MEG source-space study. Neuroimage Clin, 2013; 2:727–734.
59.
HarmonyT. The functional significance of delta oscillations in cognitive processing. Front Integr Neurosci, 2013; 7:83.
60.
Vazquez-MarrufoM, Sarrias-ArrabalE, Martin-ClementeR, et al.Altered phase and nonphase EEG activity expose impaired maintenance of a spatial-object attentional focus in multiple sclerosis patients. Sci Rep, 2020; 10(1):20721.
RoccaMA, AmatoMP, De StefanoN, et al.; MAGNIMS Study Group. Clinical and imaging assessment of cognitive dysfunction in multiple sclerosis. Lancet Neurol, 2015; 14(3):302–317.
63.
NiuS, LiuM, LiuY, et al.Distant domain transfer learning for medical imaging. IEEE J Biomed Health Inform, 2021; 25(10):3784–3793.
