To date, brain–computer interfaces (BCIs) have not achieved reliable real-time communication through auditory or tactile modalities. Such interfaces would be crucial for brain-injured patients with severe motor impairments who are also blind or deaf. This study validates the functionality of the NeuroCommTrainer, a mobile and easy-to-use multimodal BCI with flex-printed electrode strips that does not require vision and adapts to users’ attentiveness levels to initiate stimulation.
Methods:
In a study of 20 healthy participants, we evaluated auditory and vibrotactile oddball paradigms to train the system to differentiate rare and frequent event-related potentials (ERPs). In real-time online sessions, the system detected participants’ mental focus to adaptively initiate stimulation through attentiveness monitoring.
Results:
The NeuroCommTrainer successfully captured auditory and tactile ERPs, achieving a classification accuracy of 75% for stimuli in the calibration session, which is not yet reflected in the online session with 34% of found targets (chance level = 16.7%).
Discussion:
The presented early-stage prototype of the NeuroCommTrainer requires several improvements before clinical application in brain-damaged patients, which include refined algorithms to reduce classification variance across participants, and enhanced attentiveness detection specifically tuned to brain activity of the targeted patient group. The present study makes a critical step in this direction and shows that a transition into a practicable communication system for brain-damaged patients may be achievable in the future.
Impact Statement
Efforts to facilitate communication through brain–computer interfaces for patients with brain injuries have thus far achieved limited success. This is primarily due to the systems’ lack of adaptability and responsiveness to the users’ unique mental states and fluctuating arousal levels throughout the day. Our multimodal NeuroCommTrainer system combines long-term electroencephalography-based attentiveness monitoring with state-dependent stimulus delivery and neural response classification. We show that we can detect chosen targets and thus user responses significantly better than chance. Our system opens up an avenue for brain-damaged patients to eventually communicate yes/no answers via the BCI.
BandtC, PompeB. Permutation entropy: A natural complexity measure for time series. Phys Rev Lett, 2002; 88(17):174102; doi: 10.1103/PhysRevLett.88.174102
2.
BlankertzB, SannelliC, HalderS, et al.Neurophysiological predictor of SMR-based BCI performance. Neuroimage, 2010; 51(4):1303–1309; doi: 10.1016/j.neuroimage.2010.03.022
3.
BlumS, EmkesR, MinowF, et al.Flex-printed forehead EEG sensors (fEEGrid) for long-term EEG acquisition. J Neural Eng, 2020; 17(3):034003; doi: 10.1088/1741-2552/ab914c
4.
BlumS, JacobsenNS, BleichnerMG, et al.A Riemannian modification of artifact subspace reconstruction for EEG artifact handling. Front Hum Neurosci, 2019; 13:141; doi: 10.3389/fnhum.2019.00141
5.
BrouwerAM, Van ErpJB. A tactile P300 brain-computer interface. Front Neurosci, 2010; 4:19; doi: 10.3389/fnins.2010.00019
6.
BulárkaS, GonteanA. Brain-computer interface review. In 2016 12th IEEE international symposium on electronics and telecommunications (ISETC), IEEE. 2016; (pp. 219–222); doi: 10.1109/ISETC.2016.7781096
7.
ChangCY, HsuSH, Pion-TonachiniL, et al.Evaluation of artifact subspace reconstruction for automatic EEG artifact removal. In 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC) 2018; (pp. 1242–1245); doi: 10.1109/EMBC.2018.8512547
8.
ChemaliJ, ChingS, PurdonPL, et al.Burst suppression probability algorithms: State-space methods for tracking EEG burst suppression. J Neural Eng, 2013; 10(5):056017; doi: 10.1088/1741-2560/10/5/056017
9.
CipressoP, CarelliL, SolcaF, et al.The use of P300‐based BCIs in amyotrophic lateral sclerosis: From augmentative and alternative communication to cognitive assessment. Brain Behav, 2012; 2(4):479–498; doi: 10.1002/brb3.57
10.
DalyJJ, HugginsJE. Brain-computer interface: Current and emerging rehabilitation applications. Arch Phys Med Rehabil, 2015; 96(3 Suppl):S1–S7; doi: 10.1016/j.apmr.2015.01.007
11.
DebenerS, EmkesR, De VosM, et al.Unobtrusive ambulatory EEG using a smartphone and flexible printed electrodes around the ear. Sci Rep, 2015; 5(1):16743; doi: 10.1038/srep16743
12.
DelormeA. EEG is better left alone. Sci Rep, 2023; 13(1):2372; doi: 10.1038/s41598-023-27528-0
13.
Duncan‐JohnsonCC. Young psychophysiologist award address, 1980. Psychophysiology, 1981; 18(3):207–215; doi: 10.1111/j.1469-8986.1981.tb03020.x
14.
FarwellLA, DonchinE. Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol, 1988; 70(6):510–523; doi: 10.1016/0013-4694(88)90149-6
15.
FawcettT. An introduction to ROC analysis. Pattern Recognit Lett, 2006; 27(8):861–874; doi: 10.1016/j.patrec.2005.10.010
16.
FerrezPW, MillánJD. Error-related EEG potentials generated during simulated brain–computer interaction. IEEE Trans Biomed Eng, 2008; 55(3):923–929; doi: 10.1109/TBME.2007.908083
17.
FinkeA, LenhardtA, RitterH. The MindGame: A P300-based brain–computer interface game. Neural Netw, 2009; 22(9):1329–1333; doi: 10.1016/j.neunet.2009.07.003
18.
FinkeA, SteppacherI, KisslerJ, et al.Frequency band variations predict EEG single-trial classification performance in disorder of consciousness patients. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) 2018; (pp. 1927–1930); doi: 10.1109/EMBC.2018.8512719
19.
FurdeaA, HalderS, KrusienskiDJ, et al.An auditory oddball (P300) spelling system for brain‐computer interfaces. Psychophysiology, 2009; 46(3):617–625; doi: 10.1111/j.1469-8986.2008.00783.x
20.
GugerC, DabanS, SellersE, et al.How many people are able to control a P300-based brain–computer interface (BCI)? Neurosci Lett, 2009; 462(1):94–98; doi: 10.1016/j.neulet.2009.06.045
21.
HeH, WuD. Transfer learning for brain–computer interfaces: A Euclidean space data alignment approach. IEEE Trans Biomed Eng, 2020; 67(2):399–410; doi: 10.1109/TBME.2019.2913914
22.
HedgesLV. Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics, 1981; 6(2):107–128; doi: 10.3102/10769986006002107
23.
HillNJ, SchölkopfB. An online brain–computer interface based on shifting attention to concurrent streams of auditory stimuli. J Neural Eng, 2012; 9(2):026011; doi: 10.1088/1741-2560/9/2/026011
24.
HillyardSA, HinkRF, SchwentVL, et al.Electrical signs of selective attention in the human brain. Science, 1973; 182(4108):177–180; doi: 10.1126/science.182.4108.177
25.
HorváthJ, CziglerI, BirkásE, et al.Age-related differences in distraction and reorientation in an auditory task. Neurobiol Aging, 2009; 30(7):1157–1172; doi: 10.1016/j.neurobiolaging.2007.10.003
26.
HsuSH, MullenT, JungTP, et al.Online recursive independent component analysis for real-time source separation of high-density EEG. In 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE. 2014;(pp. 3845–3848); doi: 10.1109/EMBC.2014.6944462
27.
HuangJ, QiuL, LinQ, et al.Hybrid asynchronous brain–computer interface for yes/no communication in patients with disorders of consciousness. J Neural Eng, 2021; 18(5):056001; doi: 10.1088/1741-2552/abf00c
28.
JeunetC, LotteF, BatailJM, et al.Using recent BCI literature to deepen our understanding of clinical neurofeedback: A short review. Neuroscience, 2018; 378:225–233; doi: 10.1016/j.neuroscience.2018.03.013
29.
JoliffeIT, MorganBJ. Principal component analysis and exploratory factor analysis. Stat Methods Med Res, 1992; 1(1):69–95; doi: 10.1177/096228029200100105
30.
KerbyDS. The simple difference formula: An approach to teaching nonparametric correlation. Comprehensive Psychology, 2014; 3:11.IT.3.1; doi: 10.2466/11.IT.3.1
KisslerJ, MüllerMM, FehrT, et al.MEG gamma band activity in schizophrenia patients and healthy subjects in a mental arithmetic task and at rest. Clin Neurophysiol, 2000; 111(11):2079–2087; doi: 10.1016/S1388-2457(00)00425-9
33.
KotchoubeyB, PavlovYG. A systematic review and meta-analysis of the relationship between brain data and the outcome in disorders of consciousness. Front Neurol, 2018; 9:315; doi: 10.3389/fneur.2018.00315
34.
KüblerA, BirbaumerN. Brain–computer interfaces and communication in paralysis: Extinction of goal directed thinking in completely paralysed patients? Clin Neurophysiol, 2008; 119(11):2658–2666; doi: 10.1016/j.clinph.2008.06.019
35.
KüblerA, FurdeaA, HalderS, et al.A brain–computer interface controlled auditory event‐related potential (P300) spelling system for locked‐in patients. Annals of the New York Academy of Sciences, 2009; 1157(1):90–100; doi: 10.1111/j.1749-6632.2008.04122.x
36.
KüblerA, KotchoubeyB, KaiserJ, et al.Brain–computer communication: Unlocking the locked in. Psychol Bull, 2001; 127(3):358–375; doi: 10.1037/0033-2909.127.3.358
37.
LaureysS, OwenAM, SchiffND. Brain function in coma, vegetative state, and related disorders. Lancet Neurol, 2004; 3(9):537–546; doi: 10.1016/S1474-4422(04)00852-X
38.
LeithnerC, EndischC. Evoked potentials in patients with disorders of consciousness. Handb Clin Neurol, 2025; 207:147–164; doi: 10.1016/B978-0-443-13408-1.00002-6
39.
LiY, GuanC, LiH, et al.A self-training semi-supervised SVM algorithm and its application in an EEG-based brain computer interface speller system. Pattern Recognit Lett, 2008; 29(9):1285–1294; doi: 10.1016/j.patrec.2008.01.030
40.
LindenbaumL, ZeheS, AnlauffJ, et al.Different patterns of attention modulation in early N140 and late P300 sERPs following ipsilateral vs. contralateral stimulation at the fingers and cheeks. Front Hum Neurosci, 2021; 15:781778; doi: 10.3389/fnhum.2021.781778
41.
LugoZR, RodriguezJ, LechnerA, et al.A vibrotactile p300-based brain–computer interface for consciousness detection and communication. Clin EEG Neurosci, 2014; 45(1):14–21; doi: /10.1177/1550059413505533
42.
LuléD, NoirhommeQ, KleihSC, et al.Probing command following in patients with disorders of consciousness using a brain–computer interface. Clin Neurophysiol, 2013; 124(1):101–106; doi: 10.1016/j.clinph.2012.04.030
43.
MarinouA, SaundersR, CassonAJ. Flexible inkjet printed sensors for behind-the-ear SSVEP EEG monitoring. In 2020 IEEE International Conference on Flexible and Printable Sensors and Systems (FLEPS). IEEE. 2020; (pp. 1–4); doi: 10.1109/FLEPS49123.2020.9239488
44.
MattoutJ, LuautéJ, JungJ, et al.Brain–computer interfaces in disorders of consciousness. Brain–Computer Interfaces 2: Technology and Applications, 2016:1–28; doi: 10.1002/9781119332428.ch1
45.
NäätänenR, GaillardAW, MäntysaloS. Early selective-attention effect on evoked potential reinterpreted. Acta Psychol (Amst), 1978; 42(4):313–329; doi: 10.1016/0001-6918(78)90006-9
46.
NakagawaS. A farewell to Bonferroni: The problems of low statistical power and publication bias. Behavioral Ecology, 2004; 15(6):1044–1045; doi: 10.1093/beheco/arh107
47.
NeuperC, PfurtschellerG. Neurofeedback training for BCI control. Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction, 2010:65–78; doi: 10.1007/978-3-642-02091-9_4
NijboerF, SellersEW, MellingerJ, et al.A P300-based brain–computer interface for people with amyotrophic lateral sclerosis. Clin Neurophysiol, 2008; 119(8):1909–1916; doi: 10.1016/j.clinph.2008.03.034
51.
OldfieldRC. The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 1971; 9(1):97–113; doi: 10.1016/0028-3932(71)90067-4
52.
PacharraM, DebenerS, WascherE. Concealed around-the-ear EEG captures cognitive processing in a visual simon task. Front Hum Neurosci, 2017; 11:290; doi: 10.3389/fnhum.2017.00290
PicotA, CharbonnierS, CaplierA. On-line detection of drowsiness using brain and visual information. IEEE Trans Syst, Man, Cybern A, 2012; 42(3):764–775; doi: 10.1109/TSMCA.2011.2164242
55.
PokornyC, KlobassaDS, PichlerG, et al.The auditory P300-based single-switch brain–computer interface: Paradigm transition from healthy subjects to minimally conscious patients. Artif Intell Med, 2013; 59(2):81–90; doi: 10.1016/j.artmed.2013.07.003
56.
RacitiL, CoralloF, CalabròRS, et al.Nursing, caregiving and psychological support in chronic disorders of consciousness: A scoping review. Acta Biomed, 2021; 92(S2):e2021013; doi: 10.23750/abm.v92iS2.11329
57.
RavdenD, PolichJ. On P300 measurement stability: Habituation, intra-trial block variation, and ultradian rhythms. Biol Psychol, 1999; 51(1):59–76; doi: 10.1016/S0301-0511(99)00015-0
58.
SchnakersC. Assessing consciousness and cognition in disorders of consciousness. NeuroRehabilitation, 2024; 54(1):11–21; doi: 10.3233/NRE-230140
SchrögerE, WolffC. Attentional orienting and reorienting is indicated by human event-related brain potentials. Neuroreport, 1998; 9(15):3355–3358; doi: 10.1097/00001756-199810260-00003
61.
SebastianoDR, PanzicaF, VisaniE, et al.Significance of multiple neurophysiological measures in patients with chronic disorders of consciousness. Clinical Neurophysiology, 2015; 126(3):558–564; doi: 10.1016/j.clinph.2014.07.004
62.
SellersEW, DonchinE. A P300-based brain–computer interface: Initial tests by ALS patients. Clin Neurophysiol, 2006; 117(3):538–548; doi: 10.1016/j.clinph.2005.06.027
63.
ShenoyP, KrauledatM, BlankertzB, et al.Towards adaptive classification for BCI. J Neural Eng, 2006; 3(1):R13–R23; doi: 10.1088/1741-2560/3/1/R02
64.
SiohanO. On the robustness of linear discriminant analysis as a preprocessing step for noisy speech recognition. In 1995 International Conference on Acoustics, Speech, and Signal Processing. IEEE. 1995; 1, pp. 125–128; doi: 10.1109/ICASSP.1995.479289
65.
SittJD, KingJR, El KarouiI, et al.Large scale screening of neural signatures of consciousness in patients in a vegetative or minimally conscious state. Brain, 2014; 137(Pt 8):2258–2270; doi: 10.1093/brain/awu141
66.
SmithDB, DonchinE, CohenLet al.Auditory averaged evoked potentials in man during selective binaural listening. Electroencephalogr Clin Neurophysiol, 1970; 28(2):146–152; doi: 10.1016/0013-4694(70)90182-3
SpataroR, HeilingerA, ChoW, et al.PROBING CONSCIOUSNESS IN UNRESPONSIVE WAKEFULNESS SYNDROME PATIENTS WITH A VIBROTACTILE P300-BASED BRAIN-COMPUTER INTERFACE (P5. 032). Neurology, 2017; 88(16_supplement); P5-032; doi: 10.1212/WNL.88.16_supplement.P5.032
69.
SpataroR, XuY, XuR, et al.How brain-computer interface technology may improve the diagnosis of the disorders of consciousness: A comparative study. Front Neurosci, 2022; 16:959339; doi: 10.3389/fnins.2022.959339
70.
SquiresNK, SquiresKC, HillyardSA. Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. Electroencephalogr Clin Neurophysiol, 1975; 38(4):387–401; doi: 10.1016/0013-4694(75)90263-1
71.
SteppacherI, KisslerJ. A problem shared is a problem halved? Comparing burdens arising for family caregivers of patients with disorders of consciousness in institutionalized versus at home care. BMC Psychol, 2018; 6(1):58; doi: 10.1186/s40359-018-0272-x
72.
SuttonS, BrarenM, ZubinJ, et al.Evoked-potential correlates of stimulus uncertainty. Science, 1965; 150(3700):1187–1188; doi: 10.1126/science.150.3700.1187
73.
VidaurreC, SannelliC, MüllerKR, et al.Machine-learning-based coadaptive calibration for brain-computer interfaces. Neural Comput, 2011; 23(3):791–816; doi: 10.1162/NECO_a_00089
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
