The effectiveness of neurophysiologically triggered adaptive systems hinges on reliable and effective signal processing and cognitive state classification. Although this presents a difficult technical challenge in any context, these concerns are particularly pronounced in a system designed for mobile contexts. This paper describes a neurophysiologically derived cognitive state classification approach designed for ambulatory task contexts. We highlight signal processing and classification components that render the electroencephalogram (EEG) -based cognitive state estimation system robust to noise. Field assessments show classification performance that exceeds 70% for all participants in a context that many have regarded as intractable for cognitive state classification using EEG.
Get full access to this article
View all access options for this article.
References
1.
BacksR. W.SeljosK. A. (1994). Metabolic and cardiorespiratory measures of mental effort: The effects of level of difficulty in a working-memory task. International Journal of Psychophysiology, 16, 57–68.
2.
BeattyJ. (1982). Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin, 91, 276–292.
3.
BerkaC.LevendowskiC.CvetinovicM. M.PetrovicM. M.DavisG.LumicaoM. N.ZivkovicV. T.PopovicM. V.OlmsteadR. (2004). Real-time analysis of EEG indices of alertness, cognition, and memory acquired with a wireless EEG headset. International Journal of Human-Computer Interaction, 17(2), 151–170.
4.
BoitenF. A. (1998). The effects of emotional behaviour on components of the respiratory cycle. Biological Psychology, 49(1–2), 29–51.
5.
DempsterA. P.LairdN. M.RubinD. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, 39, 1–38.
6.
DixonS. R.WickensC. D. (2004). Automation reliability in unmanned aerial vehicle flight control. In VincenziD. (Ed.), Proceedings of Human Performance and Situation Awareness and Automation (HPSAA) II. Daytona Beach, FL: HPSAA.
7.
DorneichM. C.WhitlowS. D.MathanS.CarciofiniJ.VerversP. M. (2005). The communications scheduler: A task scheduling mitigation for a closed loop adaptive system. Proceedings of the 11th International Conference on Human-Computer Interaction (HCI International2005). Mahwah, NJ: Erlbaum.
8.
DorneichM. C.WhitlowS. D.MathanS.VerversP. M.PavelM.ErdogmusD. (2005). DARPA improving warfighter information intake under stress: Augmented cognition phase III final report. Technical report for DARPA Augmented Cognition Phase 3 (contract DAAD 16–03–C–0054).
9.
DorneichM. C.WhitlowS. D.VerversP. M.CarciofiniJ.CreaserJ. (2004). Closing the loop of an adaptive system with cognitive state. Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting (pp. 590–594). Santa Monica, CA: SAGE Publications.
10.
DorneichM. C.WhitlowS. D.VerversP. M.MathanS.RajA.MuthE.HooverA.DuRousseauD.ParraL.SajdaP. (2004). DARPA improving warfighter information intake under stress. Augmented cognition: Concept validation experiment (CVE) analysis report for the Honeywell team. Technical report for DARPA Augmented Cognition Phase 2B (contract DAAD 16–03-C-0054).
11.
ErdogmusD.AdamiA.PavelM.LanT.MathanS.WhitlowS.DorneichM. (2005). Cognitive state estimation based on EEG for augmented cognition, presented at the 2nd IEEE EMBS International Conference on Neural Engineering, Arlington, VA, March 16–19.
12.
GaravanH.RossT. J.LiS.-J.SteinE. A. (2000). A parametric manipulation of central executive functioning using fMRI. Cerebral Cortex, 10, 585–592.
13.
GevinsA.SmithM. E.McEvoyL.YuD. (1997). High resolution EEG mapping of cortical activation related to working memory: Effects of task difficulty, type of processing, and practice. Cerebral Cortex, 7, 374–385.
14.
GevinsA.SmithM. E. (2000). Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style. Cerebral Cortex, 10, 829–839.
15.
GevinsA.SmithM. (2003). Neurophysiological measure of cognitive workload during human-computer interaction. Theoretical Issues in Ergonomics Science, 4(1–2), 113–132.
HartS. G.StavelandL. E. (1988). Development of a multi-dimensional workload rating scale: Results of empirical and theoretical research. In HancockP.MeshkatiN. (Eds.), Human mental workload. Amsterdam: Elsevier.
18.
HeslegraveR. J.FuredyJ. J. (1979). Sensitivities of HR and T-wave amplitude for detecting cognitive and anticipatory stress. Physiology & Behavior, 22(1), 17–23.
19.
Institute of Medicine of the National Academies. (2004). Strategies for monitoring cognitive performance. In Monitoring metabolic status: Predicting decrements in physiological and cognitive performance (pp. 171–172). Washington, DC: National Academy Press.
20.
IzzetogluK.BunceS.OnaralB.PourrezaeiK.ChangemB. (2004). Functional optical brain imaging using near-infrared during cognitive tasks. International Journal of Human-Computer Interaction, 17, 211–227.
21.
KalsbeekJ. W. H.EttemaJ. H. (1963). Scored irregularity of the heart pattern and measurement of perceptual or mental load. Ergonomics, 6, 306–307.
22.
KittlerM.HatefR.DuinR. P. W.MatasJ. (1998). On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(3), 226–239.
23.
KramerA. (1991). Physiological metrics of mental workload: A review of recent progress. In DamosD. (Ed.), Multiple task performance (pp. 279–328). London: Taylor & Francis.
24.
LanT.ErdogmusD.AdamiA.PavelM.MathanS. (2005). Salient EEG channel selection in brain computer interfaces by mutual information maximization. 27th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (pp. 7064–7047). Los Alamitos, CA: IEEE.
25.
MakeigS.JungT.-P. (1995). Changes in alertness are a principal component of variance in the EEG spectrum. NeuroReport, 7(1), 213–216.
26.
MathanS.MazaevaN.WhitlowS.AdamiA.ErdogmusD.LanT.PavelM. (2005). Sensor-based cognitive state assessment in a mobile environment. Proceedings of the 11th International Conference on Human-Computer Interaction. Mahwah, NJ: Erlbaum.
27.
MikulkaP.HadleyG.FreemanF.ScerboM. (1999). The effects of a biocybernetic system on vigilance decrement. Proceedings of the Human Factors and Ergonomics Society 43rd Annual Meeting (p. 1410, abstract). Santa Monica, CA: SAGE Publications.
28.
NeumannD. L. (2002). Effect of varying levels of mental workload on startle eyeblink modulation. Ergonomics, 45, 583–602.
29.
ParasuramanR.MoulouaM.HilburnB. (1999). Adaptive aiding and adaptive task allocation enhance human-machine interaction. In ScerboM. W.MoulouaM. (Eds.), Automation technology and human performance: Current research and trends (pp. 129–133). Mahwah, NJ: Erlbaum.
30.
PartalaT.SurakkaV. (2003). Pupil size variation as an indication of affective processing. International Journal of Human-Computer Studies, 59(1–2), 185–198.
31.
ParzenE. (1967). On estimation of a probability density function and mode. In Time series analysis papers. San Diego, CA: Holden-Day, Inc.
32.
PopeA. T.BogartE. H.BartolomeD. (1995). Biocybernetic system evaluates indices of operator engagement. Biological Psychology, 40, 187–196.
33.
PopivanovD.MinevaA. (1999). Testing procedures for non-stationarity and non-linearity in physiological signals. Mathematical Biosciences.157(1–2), 303–320.
34.
PorgesS. W.ByrneE. A. (1992). Research methods for measurement of heart-rate and respiration. Biological Psychology, 34(2–3), 93–130.
35.
PrinzelL. J.FreemanF. G.ScerboM. W.MikulkaP. J.PopeA. T. (2000). A closed-loop system for examining psychophysiological measures for adaptive automation. International Journal of Aviation Psychology, 10, 393–410.
36.
RoviraE.ZinniM.ParasuramanR. (2002). Effects of information and decision automation on multi-task performance. Proceedings of the Human Factors Society 26th Annual Meeting (pp. 327–331). Santa Monica, CA: SAGE Publications.
37.
RussellC. A.GustafsonS. G. (2001). Selecting salient features of psychophysiological measures (Air Force Research Laboratory Technical Report AFRL-HE-WP-TR-2001–0136).
38.
ScerboM. W.FreemanF. G.MikulkaP. J.ParasuramanR.DiNoceroF.PrinzelL. J. (2001). The efficacy of psychophysiological measures for implementing adaptive technology (NASA Technical Report TP-2001–211018, June 2001). Hampton, VA: National Aeronautics and Space Administration, Langley Research Center.
39.
SchmorrowD. D.KruseA. A. (2002). Improving human performance through advanced cognitive system technology. Proceedings of the Interservice/Industry Training, Simulation and Education Conference. Washington, DC: National Defense Industrial Association.
40.
SchmorrowD.RaleyC.VerversP. (2004). Toward effective warfighting in stressful environments. Poster presented at Human Performance and Situation Awareness and Automation (HPSAA) II. Daytona Beach, FL.
41.
ScottW. B. (1999). Automatic GCAS: You can't fly any lower. Aviation Week and Space Technology, 150(5), 76–79.
42.
SternJ. A.BoyerD.SchroederD. (1994). Blink rate: A possible measure of fatigue. Human Factors, 36, 285–297.
43.
ThorpeS.FizeD.MarlotC. (1996). Speed of processing in the human visual system. Nature, 381, 520–522.
44.
VeltmanJ. A.GaillardA. W. K. (1998). Physiological workload reactions to increasing levels of task difficulty. Ergonomics, 41, 656–669.
45.
VerweyW. B.VeltmanH. A. (1996). Detecting short periods of elevated workload: A comparison of nine workload assessment techniques. Journal of Experimental Psychology-Applied, 2, 270–285.
46.
WelchP. D. (1967). The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short modified periodograms. IEEE Transactions on Audio and Electroacoustics, 15(2), 70–73.
47.
WickensC. D.HollandsJ. (2000). Engineering psychology and human performance (3rd ed.). New York: Prentice Hall.
48.
WidrowB.HoffM. E. (1960). Adaptive switching circuits. IRE WESCON Convention Record (pp. 96–104). New York: Institute of Radio Engineers.
49.
WientjesC. J. E. (1992). Respiration in psychophysiology: Methods and applications. Biological Psychology, 34(2–3), 179–203.
50.
WildervanckC.MulderG.MichonJ. A. (1978). Mapping mental load in car driving. Ergonomics, 21, 225–229.
51.
WilsonG.RussellC. (2003). Operator functional state classification using multiple psychophysiological features in an air traffic control task. Human Factors, 45, 381–389.
52.
WilsonG. F.EggemeierF. T. (1991). Physiological measures of workload in multi-task environments. In DamosD. (Ed.), Multiple-task performance (pp. 329–360). London: Taylor & Francis.
53.
YamadaF. (1998). Frontal midline theta rhythm and eyeblinking activity during a VDT task and a video game: Useful tools for psychophysiology in ergonomics. Ergonomics, 41, 678–688.
