This article presents a new database of speech produced under cognitive load for the purpose of non-invasive psychological stress monitoring. The voices and the heart rates of eight airline pilots were recorded while completing an advanced flight simulation programme in a level D full flight simulator. Focusing on real-world applicability, the experiments were designed to yield the maximum degree of realism possible. Evaluation of physiological reference measures in pilots demonstrates that several heart rate variability parameters correlate with speech features derived from the recorded data. The article discusses the evolution of speech monitoring in aviation and proposes that application-orientated research methods can be useful in designing a system for real-world monitoring.
WierwilleWWEggemeierFT. Recommendations for mental workload measurement in a test and evaluation environment. Human Factors1993; 35(2): 263–281.
2.
HansenJHLPatilS. Speech under stress: analysis, modeling and recognition. Speaker Classif I: Fundament Feat Meth2007; 1: 108–108.
3.
MurrayIRBaberCSouthA. Towards a definition and working model of stress and its effects on speech. Speech Commun1996; 20(1): 3–12.
4.
HagmüllerMRankEKubinG. Evaluation of the human voice for indications of workload-induced stress in the aviation environment. EEC Note No. 18/062006.
5.
Luig J. Investigations on a robust feature set for classification of speech under stress. Master’s Thesis, Graz University of Technology, 2009.
6.
Luig J. Speech features for stress recognition: universal applicability of state-of-the-art methods. Saarbrücken: VDM Verlag, 2010.
7.
SchullerBBatlinerASteidlS. Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun2011; 53(9): 1062–1087.
HansenJHLBou-GhazaleS. Getting started with SUSAS: a speech under simulated and actual stress database, 1997. Fifth European Conference on Speech Communication and Technology (EUROSPEECH), vol 97, Rhodes, Greece, 22--25 September. 1743–1746.
10.
HansenJHLZhangXAkbacakM. CU-Move: advanced in-vehicle speech systems for route navigation. In: Abut H, Hansen JHL and Takeda K (eds). DSP for In-Vehicle and Mobile Systems, New York: Springer, 2005, pp. 19–45.
11.
IkenoAVaradarajanVPatilSA. UT-Scope: speech under lombard effect and cognitive stress, 2007. IEEE Aerospace Conference, Big Sky, Montana, 3–10 March. 1–7.
12.
BorilHSadjadiSOHansenJHL. UTDrive: emotion and cognitive load classification for in-vehicle scenarios, 2011. 5th Biennial Workshop on DSP for In-Vehicle Systems, Kiel, Germany.
13.
McGrathJE. Social and psychological factors in stress, Holt: Rinehart & Winston, 1970.
14.
Douglas-CowieECampbellNCowieR. Emotional speech: towards a new generation of databases. Speech Commun2003; 40(1): 33–60.
15.
Johns-LewisC. Prosodic differentiation of discourse modes. Intonation in Discourse1986, pp. 199–220.
16.
WilsonGFEggemeierFTPsychophysiological assessment of workload in multi-task environments. In: DamosDL (ed). Multiple-task performance, New York: Taylor & Francis, 1991, pp. 329–360.
17.
HuttunenKKeränenHVäyrynenE. Effect of cognitive load on speech prosody in aviation: evidence from military simulator flights. Appl Ergon2011; 42(2): 348–357.
18.
Luig J. IEM Report 45/11: The IEM Pilot Speech Database. Technical report, Institute of Electronic Music and Acoustics, 2011.
19.
MalikMBiggerJCammA. Guidelines: heart rate variability, standards of measurement, physiological interpretation, and clinical use. Eur Heart J1996; 17: 354–381.
20.
BettermannHAmponsahDCysarzD. Musical rhythms in heart period dynamics: a cross-cultural and interdisciplinary approach to cardiac rhythms. Am J Physiol-Heart Circulat Physiol1999; 277(5): H1762–H1770.
21.
SteyerRSchwenkmezgerPNotzP. Der Mehrdimensionale Befindlichkeitsfragebogen (in German), Hogrefe Verlag für Psychologie, 1997.
22.
StevensKNHansonHMClassification of glottal vibration from acoustic measurements. In: FujimuraOHiranoH (eds). Vocal fold physiology: voice quality control, San Diego: Singular, 1995, pp. 147–170.
23.
Lugger M, Yang B and Wokurek W. Robust estimation of voice quality parameters under real-world disturbances. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), vol. 1, Toulouse, France, 14--19 May 2006, pp. I–I.
24.
Frank M. Perzeptiver Vergleich von Schallfeldreproduktionsverfahren unterschiedlicher räumlicher Bandbreite (in German). Master’s Thesis, University of Technology, 2009.
