Speech Emotion Recognition (SER) has been widely used in many fields, such as smart home assistants commonly found in the market. Smart home assistants that could detect the user’s emotion would improve the communication between a user and the assistant enabling the assistant to offer more productive feedback. Thus, the aim of this work is to analyze emotional states in speech and propose a suitable algorithm considering performance verses complexity for deployment in smart home devices. The four emotional speech sets were selected from the Berlin Emotional Database (EMO-DB) as experimental data, 26 MFCC features were extracted from each type of emotional speech to identify the emotions of happiness, anger, sadness and neutrality. Then, speaker-independent experiments for our Speech emotion Recognition (SER) were conducted by using the Back Propagation Neural Network (BPNN), Extreme Learning Machine (ELM), Probabilistic Neural Network (PNN) and Support Vector Machine (SVM). Synthesizing the recognition accuracy and processing time, this work shows that the performance of SVM was the best among the four methods as a good candidate to be deployed for SER in smart home devices. SVM achieved an overall accuracy of 92.4% while offering low computational requirements when training and testing. We conclude that the MFCC features and the SVM classification models used in speaker-independent experiments are highly effective in the automatic prediction of emotion.
PoriaS., CambriaE., BajpaiR. and HussainA., A review of affective computing: from unimodal analysis to multimodal fusion, Information Fusion37 (2017), 98–125, 10.1016/j.inffus.2017.02.003
2.
SunY. and ZhangX.Y., Characteristics of human auditory model based on compensation of glottal features in speech emotion recognition, Future Generation Computer Systems81 (2018), 291–296, DOI. 10.1016/j.future.2017.10.002
3.
özsevenT., Investigation of the effect of spectrogram images and different texture analysis methods on speech emotion recognition, Applied Acoustics142 (2018), 70–77, 10.1016/j.apacoust.2018.08.003
4.
HuiT.K.L. and SherrattS., Coverage of emotion recognition for common wearable biosensors, Sensors8(2) (2018), 30, 10.3390/bios8020030
5.
ZhuX., ShiY.-Y., KimH.-G. and EomK.-W., An integrated music recommendation system, IEEE Trans Consum ElectronCE-52(3) (2006), 612–618, 10.1109/TCE.2006.1706489
6.
BacivarovI., CorcoranP. and IonitaM., Smart cameras: 2D affine models for determining subject facial expressions, IEEE Trans Consum ElectronCE-56(2) (2010), 298–297, 10.1109/TCE.2010.5505930
7.
YoonW.-J. and ParkK.-S., Building robust emotion recognition system on heterogeneous speech databases, IEEE Trans Consum ElectronCE-57(2) (2011), 747–750, 10.1109/TCE.2011.5955217
8.
KimD.K., KimJ., LimE.C., WhangM. and ChoY., Interactive emotional content communications system using portable wireless biofeedback device, IEEE Trans Consum ElectronCE-57(4) (2011), 1929–1936, 10.1109/TCE.2011.6131173
9.
YoonK., LeeJ. and KimM.-U., Music recommendation system using emotion triggering low-level features, IEEE Trans Consum ElectronCE-58(2) (2012), 612–618, 10.1109/TCE.2012.6227467
10.
RosaR.L., RodriguezD.Z. and BressanG., Music recommendation system based on user’s sentiments extracted from social networks, IEEE Trans Consum ElectronCE-61(3) (2015), 359–367, 10.1109/TCE.2015.7298296
11.
KimD.K., AhnS., ParkS. and WhangM., Interactive emotional lighting system using physiological signals, IEEE Trans Consum ElectronCE-59(4) (2013), 765–771, 10.1109/TCE.2013.6689687
12.
ParkJ.-S., KimJ.-H. and OhY.-H., Feature vector classification-based speech emotion recognition for service robots, IEEE Trans Consum ElectronCE-55(3) (2009), 1590–1596, 10.1109/TCE.2009.5278031
13.
KimD.-S., LeeS.-S. and CholB.-H., A real-time stereo depth extraction hardware for intelligent home assistant robot, IEEE Trans Consum ElectronCE-56(3) (2010), 1782–1788, 10.1109/TCE.2010.5606326
14.
Rubio-DrosdovE., Diaz-SanchezD., AlmenarezF., Arias-CabarcosP. and MarinA., Seamless human-device interaction in the internet of things, IEEE Trans Consum ElectronCE-63(4) (2017), 490–498, 10.1109/TCE.2017.015076
15.
PerumalT., RamliA.R. and LeongC.Y., Design and implementation of SOAP-based residential management for smart home systems, IEEE Trans Consum ElectronCE-54(2) (2008), 453–459, 10.1109/TCE.2008.4560114.
16.
WangJ., ZhangZ., LiB., LeeS. and SherrattR.S., An enhanced fall detection system for elderly person monitoring using consumer home networks, IEEE Trans Consum ElectronCE-60(1) (2014), 23–29, 10.1109/TCE.2014.6780921
17.
DeyN., AshourA.S., ShiF., FongS.J. and SherrattR.S., Developing residential wireless sensor networks for ECG healthcare monitoring, IEEE Trans Consum ElectronCE-63(4) (2017), 442–449, 10.1109/TCE.2017.015063
18.
McNallyR.J., Handbook of cognition and emotion, British J Psychiatry176(5) (1999), 10.1002/0470013494
19.
HamannS., Mapping discrete and dimensional emotions onto the brain: controversies and consensus, Trends in Cognitive Sciences16(9) (2012), 458–466, 10.1016/j.tics.2012.07.006
20.
Chih-HaoC., Wei-PoL. and Jhih-YuanH., Tracking and recognizing emotions in short text messages from online chatting services, Information Processing & Management54(6) (2018), 1325–134410.1016/j.ipm.2018.05.008
21.
ShiF., DeyN., AshourA.S., Sifaki-PistollaD. and SherrattR.S., Meta-KANSEI modeling with valence-arousal fMRI dataset of brain, Cognitive Computation (2018), 10.1007/s12559-018-9614-5.
22.
DaiW., HanD., DaiY. and XuD., Emotion recognition and affective computing on vocal social media, Information & Management52(7) (2015), 777–788, 10.1016/j.im.2015.02.003
23.
XingB., ZhangK., SunS., ZhangL., GaoZ. and WangJ., Emotion-driven Chinese folk music-image retrieval based on DE-SVM, Neurocomputing148 (2015), 619–627, 10.1016/j.neucom.2014.08.007
24.
ZualkernanI., AloulF., ShapsoughS., HeshamA. and El-KhorzatyY., Emotion recognition using mobile phones, Computers & Electrical Engineering60 (2017), 1–13, 10.1016/j.compeleceng.2017.05.004
25.
AlonsoJ.B., CabreraJ., TraviesoC.M., López-de-IpiñaK. and Sánchez-MedinaA., Continuous tracking of the emotion temperature, Neurocomputing255 (2017), 17–25, 10.1016/j.neucom.2016.06.093
26.
NanniL., CostaY.M.G., LucioD.R., SillaC.N. and BrahnamS., Combining visual and acoustic features for audio classification tasks, Pattern Recognition Lett88 (2017), 49–56, 10.1016/j.patrec.2017.01.013
27.
KraxenbergerM., MenninghausW., RothA. and ScharingerM., Prosody-based sound-emotion associations in poetry, Frontiers in Psychology9 (2018), 1284–1284, 10.3389/fpsyg.2018.01284
28.
LalithaS., GeyasrutiD., NarayananR. and ShravaniM., Emotion Detection Using MFCC and Cepstrum Features, Procedia Computer Science70 (2015), 29–35, 10.1016/j.procs.2015.10.020
29.
JacobA., Speech emotion recognition based on minimal voice quality features, in Proc. ICCSP, Melmaruvathur, India, (2016), pp. 0886–0890, 10.1109/ICCSP.2016.7754275
30.
Perez-GasparL.A., Caballero-MoralesS.O. and Trujillo-RomeroF., Multimodal emotion recognition with evolutionary computation for human-robot interaction, Expert Systems with Applications66 (2016), 42–61, 10.1016/j.eswa.2016.08.047
31.
DavletcharovaA., SugathanS., AbrahamB. and JamesA.P., Detection and analysis of emotion from speech signals, Procedia Computer Science58 (2015), 91–96, 10.1016/j.procs.2015.08.032
32.
AbhangP.A., GawaliB.W. and MehrotraS.C., Chapter 7 - Proposed EEG/speech-based emotion recognition system: a case study, Introduction to EEG- and Speech-Based Emotion Recognition, (2016), 127–163, 10.1016/B978-0-12-804490-2.00007-5
33.
BasuS., ChakrabortyJ., BagA. and AftabuddinM., A review on emotion recognition using speech, in Proc ICICCT, Coimbatore, India, (2017), 109–114, 10.1109/ICICCT.2017.7975169
34.
GuidoR.C., PereiraJ.C. and SlaetsJ.F.W., Emergent artificial intelligence approaches for pattern recognition in speech and language processing, Computer Speech & Language24(3) (2010), 431–432, 10.1016/j.csl.2010.03.002
35.
RajishaT.M., SunijaA.P. and RiyasK.S., Performance analysis of Malayalam language speech emotion recognition system using ANN/SVM, Procedia Technology24 (2016), 1097–1104, 10.1016/j.protcy.2016.05.242
36.
SujathaB. and AmeenaO., Speech Emotion Recognition using HMM, GMM and SVM, Int J Professional Engineering Studies6(3) (2016), 311–318.
37.
LanjewarR.B., MathurkarS. and PatelN., Implementation and comparison of speech emotion recognition system using Gaussian mixture model (GMM) and k-nearest neighbor (K-NN) techniques, Procedia Computer Science49(1) (2015), 50–57, 10.1016/j.procs.2015.04.226
LiuZ.T., WuM., CaoW.H., MaoJ.W., XuJ.P. and TanG.Z., Speech emotion recognition based on feature selection and extreme learning machine decision tree, Neurocomputing273 (2018), 271–280, 10.1016/j.neucom.2017.07.050
40.
AyadiM.E., KamelM.S. and KarrayF., Survey on speech emotion recognition: features, classification schemes, and databases, Pattern Recognition44(3) (2011), 10.1016/j.patcog.2010.09.020
41.
SharmaN.K. and SreenivasT.V., Time-varying sinusoidal demodulation for non-stationary modeling of speech, Speech Communication105 (2018), 77–91, 10.1016/j.specom.2018.10.008
42.
TzanetakisG., Music analysis, retrieval and synthesis of audio signals MARSYAS, in Proc. Multimedia, Vancouver, British Columbia, Canada, (2009), pp. 931–932.
43.
özsevenT. and DüğenciM., Speech acoustic (SPAC): A novel tool for speech feature extraction and classification, Applied Acoustics136 (2018), 1–8, 10.1016/j.apacoust.2018.02.009
44.
HemanthD.J., AnithaJ. and SonL.H., Brain signal based human emotion analysis by circular back propagation and deep Kohonen neural networks, Computers & Electrical Engineering68 (2018), 170–180, 10.1016/j.compeleceng.2018.04.006
45.
CaoW., WangX., MingZ. and GaoJ., A review on neural networks with random weights, Neurocomputing275 (2018), 278–287, 10.1016/j.neucom.2017.08.040
46.
DongX. and ZhouD.X., Learning gradients by a gradient descent algorithm, J Mathematical Analysis and Applications341(2) (2008), 1018–1027, 10.1016/j.jmaa.2007.10.044
47.
LuoF., GuoW., YuY. and ChenG., A multi-label classification algorithm based on kernel extreme learning machine, Neurocomputing260 (2017), 313–320, 10.1016/j.neucom.2017.04.052
48.
LendasseA., VongC.M., TohK.A., MicheY. and HuangG.B., Advances in extreme learning machines, Neurocomputing261 (2017), 1–3, 10.1016/j.neucom.2017.01.089
49.
NishanthK.J. and RaviV., Probabilistic neural network based categorical data imputation, Neurocomputing218 (2016), 17–25, 10.1016/j.neucom.2016.08.044
50.
GrimJ. and HoraJ., Iterative principles of recognition in probabilistic neural networks, Neural Networks21(6) (2008), 838–846, 10.1016/j.neunet.2008.03.002
51.
González-SerranoF.-J., Navia-VázquezÁ. and Amor-MartínA., Training support vector machines with privacy-protected data, Pattern Recognition72 (2017), 93–107, 10.1016/j.patcog.2017.06.016
52.
DahakeP.P., ShawK. and MalathiP., Speaker dependent speech emotion recognition using MFCC and Support Vector Machine, in Proc. ICACDOT, Pune, India, (2016), 1080–1084, 10.1109/ICACDOT.2016.7877753
53.
PatoomsiriS., VladimirC. and BoonsermK., Universum selection for boosting the performance of multiclass support vector machines based on one-versus-one strategy, Knowledge-Based Systems159 (2018), 9–19, 10.1016/j.knosys.2018.05.025
54.
FanR.E., ChenP.H. and LinC.J., Working set selection using second order information for training support vector machines, J Machine Learning Research6(4) (2005), 1889–1918, 10.1115/1.1898234
55.
JiangL., TanP. and YangJ., Speech emotion recognition using emotion perception spectral feature, Concurrency and Computation Practice and Experience 11 2019, https://doi.org/10.1002/cpe.5427.
56.
RamyaH.R. and BhattM.R., Personalised Emotion Recognition Utilising Speech Signal and Linguistic Cues, 2019 11th International Conference on Communication Systems & Networks, Bengaluru, India, Jan. 2019.
57.
LotfianR. and BussoC., Curriculum learning for speech emotion recognition from crowdsourced labels, IEEE/ACM Transactions on Audio, Speech, and Language Processing27 (2019), 815–826, 10.1109/TASLP.2019.2898816.
