In speech emotion recognition, most emotional corpora generally have problems such as inconsistent sample length and imbalance of sample categories. Considering these problems, in this paper, a variable length input CRNN deep learning model based on Focal Loss is proposed for speech emotion recognition of anger, happiness, neutrality and sadness in IEMOCAP emotional corpus. In this model, Firstly, a variable-length strategy is introduced to input the speech spectra of the filled speech samples into CNN. Then the effective part of the input sequence is preserved and output by masking matrix and convolution layer. Thirdly, the effective output of input sequence is input into BiGRU network for learning. Finally, the focal loss is used for network training to control and adjust the contribution of various samples to the total loss. Compared with the traditional speech emotion recognition model, simulations show that our method can effectively improve the accuracy and performance of emotion recognition.
BjörnW. and LiuSchuller., Speech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends, J. Communications of the Acm61(5) (2018), 90–99.
2.
SchmidtE.M. and KimY.E., Learning emotion-based acoustic features with deep belief networks [C], (2011), 65–68.
3.
HanK., YuD. and TasheyI., Speech emotion recognition using deep neural network and extreme learning machine [C]//Fifteenth annual conference of the international speech communication association, 2014.
4.
ZhengW.Q., YuJ.S. and ZouY.X., An experimental study of speech emotion recognition based on deep convolutional neural networks [C]//2015 international conference on affective computing and intelligent interaction (ACII), IEEE (2015), 827–831.
5.
HuangC., GongW., FuW., et al. A research of speech emotion recognition based on deep belief network and SVM [J], Mathematical Problems in Engineering (2014), 2014.
6.
MaoQ., DongM. and HuangZ., et al. Learning salient features for speech emotion recognition using convolutional neural networks [J], IEEE Transactions on Multimedia16(8) (2014), 2203–2213.
7.
SattA., RozenbergS. and HooryR., Efficient Emotion Recognition from Speech Using Deep Learning on Spectrograms [C]//INTERSPEECH, (2017), 1089–1093.
8.
LimW., JangD. and LeeT., Speech emotion recognition using convolutional and recurrent neural networks [C]//2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), IEEE (2016), 1–4.
9.
MirsamadiS., BarsoumE. and ZhangC., Automatic speech emotion recognition using recurrent neural networks with local attention [C]//2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE (2017), 2227–2231.
10.
AytarY., VondrickC. and TorralbaA., Soundnet: Learning sound representations from unlabeled video [C]//Advances in neural information processing systems, (2016), 892–900.
11.
SainathT.N., VinyalO. and SakH., ”Convolutional, long short-term memory,fully connected deep neural networks”, in IEEE International Conference on Acoustics, Speech and Signal Processing, (2015), 4580–4584.
12.
LeeJ. and TashevI., “High-level feature representation using recurrent neural network for speech emotion recognition,” in INTERSPEECH, 2015.
13.
RanaR., ”Emotion classification from noisy speech – a deep learning approach,”arXiv preprint arXiv:1603.05901, 2016.
14.
ChernykhV., SterlingG. and PrihodkoP., “Emotion recognition from speech with recurrent neural networks,”arXiv preprint arXiv:1701.08071, 2017.
15.
SainathT.N., WeissR.J., SeniorA.W., WilsonK.W. and VinyalsO., ”Learning the speech front-end with raw waveform cldnns”, in INTERSPEECH, 2015, pp. 1–5.
16.
HunnunA., CaseC., CasperJ., CatanzaroB., DiamosG., ElsenE., PrengerR., SatheeshS., SenguptaS. and Coates”A., Deep speech: Scaling up end-to-end speech recognition,”Computer Science, 2014.
17.
AmodeiD., AnubhaiR., BattenbergE., CaseC., CasperJ., CatanzaroB., ChenJ., ChrzanowskiM., CoatesA. and DiamosG., ”Deep speech 2: End-to-end speech recognition in english and mandarin,”, Computer Science2015.
18.
VarianiE., LeiX., McdermottE., MorenoI.L. and Gonzalez-DominguezJ., “Deep neural networks for small footprint text-dependent speaker verification,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2014, pp. 4052–4056.
19.
ZhaoJianfeng,
MaoXia and ChenLijiang, Speech emotion recognition using deep 1D & 2D CNN LSTM networks, in Biomedical Signal Processing and Control, 2019, pp. 312–323.
20.
TrigeorrgisG., RingevalF., BruecknerR., MarchiE., NicolaouM.A., SchullerB. and ZafeiriouS., ”Adieu features? end- to-end speech emotion recognition using a deep convolutional recurrent network,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2016.
21.
RedmonJ., DivvalaS., GirshickR. and FarhadiA., You only look once: Unified, real-time object detection. In CVPR, 2016.
22.
RedmonJ. and FarhadiA., YOLO9000: Better, faster, stronger. In CVPR, 2017.
23.
LiuW., AnguelovD., ErhanD., SzegedyC. and ReedS., SSD: Single shot multibox detector. In ECCV, 2016.
24.
FuC.-Y., LiuW., RangaA., TyagiA. and BergA.C., DSSD: Deconvolutional single shot detector. arXiv: 1701. 06659, 2016.
25.
LinT.Y., GoyalP. and GirshickR., et al. Focal loss for dense object detection [C]//Proceedings of the IEEE International Conference on Computer Vision, (2017), 2980–2988.
26.
MaXi,
WuZhiyong and
JiaJia, et al.
Emotion Recognition from Variable-Length Speech Segments Using Deep Learing on Sepectrograms, In INTERSPEECH, 2018, pp. 3683–3687.
