The current challenges of automatic recognition of facial expressions: A systematic review

Abstract

In recent years, due to its great economic and social potential, the recognition of facial expressions linked to emotions has become one of the most flourishing applications in the field of artificial intelligence, and has been the subject of many developments. However, despite significant progress, this field is still subject to many theoretical debates and technical challenges. It therefore seems important to make a general inventory of the different lines of research and to present a synthesis of recent results in this field. To this end, we have carried out a systematic review of the literature according to the guidelines of the PRISMA method. A search of 13 documentary databases identified a total of 220 references over the period 2014–2019. After a global presentation of the current systems and their performance, we grouped and analyzed the selected articles in the light of the main problems encountered in the field of automated facial expression recognition. The conclusion of this review highlights the strengths, limitations and main directions for future research in this field.

Keywords

Artificial intelligence affective computing facial expression AFER emotion theories

Get full access to this article

View all access options for this article.

References

Acevedo,

Negri,

M.E.

Buemi and

Mejail, Facial expression recognition based on static and dynamic approaches, in: 2016 23rd International Conference on Pattern Recognition (ICPR), 2016, pp. 4124–4129. doi:10.1109/ICPR.2016.7900280.

Adams,

Mahmoud,

Baltrusaitis and

Robinson, Decoupling facial expressions and head motions in complex emotions, in: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), 2015, pp. 274–280. doi:10.1109/ACII.2015.7344583.

Afdhal,

Ejbali,

Zaied and

Ben Amar, Emotion recognition using features distances classified by wavelets network and trained by fast wavelets transform, in: 2014 14th International Conference on Hybrid Intelligent Systems, 2014, pp. 238–241. doi:10.1109/HIS.2014.7086205.

Al-Darraji,

Berns and

Rodić, Action unit based facial expression recognition using deep learning, in: Advances in Robot Design and Intelligent Control,

Rodić and

Borangiu, eds, Vol. 540, 2017, pp. 413–420. doi:10.1007/978-3-319-49058-8-45.

Alam,

L.S.

Vidyaratne and

K.M.

Iftekharuddin, Sparse simultaneous recurrent deep learning for robust facial expression recognition, IEEE Transactions on Neural Networks and Learning Systems 29(10) (2018), 4905–4916. doi:10.1109/TNNLS.2017.2776248.

Ali,

Hariharan,

Yaacob,

A.H.

Adom,

S.K.

Za’ba and

Elshaikh, Facial emotion recognition under partial occlusion using empirical mode decomposition, in: 2016 2nd IEEE International Symposium on Robotics and Manufacturing Automation (ROMA), 2016, pp. 1–6. doi:10.1109/ROMA.2016.7847818.

A.M.

Ashir,

Eleyan and

Akdemir, Facial expression recognition with dynamic cascaded classifier, Neural Computing and Applications. (2019). doi:10.1007/s00521-019-04138-4.

Avots,

Sapiński,

Bachmann and

Kamińska, Audiovisual emotion recognition in wild, Machine Vision and Applications 30(5) (2019), 975–985. doi:10.1007/s00138-018-0960-9.

Azazi,

S.L.

Lutfi and

Venkat, Analysis and evaluation of SURF descriptors for automatic 3D facial expression recognition using different classifiers, in: 2014 4th World Congress on Information and Communication Technologies (WICT 2014), 2014, pp. 23–28. doi:10.1109/WICT.2014.7077296.

10.

Bahreini,

van der Vegt and

Westera, A fuzzy logic approach to reliable real-time recognition of facial emotions, Multimedia Tools and Applications. (2019). doi:10.1007/s11042-019-7250-z.

11.

Baltrusaitis,

Li and

L.-P.

Morency, Local-global ranking for facial expression intensity estimation, in: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), 2017, pp. 111–118. doi:10.1109/ACII.2017.8273587.

12.

Bandrabur,

Florea,

Florea and

Mancas, Late fusion of facial dynamics for automatic expression recognition, Turkish Journal of Electrical Engineering & Computer Sciences 25 (2017), 2696–2707. doi:10.3906/elk-1607-113.

13.

E.I.

Barakova,

Gorbunov and

Rauterberg, Automatic interpretation of affective facial expressions in the context of interpersonal interaction, IEEE Transactions on Human-Machine Systems 45(4) (2015), 409–418. doi:10.1109/THMS.2015.2419259.

14.

Barman and

Dutta, Facial expression recognition using shape signature feature, in: 2017 Third International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), 2017, pp. 174–179. doi:10.1109/ICRCICN.2017.8234502.

15.

Barrett,

Adolphs,

Marsella,

A.-M.

Martinez and

S.-D.

Pollak, Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements, Psychological Science in the Public Interest 20(1) (2019), 1–68. doi:10.1177/1529100619832930.

16.

Barros,

Jirak,

Weber and

Wermter, Multimodal emotional state recognition using sequence-dependent deep hierarchical features, Neural Networks 72 (2015), 140–151. doi:10.1016/j.neunet.2015.09.009.

17.

J.C.

Batista,

Albiero,

O.R.P.

Bellon and

Silva, AUMPNet: Simultaneous action units detection and intensity estimation on multipose facial images using a single convolutional neural network, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 866–871. doi:10.1109/FG.2017.111.

18.

Battini Sönmez and

Cangelosi, Convolutional neural networks with balanced batches for facial expressions recognition, in: Proceedings Volume 10341, Ninth International Conference on Machine Vision (ICMV 2016),

Verikas,

Radeva,

D.P.

Nikolaev,

Zhang and

Zhou, eds, 2017. doi:10.1117/12.2268412.

19.

Ben Amor,

Drira,

Berretti,

Daoudi and

Srivastava, 4-D facial expression recognition by learning geometric deformations, IEEE Transactions on Cybernetics 44(12) (2014), 2443–2457. doi:10.1109/TCYB.2014.2308091.

20.

Ben Tanfous,

Drira and

Ben Amor, Sparse coding of shape trajectories for facial expression and action recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 1(1) (2019). doi:10.1109/TPAMI.2019.2932979.

21.

K.-I.

Benta and

M.-F.

Vaida, Towards real-life facial expression recognition systems, Advances in Electrical and Computer Engineering 15(2) (2015), 93–102. doi:10.4316/AECE.2015.02012.

22.

Bernin,

Müller,

Ghose,

Grecos,

Wang,

Jettke and

Vogt, Automatic classification and shift detection of facial expressions in event-aware smart environments, in: Proceedings of the 11th Pervasive Technologies Related to Assistive Environments Conference On–PETRA’18, 2018, pp. 194–201. doi:10.1145/3197768.3201527.

23.

Bhandari,

R.K.

Bijarniya,

Chatterjee and

Kolekar, Analysis for self-taught and transfer learning based approaches for emotion recognition, in: 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN), 2018, pp. 509–512. doi:10.1109/SPIN.2018.8474199.

24.

Bishay and

Patras, Fusing multilabel deep networks for facial action unit detection, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 681–688. doi:10.1109/FG.2017.86.

25.

Borah and

Konwar, ANN based human facial expression recognition in color images, in: 2014 International Conference on High Performance Computing and Applications (ICHPCA), 2014, pp. 1–6. doi:10.1109/ICHPCA.2014.7045337.

26.

Borza,

Danescu,

Itu and

Darabant, High-speed video system for micro-expression detection and recognition, Sensors 17(12) (2017), 2913. doi:10.3390/s17122913.

27.

Boubenna and

Lee, Image-based emotion recognition using evolutionary algorithms, Biologically Inspired Cognitive Architectures 24 (2018), 70–76. doi:10.1016/j.bica.2018.04.008.

28.

Boughrara,

Chtourou,

Ben Amar and

Chen, Facial expression recognition based on a mlp neural network using constructive training algorithm, Multimedia Tools and Applications 75(2) (2016), 709–731. doi:10.1007/s11042-014-2322-6.

29.

L.F.

Bringmann,

Ferrer,

E.L.

Hamaker,

Borsboom and

Tuerlinckx, Modeling nonstationary emotion dynamics in dyads using a time-varying vector-autoregressive model, Multivariate Behavioral Research 53(3) (2018), 293–314. doi:10.1080/00273171.2018.1439722.

30.

J.P.

Canario and

Oliveira, Recognition of facial expressions based on deep conspicuous net, in: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications,

Pardo and

Kittler, eds, Vol. 9423, 2015, pp. 255–262. doi:10.1007/978-3-319-25751-8-31.

31.

Chavan and

Kulkarni, Optimizing deep convolutional neural network for facial expression recognitions, in: Data Management, Analytics and Innovation,

V.E.

Balas,

Sharma and

Chakrabarti, eds, Vol. 808, 2019, pp. 185–196. doi:10.1007/978-981-13-1402-5-14.

32.

Chen,

Lv,

Xu and

Xu, Automatic social signal analysis: Facial expression recognition using difference convolution neural network, Journal of Parallel and Distributed Computing 131 (2019), 97–102. doi:10.1016/j.jpdc.2019.04.017.

33.

Chen,

Ou,

Chi and

Fu, Smile detection in the wild with deep convolutional neural networks, Machine Vision and Applications 28(1–2) (2017), 173–183. doi:10.1007/s00138-016-0817-z.

34.

M.I.H.

Chowdhury and

F.I.

Alam, A probabilistic approach to support Self-Organizing Map (SOM) driven facial expression recognition, in: 2014 17th International Conference on Computer and Information Technology (ICCIT), 2014, pp. 210–216. doi:10.1109/ICCITechn.2014.7073131.

35.

Corneanu,

Madadi and

Escalera, Deep Structure Inference Network for Facial Action Unit Recognition,

Ferrari,

Hebert,

Sminchisescu and

Weiss, eds, Vol. 11216, Computer Vision – ECCV, 2018, pp. 309–324. doi:10.1007/978-3-030-01258-8-19.

36.

C.A.

Corneanu,

M.O.

Simon,

J.F.

Cohn and

S.E.

Guerrero, Survey on RGB, 3D, thermal, and multimodal approaches for facial expression recognition: History, trends, and afect-related applications, IEEE Transactions on Pattern Analysis and Machine Intelligence 38(8) (2016), 1548–1568. doi:10.1109/TPAMI.2016.2515606.

37.

J.Y.R.

Cornejo,

Pedrini and

Flórez-Revuelta, Facial expression recognition with occlusions based on geometric representation, in: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications,

Pardo and

Kittler, eds, Vol. 9423, 2015, pp. 263–270. doi:10.1007/978-3-319-25751-8-32.

38.

M.D.

Costa-Abreu and

G.S.

Bezerra, FAMOS: A framework for investigating the use of face features to identify spontaneous emotions, Patern Analusis and Applications. (2017). doi:10.1007/sl0044-017-0675-y.

39.

Cui,

Liu and

Liu, Facial expression recognition based on ensemble of mulitple CNNs, in: Biometric Recognition,

You,

Zhou,

Wang,

Sun,

Shan,

Zheng and

Zhao, eds, Vol. 9967, 2016, pp. 511–518. doi:10.1007/978-3-319-46654-5-56.

40.

F.A.M.

da Silva and

Pedrini, Effects of cultural characteristics on building an emotion classifier through facial expression analysis, Journal of Electronic Imaging 24(2) (2015), 023015. doi:10.1117/1.JEI.24.2.023015.

41.

Darwin, The Expression of the Emotions in Man and Animals, John Murray, London, 1872/1965.

42.

Das,

M.M.

Hoque,

J.F.

Ara,

M.A.F.M.R.

Hasan,

Kobayashi and

Kuno, Automatic face parts extraction and facial expression recognition, in: 2014 9th International Forum on Strategic Technology (IFOST), 2014, 128131. doi:10.1109/IFOST.2014.6991087.

43.

Del Líbano,

M.G.

Calvo,

Fernández-Martín and

Recio, Discrimination between smiling faces: Human observers vs. automated face analysis, Acta Psychologica 187 (2018), 1929. doi:10.1016/j.actpsy.2018.04.019.

44.

Deng,

Hu,

Zhang and

Guo, DeepEmo: Real-world facial expression analysis via deep learning, 2015 Visual Communications and Image Processing (VCIP) (2015), 14. doi:10.1109/VCIP.2015.7457876.

45.

J.-L.

Dessalles, Des Intelligences TRES Artificielles, Odile Jacob, 2019.

46.

Ding,

S.K.

Zhou and

Chellappa, FaceNet2ExpNet: Regularizing a deep face REcognition net for expression recognition, in: 2017 12th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2017), 2017, pp. 118–126. doi:10.1109/FG.2017.23.

47.

H.I.

Dino and

M.B.

Abdulrazzaq, Facial expression classification based on SVM, KNN and MLP classifiers, in: 2019 International Conference on Advanced Science and Engineering (ICOASE), 2019, pp. 70–75. doi:10.1109/ICOASE.2019.8723728.

48.

Dou,

Zhou,

Wang and

Qiang, Facial expression recognition based-on saliency guided support vector machine, in: 2016 9th International Symposium on Computational Intelligence and Design (ISCID), 2016, pp. 389–393. doi:10.1109/ISCID.2016.2098.

49.

Ekman, An argument for basic emotions, Cognition and emotion 6(3–4) (1992), 169–200. doi:10.1080/02699939208411068.

50.

Ekundayo and

Viriri, Facial expression recognition: A review of methods, performances and limitations, in: 2019 Conference on Information Communications Technology and Society (ICTAS), 2019, pp. 1–6. doi:10.1109/ICTAS.2019.8703619.

51.

Eleftheriadis,

Rudovic and

Pantic, View-constrained latent variable model for multi-view facial expression classification, in: Advances in Visual Computing,

Bebis,

Boyle,

Parvin,

Koracin,

McMahan,

Jerald and

Carlson, eds, Vol. 8888, 2014, pp. 292–303. doi:10.1007/978-3-319-14364-4-28.

52.

Fan,

J.C.K.

Lam and

V.O.K.

Li, Multi-region ensemble convolutional neural network for facial expression recognition, in: Artificial Neural Networks and Machine Learning – ICANN 2018,

Kůrková,

Manolopoulos,

Hammer,

Iliadis and

Maglogiannis, eds, Vol. 11139, 2018, pp. 84–94. doi:10.1007/978-3-030-01418-6-9.

53.

Fang,

Mac Parthalain,

A.J.

Aubrey,

G.K.L.

Tam,

Borgo,

P.L.

Rosin and

Chen, Facial expression recognition in dynamic sequences: An integrated approach, Pattern Recognition 47(3) (2014), 1271–1281. doi:10.1016/j.patcog.2013.09.023.

54.

Farajzadeh,

Pan and

Wu, Facial expression recognition based on meta probability codes, Pattern Analysis and Applications 17(4) (2014), 763–781. doi:10.1007/s10044-012-0315-5.

55.

Fathallah,

Abdi and

Douik, Facial expression recognition via deep learning, in: 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), 2017, pp. 745–750. doi:10.1109/AICCSA.2017.124.

56.

J.M.

Garcia-Garcia,

V.M.R.

Penichet and

M.D.

Lozano, Emotion detection: A technology review, in: Proceedings of the XVIII International Conference on Human Computer Interaction, 2017, pp. 1–8. doi:10.1145/3123818.3123852.

57.

J.M.

Girard,

J.F.

Cohn and

De la Torre, Estimating smile intensity: A better way, Pattern Recognition Letters 66 (2015), 13–21. doi:10.1016/j.patrec.2014.10.004.

58.

Gonzalez,

Cartella,

Enescu and

Sahli, Recognition of facial actions and their temporal segments based on duration models, Multimedia Tools and Applications 74(22) (2015), 10001–10024. doi:10.1007/s11042-014-2320-8.

59.

Grobova,

Colovic,

Marjanovic,

Njegus,

Demire and

Anbarjafari, Automatic hidden sadness detection using micro-expressions, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 828–832. doi:10.1109/FG.2017.105.

60.

Gui,

Baltrusaitis and

L.-P.

Morency, Curriculum learning for facial expression recognition, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 505–511. doi:10.1109/FG.2017.68.

61.

Gunes and

Hung, Is automatic facial expression recognition of emotions coming to a dead end? The rise of the new kids on the block, Image and Vision Computing 55 (2016), 6–8. doi:10.1016/j.imavis.2016.03.013.

62.

Hammal and

M.T.

Suarez, Towards context based affective computing introduction to the third international, in: CBAR 2015 Workshop. 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, pp. 1–2. doi:10.1109/FG.2015.7284841.

63.

Hao,

Wang,

Peng and

Ji, Facial action unit recognition augmented by their dependencies, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018, pp. 187–194. doi:10.1109/FG.2018.00036.

64.

S.L.

Happy,

Dantcheva and

Bremond, A weakly supervised learning technique for classifying facial expressions, Pattern Recognition Letters 128 (2019), 162–168. doi:10.1016/j.patrec.2019.08.025.

65.

S.L.

Happy and

Routray, Automatic facial expression recognition using features of salient facial patches, IEEE Transactions on Affective Computing 6(1) (2015), 1–12. doi:10.1109/TAFFC.2014.2386334.

66.

S.L.

Happy and

Routray, Recognizing subtle micro-facial expressions using fuzzy histogram of optical flow orientations and feature selection methods, in: Computational Intelligence for Pattern Recognition,

Pedrycz and

S.-M.

Chen, eds, Vol. 777, 2018, pp. 341–368. doi:10.1007/978-3-319-89629-8-13.

67.

Hasani and

M.H.

Mahoor, Spatio-temporal facial expression recognition using convolutional neural networks and conditional random fields, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 790–795. doi:10.1109/FG.2017.99.

68.

He,

Li,

Yang,

Cao,

Sun and

Yu, Multi view facial action unit detection based on CNN and BLSTM-RNN, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 848–853. doi:10.1109/FG.2017.108.

69.

Hernandez-Garcia, Perceived emotion from images through deep neural networks, in: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), 2017, pp. 566–570. doi:10.1109/ACII.2017.8273656.

70.

M.S.

Hossain and

Muhammad, Emotion recognition using deep learning approach from audio–visual emotional big data, Information Fusion 49 (2019), 69–78. doi:10.1016/j.inffus.2018.09.008.

71.

M.S.

Hossain and

Muhammad, Emotion recognition using secure edge and cloud computing, Information Sciences 504 (2019), 589–601. doi:10.1016/j.ins.2019.07.040.

72.

Hu,

Liu,

Yuan,

Yu,

Hua,

Zhang and

Yang, Deep Multi-Task Learning to Recognise Subtle Facial Expressions of Mental States,

Ferrari,

Hebert,

Sminchisescu and

Weiss, eds, Vol. 11216, Computer Vision – ECCV, 2018, pp. 106–123. doi:10.1007/978-3-030-01258-8-7.

73.

J.C.

Hung,

K.-C.

Lin and

N.-X.

Lai, Recognizing learning emotion based on convolutional neural networks and transfer learning, Applied Soft Computing 84 (2019), 105724. doi:10.1016/j.asoc.2019.105724.

74.

Hupont and

Chetouani, Region-based facial representation for real-time action units intensity detection across datasets, Pattern Analysis and Applications. (2017). doi:10.1007/s10044-017-0645-4.

75.

Hussain,

Ujir,

Hipiny and

J.-L.

Minoi, 3D Facial Action Units Recognition for Emotional Expression, 2017, arXiv:1712.00195.

76.

Hussein,

Naqvi and

Chambers, Study of image-based expression recognition techniques on three recent spontaneous databases, in: 2017 22nd International Conference on Digital Signal Processing (DSP), 2017, pp. 1–5. doi:10.1109/ICDSP.2017.8096062.

77.

X.-P.

Huynh,

T.-D.

Tran and

Y.-G.

Kim, Convolutional neural network models for facial expression recognition using BU-3DFE database, in: Information Science and Applications (ICISA) 2016,

K.J.

Kim and

Joukov, eds, Vol. 376, 2016, pp. 441–450. doi:10.1007/978-981-10-0557-2-44.

78.

Imani and

G.A.

Montazer, A survey of emotion recognition methods with emphasis on E-Learning environments, Journal of Network and Computer Applications 147 (2019), 102423. doi:10.1016/j.jnca.2019.102423.

79.

Jain,

Durgesh and

Ramesh, Facial expression recognition using variants of LBP and classifier fusion, in: Proceedings of International Conference on ICT for Sustainable Development,

S.C.

Satapathy,

Joshi,

Modi and

Pathak, eds, Vol. 408, 2016, pp. 725–732. doi:10.1007/978-981-10-0129-1-75.

80.

Jameel,

Singhal and

Bansal, A comparison of performance of crisp logic and probabilistic neural network for facial expression recognition, in: 2015 1st International Conference on Next Generation Computing Technologies (NGCT), 2015, pp. 841–846. doi:10.1109/NGCT.2015.7375238.

81.

Jameel,

Singhal and

Bansal, A comprehensive study on facial expressions recognition techniques, in: 2016 6th International Conference – Cloud System and Big Data Engineering (Confluence), 2016, pp. 478–483. doi:10.1109/CONFLUENCE.2016.7508167.

82.

Jan and

Meng, Automatic 3D facial expression recognition using geometric and textured feature fusion, in: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, pp. 1–6. doi:10.1109/FG.2015.7284860.

83.

Jiang,

Martinez,

M.F.

Valstar and

Pantic, Decision level fusion of domain specific regions for facial action recognition, in: 2014 22nd International Conference on Pattern Recognition, 2014, pp. 1776–1781. doi:10.1109/ICPR.2014.312.

84.

Jiang,

Valstar,

Martinez and

Pantic, A dynamic appearance descriptor approach to facial actions temporal modeling, IEEE Transactions on Cybernetics 44(2) (2014), 161–174. doi:10.1109/TCYB.2013.2249063.

85.

Jin,

Wang,

Lian and

Hua, Emotion information visualization through learning of 3D morphable face model, The Visual Computer 35(4) (2019), 535–548. doi:10.1007/s00371-018-1482-1.

86.

Kacem,

Daoudi and

J.-C.

Alvarez-Paiva, Barycentric representation and metric learning for facial expression recognition, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018, pp. 443–447. doi:10.1109/FG.2018.00071.

87.

Khan,

Samyan,

M.U.G.

Khan,

Shahid and

S.Q.

Wahla, A survey on analysis of human faces and facial expressions datasets, International Journal of Machine Learning and Cybernetics. (2019). doi:10.1007/s13042-019-00995-6.

88.

H.-Q.

Khor,

See,

R.C.W.

Phan and

Lin, Enriched long-term recurrent convolutional network for facial micro-expression recognition, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018, pp. 667–674. doi:10.1109/FG.2018.00105.

89.

Kim, Exploring sources of variation in human behavioral data: Towards automatic audio-visual emotion recognition, in: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), 2015, pp. 748–753. doi:10.1109/ACII.2015.7344653.

90.

Ko, A brief review of facial emotion recognition based on visual information, Sensors 18(2) (2018), 401. doi:10.3390/s18020401.

91.

Kong, Facial expression recognition method based on deep convolutional neural network combined with improved LBP features, Personal and Ubiquitous Computing 23(3–4) (2019), 531–539. doi:10.1007/s00779-019-01238-9.

92.

Kukla and

Nowak, Facial emotion recognition based on cascade of neural networks, in: New Research in Multimedia and Internet Systems,

Zgrzywa,

Choroś and

Siemiński, eds, Vol. 314, 2015, pp. 67–78. doi:10.1007/978-3-319-10383-9-7.

93.

Kulkarni,

Corneanu,

Ofodile,

Escalera,

Baro,

Hyniewska and

Anbarjafari, Automatic recognition of facial displays of unfelt emotions, IEEE Transactions on Affective Computing 1(1) (2018). doi:10.1109/TAFFC.2018.2874996.

94.

K.R.

Kulkarni and

S.B.

Bagal, Facial expression recognition, in: 2015 Annual IEEE India Conference (INDICON), 2015, pp. 1–5. doi:10.1109/INDICON.2015.7443572.

95.

M.P.

Kumar and

M.K.

Rajagopal, Detecting facial emotions using normalized minimal feature vectors and semi-supervised twin support vector machines classifier, Applied Intelligence. (2019). doi:10.1007/s10489-019-01500-w.

96.

Kumar and

Sharma, A systematic survey of facial expression recognition techniques, in: 2017 International Conference on Computing Methodologies and Communication (ICCMC), 2017, pp. 1074–1079. doi:10.1109/ICCMC.2017.8282636.

97.

Kundu and

Saravanan, Advancements and recent trends in emotion recognition using facial, in: Image Analysis and Machine Learning Models. 2017 International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT), 2017, pp. 1–6. doi:10.1109/ICEECCOT.2017.8284512.

98.

S.M.

Lajevardi, Structural similarity classifier for facial expression recognition, Signal, Image and Video Processing 8(6) (2014), 1103–1110. doi:10.1007/s11760-014-0639-2.

99.

A.C.

Le Ngo,

See and

R.C.-W.

Phan, Sparsity in dynamics of spontaneous subtle emotions: Analysis and application, IEEE Transactions on Affective Computing 8(3) (2017), 396–411. doi:10.1109/TAFFC.2016.2523996.

100.

Li,

Soladie and

Seguier, LTP-ML: Micro-expression detection by recognition of local temporal pattern of facial movements, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018, pp. 634–641. doi:10.1109/FG.2018.00100.

101.

Li,

Soladie,

Seguier,

S.-J.

Wang and

M.H.

Yap, Spotting micro-expressions on long videos sequences, in: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), 2019, pp. 1–5. doi:10.1109/FG.2019.8756626.

102.

Li,

Wang,

See and

Liu, Micro-expression recognition based on 3D flow convolutional neural network, Pattern Analysis and Applications 22(4) (2019), 1331–1339. doi:10.1007/s10044-018-0757-5.

103.

Li,

Yu,

Kurihara and

Zhan, Micro-expression analysis by fusing deep convolutional neural network and optical flow, in: 2018 5th International Conference on Control, Decision and Information Technologies (CoDIT), 2018, pp. 265–270. doi:10.1109/CoDIT.2018.8394868.

104.

Li,

Zhan,

Xu and

Wu, Facial micro-expression recognition based on the fusion of deep learning and enhanced optical flow, Multimedia Tools and Applications. (2018). doi:10.1007/s11042-018-6857-9.

105.

Li and

Deng, Blended emotion in-the-wild: Multi-label facial expression recognition using crowdsourced annotations and deep locality feature learning, International Journal of Computer Vision 127(6–7) (2019), 884–906. doi:10.1007/s11263-018-1131-1.

106.

Li,

Huang,

Li and

Wang, Automatic 4D facial expression recognition using dynamic geometrical image network, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018, pp. 24–30. doi:10.1109/FG.2018.00014.

107.

Li,

Chen and

Jin, Facial action units detection with multi-features and -AUs fusion, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 860–865. doi:10.1109/FG.2017.110.

108.

Li,

Hong,

Moilanen,

Huang,

Pfister,

Zhao and

Pietikainen, Towards reading hidden emotions: A comparative study of spontaneous micro-expression spotting and recognition methods, IEEE Transactions on Affective Computing 9(4) (2018), 563–577. doi:10.1109/TAFFC.2017.2667642.

109.

Li,

S.M.

Mavadati,

M.H.

Mahoor,

Zhao and

Ji, Measuring the intensity of spontaneous facial action units with dynamic Bayesian network, Pattern Recognition 48(11) (2015), 3417–3427. doi:10.1016/j.patcog.2015.04.022.

110.

M.Z.

Lifkooee,

Ö.M.

Soysal and

Sekeroglu, Video mining for facial action unit classification using statistical spatial–temporal feature image and LoG deep convolutional neural network, Machine Vision and Applications 30(1) (2018), 41–57. doi:10.1007/s00138-018-0967-2.

111.

D.Y.

Liliana,

Basaruddin and

M.R.

Widyanto, Mix emotion recognition from facial expression using SVM-CRF sequence classifier, in: Proceedings of the International Conference on Algorithms, Computing and Systems – ICACS’17, 2017, pp. 27–31. doi:10.1145/3127942.3127958.

112.

D.Y.

Liliana,

Basaruddin,

M.R.

Widyanto and

I.I.D.

Oriza, Fuzzy emotion: A natural approach to automatic facial expression recognition from psychological perspective using fuzzy system, Cognitive Processing. (2019). doi:10.1007/s10339-019-00923-0.

113.

Liu,

Li,

Ma and

Song, Facial expression recognition with fusion features extracted from salient facial areas, Sensors 17(4) (2017), 712. doi:10.3390/s17040712.

114.

Liu,

Li and

Deng, Boosting-POOF: Boosting part based one vs one feature for facial expression recognition in the wild, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 967–972. doi:10.1109/FG.2017.120.

115.

Lo Presti and

La Cascia, Boosting Hankel matrices for face emotion recognition and pain detection, Computer Vision and Image Understanding 156 (2017), 19–33. doi:10.1016/j.cviu.2016.10.007.

116.

Lu,

Luo,

Zheng,

Chen and

Li, A Delaunay-based temporal coding model for micro-expression recognition, in: Computer Vision – ACCV 2014 Workshops,

C.V.

Jawahar and

Shan, eds, Vol. 9009, 2015, pp. 698–711. doi:10.1007/978-3-319-16631-5-51.

117.

Ma,

Mahmoud,

Robinson,

Dias and

Skrypchuk, Automatic detection of a driver’s complex mental states, in: Computational Science and Its Applications – ICCSA 2017,

Gervasi,

Murgante,

Misra,

Borruso,

C.M.

Torre,

A.M.A.C.

Rocha and

Cuzzocrea, eds, Vol. 10406, 2017, pp. 678–691. doi:10.1007/978-3-319-62398-6-48.

118.

Mandal,

Lee and

Ouarti, Distinguishing posed and spontaneous smiles by facial dynamics, in: Computer Vision – ACCV 2016 Workshops,

C.-S.

Chen,

Lu and

K.-K.

Ma, eds, Vol. 10116, 2017, pp. 552–566. doi:10.1007/978-3-319-54407-6-37.

119.

Mansouri-Benssassi and

Ye, Bio-inspired spiking neural networks for facial expression recognition: Generalisation investigation, in: Theory and Practice of Natural Computing,

Fagan,

Martín-Vide,

O’Neill and

M.A.

Vega-Rodríguez, eds, Vol. 11324, 2018, pp. 426–437. doi:10.1007/978-3-030-04070-3-33.

120.

Marrero-Fernández,

Montoya-Padrón,

Jaume-i-Capó and

J.M.

Buades Rubio, Evaluating the research in automatic emotion recognition, IETE Technical Review 31(3) (2014), 220–232. doi:10.1080/02564602.2014.906863.

121.

Mattela and

S.K.

Gupta, Facial expression recognition using Gabor-mean-DWT feature extraction technique, in: 2018 5th International Conference on Signal Processing and Integrated Networks (SPIN), 2018, pp. 575–580. doi:10.1109/SPIN.2018.8474206.

122.

Mayya,

R.M.

Pai and

M.M.

Manohara Pai, Automatic facial expression recognition using DCNN, Procedia Computer Science 93 (2016), 453–461. doi:10.1016/j.procs.2016.07.233.

123.

Mehta,

Siddiqi and

Javaid, Facial emotion recognition: A survey and real-world user experiences in mixed reality, Sensors 18(2) (2018), 416. doi:10.3390/s18020416.

124.

Mehta and

Jadhav, Facial emotion recognition using Log Gabor filter and PCA, in: 2016 International Conference on Computing Communication Control and Automation (ICCUBEA), 2016, pp. 1–5. doi:10.1109/ICCUBEA.2016.7860054.

125.

Miao,

Dong,

J.M.A.

Jaam and

A.E.

Saddik, A deep learning system for recognizing facial expression in real-time, ACM Transactions on Multimedia Computing, Communications, and Applications 15(2) (2019), 1–20. doi:10.1145/3311747.

126.

Mou,

Gunes and

Patras, Alone versus in-a-group: A multi-modal framework for automatic affect recognition, ACM Transactions on Multimedia Computing, Communications, and Applications 15(2) (2019), 1–23. doi:10.1145/3321509.

127.

Muna,

U.D.

Rosiani,

E.M.

Yuniamo and

M.H.

Pumomo, Subpixel subtle motion estimation of micro-expressions multiclass classification, in: 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), 2017, pp. 325–330. doi:10.1109/SIPROCESS.2017.8124558.

128.

Naik and

R.P.K.

Jagannath, GCV-based regularized extreme learning machine for facial expression recognition, in: Advances in Machine Learning and Data Science,

Reddy Edla,

Lingras and

Venkatanareshbabu, eds, Vol. 705, 2018, pp. 129–138. doi:10.1007/978-981-10-8569-7-14.

129.

Natarajan and

Muthuswamy, Multi-view face expression recognition – a hybrid method, in: Artificial Intelligence and Evolutionary Algorithms in Engineering Systems,

L.P.

Suresh,

S.S.

Dash and

B.K.

Panigrahi, eds, Vol. 325, 2015, pp. 799–808. doi:10.1007/978-81-322-2135-7-84.

130.

Nguyen,

Sridharan,

Abbasnejad,

Dean and

Fookes, Meta transfer learning for facial emotion recognition, in: 2018 24th International Conference on Pattern Recognition (ICPR), 2018, pp. 3543–3548. doi:10.1109/ICPR.2018.8545411.

131.

T.-L.

Nguyen,

Kavuri and

Lee, A multimodal convolutional neuro-fuzzy network for emotion understanding of movie clips, Neural Networks 118 (2019), 208–219. doi:10.1016/j.neunet.2019.06.010.

132.

M.F.

O’Connor and

L.D.

Riek, Detecting social context: A method for social event classification using naturalistic multimodal data, in: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, pp. 1–7. doi:10.1109/FG.2015.7284843.

133.

Y.-H.

Oh,

See,

A.C.

Le Ngo,

R.C.-W.

Phan and

V.M.

Baskaran, A survey of automatic facial micro-expression analysis: Databases, Methods, and Challenges. Frontiers in Psychology 9 (2018), 1128. doi:10.3389/fpsyg.2018.01128.

134.

Patel,

Zhao and

Pietikäinen, Spatiotemporal integration of optical flow vectors for micro-expression detection, in: Advanced Concepts for Intelligent Vision Systems,

Battiato,

Blanc-Talon,

Gallo,

Philips,

Popescu and

Scheunders, eds, Vol. 9386, 2015, pp. 369–380. doi:10.1007/978-3-319-25903-1-32.

135.

M.N.

Patil,

Iyer and

Arya, Performance evaluation of PCA and ICA algorithm for facial expression recognition application, in: Proceedings of Fifth International Conference on Soft Computing for Problem Solving,

Pant,

Deep,

J.C.

Bansal,

Nagar and

K.N.

Das, eds, Vol. 436, 2016, pp. 965–976. doi:10.1007/978-981-10-0448-3-81.

136.

Peng,

Wu,

Zhang and

Chen, From macro to micro expression recognition: Deep learning on small datasets using transfer learning, in: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018, pp. 657–661. doi:10.1109/FG.2018.00103.

137.

Perikos,

Paraskevas and

Hatzilygeroudis, Facial expression recognition using adaptive neuro-fuzzy inference systems, in: 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS), 2018, pp. 1–6. doi:10.1109/ICIS.2018.8466438.

138.

Perikos,

Ziakopoulos and

Hatzilygeroudis, Recognizing emotions from facial expressions using neural network, in: Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications,

Bayro-Corrochano and

Hancock, eds, Vol. 8827, 2014, pp. 236–245. doi:10.1007/978-3-662-44654-6-23.

139.

Perikos,

Ziakopoulos and

Hatzilygeroudis, Recognize emotions from facial expressions using a SVM and neural network schema, in: Engineering Applications of Neural Networks,

Iliadis and

Jayne, eds, Vol. 517, 2015, pp. 265–274. doi:10.1007/978-3-319-23983-5-25.

140.

Piparsaniyan,

V.K.

Sharma and

K.K.

Mahapatra, Robust facial expression recognition using Gabor feature and Bayesian discriminating classifier, in: 2014 International Conference on Communication and Signal Processing, 2014, pp. 538–541. doi:10.1109/ICCSP.2014.6949900.

141.

Plutchik, The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice, American Scientist 89(4) (2001), 344–350. doi:10.1511/2001.4.344.

142.

R.K.

Prasada,

S.R.

Chandra and

H.N.

Chowdary, An integrated approach to emotion recognition and gender classification, Journal of Visual Communication and Image Representation 60 (2019), 339–345. doi:10.1016/j.jvcir.2019.03.002.

143.

Quan,

Qian and

Ren, Dynamic facial expression recognition based on K-order emotional intensity model, in: 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014), 2014, pp. 1164–1168. doi:10.1109/ROBIO.2014.7090490.

144.

Racoviteanu,

Florea,

Badea and

Vertan, Spontaneous emotion detection by combined learned and fixed descriptors, in: 2019 International Symposium on Signals, Circuits and Systems (ISSCS), 2019, pp. 1–4. doi:10.1109/ISSCS.2019.8801755.

145.

Rahdari,

Rashedi and

Eftekhari, A multimodal emotion recognition system using facial landmark analysis, Iranian Journal of Science and Technology, Transactions of Electrical Engineering 43(S1) (2019), 171–189. doi:10.1007/s40998-018-0142-9.

146.

Rao,

Li,

Zhang and

Xu, Multi-level region-based convolutional neural network for image emotion classification, Neurocomputing 333 (2019), 429–439. doi:10.1016/j.neucom.2018.12.053.

147.

T.A.

Rashid, Convolutional neural networks based method for improving facial expression recognition, in: Intelligent Systems Technologies and Applications,

J.M.

Corchado Rodriguez,

Mitra,

S.M.

Thampi and

E.-S.

El-Alfy, eds, Vol. 530, 2016, pp. 73–84. doi:10.1007/978-3-319-47952-1-6.

148.

Rathee and

Ganotra, An efficient approach for facial action unit intensity detection using distance metric learning based on cosine similarity, Signal, Image and Video Processing 12(6) (2018), 1141–1148. doi:10.1007/s11760-018-1255-3.

149.

R.P.

Reddy,

P.M.

Krishna,

Narayanan and

Lalitha, Affective state recognition using image cues, in: 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2018, pp. 928–933. doi:10.1109/ICACCI.2018.8554441.

150.

Richhariya and

Gupta, Facial expression recognition using iterative universum twin support vector machine, Applied Soft Computing 76 (2019), 53–67. doi:10.1016/j.asoc.2018.11.046.

151.

P.V.

Rouast,

Adam and

Chiong, Deep learning for human affect recognition: Insights and new developments, IEEE Transactions on Affective Computing 1(1) (2019). doi:10.1109/TAFFC.2018.2890471.

152.

Rudovic,

Pavlovic and

Pantic, Context-sensitive dynamic ordinal regression for intensity estimation of facial action units, IEEE Transactions on Pattern Analysis and Machine Intelligence 37(5) (2015), 944–958. doi:10.1109/TPAMI.2014.2356192.

153.

Saabni, Facial expression recognition using multi radial bases function networks and 2-D Gabor filters, in: 2015 Fifth International Conference on Digital Information Processing and Communications (ICDIPC), 2015, pp. 225–230. doi:10.1109/ICDIPC.2015.7323033.

154.

Sadeghi and

A.-A.

Raie, Human vision inspired feature extraction for facial expression recognition, Multimedia Tools and Applications. (2019). doi:10.1007/s11042-019-07863-z.

155.

Said,

Jemai,

Zaied and

Ben Amar, Wavelet networks for facial emotion recognition, in: 2015 15th International Conference on Intelligent Systems Design and Applications (ISDA), 2015, pp. 295–300. doi:10.1109/ISDA.2015.7489242.

156.

Sajjad,

Shah,

Jan,

S.I.

Shah,

S.W.

Baik and

Mehmood, Facial appearance and texture feature-based robust facial expression recognition framework for sentiment knowledge discovery, Cluster Computing 21(1) (2018), 549–567. doi:10.1007/s10586-017-0935-z.

157.

Salam and

Chetouani, A multi-level context-based modeling of engagement in human-robot interaction, in: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, pp. 1–6. doi:10.1109/FG.2015.7284845.

158.

M.G.

Salido Ortega,

L.-F.

Rodríguez and

J.O.

Gutierrez-Garcia, Towards emotion recognition from contextual information using machine learning, Journal of Ambient Intelligence and Humanized Computing. (2019). doi:10.1007/s12652-019-01485-x.

159.

F.Z.

Salmam,

Madani and

Kissi, Facial expression recognition using decision trees, in: 2016 13th International Conference on Computer Graphics, Imaging and Visualization (CGiV), 2016, pp. 125–130. doi:10.1109/CGiV.2016.33.

160.

F.Z.

Salmam,

Madani and

Kissi, Fusing multi-stream deep neural networks for facial expression recognition, Signal, Image and Video Processing 13(3) (2019), 609–616. doi:10.1007/s11760-018-1388-4.

161.

Samadiani,

Huang,

Cai,

Luo,

C.-H.

Chi,

Xiang and

He, A review on automatic facial expression recognition systems assisted by multimodal sensor data, Sensors 19(8) (2019), 1863. doi:10.3390/s19081863.

162.

Samara,

Galway,

Bond and

Wang, Affective state detection via facial expression analysis within a human–computer interaction context, Journal of Ambient Intelligence and Humanized Computing 10(6) (2019), 2175–2184. doi:10.1007/s12652-017-0636-8.

163.

Scarantino, How to do things with emotional expressions: The theory of affective pragmatics, Psychological Inquiry 28(2–3) (2017), 165–185. doi:10.1080/1047840X.2017.1328951.

164.

Selvaraj and

N.S.

Russel, Bimodal recognition of affective states with the features inspired from human visual and auditory perception system, International Journal of Imaging Systems and Technology. (2019). doi:10.1002/ima.22338.

165.

Sen,

Datta and

Balasubramanian, Facial emotion classification using concatenated geometric and textural features, Multimedia Tools and Applications 78(8) (2019), 10287–10323. doi:10.1007/s11042-018-6537-9.

166.

Senechal,

Bailly and

Prevost, Impact of action unit detection in automatic emotion recognition, Pattern Analysis and Applications 17(1) (2014), 51–67. doi:10.1007/s10044-012-0279-5.

167.

Shan,

Guo,

You,

Lu and

Bie, Automatic facial expression recognition based on a deep convolutional-neural-network structure, in: 2017 IEEE 15th International Conference on Software Engineering Research, Management and Applications (SERA), 2017, pp. 123–128. doi:10.1109/SERA.2017.7965717.

168.

Shao,

Gori,

Wan and

J.K.

Aggarwal, 3D dynamic facial expression recognition using low-resolution videos, Pattern Recognition Letters 65 (2015), 157–162. doi:10.1016/j.patrec.2015.07.039.

169.

Sharma,

Singh and

Gautam, Automatic facial expression recognition using combined geometric features, 3D Research 10(2) (2019). doi:10.1007/s13319-019-0224-0.

170.

Sharma,

A.S.

Jalal and

Khan, Emotion recognition using facial expression by fusing key points descriptor and texture features, Multimedia Tools and Applications 78(12) (2019), 16195–16219. doi:10.1007/s11042-018-7030-1.

171.

Shi,

Lv,

Bi and

Zhang, An improved SIFT algorithm for robust emotion recognition under various face poses and illuminations, Neural Computing and Applications. (2019). doi:10.1007/s00521-019-04437-w.

172.

Shim,

K.-H.

Cho,

K.-E.

Ko,

I.-H.

Jang and

K.-B.

Sim, Multi-tasking deep convolutional network architecture design for extracting nonverbal communicative information from a face, Cognitive Systems Research 52 (2018), 658–667. doi:10.1016/j.cogsys.2018.08.006.

173.

M.H.

Siddiqi, Accurate and robust facial expression recognition system using real-time YouTube-based datasets, Applied Intelligence 48(9) (2018), 2912–2929. doi:10.1007/s10489-017-1121-y.

174.

M.H.

Siddiqi,

Ali,

M.E.

Abdelrahman Eldib,

Khan,

Banos,

A.M.

Khan and

Choo, Evaluating real-life performance of the state-of-the-art in facial expression recognition using a novel YouTube-based datasets, Multimedia Tools and Applications 77(1) (2018), 917–937. doi:10.1007/s11042-016-4321-2.

175.

Siritanawan,

Kotani and

Chen, Independent subspace of dynamic Gabor features for facial expression classification, in: 2014 IEEE International Symposium on Multimedia, 2014, pp. 47–54. doi:10.1109/ISM.2014.48.

176.

Slimani,

Lekdioui,

Messoussi and

Touahni, Compound facial expression recognition based on highway CNN, in: Proceedings of the New Challenges in Data Sciences: Acts of the Second Conference of the Moroccan Classification Society on ZZZ–SMC’19, 2019, pp. 1–7. doi:10.1145/3314074.3314075.

177.

Song,

McDuff,

Vasisht and

Kapoor, Exploiting sparsity and co-occurrence structure for action unit recognition, in: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, pp. 1–8. doi:10.1109/FG.2015.7163081.

178.

Sönmez and

Albayrak, A facial component-based system for emotion classification, Turkish Journal of Electrical Engineering & Computer Sciences 24 (2016), 1663–1673. doi:10.3906/elk-1401-18.

179.

O.M.

Soysal,

Shirzad and

Sekeroglu, Facial action unit recognition using data mining integrated deep learning, in: 2017 International Conference on Computational Science and Computational Intelligence (CSCI), 2017, pp. 437–443. doi:10.1109/CSCI.2017.74.

180.

Starostenko,

Cruz-Perez,

Alarcon-Aquino and

Rosas-Romero, Real-time facial expression recognition using local appearance-based descriptors, Journal of Intelligent & Fuzzy Systems 36(5) (2019), 5037–5049. doi:10.3233/JIFS-179049.

181.

Stöckli,

Schulte-Mecklenbeck,

Borer and

A.C.

Samson, Facial expression analysis with AFFDEX and FACET: A validation study, Behavior Research Methods 50(4) (2018), 1446–1460. doi:10.3758/s13428-017-0996-1.

182.

Stratou and

L.-P.

Morency, MultiSense — context-aware nonverbal behavior analysis framework: A psychological distress use case, IEEE Transactions on Affective Computing 8(2) (2017), 190–203. doi:10.1109/TAFFC.2016.2614300.

183.

Stratou,

Van Der Schalk,

Hoegen and

Gratch, Refactoring facial expressions: An automatic analysis of natural occurring facial expressions in iterative social dilemma, in: 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), 2017, pp. 427–433. doi:10.1109/ACII.2017.8273635.

184.

Suja,

Tripathi and

Deepthy, Emotion recognition from facial expressions using frequency domain techniques, in: Advances in Signal Processing and Intelligent Recognition Systems,

S.M.

Thampi,

Gelbukh and

Mukhopadhyay, eds, Vol. 264, 2014, pp. 299–310. doi:10.1007/978-3-319-04960-1-27.

185.

Sultan Zia,

Hussain and

Arfan Jaffar, A novel spontaneous facial expression recognition using dynamically weighted majority voting based ensemble classifier, Multimedia Tools and Applications 77(19) (2018), 25537–25567. doi:10.1007/s11042-018-5806-y.

186.

Sumi and

Ueda, Micro-expression recognition for detecting human emotional changes, in: Novel User Experiences,

Kurosu and

H.-C.

Interaction, eds, Vol. 9733, 2016, pp. 60–70. doi:10.1007/978-3-319-39513-5-6.

187.

Sun,

Cao,

He and

Yu, Affect recognition from facial movements and body gestures by hierarchical deep spatio-temporal features and fusion strategy, Neural Networks 105 (2018), 36–51. doi:10.1016/j.neunet.2017.11.021.

188.

Sun and

A.N.

Akansu, Automatic inference of mental states from spontaneous facial expressions, in: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp. 719–723. doi:10.1109/ICASSP.2014.6853690.

189.

Talele,

Shirsat,

Uplenchwar and

Tuckley, Facial expression recognition using general regression neural network, in: 2016 IEEE Bombay Section Symposium (IBSS), 2016, pp. 1–6. doi:10.1109/IBSS.2016.7940203.

190.

B.M.S.B.

Talukder,

Chowdhury,

Howlader and

S.M.M.

Rahman, Intelligent recognition of spontaneous expression using motion magnification of spatio-temporal data, in: Intelligence and Security Informatics,

Chau,

G.A.

Wang and

Chen, eds, Vol. 9650, 2016, pp. 114–128. doi:10.1007/978-3-319-31863-9-9.

191.

Tang,

Zheng,

Yan,

Li,

Zhang and

Cui, View-independent facial action unit detection, in: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 878–882. doi:10.1109/FG.2017.113.

192.

D.L.

Tran,

Walecki,

Rudovic,

Eleftheriadis,

Schuller and

Pantic, DeepCoder: Semi-parametric variational autoencoders for automatic facial action coding, in: 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 3209–3218. doi:10.1109/ICCV.2017.346.

193.

Uçar,

Demir and

Güzeliş, A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering, Neural Computing and Applications 27(1) (2016), 131–142. doi:10.1007/s00521-014-1569-1.

194.

Van Huynh,

H.-J.

Yang,

G.-S.

Lee,

S.-H.

Kim and

I.-S.

Na, Emotion recognition by integrating eye movement analysis and facial expression model, in: Proceedings of the 3rd International Conference on Machine Learning and Soft Computing–ICMLSC 2019, 2019, pp. 166–169. doi:10.1145/3310986.3311001.

195.

Vedantham,

Settipalli and

E.S.

Reddy, Modified back propagation neural network for facial expression classification using principal component analysis and ridgelet transform, in: Computational Intelligence: Theories, Applications and Future Directions – Volume II,

N.K.

Verma and

A.K.

Ghosh, eds, Vol. 799, 2019, pp. 175–187. doi:10.1007/978-981-13-1135-2-14.

196.

Verburg and

Menkovski, Micro-expression detection in long videos using optical flow and recurrent neural networks, in: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), 2019, pp. 1–6. doi:10.1109/FG.2019.8756588.

197.

Verma,

Sexena,

Vipparthi and

Singh, QUEST: Quadriletral senary bit pattern for facial expression recognition, in: 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2018, pp. 1498–1503. doi:10.1109/SMC.2018.00260.

198.

R.-L.

Vieriu,

Tulyakov,

Semeniuta,

Sangineto and

Sebe, Facial expression recognition under a wide range of head poses, in: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, pp. 1–7. doi:10.1109/FG.2015.7163098.

199.

Vishnu Priya,

Vijayakumar and

J.M.R.S.

Tavares, MQSMER: A mixed quadratic shape model with optimal fuzzy membership functions for emotion recognition, Neural Computing and Applications. (2019). doi:10.1007/s00521-018-3940-0.

200.

Walecki,

Rudovic,

Pavlovic,

Schuller and

Pantic, Deep Structured Learning for Facial Action Unit Intensity Estimation, 2017, arXiv:1704.04481.

201.

Wang,

Liu,

Lu and

Shen, A new facial expression recognition method based on geometric alignment and LBP features, in: 2014 IEEE 17th International Conference on Computational Science and Engineering, 2014, pp. 1734–1737. doi:10.1109/CSE.2014.318.

202.

Wang,

See,

Y.-H.

Oh,

R.C.-W.

Phan,

Rahulamathavan,

H.-C.

Ling and

Li, Effective recognition of facial micro-expressions with video motion magnification, Multimedia Tools and Applications 76(20) (2017), 21665–21690. doi:10.1007/s11042-016-4079-6.

203.

C.-H.

Wu,

J.-C.

Lin and

W.-L.

Wei, Action unit reconstruction of occluded facial expression, in: 2014 International Conference on Orange Technologies, 2014, pp. 177–180. doi:10.1109/ICOT.2014.6956628.

204.

Wu,

An and

Zhang, Lite AU convolution network driven by a small amount of samples, Electronics Letters 53(14) (2017), 920–922. doi:10.1049/el.2017.0272.

205.

Xia,

Feng,

Hong and

Zhao, Spontaneous facial micro-expression recognition via deep convolutional network, in: 2018 Eighth International Conference on Image Processing Theory, Tools and Applications (IPTA), 2018, pp. 1–6. doi:10.1109/IPTA.2018.8608119.

206.

Xiao,

Wang and

Hou, Unsupervised emotion recognition algorithm based on improved deep belief model in combination with probabilistic linear discriminant analysis, Personal and Ubiquitous Computing 23(3–4) (2019), 553–562. doi:10.1007/s00779-019-01235-y.

207.

Xiaohua,

Muzi,

Lijuan,

Min,

Chunhua and

Fuji, Two-level attention with two-stage multi-task learning for facial emotion recognition, Journal of Visual Communication and Image Representation 62 (2019), 217–225. doi:10.1016/j.jvcir.2019.05.009.

208.

Xie and

Hu, Facial expression recognition using hierarchical features with deep comprehensive multipatches aggregation convolutional neural networks, IEEE Transactions on Multimedia 21(1) (2019), 211–220. doi:10.1109/TMM.2018.2844085.

209.

Xie,

Hu and

Wu, Deep multi-path convolutional neural network joint with salient region attention for facial expression recognition, Pattern Recognition 92 (2019), 177–191. doi:10.1016/j.patcog.2019.03.019.

210.

Xudong,

Huang,

Wang and

Chen, Automatic 3D facial expression recognition using geometric scattering representation, in: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, pp. 1–6. doi:10.1109/FG.2015.7163090.

211.

Xue and

Gertner, Automatic recognition of emotions from facial expressions, in: Proceedings Volume 9090, Automatic Target Recognition XXIV,

F.A.

Sadjadi and

Mahalanobis, eds, 2014. doi:10.1117/12.2057796.

212.

Yaddaden,

Adda,

Bouzouane,

Gaboury and

Bouchard, One-class and bi-class SVM classifier comparison for automatic facial expression recognition, in: 2018 International Conference on Applied Smart Systems (ICASS), 2018, pp. 1–6. doi:10.1109/ICASS.2018.8651969.

213.

Yuce,

Gao and

J.-P.

Thiran, Discriminant multi-label manifold embedding for facial action unit detection, in: 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015, pp. 1–6. doi:10.1109/FG.2015.7284871.

214.

Yurtkan and

Demirel, Entropy-based feature selection for improved 3D facial expression recognition, Signal, Image and Video Processing 8(2) (2014), 267–277. doi:10.1007/s11760-013-0543-1.

215.

Zeng,

Zhang,

Song,

Liu,

Li and

A.M.

Dobaie, Facial expression recognition via learning deep sparse autoencoders, Neurocomputing 273 (2018), 643–649. doi:10.1016/j.neucom.2017.08.043.

216.

Zhang,

Pan,

Cui,

Zhao and

Liu, Learning affective video features for facial expression recognition via hybrid deep learning, IEEE Access 1(1) (2019). doi:10.1109/ACCESS.2019.2901521.

217.

Zhang and

Ma, Learning of complicate facial expression categories, in: Proceedings of the 2019 International Conference on Image, Video and Signal Processing–IVSP 2019, 2019, pp. 73–80. doi:10.1145/3317640.3317659.

218.

Zhang,

M.H.

Mahoor and

S.M.

Mavadati, Facial expression recognition using lp-norm MKL multiclass-SVM, Machine Vision and Applications 26(4) (2015), 467–483. doi:10.1007/s00138-015-0677-y.

219.

Zhang,

Zhang and

M.A.

Hossain, Adaptive 3D facial action intensity estimation and emotion recognition, Expert Systems with Applications 42(3) (2015), 1446–1464. doi:10.1016/j.eswa.2014.08.042.

220.

Zhang,

Luo,

C.C.

Loy and

Tang, From facial expression recognition to interpersonal relation prediction, International Journal of Computer Vision 126(5) (2018), 550–569. doi:10.1007/s11263-017-1055-1.

221.

Zhao,

Mao and

Zhang, Learning deep facial expression features from image and optical flow sequences using 3D CNN, The Visual Computer 34(10) (2018), 1461–1475. doi:10.1007/s00371-018-1477-y.

222.

Zhao,

Wen-Sheng,

De la Torre,

J.F.

Cohn and

Zhang, Joint patch and multi-label learning for facial action unit detection, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 2207–2216. doi:10.1109/CVPR.2015.7298833.

223.

Zhao and

Xu, Necessary morphological patches extraction for automatic micro-expression recognition, Applied Sciences 8(10) (2018), 1811. doi:10.3390/app8101811.

224.

Zhi,

Liu and

Zhang, A comprehensive survey on automatic facial action unit analysis, The Visual Computer. (2019). doi:10.1007/s00371-019-01707-5.

225.

Zhu and

Chen, Dual-modality spatiotemporal feature learning for spontaneous facial expression recognition in e-learning using hybrid deep neural network, The Visual Computer. (2019). doi:10.1007/s00371-019-01660-3.

226.

M.S.

Zia and

M.A.

Jaffar, An adaptive training based on classification system for patterns in facial expressions using SURF descriptor templates, Multimedia Tools and Applications 74(11) (2015), 3881–3899. doi:10.1007/s11042-013-1803-3.