Sage Journals: Discover world-class research

Abstract

The work described in this paper attempts to contribute to one of the most stimulating and promising sectors in the field of emotion recognition, which is health care. Multidisciplinary studies in artificial intelligence, augmented reality, and psychology stressed out the importance of emotions in communication and awareness. The intent is the recognition of human emotions, processing images streamed in real-time from a mobile device. The proposed techniques involve the use of open source libraries of visual recognition and machine learning approaches based on convolutional neural networks (CNN).

Keywords

Image recognition emotion recognition face detection machine learning convolutional neural networks

Get full access to this article

View all access options for this article.

References

Alkuhlani,

Nassef and

Farag, Multistage feature selection approach for high-dimensional cancer data, Soft Computing21(22) (2017), 6895–6906. doi:10.1007/s00500-016-2439-9.

Angelov,

Gu,

J.A.

Iglesias,

Ledezema,

Sanchis,

Sipele and

Ramezani, Cybernetics of the mind, learning individual’s perceptions autonomously, IEEE Systems, Man, and Cybernetics Magazine3(2) (2017), 6–17. doi:10.1109/MSMC.2017.2664478. ISSN: 2333-942X.

Bastianelli,

Nardi,

L.C.

Aiello,

Giacomelli and

Manes, Speaky for robots: The development of vocal interfaces for robotic applications, Applied Intelligence44(1) (2016), 43–66. doi:10.1007/s10489-015-0695-5.

Bertola and

Patti, Emotional responses to artworks in online collections, UMAP Workshops Proceedings997 (2013).

Biondi,

Franzoni,

Li and

Milani, Web-based similarity for emotion recognition in web objects, in: Proceedings of the 9th International Conference on Utility and Cloud Computing, UCC 2016, Shanghai, China, December 6–9, 2016, pp. 327–332. doi:10.1145/2996890.3007883.

Caffe Framework: Github: https://github.com/BVLC/caffe, last visited on Sept 12, 2018.

Canossa,

Badler,

M.S.

El-Nasr and

Anderson, Eliciting emotions in design of games – a theory driven approach, CEUR Workshop Proceedings1680 (2016), 34–40.

Chmiel,

Sienkiewicz,

Thelwall,

Paltoglou,

Buckley,

Kappas and

J.A.

Hołyst, Collective emotions online and their influence on community life, PLoS ONE6(7) (2011), e22207. doi:10.1371/journal.pone.0022207.

Cieliebak,

Dürr and

Uzdilli, Potential and limitations of commercial sentiment detection tools, in: CEUR Workshop Proceedings, Vol. 1096, 2013, pp. 47–58.

10.

T.F.

Cootes,

C.J.

Taylor,

D.H.

Cooper and

Graham, Active shape models-their training and application, Computer Vision and Image Understanding61(1) (1995), 38–59. doi:10.1006/cviu.1995.1004.

11.

Cui,

Du,

Shen,

Zhou and

Li, Personalized microblog recommendation using sentimental features, in: 2017 IEEE International Conference on Big Data and Smart Computing, BigComp 2017, Art. No. 7881756, 2017, pp. 455–456. doi:10.1109/BIGCOMP.2017.7881756.

12.

Eveno,

Caplier and

P.Y.

Coulon, Accurate and quasi-automatic lip tracking, IEEE Transactions on Circuits and Systems for Video Technology14(5) (2004), 706–715. doi:10.1109/TCSVT.2004.826754.. ISSN:1051-8215.

13.

Franzoni, Autonomous hexapod robot with artificial vision and remote control by myo-electric gestures, Cyber-Physical Systems for Next-Generation Networks (2018), 143–162. doi:10.4018/978-1-5225-5510-0.ch007.

14.

Franzoni,

Chiancone and

Milani, A multistrain bacterial diffusion model for link prediction, International Journal of Pattern Recognition and Artificial Intelligence31(11) (2017). doi:10.1142/S0218001417590248.

15.

Franzoni and

Gervasi, Guidelines for web usability and accessibility on the Nintendo Wii, in: Transactions in Computer Science VI, Lecture Notes in Computer Science, Vol. 5730, 2009, pp. 19–40. doi:10.1007/978-3-642-10649-1_2.

16.

Franzoni,

Milani,

Pallottelli,

C.H.C.

Leung and

Li, Context-based image semantic similarity, in: 12th International Conference on Fuzzy Systems and Knowledge Discovery, FSKD 2015, Art. No. 7382127, 2015, pp. 1280–1284.

17.

Franzoni,

Milani and

Vallverdu, Emotional affordances in human-machine interactive planning and negotiation, in: Proceedings of WI 2017, Workshop on Affective Computing and Emotion Recognition (ACER), 2017, pp. 924–930. doi:10.1145/3106426.3109421.

18.

Franzoni,

Poggioni and

Zollo, Automated classification of book blurbs according to the emotional tags of the social network, in: Zazie CEUR Workshop Proceedings, Vol. 1096, 2013, pp. 83–94. doi:10.13140/RG.2.1.3194.7689.

19.

Gervasi,

Magni and

Ferri, A method for predicting words by interpreting labial movements, in: Lecture Notes in Computer Science, ICCSA 2016, Vol. 9787, Springer, Beijing (China)2016, pp. 450–464. doi:10.1007/978-3-319-42108-7_34.

20.

Gervasi,

Magni and

Zampolini, Nu!RehaVR: Virtual reality in neuro tele-rehabilitation of patients with traumatic brain injury and stroke, Virtual Reality14(2) (2010), 131–141. doi:10.1007/s10055-009-0149-7.

21.

Gervasi,

Russo and

Vella, The AES implantation based on OpenCL for multi/many core architecture, in: 2010 International Conference on Computational Science and Its Applications, Fukuoka, ICCSA 2010, Washington, DC, USA, IEEE Computer Society, 2010, pp. 129–134. doi:10.1109/ICCSA.2010.44.

22.

G.R.

Hayes,

Hirano,

Marcu,

Monibi,

D.H.

Nguyen and

Yeganyan, Interactive visual supports for children with autism, Personal and Ubiquitous Computing14(7) (2010), 663–680. doi:10.1007/s00779-010-0294-8.

23.

Karayev,

Trentacoste and

Han, Recognizing Image Style, University of California Barkley Adobe, 2014, arXiv:1311.3715 [cs.CV].

24.

Krizhevsky,

Sutskever and

Hinton, Imagenet classification with deep convolutional neural networks, in: 25th International Conference on Advance in Neural Information Processing System, 2012, pp. 1106–1114.

25.

LeCun,

Bengio and

Hinton, Deep learning, Nature521 (2015), 436–444. doi:10.1038/nature14539.

26.

V.I.

Levenshtein, Binary codes capable of correcting deletions, insertions and reversals, Cibernetics and Control Theory, Soviet Physics Doklady10 (1966), 707–710.

27.

T.W.

Lewis and

D.M.W.

Powers, Lip contour detection techniques based on front view of face, Journal of Global Research in Computer Science2(5) (2011), 43–46ISSN: 2229-371X.

28.

Lombardi,

W.R.

Butt and

Grecuccio, Lip tracking towards an automatic lip reading approach, JMPT5 (2014), 1–11. doi:10.4236/jmp.2014.51001.

29.

Lou,

Wu,

R.-D.

Vatavu and

W.T.

Tsai, Personalized gesture interactions for cyber-physical smart-home environments, Science China Information Sciences60(7) (2017). doi:10.1007/s11432-015-1014-7.

30.

P.C.

Mahalanobis, On the generalised distance in statistics,in, Proceedings of the National Institute of Sciences of India2(1) (1936), 49–55.

31.

Mariotti,

Gervasi,

Vella,

Cuzzocrea and

Costantini, Strategies and systems towards grids and clouds integration: A DBMS-based solution, Future Generation Computer Systems88 (2018), 718–729. doi:10.1016/j.future.2017.02.047.

32.

Neumann,

Patricia,

Garnett and

Kersting, Efficient graph kernels by randomization, in: Lecture Notes in Computer Science, LNAI (PART 1), Vol. 7523, (2012), 378–393. doi:10.1007/978-3-642-33460-3_30.

33.

Pantelopoulos and

N.G.

Bourbakis, A survey on wearable sensor-based systems for health monitoring and prognosis, IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews40(1) (2010), 5306098.

34.

V.M.

Patel,

Gopalan,

Li, Chellappa and

Visual, Domain adaptation: A survey of recent advances, IEEE Signal Processing Magazine32(3) (2015), 53–69. doi:10.1109/MSP.2014.2347059.

35.

K.-C.

Peng,

Chen,

Sadovnik and

Gallagher, A mixed bag of emotions: Model, predict, and transfer emotion distributions, in: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern, 2015, pp. 860–868.

36.

Phelan,

Arden,

Garcia and

Roast, Exploring virtual reality and prosthetic training, in: 2015 IEEE Virtual Reality Conference, VR 2015 – Proceedings, Art. No. 7223441, 2015, pp. 353–354.

37.

R.W.

Picard, Affective computing: Challenges, International Journal of Human Computer Studies59(1–2) (2003), 55–64. doi:10.1016/S1071-5819(03)00052-1.

38.

R.W.

Picard,

Vyzas and

Healey, Toward machine emotional intelligence: Analysis of affective physiological state, IEEE Transactions on Pattern Analysis and Machine Intelligence23(10) (2001), 1175–1191. doi:10.1109/34.954607.

39.

Riganelli,

Franzoni,

Gervasi and

Tasso, EmEx, a tool for automated emotive face recognition using convolutional neural networks, in: ICCSA 2017, Workshop on Emotion Recognition, Lecture Notes in Computer Science, Vol. 10406, 2017, pp. 692–704. doi:10.1007/978-3-319-62398-6_49.

40.

Saeed and

J.-L.

Dugelay, Combining edge detection and region segmentation for lip contour extraction, in: AMDO, 2010, 6th Conference on Articulated Motion and Deformable Objects, Andratx, Mallorca, Spain, July 7–9, 2010, Also published in LNCS, Volume 6169, pp.11–20, 2010.

41.

Saif,

Fernandez,

He and

Alani, Evaluation datasets for Twitter sentiment analysis a survey and a new dataset, the STS-Gold, in: CEUR Workshop Proceedings, Vol. 1096, 2013, pp. 9–21.

42.

Stephane and

Luc, The M2VTS multimodal face database (release 1.00) audio- and video-based biometric person authentication, in: First International Conference, AVBPA’97 Crans-Montana, Proceedings, Switzerland, March 12–14, 1997, Springer, Berlin Heidelberg1997, pp. 403–409. ISBN 978-3-540-68425-1. doi:10.1007/BFb0016021.

43.

Stiefelhagen,

Yang and

Waibel, A model-based gaze tracking system, International Journal of Artificial Intelligence Tools6(2) (1997), 193–209. doi:10.1142/S0218213097000116.

44.

Tasso,

Pallottelli,

Rui and

Laganá, Learning objects efficient handling in a federation of science distributed repositories, in: Lecture Notes in Computer Science, LNCS (PART 1), Vol. 8579, 2014, pp. 615–626. doi:10.1007/978-3-319-09144-0_42.

45.

Vella,

Neri,

Gervasi and

Tasso, A simulation framework for scheduling performance evaluation on CPU-GPU heterogeneous system, in: Lecture Notes in Computer Science, ICCSA 2012, Vol. 7336, Springer, 2012, pp. 457–469. doi:10.1007/978-3-642-31128-4_34.

46.

Voirin,

Payet,

Barrat,

Cattuto,

Khanafer,

Regis,

B.-A.

Kim,

Comte,

J.-S.

Casalegno,

Lina and

Vanhems, Combining high-resolution contact data with virological data to investigate influenza transmission in a Tertiary care hospital, Infection Control and Hospital Epidemiology36(3) (2015), 254–260. doi:10.1017/ice.2014.53.

Automating facial emotion recognition

Abstract

Keywords

Get full access to this article

References