Measurements of Mental Stress and Safety Awareness during Human Robot Collaboration -Review

Abstract

Human-robot collaboration is an emerging research area that has gained tremendous attention in both academia and industry. Yet, the feature that human and robot sharing the workplace has led to safety concerns. In particular, the psychological states of human teammates during human-robot collaboration remains unclear but is also of great importance to workplace safety. This manuscript briefly reviewed possible direct and indirect measures that can be used to evaluate workers’ mental stress and safety awareness during human robot collaboration. It was concluded that each measure reviewed in this paper has its validity and rationality, and a combination of different methods may provide a more comprehensive and accurate assessment.

Get full access to this article

View all access options for this article.

References

Affanni

Chiorboli

(2014). Wearable instrument for skin potential response analysis in AAL applications. 20th IMEKO TC4 Symposium on Measurements of Electrical Quantities: Research on Electrical and Electronic Measurement for the Economic Upturn, Together with 18th TC4 International Workshop on ADC and DCA Modeling and Testing, IWADC 2014, 807–811.

Affanni

Antonio

Bernardini

Piras

Rinaldo

Zontone

(2018). Driver’s stress detection using Skin Potential Response signals. Measurement: Journal of the International Measurement Confederation, 122(March), 264–274. https://doi.org/10.1016/j.measurement.2018.03.040

Al-Shargie

F. M.

Tang

T. B.

Badruddin

Kiguchi

(2016). Mental stress quantification using EEG signals. IFMBE Proceedings, 56, 15–19. https://doi.org/10.1007/978-981-10-0266-3_4

Arai

Kato

Fujita

(2010). Assessment of operator stress induced by robot collaboration in assembly. CIRP Annals - Manufacturing Technology, 59(1), 5–8. https://doi.org/10.1016/j.cirp.2010.03.043

Bari

D. S.

Aldosky

H. Y. Y.

Tronstad

Kalvøy

Martinsen

(2018). Electrodermal responses to discrete stimuli measured by skin conductance, skin potential, and skin susceptance. Skin Research and Technology, 24(1), 108–116. https://doi.org/10.1111/srt.12397

Bethel

C. L.

Salomon

Murphy

R. R.

Burke

J. L.

(2007). Survey of psychophysiology measurements applied to human-robot interaction. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication, 732–737. https://doi.org/10.1109/ROMAN.2007.4415182

Borghini

Astolfi

Vecchiato

Mattia

Babiloni

(2014). Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness. Neuroscience and Biobehavioral Reviews, 44, 58–75. https://doi.org/10.1016/j.neubiorev.2012.10.003

Bueno

González-Fierro

(2013). Facial Emotion Recognition and Adaptative Postural Reaction by a Humanoid based on Neural Evolution. International Journal of …, 3(10), 481–493. http://www.ijpg.org/index.php/IJACSci/article/view/488

Catherwood

Edgar

G. K.

Nikolla

Alford

Brookes

Baker

White

(2014). Mapping brain activity during loss of situation awareness: An EEG investigation of a basis for top-down influence on perception. Human Factors, 56(8), 1428–1452. https://doi.org/10.1177/0018720814537070

10.

de Merwe

D. B.

Van Maanen

Ter Haar

F. B.

Van Dijk

R. J. E.

Hoeba

der Stap

(2019). Human-Robot Interaction During Virtual Reality Mediated Teleoperation: How Environment Information Affects Spatial Task Performance and Operator Situation Awareness. In Chen

J. Y. C.

Fragomeni

(Eds.), Virtual, Augmented and Mixed Reality. Applications and Case Studies (pp. 163–177). Springer International Publishing.

11.

Dehais

Sisbot

E. A.

Alami

Causse

(2011). Physiological and subjective evaluation of a human-robot object hand-over task. Applied Ergonomics, 42(6), 785–791. https://doi.org/10.1016/j.apergo.2010.12.005

12.

Dini

Murko

Yahyanejad

Augsdorfer

Hofbaur

Paletta

(2017). Measurement and prediction of situation awareness in human-robot interaction based on a framework of probabilistic attention. IEEE International Conference on Intelligent Robots and Systems, 2017-Septe(c), 4354–4361. https://doi.org/10.1109/IROS.2017.8206301

13.

Dragan

A. D.

Bauman

Forlizzi

Srinivasa

S. S.

(2015). Effects of robot motion on human-robot collaboration. Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, 51–58.

14.

Endsley

M. R.

(2019). A Systematic Review and Meta-Analysis of Direct Objective Measures of Situation Awareness: A Comparison of SAGAT and SPAM. Human Factors. https://doi.org/10.1177/0018720819875376

15.

Gasparetto

Zanotto

(2008). A technique for time-jerk optimal planning of robot trajectories. Robotics and Computer-Integrated Manufacturing, 24(3), 415–426. https://doi.org/10.1016/j.rcim.2007.04.001

16.

Gatsoulis

Virk

G. S.

Dehghani-Sanij

A. A.

(2010). On the Measurement of Situation Awareness for Effective Human-Robot Interaction in Teleoperated Systems. Journal of Cognitive Engineering and Decision Making, 4(1), 69–98. https://doi.org/10.1518/155534310X495591

17.

Heinzmann

Zelinsky

(2003). Quantitative safety guarantees for physical human-robot interaction. The International Journal of Robotics Research, 22(7–8), 479–504.

18.

Höfling

T. T. A.

Gerdes

A. B. M.

Föhl

Alpers

G. W.

(2020). Read My Face: Automatic Facial Coding Versus Psychophysiological Indicators of Emotional Valence and Arousal. In Frontiers in Psychology (Vol. 11). https://doi.org/10.3389/fpsyg.2020.01388

19.

International Organization for Standardization (ISO). (2016). Robots and robotic devices -- Collaborative robots (ISO/TS 15066:2016).

20.

Kaber

D. B.

Endsley

M. R.

(1998). Team situation awareness for process control safety and performance. Process Safety Progress, 17(1), 43–48. https://doi.org/10.1002/prs.680170110

21.

Kästle

J. L.

Anvari

Krol

Wurdemann

H. A.

(2021). Correlation between Situational Awareness and EEG signals. Neurocomputing, 432, 70–79. https://doi.org/10.1016/j.neucom.2020.12.026

22.

Kokabe

Shibata

Yamamoto

(2008). Modeling of handing motion reflecting emotional state and its application to robots. Proceedings of the SICE Annual Conference, 495–501. https://doi.org/10.1109/SICE.2008.4654706

23.

Kulić

Croft

E. A.

(2003). Estimating Intent for Human Robot Interaction. IEEE International Conference on Advanced Robotics, 810–815.

24.

Lerner

J. S.

Dahl

R. E.

Hariri

A. R.

Taylor

S. E.

(2007). Facial Expressions of Emotion Reveal Neuroendocrine and Cardiovascular Stress Responses. Biological Psychiatry, 61(2), 253–260. https://doi.org/10.1016/j.biopsych.2006.08.016

25.

Mauss

I. B.

Robinson

M. D.

(2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209–237.

26.

Michalos

Makris

Tsarouchi

Guasch

Kontovrakis

Chryssolouris

(2015). Design considerations for safe human-robot collaborative workplaces. Procedia CIrP, 37, 248–253.

27.

Mollahosseini

Hasani

Mahoor

M. H.

(2017). Affectnet: A database for facial expression, valence, and arousal computing in the wild. ArXiv Preprint ArXiv:1708.03985.

28.

C. K. L.

Duffy

V. G.

Cheung

C. C.

(2009). Perception of safe robot idle time in virtual reality and real industrial environments. International Journal of Industrial Ergonomics, 39(5), 807–812. https://doi.org/10.1016/j.ergon.2009.01.003

29.

Orguc

Khurana

H. S.

Stankovic

K. M.

Leel

H. S.

Chandrakasan

A. P.

(2018). EMG-based Real Time Facial Gesture Recognition for Stress Monitoring. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, 2018-July, 2651–2654. https://doi.org/10.1109/EMBC.2018.8512781

30.

OSHA accident report 202475737. (2009). https://www.osha.gov/pls/imis/accidentsearch.accident_detail?id=202475737

31.

Pang

Deng

Wang

Zhang

Pang

Yang

(2018). Development of flexible robot skin for safe and natural human-robot collaboration. Micromachines, 9(11), 1–15. https://doi.org/10.3390/mi9110576

32.

Park

Kim

(2018). Wearable Photoplethysmography Sensor. TENCON 2018 - 2018 IEEE Region 10 Conference, October, 1899–1902.

33.

Pourmohammadi

Maleki

(2020). Stress detection using ECG and EMG signals: A comprehensive study. Computer Methods and Programs in Biomedicine, 193. https://doi.org/10.1016/j.cmpb.2020.105482

34.

Rani

Sarkar

Adams

(2007). Anxiety-based affective communication for implicit human-machine interaction. Advanced Engineering Informatics, 21(3), 323–334. https://doi.org/10.1016/j.aei.2006.11.009

35.

Rani

Sims

Brackin

Sarkar

(2002). Online stress detection using psychophysiological signals for implicit human-robot cooperation. Robotica, 20(6), 673–685. https://doi.org/10.1017/S0263574702004484

36.

Safta

Grigore

(2011). Emotion Detection Using Psycho- P hysiological Signal Processing. 2–5.

37.

Saidatul

Paulraj

M. P.

Yaacob

Yusnita

M. A.

(2011). Analysis of EEG signals during relaxation and mental stress condition using AR modeling techniques. Proceedings - 2011 IEEE International Conference on Control System, Computing and Engineering, ICCSCE 2011, 477–481. https://doi.org/10.1109/ICCSCE.2011.6190573

38.

Salmon

Stanton

P. N.

Walker

Green

(2006). Situation Awareness Measurement-A review of applicability.pdf. Applied Ergonomics, 37(2), 225–238.

39.

Sauppé

Mutlu

(2015). The social impact of a robot co-worker in industrial settings. Conference on Human Factors in Computing Systems - Proceedings, 2015-April, 3613–3622. https://doi.org/10.1145/2702123.2702181

40.

Schuster

Keebler

J. R.

Zuniga

Jentsch

(2012). Individual differences in SA measurement and performance in human-robot teaming. 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support, CogSIMA 2012, 187–190. https://doi.org/10.1109/CogSIMA.2012.6188378

41.

Sneddon

Mearns

Flin

(2013). Stress, fatigue, situation awareness and safety in offshore drilling crews. Safety Science, 56, 80–88. https://doi.org/10.1016/j.ssci.2012.05.027

42.

Stanton

N. A.

Piggott

(2017). Situational awareness and safety Situational Awareness and Safety. 7535(December 2001), 189–204. https://doi.org/10.1016/S0925-7535(01)00010-8

43.

Teplan

(2002). Fundamental of EEG Measurement. MEASUREMENT SCIENCE REVIEW, 2.

44.

Unhelkar

V. V.

Siu

H. C.

Shah

J. A.

(2014). Comparative performance of human and mobile robotic assistants in collaborative fetch-anddeliver tasks. ACM/IEEE International Conference on Human-Robot Interaction, 82–89. https://doi.org/10.1145/2559636.2559655

45.

Villani

Pini

Leali

Secchi

(2018). Survey on human–robot collaboration in industrial settings: Safety, intuitive interfaces and applications. Mechatronics, 55(February), 248–266. https://doi.org/10.1016/j.mechatronics.2018.02.009

46.

Wijsman

Grundlehner

Penders

Hermens

(2013). Trapezius muscle EMG as predictor of mental stress. Transactions on Embedded Computing Systems, 12(4), 1–20. https://doi.org/10.1145/2485984.2485987

47.

Yang

Jiang

Sun

Tong

(2010). Cortical synchrony change under mental stress due to time pressure. Proceedings -2010 3rd International Conference on Biomedical Engineering and Informatics, BMEI 2010, 5(Bmei), 2004–2007. https://doi.org/10.1109/BMEI.2010.5639664

48.

Yeo

L. G.

Sun

Liu

Trapsilawati

Sourina

Chen

C. H.

Mueller-Wittig

Ang

W. T.

(2017). Mobile EEG-based situation awareness recognition for air traffic controllers. 2017 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2017, 2017-Janua, 3030–3035. https://doi.org/10.1109/SMC.2017.8123090

49.

Zhang

Yang

Liang

Pitts

B. J.

Prakah-Asante

K. O.

Curry

Duerstock

B. S.

Wachs

J. P.

(2020). Physiological Measurements of Situation Awareness: A Systematic Review. Human Factors, 3, 1–22. https://doi.org/10.1177/0018720820969071