Complex Cyber-Physical-Human System (CPHS) integrate the human operator as an essential element to assist with different aspects of information monitoring and decision making across the system to achieve the desired goal. A crucial aspect of enhancing CPHS efficiency lies in understanding the interaction dynamics of human cognitive factors and Cyber-Physical Systems (CPS). This entails designing feedback mechanisms, reasoning processes, and compliance protocols with consideration of their psychological impacts on human operators, fostering shared awareness between humans and AI agents and calibrating feedback levels to ensure operators are informed without being overwhelmed. This study focuses on a specific CPHS scenarios involving a human operator interacting with a swarm of robots for search, rescue, and monitoring tasks. It explores the impact of swarm non-compliance rate and feedback levels from the robotic swarm on the human operators’ cognitive processes, and the extent to which individual differences in information processing influence this interaction dynamics. A human subject study with 20 participants experienced in strategic gaming involved nine scenarios with randomized robot compliance and AI feedback levels was conducted. Cognitive factors, categorized into brain-based features (mental engagement, workload, distraction) and eye-based features (pupil size, fixation, saccade, blink rate), were analyzed. Statistical analyses revealed that brain-based features, particularly mental workload, were predictive of the effect of compliance level, while feedback level and expertise affected both eye features and brain cognitive factors.
AmadoriP. V.FischerT.WangR.DemirisY. (2022). Predicting secondary task performance: A directly actionable metric for cognitive overload detection. IEEE Transactions on Cognitive and Developmental Systems, 14(4), 1474–1485. https://doi.org/10.1109/TCDS.2021.3114162
2.
BerkaC.LevendowskiD. J.CvetinovicM. M.PetrovicM. M.DavisG.LumicaoM. N.ZivkovicV. T.PopovicM. V.OlmsteadR. (2004). Real-time analysis of EEG indexes of alertness, cognition, and memory acquired with a wireless EEG headset. International Journal of Human-Computer Interaction, 17(2), 151–170. https://doi.org/10.1207/s15327590ijhc1702_3
3.
BhutoriaA. (2022). Personalized education and artificial intelligence in the United States, China, and India: A systematic review using a human-in-the-loop model. Computers and Education: Artificial Intelligence, 3, 100068. https://doi.org/10.1016/j.caeai.2022.100068
4.
BuçincaZ.MalayaM. B.GajosK. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), 1–21. https://doi.org/10.1145/3449287
5.
CamilleriA.DogramadziS.Caleb-SollyP. (2022). A study on the effects of cognitive overloading and distractions on human movement during robot-assisted dressing. Frontiers in Robotics and AI, 9, 815871. https://doi.org/10.3389/frobt.2022.815871
6.
ChenJ. Y.BarnesM. J. (2014). Human–agent teaming for multirobot control: A review of human factors issues. IEEE Transactions on Human-Machine Systems, 44(1), 13–29. https://doi.org/10.1109/THMS.2013.2293535
7.
ChenJ. Y. C.BarnesM. J.Harper-SciariniM. (2011). Supervisory control of multiple robots: Human performance issues and user-interface design. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 41(4), 435–454. https://doi.org/10.1109/TSMCC.2010.2056682
8.
ChenJ. Y. C.LakhmaniS. G.StowersK.SelkowitzA. R.WrightJ. L.BarnesM. (2018). Situation awareness-based agent transparency and human-autonomy teaming effectiveness. Theoretical Issues in Ergonomics Science, 19(3), 259–282. https://doi.org/10.1080/1463922X.2017.1315750
9.
de VisserE.ParasuramanR. (2011). Adaptive aiding of human-robot teaming: Effects of imperfect automation on performance, trust, and workload. Journal of Cognitive Engineering and Decision Making, 5(2), 209–231. https://doi.org/10.1177/1555343411410160
10.
DistefanoJ. P.ManjunathaH.ChowdhuryS.DantuK.DoermannD.EsfahaniE. T. (2021). Using physiological information to classify task difficulty in human-swarm interaction. In 2021 IEEE International Conference on Systems, Man, and Cybernetics (pp. 1198–1203). IEEE.
11.
DuanS.AyyubB. M. (2020). Assessment methods of network resilience for cyber-human-physical systems. ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems. Part A: Civil Engineering, 6(1), 03119001. https://doi.org/10.1061/AJRUA6.0001021
EndsleyM. R. (2023). Supporting Human-AI Teams: Transparency, explainability, and situation awareness. Computers in Human Behavior, 140, 107574. https://doi.org/10.1016/j.chb.2022.107574
14.
EskandarS.WangJ.RazaviS. (2020). Human-in-the-loop cyber-physical systems for construction safety. Cyber-physical Systems in the Built Environment, 161–173. https://doi.org/10.1007/978-3-030-41560-0_9
15.
FaureV.LobjoisR.BenguiguiN. (2016). The effects of driving environment complexity and dual tasking on drivers’ mental workload and eye blink behavior. Transportation Research Part F: Traffic Psychology and Behaviour, 40, 78–90. https://doi.org/10.1016/j.trf.2016.04.007
16.
GarciaM. A. R.RojasR.GualtieriL.RauchE.MattD. (2019). A human-in-the-loop cyber-physical system for collaborative assembly in smart manufacturing. Procedia CIRP, 81, 600–605. https://doi.org/10.1016/j.procir.2019.03.162
KhalafM.AyadA.TusharM. H. K.KassoufM.KundurD. (2024). A survey on cyber-physical security of active distribution networks in smart grids. IEEE Access, 12, 29414–29444. https://doi.org/10.1109/ACCESS.2024.3364362
19.
KimS. S.LiaoQ. V.VorvoreanuM.BallardS.VaughanJ. W. (2024). “ I'm not sure, but…": Examining the impact of large language models’ uncertainty expression on user reliance and trust. In The 2024 ACM Conference on Fairness, Accountability, and Transparency (pp. 822–835).
20.
LongY.WeiW.HuangT.WangY.DouQ. (2023). Human-in-the-loop embodied intelligence with interactive simulation environment for surgical robot learning. IEEE Robotics and Automation Letters, 8(8), 4441–4448. https://doi.org/10.1109/LRA.2023.3284380
21.
LyonsJ. B.HavigP. R. (2014). Transparency in a human-machine context: Approaches for fostering shared awareness/intent. In Virtual, augmented and mixed reality. Designing and developing virtual and augmented environments (pp. 181–190). Springer International Publishing.
22.
ManjunathaH.KrisshnaKumarP.DistefanoJ. P.JaniA.BehjatA.GhassemiP.ChowdhuryS.DantuK.EsfahaniE. T. (2024). SHaSTA: An open-source simulator for human and swarm team applications. Available at Research Square: https://doi.org/10.21203/rs.3.rs-4338790/v1
23.
MarquartG.CabrallC.De WinterJ. (2015). Review of eye-related measures of drivers’ mental workload. Procedia Manufacturing, 3, 2854–2861. https://doi.org/10.1016/j.promfg.2015.07.783
24.
McKendrickR.ShawT.de VisserE.SaqerH.KidwellB.ParasuramanR. (2014). Team performance in networked supervisory control of unmanned air vehicles: Effects of automation, working memory, and communication content. Human Factors, 56(3), 463–475. https://doi.org/10.1177/0018720813496269
25.
MemarA. H.EsfahaniE. T. (2018). Physiological measures for human performance analysis in human-robot teamwork: Case of tele-exploration. IEEE access, 6, 3694–3705. https://doi.org/10.1109/ACCESS.2018.2790838
26.
MhapsekarR. U.UmraniM. I.FaizanM.AliO.AbrahamL. (2024). Building trust in AI-driven decision making for cyber-physical Systems (CPS): A comprehensive review. arXiv preprint arXiv:2405.06347.
27.
NamC.LiH.LiS.LewisM.SycaraK. (2018). Trust of humans in supervisory control of swarm robots with varied levels of autonomy. In 2018 IEEE international conference on systems, man, and cybernetics (pp. 825–830). IEEE.
28.
OkoloC. T.AgarwalD.DellN.VashisthaA. (2024). “If it is easy to understand then it will have value”: Examining perceptions of explainable AI with community health workers in rural India. Proceedings of the ACM on Human-Computer Interaction, 8(CSCW1), 1–28. https://doi.org/10.1145/3637348
PratiE.PeruzziniM.PellicciariM.RaffaeliR. (2021). How to include user experience in the design of human-robot interaction. Robotics and Computer-Integrated Manufacturing, 68, 102072. https://doi.org/10.1016/j.rcim.2020.102072
31.
RecarteMÁPérezE.ConchilloÁNunesL. M. (2008). Mental workload and visual impairment: Differences between pupil, blink, and subjective rating. The Spanish Journal of Psychology, 11(2), 374–385. https://doi.org/10.1017/S1138741600004406
32.
SchemmerM.KuehlN.BenzC.BartosA.SatzgerG. (2023). Appropriate reliance on AI advice: Conceptualization and the effect of explanations. In Proceedings of the 28th International Conference on Intelligent User Interfaces (pp. 410–422).
33.
SreeramM.NofS. Y. (2021). Human-in-the-loop: Role in cyber physical agricultural systems. International Journal of Computers Communications & Control, 16, 2. https://doi.org/10.15837/ijccc.2021.2.4166
34.
SuH.QiW.SchmiranderY.OvurS. E.CaiS.XiongX. (2022). A human activity-aware shared control solution for medical human–robot interaction. Assembly Automation, 42(3), 388–394. https://doi.org/10.1108/AA-12-2021-0174
35.
VasconcelosH.JörkeM.Grunde-McLaughlinM.GerstenbergT.BernsteinM. S.KrishnaR. (2023). Explanations can reduce overreliance on AI systems during decision-making. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1), 1–38. https://doi.org/10.1145/3579605
