A Longitudinal Study Investigating the Effects of Workload and Exposure on 3D Gestural Human Computer Interaction

Abstract

3D gestural input technology has the ability to expand human-computer interaction (HCI) beyond traditional input modalities. It is known that context and domain expertise are influential to gesture development, but there is little known about other individual factors such as workload and exposure. Therefore, the objective of this work is to explore the effects of workload and exposure on intuitive gesture choice and reaction time under a general HCI context. A longitudinal study was conducted to investigate the differences in intuitive mappings for high and low workload conditions as well as across three separate experimental sessions. There were no differences in the intuitive mappings for either workload conditions or different experimental sessions. However, there was a difference in reaction times between all experimental sessions indicating there was a learning effect from the first to the last session in that the participants became faster in generating intuitive mappings.

Get full access to this article

View all access options for this article.

References

Anderson

Bischof

W. F.

(2013). Learning and performance with gesture guides. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1109–1118). ACM.

Ardito

Costabile

M. F.

Jetter

H.-C.

(2014). Gestures that people can understand and use. Journal of Visual Languages & Computing, 25(5), 572–576.

Bates

Mächler

Bolker

Walker

(2014). Fitting linear mixed- effects models using lme4. Journal of Statistical Software, 67(1).

Baudel

Beaudouin-Lafon

(1993). Charade: remote control of objects using free-hand gestures. Communications of the ACM, 36(7), 28–35.

Hinckley

Jacob

R. J. K.

Ware

Wobbrock

J. O.

Wigdor

(2014). Input/Output Devices and Interaction Techniques.

Hothorn

Bretz

Westfall

(2008). Simultaneous inference in general parametric models. Biometrical Journal, 50(3), 346–363.

Huang

Zhou

(2015). Sign language recognition using real-sense. 2015 IEEE China Summit and International Conference on Signal and Information Processing (ChinaSIP). https://doi.org/10.1109/ChinaSIP.2015.7230384

Jacob

M. G.

Wachs

J. P.

(2014). Context-based hand gesture recognition for the operating room. Pattern Recognition Letters, 36, 196–203.

Jacob

M. G.

Wachs

J. P.

Packer

R. A.

(2013). Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images. Journal of the American Medical Informatics Association, 20(1), 183–186.

10.

Jurewicz

K. A.

Neyens

D. M.

(2017). Mapping 3D Gestural Inputs to Traditional Touchscreen Interface Designs within the Context of Anesthesiology. In Proceedings of the Human Factors and Ergonomics Society 61th Annual Meeting. Austin, TX. https://doi.org/10.1177/1541931213601660

11.

Jurewicz

K. A.

Neyens

D. M.

Catchpole

Reeves

S. T.

(2018). Developing a 3D gestural interface for anesthesia-related human-computer interaction tasks using both experts and novices. Human Factors, 60(7), 992–1007. https://doi.org/10.1177/0018720818780544

12.

Meyer

Dimitriadou

Hornik

Weingessel

Leisch

(2015). e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien, 2015. R Package Version, 1–6.

13.

Nielsen

Störring

Moeslund

Granum

(2004). A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for HCI. In Camurri

Volpe

(Eds.), Gesture-Based Communication in Human-Computer Interaction SE - 38 (vol. 2915, pp. 409–420). Springer Berlin Heidelberg.

14.

Nieuwenhuis

te Grotenhuis

H. F.

Pelzer

B. J.

(2012). Influence. ME: tools for detecting influential data in mixed effects models.

15.

Pickering

C. A.

Burnham

K. J.

Richardson

M. J.

(2007). A research study of hand gesture recognition technologies and applications for human vehicle interaction. In 3rd Conference on Automotive Electronics.

16.

Pisharady

P. K.

Saerbeck

(2015). Recent methods and databases in vision-based hand gesture recognition: A review. Computer Vision and Image Understanding, 141, 152–165. https://doi.org/https://doi.org/10.1016/j.cviu.2015.08.004

17.

Rautaray

S. S.

Agrawal

(2015). Vision based hand gesture recognition for human computer interaction: a survey. Artificial Intelligence Review, 43(1), 1–54. https://doi.org/10.1007/s10462-012-9356-9

18.

Stern

H. I.

Wachs

J. P.

Edan

(2008). Optimal Consensus Intuitive Hand Gesture Vocabulary Design. 2008 IEEE International Conference on Semantic Computing. https://doi.org/10.1109/ICSC.2008.29

19.

Wachs

J. P.

Kölsch

Stern

Edan

(2011). Vision-based hand-gesture applications. Communications of the ACM, 54(2), 60–71.

20.

Wigdor

Wixon

(2011). Brave NUI world: designing natural user interfaces for touch and gesture. Elsevier.