Developing a 3D Gestural Interface for Anesthesia-Related Human-Computer Interaction Tasks Using Both Experts and Novices

Abstract

Objective:

The purpose of this research was to compare gesture-function mappings for experts and novices using a 3D, vision-based, gestural input system when exposed to the same context of anesthesia tasks in the operating room (OR).

Background:

3D, vision-based, gestural input systems can serve as a natural way to interact with computers and are potentially useful in sterile environments (e.g., ORs) to limit the spread of bacteria. Anesthesia providers’ hands have been linked to bacterial transfer in the OR, but a gestural input system for anesthetic tasks has not been investigated.

Methods:

A repeated-measures study was conducted with two cohorts: anesthesia providers (i.e., experts) (N = 16) and students (i.e., novices) (N = 30). Participants chose gestures for 10 anesthetic functions across three blocks to determine intuitive gesture-function mappings. Reaction time was collected as a complementary measure for understanding the mappings.

Results:

The two gesture-function mapping sets showed some similarities and differences. The gesture mappings of the anesthesia providers showed a relationship to physical components in the anesthesia environment that were not seen in the students’ gestures. The students also exhibited evidence related to longer reaction times compared to the anesthesia providers.

Conclusion:

Domain expertise is influential when creating gesture-function mappings. However, both experts and novices should be able to use a gesture system intuitively, so development methods need to be refined for considering the needs of different user groups.

Application:

The development of a touchless interface for perioperative anesthesia may reduce bacterial contamination and eventually offer a reduced risk of infection to patients.

Keywords

gestures human-computer interaction expertise anesthesiology and perioperative care

Get full access to this article

View all access options for this article.

References

Aigner

Wigdor

Benko

Haller

Lindbauer

Ion

. . . Koh

(2012). Understanding mid-air hand gestures: A study of human preferences in usage of gesture types for HCI. Microsoft Research TechReport MSR-TR-2012-111. Retrieved from https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/GesturesTR-20121107-RoA.pdf

Ardito

Costabile

M. F.

Jetter

H.-C.

(2014). Gestures that people can understand and use. Journal of Visual Languages & Computing, 25(5), 572–576.

Bates

Mächler

Bolker

Walker

(2014). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48.

Baudel

Beaudouin-Lafon

(1993). Charade: Remote control of objects using free-hand gestures. Communications of the ACM, 36(7), 28–35.

Betza

S. M.

Jurewicz

K. A.

Neyens

D. M.

Riggs

S. L.

Abernathy

J. H.

Reeves

S. T.

(2016). Anesthesia maintenance and vigilance. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (pp. 608–612). Santa Monica, CA: Human Factors and Ergonomics Society.

Biddle

Shah

(2012). Quantification of anesthesia providers’ hand hygiene in a busy metropolitan operating room: What would Semmelweis think? American Journal of Infection Control, 40(8), 756–759.

Birnbach

D. J.

Rosen

L. F.

Fitzpatrick

Carling

Munoz-Price

L. S.

(2015). The use of a novel technology to study dynamics of pathogen transmission in the operating room. Anesthesia & Analgesia, 120(4), 844–847.

Bizzotto

Costanzo

Bizzotto

Regis

Sandri

Magnan

(2014). Leap motion gesture control with OsiriX in the operating room to control imaging: First experiences during live surgery. Surgical Innovation, 1, 655–666.

Efron

(1972). Gesture, race and culture. The Hague, the Netherlands: Mouton.

10.

Freedman

(1972). The analysis of movement behavior during the clinical interview. In Siegman

A. W.

Pope

(Eds.), Studies in Dyadic Communication (pp. 153–175). New York, NY: Pergamon.

11.

Freeman

Benko

Morris

M. R.

Wigdor

(2009). ShadowGuides: Visualizations for in-situ learning of multi-touch and whole-hand gestures. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (pp. 165–172). New York, NY: ACM.

12.

Galitz

W. O.

(2007). The essential guide to user interface design: An introduction to GUI design principles and techniques (3rd ed.). New York, NY: Wiley.

13.

Horsky

Kaufman

D. R.

Oppenheim

M. I.

Patel

V. L.

(2003). A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics, 36(1), 4–22.

14.

Höysniemi

Hämäläinen

Turkki

(2004). Wizard of Oz prototyping of computer vision based action games for children. In Proceedings of the 2004 Conference on Interaction Design and Children: Building a Community (pp. 27–34). New York, NY: ACM.

15.

Intel Corporation. (2016). Handling gestures. Intel® RealSense^TM SDK 2016 R2 documentation. Retrieved from https://software.intel.com/sites/landingpage/realsense/camera-sdk/v1.1/documentation/html/index.html?doc_devguide_whats_new_in_sdk.html

16.

Jacob

M. G.

Wachs

J. P.

(2014). Context-based hand gesture recognition for the operating room. Pattern Recognition Letters, 36, 196–203. Retrieved from https://doi.org/10.1016/j.patrec.2013.05.024

17.

Jacob

M. G.

Wachs

J. P.

Packer

R. A.

(2013). Hand-gesture-based sterile interface for the operating room using contextual cues for the navigation of radiological images. Journal of the American Medical Informatics Association, 20(1), 183–186.

18.

Karam

Schraefel

M. C.

(2005). A taxonomy of gestures in human computer interactions. Southampton, UK: University of Southampton, Computing Service.

19.

Kendon

(1988). How gestures can become like words. Cross-Cultural Perspectives in Nonverbal Communication, 1, 131–141.

20.

Loftus

R. W.

Brown

J. R.

Koff

M. D.

Reddy

Heard

S. O.

Patel

H. M.

. . . Jensen

J. T.

(2012). Multiple reservoirs contribute to intraoperative bacterial transmission. Anesthesia & Analgesia, 114(6), 1236–1248.

21.

Loftus

R. W.

Koff

M. D.

Burchman

C. C.

Schwartzman

J. D.

Thorum

Read

M. E.

. . . Beach

M. L.

(2008). Transmission of pathogenic bacterial organisms in the anesthesia work area. Anesthesiology, 109(3), 399–407. Retrieved from http://dx.doi.org/10.1097/ALN.0b013e318182c855

22.

Loftus

R. W.

Muffly

M. K.

Brown

J. R.

Beach

M. L.

Koff

M. D.

Corwin

H. L.

. . . Yeager

M. P.

(2011). Hand contamination of anesthesia providers is an important risk factor for intraoperative bacterial transmission. Anesthesia & Analgesia, 112(1), 98–105.

23.

Maggioni

Kämmerer

(1998). GestureComputer—History, design and applications. In Cipolla

Pentland

(Eds.), Computer vision for human-machine interaction (pp. 23–52). Cambridge, UK: Cambridge University Press.

24.

McNeill

(1992). Hand and mind: What gestures reveal about thought. Chicago, IL: University of Chicago Press.

25.

Mewes

Saalfeld

Riabikin

Skalej

Hansen

(2016). A gesture-controlled projection display for CT-guided interventions. International Journal of Computer Assisted Radiology and Surgery, 11(1), 157–164.

26.

Miller

R. D.

Pardo

(2011). Basics of anesthesia (6th ed.). Philadelphia, PA: Elsevier.

27.

Morris

M. R.

Wobbrock

J. O.

Wilson

A. D.

(2010). Understanding users’ preferences for surface gestures. In Proceedings of graphics interface 2010 (pp. 261–268). Mississauga: Canadian Information Processing Society.

28.

Munoz-Price

L. S.

Lubarsky

D. A.

Arheart

K. L.

Prado

Cleary

Fajardo-Aquino

. . . Birnbach

D. J.

(2013). Interactions between anesthesiologists and the environment while providing anesthesia care in the operating room. American Journal of Infection Control, 41(10), 922–924. Retrieved from https://doi.org/http://dx.doi.org/10.1016/j.ajic.2013.03.306

29.

Nielsen

Störring

Moeslund

Granum

(2004). A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In Camurri

Volpe

(Eds.), Gesture-based communication in human-computer interaction SE-38 (Vol. 2915, pp. 409–420). Heidelberg, Germany: Springer Berlin.

30.

O’Hara

Gonzalez

Sellen

Penney

Varnavas

Mentis

. . . Dastur

(2014). Touchless interaction in surgery. Communications of the ACM, 57(1), 70–77.

31.

Oxford Grice

Vogel

K. A.

Mitchell

Muniz

Vollmer

M. A.

(2003). Adult norms for a commercially available nine hole peg test for finger dexterity. American Journal of Occupational Therapy, 57(5), 570–573. Retrieved from http://dx.doi.org/10.5014/ajot.57.5.570

32.

Pereira

Wachs

J. P.

Park

Rempel

(2015). A user-developed 3-D hand gesture set for human-computer interaction. Human Factors: The Journal of the Human Factors and Ergonomics Society, 57(4), 607–621.

33.

Rowlands

Yeager

M. P.

Beach

Patel

H. M.

Huysman

B. C.

Loftus

R. W.

(2014). Video observation to map hand contact and bacterial transmission in operating rooms. American Journal of Infection Control, 42(7), 698–701.

34.

Schneiderman

Plaisant

(2003). Designing the user interface (4th ed.). New York, NY: Pearson Education.

35.

Seagull

J. F.

Sanderson

P. M.

(2001). Anesthesia alarms in context: An observational study. Human Factors, 43, 66–78. Retrieved from https://doi.org/10.1518/001872001775992453

36.

Stern

H. I.

Wachs

J. P.

Edan

(2006). Human factors for design of hand gesture human-machine interaction. IEEE International Conference on Systems Man and Cybernetics, 5, 4052–4056. Retrieved from http://web.ics.purdue.edu/~jpwachs/papers/smc06_human.pdf

37.

Stern

H. I.

Wachs

J. P.

Edan

(2008). Optimal consensus intuitive hand gesture vocabulary design. 2008 IEEE International Conference on Semantic Computing. Retrieved from https://doi.org/10.1109/ICSC.2008.29

38.

Wachs

J. P.

Stern

H. I.

Edan

Gillam

Handler

Feied

Smith

(2008). A gesture-based tool for sterile browsing of radiology images. Journal of the American Medical Informatics Association, 15(3), 321–323.

39.

Wickham

(2009). ggplot2: Elegant graphics for data analysis. New York, NY: Springer-Verlag. Retrieved from http://ggplot2.org

40.

Wigdor

Wixon

(2011). Brave NUI world: Designing natural user interfaces for touch and gesture. Burlington, MA: Elsevier.

41.

World Health Organization. (2009). WHO guidelines on hand hygiene in health care: First global patient safety challenge. Clean care is safer care. Geneva, Switzerland: Author.