Usable Without Touch? Revealing the Relationship Between Usability and Mid-Air Gestures

Abstract

Mid-air gestures are an interaction method where the participant uses their hand(s) to perform a movement or sequence that represents control. This method of interaction is becoming more common for public use, yet the impact on usability lacks summarization. This systematic review explores themes of usability testing methods, applications, gesture design, multimodal combinations, input comparisons, and the participants. Findings reveal that the usability outcomes (effectiveness, efficiency, and user satisfaction) vary based on the type of evaluation method, application, design of gestures, and testing methods used. Furthermore, participant differences such as prior experience, training, and physical ability affect perceived usability. These insights reveal the need for inclusive, adaptive, and context-aware design strategies. Future research should standardize usability evaluation practices and prioritize user-centered approaches to mid-air gesture development. We propose guidelines for future research that will improve gesture systems, feedback, contextual fit, and usability evaluations.

Keywords

mid-air gestures usability human-computer interaction

Get full access to this article

View all access options for this article.

References

Bevan

Kirakowski

Maisse

L. J.

(1991). What is usability. In Proceedings of the 4th International Conference on Human–Computer Interaction (HCI International ’91, Stuttgart, Germany). Taylor & Francis. https://doi.org/10.1201/9781482272574-76

Cao

Yuan

Jeon

(2024). Head-up displays improve drivers’ performance and subjective perceptions with the in-vehicle gesture interaction system. International Journal of Human–Computer Interaction, 1–15. https://doi.org/10.1080/10447318.2024.2387397

Chen

L.-C.

Cheng

Y.-M.

Chu

P.-Y.

Sandnes

F. E.

(2017). Identifying the usability factors of mid-air hand gestures for 3D virtual model manipulation. In Antona

Stephanidis

(Eds.), Universal access in human–computer interaction. Designing novel interactions (UAHCI 2017, Lecture Notes in Computer Science, Vol. 10278, pp. 393–402). Springer. https://doi.org/10.1007/978-3-319-58703-5_29

Cui

Shen

Wang

Zhang

Kong

(2023). Impact of in-air gestures on in-car task’s driver distraction. Electronics, 12(7), 1626. https://doi.org/10.3390/electronics12071626

Detjen

Geisler

Schneegass

(2020). Maneuver-based control interventions during automated driving: Comparing touch, voice, and mid-air gestures as input modalities. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) (pp. 3268–3274). IEEE. https://doi.org/10.1109/SMC42975.2020.9283431

Evan

Kirakowski

Maisse

L. J.

(1991). What is usability. In Proceedings of the 4th International Conference on Human–Computer Interaction (HCI International ’91, Stuttgart, 1991). Taylor & Francis. https://doi.org/10.1201/9781482272574-76

Fink

Dimitrov

Yasuda

Chen

T. L.

Corey

R. R.

Giudice

N. A.

Sumner

E. S.

(2023). Autonomous is not enough: Designing multisensory mid-air gestures for vehicle interactions among people with visual impairments. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23, Article 74, pp. 1–13). ACM. https://doi.org/10.1145/3544548.3580762

Guérit

Cierro

Vanderdonckt

Pérez-Medina

J. L.

(2019). Gesture elicitation and usability testing for an armband interacting with Netflix and Spotify. In Advances in Intelligent Systems and Computing (Vol. 918, pp. 625–637). Springer. https://doi.org/10.1007/978-3-030-11890-7_60

Koutsabasis

Vogiatzidakis

(2019). Empirical research in mid-air interaction: A systematic review. International Journal of Human–Computer Interaction, 35(18), 1747–1768. https://doi.org/10.1080/10447318.2019.1572352

10.

Kronester

M. J.

Riener

Babic

(2021). Potential of wrist-worn vibrotactile feedback to enhance the perception of virtual objects during mid-air gestures. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems (Article 256, pp. 1–7). ACM. https://doi.org/10.1145/3411763.3451655

11.

Page

M. J.

McKenzie

J. E.

Bossuyt

P. M.

Boutron

Hoffmann

T. C.

Mulrow

C. D.

(2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71

12.

Tabbarah

Cao

Shamat

A. A.

Fang

Jeon

(2023). Novel in-vehicle gesture interactions: Design and evaluation of auditory displays and menu generation interfaces. In Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’23, pp. 224–233). ACM. https://doi.org/10.1145/3580585.3607164

13.

Vaidyanathan

Rosenberg

(2014). “Will use it, because I want to look cool”: A comparative study of simple computer interactions using touchscreen and in-air hand gestures. In Human–Computer Interaction. Advanced interaction modalities and techniques (HCI 2014, Lecture Notes in Computer Science, Vol. 8511, pp. 170–181). Springer. https://doi.org/10.1007/978-3-319-07230-2_17

14.

Waugh

Robertson

(2021). Don’t touch me! A comparison of usability on touch and non-touch inputs. In Ardito

, et al (Eds.), Human–Computer Interaction – INTERACT 2021: 18th IFIP TC 13 International Conference (Lecture Notes in Computer Science, Vol. 12936, pp. 400–404). Springer. https://doi.org/10.1007/978-3-030-85607-6_46

15.

Zhang

Stellmach

Sellen

Blake

(2015). The costs and benefits of combining gaze and hand gestures for remote interaction. In Human–Computer Interaction – INTERACT 2015 (Lecture Notes in Computer Science, Vol. 9298, pp. 570–577). Springer. https://doi.org/10.1007/978-3-319-22698-9_39