The purpose of this study was to develop a lexicon for 3-D hand gestures for common human–computer interaction (HCI) tasks by considering usability and effort ratings.
Background:
Recent technologies create an opportunity for developing a free-form 3-D hand gesture lexicon for HCI.
Method:
Subjects (N = 30) with prior experience using 2-D gestures on touch screens performed 3-D gestures of their choice for 34 common HCI tasks and rated their gestures on preference, match, ease, and effort. Videos of the 1,300 generated gestures were analyzed for gesture popularity, order, and response times. Gesture hand postures were rated by the authors on biomechanical risk and fatigue.
Results:
A final task gesture set is proposed based primarily on subjective ratings and hand posture risk. The different dimensions used for evaluating task gestures were not highly correlated and, therefore, measured different properties of the task–gesture match.
Application:
A method is proposed for generating a user-developed 3-D gesture lexicon for common HCIs that involves subjective ratings and a posture risk rating for minimizing arm and hand fatigue.
AlpernM.MinardoK. (2003). Developing a car gesture interface for use as a secondary task. In CHI 2003 Extended Abstracts on Human Factors in Computing Systems (pp. 932–933). New York, NY: ACM.
2.
BaoS.HowardN.SpielholzP.SilversteinB.PolissarN. (2009). Interrater reliability of posture observations. Human Factors, 51, 292–309.
3.
BaudelT.Beaudouin-LafonM. (1993). Charade: Remote control of objects using freehand gestures. Communications of the ACM, 36, 28–35.
4.
BowmanD.WingraveC. (2001). Design and evaluation of menus systems for immersive virtual environments. In Proceedings of IEEE Virtual Reality 2001 (pp. 149–156). Piscataway, NJ: IEEE.
5.
de la BarréR.ChojeckiP.LeinerU.MühlbachL.RuschinD. (2009). Touchless interaction: Novel chances and challenges. In KurosuM. (Ed.), Human–computer interaction: Novel interaction methods and techniques (pp. 161–169). Berlin, Germany: Springer.
6.
Farhadi-NiakiF.EtemadS. A.AryaA. (2013). Design and usability analysis of gesture-based control for common desktop tasks. In KurosuM. (Ed.), Human–computer interaction: Interaction modalities and techniques (pp. 215–224). Berlin, Germany: Springer.
7.
FeuersteinM.CarosellaA. M.BurrellL. M.MarshallL.DecaroJ. (1997). Occupational upper extremity symptoms in sign language interpreters: Prevalence and correlates of pain, function, and work disability. Journal of Occupational Rehabilitation, 7, 187–205.
8.
FikkertW.VetP. van derVeerG. van derNijholtA. (2010). Gestures for large display control. In KoppS.WachsmuthI. (Eds.), Lecture notes in computer science: Vol. 5934. Gesture in embodied communication and human–computer interaction (pp. 245–256). Berlin, Germany: Springer.
9.
GuiardY. (1987). Asymmetric division of labor in human skilled bimanual action: The kinematic chain as a model. Journal of Motor Behavior, 19, 486–517.
10.
GuiardY.FerrandT. (1996). Asymmetry in bimanual skills. In ElliottD.RoyE. A. (Eds.), Manual asymmetries in motor performance (pp. 175–195). Boca Raton, FL: CRC Press.
HorskyJ.KaufmanD. R.OppenheimM. I.PatelV. L. (2003). A framework for analyzing the cognitive complexity of computer-assisted clinical ordering. Journal of Biomedical Informatics, 36(1), 4–22.
13.
KaiserE.OlwalA.McGeeD.BenkoH.CorradiniA.LiX.. . . FeinerS. (2003). Mutual disambiguation of 3D multimodal interaction in augmented and virtual reality. In Proceedings of the 5th International Conference on Multimodal Interfaces (pp. 12–19). New York, NY: ACM.
14.
KaramM.SchraefelM. (2005). A study on the use of semaphoric gestures to support secondary task interactions. In Proceedings of CHI ’05 Extended Abstracts on Human Factors in Computing Systems (pp. 1961–1964). New York, NY: ACM.
15.
KeirP. J.BachJ. M.RempelD. M. (1998). Effects of finger posture on carpal tunnel pressure during wrist motion. Journal of Hand Surgery, 23, 1004–1009.
16.
KeskinC.ErkanA.AkarunL. (2003). Real time hand tracking and 3D gesture recognition for interactive interfaces using HMM. In ICANN/ICONIPP (pp. 26–29). Berlin, Germany: Springer.
MillerG. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81–97.
19.
NacentaM. A.KamberY.QiangY.KristenssonP. O. (2013). Memorability of pre-designed and user-defined gesture sets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1099–1108). New York, NY: ACM.
20.
NiT. (2011). A framework of freehand gesture interaction: Techniques, guidelines, and applications (PhD dissertation). Virginia Polytechnic Institute and State University, Blacksburg.
21.
NiT.BowmanD.NorthC.McMahanR. (2011). Design and evaluation of freehand menus selection interfaces using tilt and pinch gestures. Interaction Journal of Human–Computer Studies, 69, 551–662.
22.
NielsenM.StorringM.MoeslundT. B.GranumE. (2004). A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In Gesture-based communication in human–computer interaction: 5th International Gesture Workshop (pp. 409–420). Berlin, Germany: Springer.
23.
RempelD.BachJ. M.GordonL.SoY. (1998). Effects of forearm pronation/supination on carpal tunnel pressure. Journal of Hand Surgery, 23, 38–42.
24.
RempelD.LeeD. L.CamilleriM. (2014). The design of gestures for human–computer interaction: Lessons from sign language interpreters. International Journal of Human–Computer Studies, 72, 728–735.
25.
SmithM.SchraefelM. (2004). The radial scroll tool: Scrolling support for stylus- or touch-based document navigation. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology. New York, NY: ACM.
26.
SternH.WachsJ.EdanY. (2006). Optimal hand gesture vocabulary design using psycho-physiological and technical factors. In Proceedings from 7th International Conference on Automatic Face and Gesture Recognition (pp. 257–262). Piscataway, NJ: IEEE.
27.
VatavuR. D. (2012, July). User-defined gestures for free-hand TV control. In Proceedings of the 10th European Conference on Interactive TV and Video (pp. 45-48). New York, NY: ACM.
28.
VogelD.BalakrishnanR. (2005). Distant freehand pointing and clicking on very large, high-resolution displays. In UIST ’04: Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (pp. 137–146). New York, NY: ACM.
29.
WachsJ. (2006). Optimal hand gesture vocabulary design methodology for virtual robotic control (PhD dissertation). Ben-Gurion University of the Negev, Beer-Sheva, Israel.
30.
WachsJ. P.KolschM.SternH.EdanY. (2011). Vision-based hand-gesture applications. Communications of the ACM, 54, 60–71.
31.
WachsJ. P.SternH. I.EdanY.GillamM.HandlerJ.FeiedC.SmithM. (2008). A gesture-based tool for sterile browsing of radiology images. Journal of the American Medical Informatics Association, 15, 321–323.
32.
WebsterB. S.SnookS. H. (1994). The cost of compensable upper extremity cumulative trauma disorders. Journal of Occupational Medicine, 36, 713–717.
33.
WobbrockJ. O.MorrisM. R.WilsonA. D. (2009). User-defined gestures for surface computing. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI ’09) (pp. 1083-1092). New York, NY: ACM.
34.
WrightM.LinC. J.O’NeillE.CoskerD.JohnsonP. (2011). 3D gesture recognition: An evaluation of user and system performance. In Proceedings of Pervasive 2011 (pp. 294-313). Berlin, Germany: Springer.
35.
WuM.BalakrishnanR. (2003). Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In Proceedings of the 16th Annual ACM symposium on User Interface Software and Technology (pp. 193–202). New York, NY: ACM.
36.
ZhangZ. (2012). Microsoft Kinect sensor and its effects. IEEE MultiMedia, 19, 4–10.
