To determine whether response-effect (R-E) compatibility or stimulus-response (S-R) compatibility is more critical for touchless gesture responses.
Background:
Content on displays can be moved in the same direction (S-R incompatible but R-E compatible) or opposite direction (S-R compatible but R-E incompatible) as the touchless gesture that produces the movement. Previous studies suggested that it is easier to produce a button-press response when it is R-E compatible (and S-R incompatible). However, whether this R-E compatibility effect also occurs for touchless gesture responses is unknown.
Method:
Experiments 1 and 2 employed an R-E compatibility manipulation in which participants made responses with an upward or downward touchless gesture that resulted in the display content moving in the same (compatible) or opposite (incompatible) direction. Experiment 3 employed an S-R compatibility manipulation in which the stimulus occurred at the upper or lower location on the screen.
Results:
Overall, only negligible influences of R-E compatibility on performing the touchless gestures were observed (in contrast to button-press responses), whereas S-R compatibility heavily affected the gestural responses.
Conclusion:
The R-E compatibility obtained in many previous studies with various types of responses appears not to hold for touchless gestures as responses.
Application:
The results suggest that in the design of touchless interfaces, unique factors may contribute to determining which mappings of gesture and display movements are preferred by users.
AshleyS. (2014). Touch-less control coming to cars. Automotive Engineering, 4, 20–23.
2.
BerlyneD. E. (1957). Conflict and choice time. British Journal of Psychology, 48, 106–118.
3.
BragdonA.NelsonE.LiY.HinckleyK. (2011). Experimental analysis of touch-screen gesture designs in mobile environments. In Proceedings of the CHI’11 Conference on Human Factors in Computing Systems (pp. 403–412). New York, NY: ACM.
4.
BuryK. F.BoyleJ. M.EveyR. J.NealA. S. (1982). Windowing versus scrolling on a visual display terminal. Human Factors, 24, 385–394.
5.
ChenJ.ProctorR. W. (2013). Response–effect compatibility defines the natural scrolling direction. Human Factors, 55, 1112–1129.
6.
ElsnerB.HommelB. (2001). Effect anticipation and action control. Journal of Experimental Psychology: Human Perception and Performance, 27, 229–240.
7.
FittsP. M.DeiningerR. L. (1954). S–R compatibility: Correspondence among paired elements within stimulus and response codes. Journal of Experimental Psychology, 48, 483–492.
8.
FittsP. M.SeegerC. M. (1953). S–R compatibility: Spatial characteristics of stimulus and response codes. Journal of Experimental Psychology, 46, 199–210.
9.
GentileV.MaliziaA.SorceS.GentileA. (2015). Designing touchless gestural interactions for public displays in-the-wild. In KurosuM. (Ed.), Human-computer interaction: Interaction technologies. HCI 2015. Lecture Notes in Computer Science (vol. 9170; pp. 24–34). Cham, Switzerland: Springer.
HarleßE. (1861). Der Apparat des Willens [The apparatus of will]. Zeitschrift für Philosophie und Philosophische Kritik, 38, 50–73.
12.
HartS. G.StavelandL. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In HancockP. A.MeshkatiN. (Eds.), Human mental workload (pp. 139–183). Amsterdam, the Netherlands: North-Holland.
13.
HerwigA.PrinzW.WaszakF. (2007). Two modes of sensorimotor integration in intention-based and stimulus-based actions. Quarterly Journal of Experimental Psychology, 60, 1540–1554.
14.
HerwigA.WaszakF. (2009). Short article: Intention and attention in ideomotor learning. Quarterly Journal of Experimental Psychology, 62, 219–227.
15.
HommelB.MüsselerJ.AscherslebenG.PrinzW. (2001). The theory of event coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences, 24, 849–937.
16.
JamesW. (1981). The principles of psychology. Cambridge, MA: Harvard University Press. (Original work published 1890).
17.
JanczykM.DurstM.UlrichR. (2017). Action selection by temporally distal goal-states. Psychonomic Bulletin and Review, 24, 467–473.
18.
JanczykM.LercheV. (2019). A diffusion model analysis of the response-effect compatibility effect. Journal of Experimental Psychology: General, 148(2), 237–251.
19.
JanczykM.YamaguchiM.ProctorR.W.PfisterR. (2015). Response-effect compatibility with complex actions: The case of wheel rotations. Attention, Perception, & Psychophysics, 77, 930–940.
20.
KimH.SongH. (2014). Evaluation of the safety and usability of touch gestures in operating in-vehicle information systems with visual occlusion. Applied Ergonomics, 45, 789–798.
KundeW. (2001). Response-effect compatibility in manual choice reaction tasks. Journal of Experimental Psychology: Human Perception and Performance, 27, 387–394.
23.
KundeW.MüsselerJ.HeuerH. (2007). Spatial compatibility effects with tool use. Human Factors, 49, 661–670.
24.
LukasS.BrauH.KochI. (2010). Anticipatory movement compatibility for virtual reality interaction devices. Behaviour & Information Technology, 29, 165–174.
25.
MassonM. E. (2011). A tutorial on a practical Bayesian alternative to null-hypothesis significance testing. Behavior Research Methods, 43, 679–690.
26.
MüsselerJ.SkottkeE. (2011). Compatibility relationships with simple lever tools. Human Factors, 53, 383–390.
27.
NaefgenC.DambacherM.JanczykM. (2018). Why free choices take longer than forced choices: Evidence from response threshold manipulations. Psychological Research, 82, 1039–1052.
28.
O’HaraK.HarperR.MentisH.SellenA.TaylorA. (2013). On the naturalness of touchless: Putting the “interaction” back into NUI. ACM Transactions on Computer-Human Interaction, 20(1), Article 5. doi:10.1145/2442106.2442111
29.
OlsonR. L.HanowskiR. J.HickmanJ. S.BocanegraJ. L. (2009). Driver distraction in commercial vehicle operations (Report No. FMCSA-RRR-09-042). Washington, DC: U.S. Department of Transportation.
30.
PfisterR.JanczykM. (2012). Harleß’ Apparatus of Will: 150 years later. Psychological Research, 76, 561–565.
31.
PfisterRJanczykM. (2013). Confidence intervals for two sample means: Calculation, interpretation, and a few simple rules. Advances in Cognitive Psychology, 9, 74–80.
32.
PfisterR.KundeW. (2013). Dissecting the response in response-effect compatibility. Experimental Brain Research, 224, 647–655.
33.
PickeringC. A.BurnhamK. J.RichardsonM. J. (2007). A research study of hand gesture recognition technologies and applications for human vehicle interaction. In Automotive Electronics, 2007 3rd Institution of Engineering and Technology Conference on Automotive Electronics (pp. 1–15). IET.
34.
ProctorR. W.VuK.-P. L. (2006). Stimulus-response compatibility principles: Data, theory, and application. Boca Raton, FL: CRC Press.
35.
ProctorR. W.VuK.-P. L. (2016). Principles for designing interfaces compatible with human information processing. International Journal of Human-Computer Interaction, 32, 2–22.
36.
RakubutuT.GelderblomH.CohenJ. (2014, September). Participatory design of touch gestures for informational search on a tablet device. In SAICSIT 2014: Proceedings of the Southern African Institute for Computer Scientist and Information Technologists Annual Conference 2014 (pp. 276–285). New York, NY: ACM.
37.
RöttgerE.HaiderH. (2017). Investigating the characteristics of “not responding”: Backward crosstalk in the PRP paradigm with forced vs. free no-go decisions. Psychological Research, 81, 596–610.
38.
ShinY. K.ProctorR. W.CapaldiE. J. (2010). A review of contemporary ideomotor theory. Psychological Bulletin, 136, 943–974.
39.
WangD.-Y, D.ProctorR. W.PickD. F. (2003). The Simon effect with wheel-rotation responses. Journal of Motor Behavior, 35, 261–273.
40.
WangD.-Y. D.ProctorR. W.PickD. F. (2007). Coding controlled and triggered cursor movements as action effects: Influences on the auditory Simon effect for wheel-rotation responses. Journal of Experimental Psychology: Human Perception and Performance, 33, 657–669.
