This paper addresses the question of how large the temporal delay between a visual and a haptic stimulus may be such that the stimuli are still perceived as being synchronous. Participants had to judge whether the moment at which a graphical object collided with a virtual wall occurred simultaneously with the moment at which a force was felt through a force feedback joystick. Participants either moved the joystick to drive the object (active touch) or held the joystick in a steady position while the object moved by itself (passive touch). Participants were found to be very sensitive to visual-haptic time delays. Sensitivity was higher for passive touch than for active touch. The minimum delay at which participants judged the stimuli as asynchronous was on average 45 ms. The delay at which the proportion of synchronous judgments reached a maximum was on average close to zero. The results indicate that the temporal accuracy of visual-haptic interfaces has to meet stringent requirements in order to optimize the overall realism that users experience. Actual or potential applications of this research include teleoperation, medical training, computer-aided-design, and scientific visualization.
Get full access to this article
View all access options for this article.
References
1.
Blakemore, S. J., Frith, C. D., & Wolpert, D. M. (1999). Spatiotemporal prediction modulates the perception of self-produced stimuli. Journal of Cognitive Neuroscience, 11, 551–559.
2.
Craig, J. C., & Baihua, X. (1990). Temporal order and tactile patterns. Perception and Psychophysics, 47, 22–32.
3.
Dinnerstein, A. J., & Zlotogura, P. (1968). Intermodal perception of temporal order and motor skills: Effects of age. Perceptual and Motor Skills, 26, 987–1000.
4.
Dixon, N. F., & Spitz, L. (1980). The detection of audio-visual desynchrony. Perception, 9, 719–721.
5.
Exner, S. (1875). Experimentelle Untersuchung der einfachsten psychischen Processe: III. Der persönlichen Gleichung zweiter Theil [trans. n.a.]. Archiv f?r die gesammte Physiologie des Menschen und der Thiere, 11, 403–432.
6.
Ferrell, W. R. (1965). Remote manipulation with transmission delay. IEEE Transactions on Human Factors in Electronics, HFE-6, 24–32.
7.
Ferrell, W. R. (1966). Delayed force feedback. Human Factors, 8, 449–455.
8.
Gengel, R. W., & Hirsh, I. J. (1970). Temporal order: The effect of single versus repeated presentations, practice and verbal feedback. Perception and Psychophysics, 7, 209–211.
9.
Gescheider, G. A. (1974). Temporal relations in cutaneous stimulation. In F. A. Geldard (Ed.), Cutaneous communication systems and devices (pp. 33–37.). Austin, TX: Psychonomic Society.
10.
Halliday, A. M., & Mingay, R. (1965). On the resolution of small time intervals and the effect of conduction delays on the judgement of simultaneity. Quarterly Journal of Experimental Psychology, 18, 35–46.
11.
Held, R. M., & Durlach, N. I. (1993). Telepresence, time delay and adaptation. In S. R. Ellis, M. K. Kaiser & A. J. Grunwald (Eds.), Pictorial communication in virtual and real environments (pp. 232–246). London: Taylor & Francis.
12.
Heller, M. A., & Myers, D. S. (1983). Active and passive tactual recognition of form. Journal of General Psychology, 108, 225–229.
13.
Hirsh, I. J., & Sherrick, C. E. (1961). Perceived order in different sense modalities. Journal of Experimental Psychology, 62, 423–432.
14.
Jáskowski, P. (1992). Temporal-order judgement and reaction time for short and long stimuli. Psychological Research, 54, 141–145.
15.
Lewkowicz, D. J. (1996). The development of temporally-based intersensory perception in human infants. In F. Macar, V. Pothas, & W. J. Friedman (Eds.), Time, action, and cognition: Towards bridging the gap (pp. 1094–1106). Dordrecht, Netherlands: Kluwer Academic.
16.
Müller, K., Aschersleben, G., Koch, R., Freund, H., & Prinz, W. (1999). Action timing in an isochronous tapping task. In G. Aschersleben, T. Bachmann, & J. Müller (Eds.), Cognitive contributions to the perception of temporal events (pp. 233–263). Amsterdam: Elsevier Science.
17.
Müsseler, J. (1999). How independent from action control is perception? An event-coding account for more equally-ranked crosstalks. In G. Aschersleben, T. Bachmann, & J. Müller (Eds.), Cognitive contributions to the perception of temporal events (pp. 121–147). Amsterdam: Elsevier Science.
18.
Petrosino, L., & Fucci, D. (1989). Temporal resolution of the aging tactile sensory system. Perceptual and Motor Skills, 68, 288–290.
19.
Richardson, B. L., Wuillemin, D. B., & MacKintosh, G. J. (1981). Can passive touch be better than active touch? A comparison of active and passive tactile maze learning. British Journal of Psychology, 72, 353–362.
20.
Sandvad, J. (1996, May). Dynamic aspects of auditory virtual environments. Presented at the 100th Convention of the Audio Engineering Society, Copenhagen.
21.
Sheridan, T. B. (1993). Space teleoperation through time delay: Review and prognosis. IEEE Transactions on Robotics and Automation, 9, 592–606.
22.
Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge, MA: MIT Press.
23.
Stelmach, L. B., & Herdman, C. M. (1991). Directed attention and perception of temporal order. Journal of Experimental Psychology: Human Perception and Performance, 17, 539–550.
24.
Sternberg, S., & Knoll, R. L. (1973). The perception of temporal order: Fundamental issues and a general model. In S. Korenblum (Ed.), Attention and performance IV (pp. 629–685). New York: Academic.
25.
Stone, J. V., Hunkin, N. M., Porrill, J., Wood, R., Keeler, V., Beanland, M., et al. (2001). When is now? Perception of simultaneity. Proceedings of the Royal Society of London, 268, 31–38.
26.
Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212–215.
27.
Thompson, D. A. (1977). The development of a six degree-offreedom robot evaluation test. Presented at the 13th Annual Conference on Manual Control, Cambridge, MA.
28.
Ulrich, R. (1987). Threshold models of temporal-order judgments evaluated by ternary response task. Perception and Psychophysics, 43, 224–239.
29.
Van de Par, S., & Kohlrausch, A. (2000). Sensitivity to auditory-visual asynchrony and to jitter in auditory-visual timing. In B. E. Rogowitz & T. N. Pappas (Eds), Proceedings of SPIE: Human Vision and Electronic Imaging (Vol. 3959, pp. 234–242). Bellingham, WA: SPIE --- The International Society for Optical Engineering.
