Mapping the Range of Information Contained in the Iconic Hand Gestures that Accompany Spontaneous Speech

Abstract

This study tested McNeill’s theory that the iconic gestures that accompany speech in everyday talk convey critical information in interpersonal communication. Using a structured interview to measure the amount of information respondents receive from clause-length clips depicting aspects of a cartoon story, we discovered that when respondents could see the iconic gestures as well as hear the speech they received significantly more accurate information about those aspects of the original story depicted in the clips than when they just heard the speech. We also discovered that it was only with respect to certain semantic categories, namely, the relative position and the size of objects, that the beneficial effect of gestural communication was significant. We then considered in detail how specific attributes of actions and of objects are communicated via the iconic representation within individual gestures. Lastly, we discuss the implications of these findings for models of human communication in this area.

Get full access to this article

View all access options for this article.

References

Beattie, G. (1982). Unnatural behaviour in the laboratory. New Scientist, 96, 181.

Beattie, G. (1983). Talk: An analysis of speech and nonverbal behaviour in conversation. Milton Keynes, UK: Open University Press.

Beattie, G. , & Aboudan, R. (1994). Gestures, pauses and speech: An experimental investigation of the effects of changing social context on their precise temporal relationships. Semiotica, 99, 239-272.

Beattie, G. , & Butterworth, B. (1979). Contextual probability and word frequency as determinants of pauses and errors in spontaneous speech. Language and Speech, 22, 201-211.

Beattie, G. , & Coughlan, J. (1998). Do iconic gestures have a functional role in lexical access? An experimental study of the effects of repeating a verbal message on gesture production. Semiotica, 119, 221-249.

Beattie, G. , & Coughlan, J. (1999). An experimental investigation of the role of iconic gestures in lexical access using the tip-of-the-tongue phenomenon. British Journal of Psychology, 90, 35-56.

Beattie, G. , & Shovelton, H. (1999). Do iconic hand gestures really contribute anything to the semantic information conveyed by speech? An experimental investigation. Semiotica, 123, 1-30.

Butterworth, B. , & Beattie, G. (1978). Gesture and silence as indicators of planning in speech. In R. N. Campbell & P. T. Smith (Eds.), Recent advances in the psychology of language (pp. 347-360). New York: Plenum.

Butterworth, B. , & Hadar, U. (1989). Gesture, speech, and computational stages: A reply to McNeill. Psychological Review, 96, 168-174.

10.

Ellis, A. , & Beattie, G. (1986). The psychology of language and communication. Hove, UK: Lawrence Erlbaum.

11.

Goldman-Eisler, F. (1958). Speech production and the predictability of words in context. Quarterly Journal of Experimental Psychology, 10, 96-106.

12.

Goldman-Eisler, F. (1968). Psycholinguistics: Experiments in spontaneous speech. London: Academic Press.

13.

Hadar, U. , & Butterworth, B. (1997). Iconic gestures, imagery and word retrieval in speech. Semiotica, 115, 147-172.

14.

Jefferson, G. (1985). An exercise in the transcription and analysis of laughter. In T. Van Dijk (Ed.), Handbook of discourse analysis (Vol. 3, pp. 25-34). London: Academic Press.

15.

Krauss, R. M. , Morrel-Samuels, P. , & Colasante, C. (1991). Do conversational hand gestures communicate? Journal of Personality and Social Psychology, 61, 743-754.

16.

Lee, V. , & Beattie, G. (1998). The rhetorical organization of verbal and nonverbal behavior in emotion talk. Semiotica, 120, 39-92.

17.

Martin, J. G. (1970). On judging pauses in spontaneous speech. Journal of Verbal Learning and Verbal Behaviour, 9, 75-78.

18.

McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92, 350-371.

19.

McNeill, D. (1989). A straight path to where? Reply to Butterworth and Hadar. Psychological Review, 96, 175-179.

20.

McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.

21.

McNeill, D. , & Levy, E. (1982). Conceptual representations in language activity and gesture. In R. Jarvella & W. Klein (Eds.), Speech, place, and action: Studies in deixis and related topics (pp. 271-295). Chichester, UK: Wiley.

22.

Miller, G. A. , & Johnson-Laird, P. N. (1976). Language and perception. Cambridge, MA: Harvard University Press.

23.

Morrel-Samuels, P. , & Krauss, R. M. (1992). Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory and Cognition, 18, 615-623.

24.

Potter, J. (1996). Representing reality: Discourse, rhetoric and social construction. London: Sage Ltd.

25.

Rauscher, F. H. , Krauss, R. M. , & Chen, Y. (1996). Gesture, speech and lexical access: The role of lexical movements in speech production. Psychological Science, 7, 226-231.

26.

Sacks, H. , Schegloff, E. A. , & Jefferson, G. A. (1974). A simplest systematics for the organization of turn-taking for conversation. Language, 50, 697-735.

27.

Zuckerman, M. , Larrance, D. T. , Hall, J. A. , DeFrank, R. S. , & Rosenthal, R. (1979). Posed and spontaneous communication of emotion via facial and vocal cues. Journal of Personality, 47, 712-733.