Sage Journals: Discover world-class research

Abstract

To understand a speaker’s gestures, people may draw on kinesthetic working memory (KWM)—a system for temporarily remembering body movements. The present study explored whether sensitivity to gesture meaning was related to differences in KWM capacity. KWM was evaluated through sequences of novel movements that participants viewed and reproduced with their own bodies. Gesture sensitivity was assessed through a priming paradigm. Participants judged whether multimodal utterances containing congruent, incongruent, or no gestures were related to subsequent picture probes depicting the referents of those utterances. Individuals with low KWM were primarily inhibited by incongruent speech-gesture primes, whereas those with high KWM showed facilitation—that is, they were able to identify picture probes more quickly when preceded by congruent speech and gestures than by speech alone. Group differences were most apparent for discourse with weakly congruent speech and gestures. Overall, speech-gesture congruency effects were positively correlated with KWM abilities, which may help listeners match spatial properties of gestures to concepts evoked by speech.

Keywords

action perception embodiment gesture comprehension grounded cognition kinesthetic memory motor resonance multimodal discourse working memory haptic memory

Get full access to this article

View all access options for this article.

References

Baddeley

(2012). Working memory: Theories, models, and controversies. Annual Review of Psychology, 63, 1–29.

Baddeley

A. D.

Hitch

G. J.

(1974). Working memory. In Bower

G. A.

(Ed.), Recent advances in learning and motivation (Vol. 8, pp. 47–90). New York, NY: Academic Press.

Beattie

Shovelton

(1999). Mapping the range of information contained in the iconic hand gestures that accompany spontaneous speech. Journal of Language and Social Psychology, 18, 438–462.

Bernardis

Salillas

Caramelli

(2008). Behavioral and neurophysiological evidence of semantic interaction between iconic gestures and words. Cognitive Neuropsychology, 25, 1114–1128.

Cowan

(1999). An embedded-processes model of working memory. In Miyake

Shah

(Eds.), Models of working memory: Mechanisms of active maintenance and executive control (pp. 62–101). Cambridge, England: Cambridge University Press.

Cowan

(2008). What are the differences between long-term, short-term, and working memory? Progress in Brain Research, 169, 323–338.

Daneman

Carpenter

P. A.

(1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450–466.

Della Sala

Gray

Baddeley

Allamano

Wilson

(1999). Pattern span: A tool for unwelding visuo–spatial memory. Neuropsychologia, 37, 1189–1199.

Engle

R. W.

(2002). Working memory capacity as executive attention. Current Directions in Psychological Science, 11, 19–23.

10.

Engle

R. W.

Kane

M. J.

(2004). Executive attention, working memory capacity, and a two-factor theory of cognitive control. In Ross

(Ed.), The psychology of learning and motivation (pp. 145–199). New York, NY: Academic Press.

11.

Goodwin

(2003). The body in action. In Coupland

Gwyn

(Eds.), Discourse, the body, and identity (pp. 19–42). New York, NY: Palgrave Macmillan.

12.

Habets

Kita

Shao

Özyurek

Hagoort

(2011). The role of synchrony and ambiguity in speech-gesture integration during comprehension. Journal of Cognitive Neuroscience, 23, 1845–1854.

13.

Holle

Gunter

T. C.

(2007). The role of iconic gestures in speech disambiguation: ERP evidence. Journal of Cognitive Neuroscience, 19, 1175–1192.

14.

Hostetter

A. B.

Alibali

M. W.

(2007). Raise your hand if you’re spatial: Relations between verbal and spatial skills and gesture production. Gesture, 7, 73–95.

15.

Just

M. A.

Carpenter

P. A.

(1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122–149.

16.

Kelly

S. D.

Creigh

Bartolotti

(2009). Integrating speech and iconic gestures in a Stroop-like task: Evidence for automatic processing. Journal of Cognitive Neuroscience, 22, 683–694.

17.

Kelly

S. D.

Kravitz

Hopkins

(2004). Neural correlates of bimodal speech and gesture comprehension. Brain & Language, 89, 253–260.

18.

Kelly

S. D.

McDevitt

Esch

(2009). Brief training with co-speech gesture lends a hand to word learning in a foreign language. Language and Cognitive Processes, 24, 313–334.

19.

Kelly

S. D.

Özyurek

Maris

(2010). Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21, 260–267.

20.

Kelly

S. D.

Ward

Creigh

Bartolotti

(2007). An intentional stance modulates the integration of gesture and speech during comprehension. Brain & Language, 101, 222–233.

21.

Klauer

K. C.

Zhao

(2004). Double dissociations in visual and spatial short-term memory. Journal of Experimental Psychology: General, 133, 355–381.

22.

Macedonia

Knösche

T. R.

(2011). Body in mind: How gestures empower foreign language learning. Mind, Brain, and Education, 5, 196–211.

23.

Obermeier

Dolk

Gunter

(2012). The benefit of gestures during communication: Evidence from hearing and hearing-impaired individuals. Cortex, 48, 857–870.

24.

Özyurek

Willems

R. M.

Kita

Hagoort

(2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19, 605–616.

25.

Posner

(1967). Characteristics of visual and kinesthetic memory codes. Journal of Experimental Psychology: General, 75, 103–107.

26.

Seemüller

Fiehler

Rösler

(2011). Unimodal and crossmodal working memory representations of visual and kinesthetic movement trajectories. Acta Psychologica, 136, 52–59.

27.

Seemüller

Müller

E. M.

Rösler

(2012). EEG-power and -coherence changes in a unimodal and crossmodal working memory task with visual and kinesthetic stimuli. International Journal of Psychophysiology, 83, 87–95.

28.

Shah

Miyake

(1996). The separability of working memory resources for spatial thinking and language processing: An individual differences approach. Journal of Experimental Psychology: General, 125, 4–27.

29.

Smyth

Pearson

N. A.

Pendleton

L. R.

(1988). Movement and working memory: Patterns and positions in space. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 40, 497–514.

30.

Smyth

Pendleton

(1989). Working memory for movements. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 41, 235–250.

31.

Smyth

Pendleton

(1990). Space and movement in working memory. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology, 42, 291–304.

32.

Streeck

(2002). A body and its gestures. Gesture, 2, 19–44.

33.

Streeck

(2008). Depicting by gesture. Gesture, 8, 285–301.

34.

Y. C.

Coulson

(2007a). How iconic gestures enhance communication: An ERP study. Brain & Language, 101, 234–245.

35.

Y. C.

Coulson

(2007b). Iconic gestures prime related concepts: An ERP study. Psychonomic Bulletin & Review, 14, 57–63.

36.

Y. C.

Coulson

(2010a). Gestures modulate speech processing early in utterances. NeuroReport, 21, 522–526.

37.

Y. C.

Coulson

(2010b, November). Iconic gestures facilitate word and message processing: The multi-level integration model of audio-visual discourse comprehension. Paper presented at the Neurobiology of Language Conference, San Diego, CA. Abstract retrieved from http://www.neurolang.org/programs/ScientificProgramNLC2010.pdf

38.

Y. C.

Coulson

(2014). A psychometric measure of working memory for configured body movement. PLoS ONE, 9(1), Article e84834. Retrieved from http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0084834

39.

Yap

D.-F.

W.-C.

Yap

J.-M. M.

Tan

Y.-Q.

Teoh

R.-L. S.

(2011). Iconic gestures prime words. Cognitive Science, 35, 171–183.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

Iconic Gestures Facilitate Discourse Comprehension in Individuals With Superior Immediate Memory for Body Configurations

Abstract

Keywords

Get full access to this article

References

Supplementary Material