Co-speech hand gestures offer a rich avenue for research into studying emotion communication because they serve as both prominent expressive bodily cues and an integral part of language. Despite such a strategic relevance, gesture-speech integration and interaction have received less research focus on its emotional function compared to its cognitive function. This review aims to shed light on the current state of the field regarding the interplay between co-speech hand gestures and emotions, focusing specifically on the role of gestures in expressing and understanding both others’ and one's own emotions. The article concludes by addressing current limitations in the field and proposing future directions for researchers investigating gesture-emotion interaction. Our goal is to provide a roadmap to researchers in their exploration of the role of gestures in emotions, ultimately contributing to a more comprehensive understanding of how gestures and emotions intersect.
AlibaliM. W. (2005). Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition & Computation, 5(4), 307–331. https://doi.org/10.1207/s15427633scc0504_2
2.
AlibaliM. W.HeathD. C.MyersH. J. (2001). Effects of visibility between speaker and listener on gesture production: Some gestures are meant to be seen. Journal of Memory and Language, 44(2), 169–188. https://doi.org/10.1006/jmla.2000.2752
3.
AlibaliM. W.NathanM. J. (2012). Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. Journal of the Learning Sciences, 21(2), 247–286. https://doi.org/10.1080/10508406.2011.611446
4.
ArslanB.GöksunT. (2021). Ageing, working memory, and mental imagery: Understanding gestural communication in younger and older adults. Quarterly Journal of Experimental Psychology, 74(1), 29–44. https://doi.org/10.1177/1747021820944696
5.
AsalıoğluE. N.GöksunT. (2023). The role of hand gestures in emotion communication: Do type and size of gestures matter?Psychological Research. https://doi.org/10.1007/s00426-022-01774-9
AussemsS.KitaS. (2019). Seeing iconic gestures while encoding events facilitates children’s memory of these events. Child Development, 90(4), 1123–1137. https://doi.org/10.1111/cdev.12988
8.
BarrettL. F. (2006). Solving the emotion paradox: Categorization and the experience of emotion. Personality and Social Psychology Review, 10(1), 20–46. https://doi.org/10.1207/s15327957pspr1001_2
9.
BarrettL. F.MesquitaB.GendronM. (2011). Context in emotion perception. Current Directions in Psychological Science, 20(5), 286–290. https://doi.org/10.1177/0963721411422522
10.
BharadwajA.DargueN.SwellerN. (2022). A hands-on approach to learning: Gesture production during encoding and its effect on narrative recall. Cognitive Science, 46, e13214. https://doi.org/10.1111/cogs.13214
BriotK.PizanoA.BouvardM.AmestoyA. (2021). New technologies as promising tools for assessing facial emotion expressions impairments in ASD: A systematic review. Frontiers in Psychiatry, 12. https://doi.org/10.3389/fpsyt.2021.634756
13.
CalbiM.LangiulliN.SiriF.UmiltàM. A.GalleseV. (2021). Visual exploration of emotional body language: A behavioural and eye-tracking study. Psychological Research, 85, 2326–2339. https://doi.org/10.1007/s00426-020-01416-y
14.
Caldwell-HarrisC. L. (2015). Emotionality differences between a native and foreign language: Implications for everyday life. Current Directions in Psychological Science, 24(3), 214–219. https://doi.org/10.1177/0963721414566268
15.
CartmillE. A.Goldin-MeadowS. (2016). Gesture. In MatsumotoD.HwangH. C.FrankM. G. (Eds.), APA Handbook of nonverbal communication (pp. 307–333). American Psychological Association. https://doi.org/10.1037/14669-012.
16.
CasasantoD.JasminK. (2010). Good and bad in the hands of politicians: Spontaneous gestures during positive and negative speech. PLOS ONE, 5(7), e11805. https://doi.org/10.1371/journal.pone.0011805
17.
CasasantoD.JasminK. (2012). The hands of time: Temporal gestures in English speakers. Cognitive Linguistics, 23(4), 643–674. https://doi.org/10.1515/cog-2012-0020
18.
ÇatakE. N.AçıkA.GöksunT. (2018). The relationship between handedness and valence: A gesture study. Quarterly Journal of Experimental Psychology, 71(12), 2615–2626. https://doi.org/10.1177/1747021817750110
19.
ChanR. C. K.LiH.CheungE. F. C.GongQ. (2010). Impaired facial emotion perception in schizophrenia: A meta-analysis. Psychiatry Research, 178(2), 381–390. https://doi.org/10.1016/j.psychres.2009.03.035
20.
ChuM.KitaS. (2008). Spontaneous gestures during mental rotation tasks: Insights into the microdevelopment of the motor strategy. Journal of Experimental Psychology: General, 137(4), 706. https://doi.org/10.1037/a0013157
21.
ChuM.MeyerA.FoulkesL.KitaS. (2014). Individual differences in frequency and saliency of speech-accompanying gestures: The role of cognitive abilities and empathy. Journal of Experimental Psychology: General, 143(2), 694–709. https://doi.org/10.1037/a0033861
22.
ChurchR. B.KellyS.HolcombeD. (2014). Temporal synchrony between speech, action and gesture during language production. Language, Cognition and Neuroscience, 29(3), 345–354. https://doi.org/10.1080/01690965.2013.857783
23.
CienkiA.MüllerC. (2008). Metaphor and gesture. John Benjamins Publishing.
24.
ClarkH. H. (1996). Using language. Cambridge University Press.
25.
CookS. W.FennK. M. (2017). The function of gesture in learning and memory. In ChurchR. B.AlibaliM. W.KellyS. D. (Eds.), Why gesture? How the hands function in speaking, thinking and communicating (pp. 129–153). John Benjamins Publishing Company.
26.
CookS. W.YipT. K.Goldin-MeadowS. (2010). Gesturing makes memories that last. Journal of Memory and Language, 63(4), 465–475. https://doi.org/10.1016/j.jml.2010.07.002
27.
DaelN.GoudbeekM.SchererK. R. (2013). Perceived gesture dynamics in nonverbal expression of emotion. Perception, 42(6), 642–657. https://doi.org/10.1068/p7364
28.
DaelN.MortillaroM.SchererK. R. (2012). Emotion expression in body action and posture. Emotion, 12(5), 1085–1101. https://doi.org/10.1037/a0025737
29.
DargueN.SwellerN. (2020). Learning stories through gesture: Gesture’s effects on child and adult narrative comprehension. Educational Psychology Review, 32(1), 249–276. https://doi.org/10.1007/s10648-019-09505-0
30.
DargueN.SwellerN.JonesM. P. (2019). When our hands help US understand: A meta-analysis into the effects of gesture on comprehension. Psychological Bulletin, 145(8), 765–784. https://doi.org/10.1037/bul0000202
31.
de GelderB. (2009). Why bodies? Twelve reasons for including bodily expressions in affective neuroscience. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535), 3475–3484. https://doi.org/10.1098/rstb.2009.0190
32.
de GelderB.de BorstA. W.WatsonR. (2015). The perception of emotion in body expressions. Wiley Interdisciplinary Reviews: Cognitive Science, 6(2), 149–158. https://doi.org/10.1002/wcs.1335
33.
de RuiterJ. P. (2000). The production of gesture and speech. In McNeillD. (Ed.), Language and gesture (pp. 284–311). Cambridge University Press.
34.
EfronD. (1941). Gesture and environment. King’s Crown Press.
35.
EkmanP.FriesenW. V. (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1(1), 49–98. https://doi.org/10.1515/semi.1969.1.1.49
36.
FeyereisenP.HavardI. (1999). Mental imagery and production of hand gestures while speaking in younger and older adults. Journal of Nonverbal Behavior, 23(2), 153–171. https://doi.org/10.1023/A:1021487510204
37.
FlaischT.HäckerF.RennerB.SchuppH. T. (2011). Emotion and the processing of symbolic gestures: An event-related brain potential study. Social Cognitive and Affective Neuroscience, 6(1), 109–118. https://doi.org/10.1093/scan/nsq022
38.
GaoY.Bianchi-BerthouzeN.MengH. (2012). What does touch tell US about emotions in touchscreen-based gameplay?ACM Transactions on Computer-Human Interaction (TOCHI), 19(4), 1–30. https://doi.org/10.1145/2395131.2395138
39.
GhoshS.HiwareK.GangulyN.MitraB.DeP. (2019). Emotion detection from touch interactions during text entry on smartphones. International Journal of Human-Computer Studies, 130, 47–57. https://doi.org/10.1016/j.ijhcs.2019.04.005
40.
GillespieM.JamesA. N.FedermeierK. D.WatsonD. G. (2014). Verbal working memory predicts co-speech gesture: Evidence from individual differences. Cognition, 132(2), 174–180. https://doi.org/10.1016/j.cognition.2014.03.012
41.
GlowinskiD.DaelN.CamurriA.VolpeG.MortillaroM.SchererK. (2011). Toward a minimal representation of affective gestures. IEEE Transactions on Affective Computing, 2(2), 106–118. https://doi.org/10.1109/T-AFFC.2011.7
42.
Goldin-MeadowS. (2003). Hearing gesture: How our hands help us think. The Belknap Press.
43.
GrazianoM.NicoladisE.MarentetteP. (2020). How referential gestures align with speech: Evidence from monolingual and bilingual speakers. Language Learning, 70(1), 266–304. https://doi.org/10.1111/lang.12376
44.
GuY.ZhengY.SwertsM. (2019). Which is in front of Chinese people, past or future? The effect of language and culture on temporal gestures and spatial conceptions of time. Cognitive Science, 43(12), e12804. https://doi.org/10.1111/cogs.12804
45.
GuilbertD.SwellerN.Van BergenP. (2021). Emotion and gesture effects on narrative recall in young children and adults. Applied Cognitive Psychology, 35(4), 873–889. https://doi.org/10.1002/acp.3815
46.
HarrisC. L.AyçíçeğíA.GleasonJ. B. (2003). Taboo words and reprimands elicit greater autonomic reactivity in a first language than in a second language. Applied Psycholinguistics, 24(4), 561–579. https://doi.org/10.1017/S0142716403000286
47.
HasegawaY.HiroseY. (2005). What the Japanese language tells US about the alleged Japanese relational self. Australian Journal of Linguistics, 25(2), 219–251. https://doi.org/10.1080/07268600500233019
48.
HertensteinM. J.HolmesR.McCulloughM.KeltnerD. (2009). The communication of emotion via touch. Emotion, 9(4), 566. https://doi.org/10.1037/a0016108
HostetterA. (2011). When do gestures communicate? A meta-analysis. Psychological Bulletin, 137, 297–315. https://doi.org/10.1037/a0022128
51.
HostetterA. B.AlibaliM. W. (2008). Visible embodiment: Gestures as simulated action. Psychonomic Bulletin & Review, 15(3), 495–514. https://doi.org/10.3758/PBR.15.3.495
52.
JacksonJ. C.WattsJ.HenryT. R.ListJ.-M.ForkelR.MuchaP. J.GreenhillS. J.GrayR. D.LindquistK. A. (2019). Emotion semantics show both cultural variation and universal structure. Science, 366(6472), 1517–1522. https://doi.org/10.1126/science.aaw8160
53.
KellyS. D.Ngo TranQ.-A. (2023). Exploring the emotional functions of co-speech hand gesture in language and communication. Topics in Cognitive Science. https://doi.org/10.1111/tops.12657
54.
KendonA. (2004). Gesture: Visible action as utterance. Cambridge University Press.
55.
KessousL.CastellanoG.CaridakisG. (2010). Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis. Journal on Multimodal User Interfaces, 3, 33–48. https://doi.org/10.1007/s12193-009-0025-5
56.
KippM.MartinJ. (2009). Gesture and emotion: Can basic gestural form features discriminate emotions? In 3rd International conference on affective computing and intelligent interaction and workshops, Amsterdam, Netherlands. https://doi.org/10.1109/ACII.2009.5349544
57.
KirschL. P.KrahéC.BlomN.CrucianelliL.MoroV.JenkinsonP. M.FotopoulouA. (2018). Reading the mind in the touch: Neurophysiological specificity in the communication of emotions by touch. Neuropsychologia, 116, 136–149. https://doi.org/10.1016/j.neuropsychologia.2017.05.024
58.
KitaS. (2000). How representational gestures help speaking. In McNeillD. (Ed.), Language and gesture (pp. 162–185). Cambridge University Press.
59.
KitaS.AlibaliM. W.ChuM. (2017). How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis. Psychological Review, 124(3), 245–266. https://doi.org/10.1037/rev0000059
60.
KitaS.de CondappaO.MohrC. (2007). Metaphor explanation attenuates the right-hand preference for depictive co-speech gestures that imitate actions. Brain and Language, 101, 185–197. https://doi.org/10.1016/j.bandl.2006.11.006
61.
KitaS.ÖzyürekA. (2003). What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: Evidence for an interface representation of spatial thinking and speaking. Journal of Memory and Language, 48(1), 16–32. https://doi.org/10.1016/S0749-596X(02)00505-3
62.
KonishiH.KarstenA.VallottonC. D. (2018). Toddlers’ use of gesture and speech in service of emotion regulation during distressing routines. Infant Mental Health Journal, 39(6), 730–750. https://doi.org/10.1002/imhj.21740
63.
KraussR. M.ChenY.GottesmanR. F. (2000). Lexical gestures and lexical access: A process model. In McNeillD. (Ed.), Language and gesture (pp. 261–283). Cambridge University Press.
64.
KretM. E.StekelenburgJ. J.de GelderB.RoelofsK. (2017). From face to hand: Attentional bias towards expressive hands in social anxiety. Biological Psychology, 122, 42–50. https://doi.org/10.1016/j.biopsycho.2015.11.016
65.
LakoffG.JohnsonM. (1980). Metaphors we live by. University of Chicago.
66.
LakoffG.KövecsesZ. (1987). The cognitive model of anger inherent in American English. In HollandD.QuinnN. (Eds.), Cultural models in language and thought (pp. 195–221). Cambridge University Press. https://doi.org/10.1017/CBO9780511607660.009
67.
LevyR. S.KellyS. D. (2020). Emotion matters: The effect of hand gesture on emotionally valenced sentences. Gesture, 19(1), 41–71. https://doi.org/10.1075/gest.19029.lev
68.
LinW.OrtonI.LiQ.PavariniG.MahmoudM. (2021). Looking at the body: Automatic analysis of body gestures and self-adaptors in psychological distress. IEEE Transactions on Affective Computing. https://doi.org/10.1109/TAFFC.2021.3101698
69.
LindquistK. A.SatputeA. B.GendronM. (2015). Does language do more than communicate emotion?Current Directions in Psychological Science, 24(2), 99–108. https://doi.org/10.1177/0963721414553440
MarstallerL.BurianováH. (2013). Individual differences in the gesture effect on working memory. Psychonomic Bulletin & Review, 20(3), 496–500. https://doi.org/10.3758/s13423-012-0365-0
72.
McNeillD. (1992). Hand and mind: What gestures reveal about thought. University of Chicago Press.
NiewiadomskiR.BeyanC.SciuttiA. (2022). Affect recognition in hand-object interaction using object-sensed tactile and kinematic data. IEEE Transactions on Haptics, 16(1), 112–117. https://doi.org/10.1109/TOH.2022.3230643
75.
ÖzderL. E.ÖzerD.GöksunT. (2023). Gesture use in L1-turkish and L2-English: Evidence from emotional narrative retellings. Quarterly Journal of Experimental Psychology, 76(8), 1797–1816. https://doi.org/10.1177/17470218221126685
76.
ÖzerD.GöksunT. (2020). Gesture use and processing: A review on individual differences in cognitive resources. Frontiers in Psychology, 11, 573555. https://doi.org/10.3389/fpsyg.2020.573555
77.
ÖzerD.GöksunT.ChatterjeeA. (2019). Differential roles of gestures on spatial language in neurotypical elderly adults and individuals with focal brain injury. Cognitive Neuropsychology, 36(5–6), 282–299. https://doi.org/10.1080/02643294.2019.1618255
78.
PavlenkoA. (2008). Emotion and emotion-laden words in the bilingual lexicon. Bilingualism: Language and Cognition, 11(2), 147–164. https://doi.org/10.1017/S1366728908003283
PouwW. T.De NooijerJ. A.Van GogT.ZwaanR. A.PaasF. (2014). Toward a more embedded/extended perspective on the cognitive function of gestures. Frontiers in Psychology, 5, 359. https://doi.org/10.3389/fpsyg.2014.00359
81.
RedcayE.CarlsonT. A. (2015). Rapid neural discrimination of communicative gestures. Social Cognitive and Affective Neuroscience, 10(4), 545–551. https://doi.org/10.1093/scan/nsu089
82.
RossP.FlackT. (2020). Removing hand form information specifically impairs emotion recognition for fearful and angry body stimuli. Perception, 49(1), 98–112. https://doi.org/10.1177/0301006619893229
RowbothamS.HollerJ.LloydD.WeardenA. (2012). How do we communicate about pain? A systematic analysis of the semantic contribution of co-speech gestures in pain-focused conversations. Journal of Nonverbal Behavior, 36(1), 1–21. https://doi.org/10.1007/s10919-011-0122-5
85.
RowbothamS.WardyA. J.LloydD. M.WeardenA.HollerJ. (2014). Increased pain intensity is associated with greater verbal communication difficulty and increased production of speech and co-speech gestures. PLOS ONE, 9(10), e110779. https://doi.org/10.1371/journal.pone.0110779
86.
SoW. C.ShumP. L. C.WongM. K. Y. (2015). Gesture is more effective than spatial language in encoding spatial information. Quarterly Journal of Experimental Psychology, 68(12), 2384–2401. https://doi.org/10.1080/17470218.2015.1015431
ter BekkeM.DrijversL.HollerJ. (2020). The predictive potential of hand gestures during conversation: An investigation of the timing of gestures in relation to speech. PsyArXiv. https://doi.org/10.31234/osf.io/b5zq7
WeissE. M.KohlerC. G.VonbankJ.StadelmannE.KemmlerG.HinterhuberH.MarksteinerJ. (2008). Impairment in emotion recognition abilities in patients with mild cognitive impairment, early and moderate Alzheimer disease compared with healthy comparison subjects. The American Journal of Geriatric Psychiatry, 16(12), 974–980. https://doi.org/10.1097/JGP.0b013e318186bd53
91.
WespR.HesseJ.KeutmannD.WheatonK. (2001). Gestures maintain spatial imagery. The American Journal of Psychology, 114(4), 591–600. https://doi.org/10.2307/1423612
92.
WilliamsR. F. (2008). Gesture as a conceptual mapping tool. In CienkiA.MüllerC. (Eds.), Metaphor and gesture (pp. 55–92). John Benjamins. https://doi.org/10.1075/gs.3.06wil.
93.
WitkowerZ.TracyJ. L. (2019). Bodily communication of emotion: Evidence for extrafacial behavioral expressions and available coding systems. Emotion Review, 11(2), 184–193. https://doi.org/10.1177/1754073917749880
