The current study investigated what can be understood from another person’s tone of voice. Participants from five English-speaking nations (Australia, India, Kenya, Singapore, and the United States) listened to vocal expressions of nine positive and nine negative affective states recorded by actors from their own nation. In response, they wrote open-ended judgments of what they believed the actor was trying to express. Responses cut across the chronological emotion process and included descriptions of situations, cognitive appraisals, feeling states, physiological arousal, expressive behaviors, emotion regulation, and attempts at social influence. Accuracy in terms of emotion categories was overall modest, whereas accuracy in terms of valence and arousal was more substantial. Coding participants’ 57,380 responses yielded a taxonomy of 56 categories, which included affective states as well as person descriptors, communication behaviors, and abnormal states. Open-ended responses thus reveal a wide range of ways in which people spontaneously perceive the intent behind emotional speech prosody.
AbelinÅ.AllwoodJ. (2000). Cross linguistic interpretation of emotional prosody. In CowieR.Douglas-CowieE.SchröderM. (Eds.), Proceedings of the ISCA Workshop on Speech and Emotion (pp. 110–113). International Speech Communication Association.
BoucherJ. D.CarlsonG. E. (1980). Recognition of facial expression in three cultures. Journal of Cross-Cultural Psychology, 11, 263–280. https://doi.org/10.1177/0022022180113003
4.
BuckR. (2014). Emotion: A biosocial synthesis. Cambridge University Press.
5.
CalvoM. G.NummenmaaL. (2016). Perceptual and affective mechanisms in facial expression recognition: An integrative review. Cognition and Emotion, 30, 1081–1106. https://doi.org/10.1080/02699931.2015.1049124
6.
CordaroD. T.KeltnerD.TsheringS.WangchukD.FlynnL. M. (2016). The voice conveys emotion in ten globalized cultures and one remote village in Bhutan. Emotion, 16, 117–128. https://doi.org/10.1037/emo0000100
7.
CowenA. S.LaukkaP.ElfenbeinH. A.LiuR.KeltnerD. (2019). The primacy of categories in the recognition of 12 emotions in speech prosody across two cultures. Nature Human Behaviour, 3, 369–382. https://doi.org/10.1038/s41562-019-0533-6
8.
DarwinC. (1965). The expression of the emotions in man and animals. University of Chicago Press. (Original work published 1872)
9.
DiGirolamoM. A.RussellJ. A. (2017). The emotion seen in a face can be a methodological artifact: The process of elimination hypothesis. Emotion, 17, 538–546. https://doi.org/10.1037/emo0000247
EkmanP. (1997). Should we call it expression or communication?Innovation: The European Journal of Social Science Research, 10(4), 333–344. https://doi.org/10.1080/13511610.1997.9968538
12.
ElfenbeinH. A.AmbadyN. (2002). On the universality and cultural specificity of emotion recognition: A meta-analysis. Psychological Bulletin, 128, 203–235. https://doi.org/10.1037/0033-2909.128.2.203
13.
EllsworthP. C.SchererK. R. (2003). Appraisal processes in emotion. In DavidsonR. J.SchererK. R.GoldsmithH. H. (Eds.), Handbook of affective sciences (pp. 572–595). Oxford University Press.
14.
FaulF.ErdfelderE.BuchnerA.LangA.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
15.
FehrB.RussellJ. A. (1984). Concept of emotion viewed from a prototype perspective. Journal of Experimental Psychology: General, 113, 464–486. https://doi.org/10.1037/0096-3445.113.3.464
16.
FrankM. G.StennettJ. (2001). The forced-choice paradigm and the perception of facial expressions of emotion. Journal of Personality and Social Psychology, 80, 75–85. https://doi.org/10.1037/0022-3514.80.1.75
FrijdaN. H. (2007). The laws of emotion. Lawrence Erlbaum.
19.
GendronM.RobersonD.van der VyverJ. M.BarrettL. F. (2014). Cultural relativity in perceiving emotion from vocalizations. Psychological Science, 25, 911–920. https://doi.org/10.1177/0956797613517239
20.
GibsonJ. J. (1979). The ecological approach to visual perception. Houghton Mifflin.
21.
GilliozG.FontaineJ. R. J.SorianoC.SchererK. R. (2016). Mapping emotion terms into affective space: Further evidence for a four-dimensional structure. Swiss Journal of Psychology, 75, 141–148.
22.
GoudbeekM.SchererK. R. (2010). Beyond arousal: Valence and potency/control cues in the vocal expression of emotion. Journal of the Acoustical Society of America, 128, 1322–1336. https://doi.org/10.1121/1.3466853
23.
GreasleyP.SherrardC.WatermanM. (2000). Emotion in language and speech: Methodological issues in naturalistic approaches. Language and Speech, 43, 355–375. https://doi.org/10.1177/00238309000430040201
24.
HaidtJ.KeltnerD. (1999). Culture and facial expression: Open-ended methods find more expressions and a gradient of recognition. Cognition and Emotion, 13, 225–266. https://doi.org/10.1080/026999399379267
25.
HallJ. A.AndrzejewskiS. A.YopchickJ. E. (2009). Psychosocial correlates of interpersonal sensitivity: A meta-analysis. Journal of Nonverbal Behavior, 33, 149–180. https://doi.org/10.1007/s10919-009-0070-5
26.
HallJ. A.GunneryS. D.HorganT. G. (2016). Gender differences in interpersonal accuracy. In HallJ. A.Schmid MastM.WestT. V. (Eds.), The social psychology of perceiving others accurately (pp. 309–327). Cambridge University Press.
27.
HofstedeG. (2001). Culture’s consequences: Comparing values, behaviors, institutions, and organizations across nations (2nd ed.). SAGE.
28.
HönemannA.MixdorffH.RilliardA. (2015). Classification of auditory-visual attitudes in German. In Proceedings of the 1st Joint Conference on Facial Analysis, Animation, and Auditory-Visual Speech Processing (FAAVSP 2015) (pp. 202–207). International Speech Communication Association.
29.
JarvisB. G. (2008). MediaLab (Version 2008) [Computer Software]. Empirisoft Corp.
30.
JuslinP. N. (1997). Can results from studies of perceived expression in musical performances be generalized across response formats?Psychomusicology, 16, 77–101. https://doi.org/10.1037/h0094065
31.
JuslinP. N.LaukkaP. (2003). Communication of emotions in vocal expression and music performance: Different channels, same code?Psychological Bulletin, 129, 770–814. https://doi.org/10.1037/0033-2909.129.5.770
32.
JuslinP. N.LaukkaP.BänzigerT. (2018). The mirror to our soul? Comparisons of spontaneous and posed vocal expression of emotion. Journal of Nonverbal Behavior, 42, 1–40. https://doi.org/10.1007/s10919-017-0268-x
33.
LaukkaP.ElfenbeinH. A. (2012). Emotion appraisal dimensions can be inferred from vocal expressions. Social Psychological and Personality Science, 3, 529–536. https://doi.org/10.1177/1948550611428011
34.
LaukkaP.ElfenbeinH. A.ChuiW.ThingujamN. S.IrakiF. K.RockstuhlT.AlthoffJ. (2010). Presenting the VENEC corpus: Development of a cross-cultural corpus of vocal emotion expressions and a novel method of annotating emotion appraisals. In DevillersL.SchullerB.CowieR.Douglas-CowieE.BatlinerA. (Eds.), Proceedings of the LREC 2010 Workshop on Corpora for Research on Emotion and Affect (pp. 53–57). European Language Resources Association.
35.
LaukkaP.ElfenbeinH. A.ThingujamN. S.RockstuhlT.IrakiF. K.ChuiW.AlthoffJ. (2016). The expression and recognition of emotions in the voice across five nations: A lens model analysis based on acoustic features. Journal of Personality and Social Psychology, 111, 686–705. https://doi.org/10.1037/pspi0000066
36.
LaukkaP.JuslinP. N.BresinR. (2005). A dimensional approach to vocal expression of emotion. Cognition and Emotion, 19, 633–653. https://doi.org/10.1080/02699930441000445
37.
LausenA.SchachtA. (2018). Gender differences in the recognition of vocal emotions. Frontiers in Psychology, 9, Article 882.
38.
LazarusR. S. (1991). Emotion and adaptation. Oxford University Press.
39.
Limbrecht-EcklundtK.ScheckA.Jerg-BretzkeL.WalterS.HoffmanH.TraueH. C. (2013). The effect of forced choice on facial emotion recognition: A comparison to open verbal classification of emotion labels. GMS Psycho-Social-Medicine, 10, 1–8. https://doi.org/10.3205/psm000094
40.
NelsonN. L.RussellJ. A. (2016). A facial expression of pax: Assessing children’s recognition of emotion from faces. Journal of Experimental Child Psychology, 141, 49–64. https://doi.org/10.1016/j.jecp.2015.07.016
41.
NordströmH.LaukkaP.ThingujamN. S.SchubertE.ElfenbeinH. A. (2017). Appraisal dimensions inferred from vocal emotion expressions are consistent across cultures: A comparison between Australia and India. Royal Society Open Science, 4, Article 170912. https://doi.org/10.1098/rsos.170912
42.
OrtonyA.CloreG. L.CollinsA. (1988). The cognitive structure of emotions. Cambridge University Press.
43.
PlanalpS.DeFranciscoV. L.RutherfordD. (1996). Varieties of cues to emotion in naturally occurring situations. Cognition and Emotion, 10, 137–153. https://doi.org/10.1080/026999396380303
44.
RosenbergE. L.EkmanP. (1995). Conceptual and methodological issues in the judgment of facial expressions of emotion. Motivation and Emotion, 19, 111–138. https://doi.org/10.1007/BF02250566
45.
RosenthalR.HallJ. A.DiMatteoM. R.RogersP. L.ArcherD. (1979). Sensitivity to nonverbal communication: The PONS Test. The Johns Hopkins University Press.
46.
RussellJ. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178. https://doi.org/10.1037/h0077714
47.
RussellJ. A. (1993). Forced-choice response format in the study of facial expression. Motivation and Emotion, 17, 41–51. https://doi.org/10.1007/BF00995206
48.
RussellJ. A. (1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 115, 102–141. https://doi.org/10.1037/0033-2909.115.1.102
49.
RussellJ. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145–172.
50.
RussellJ. A.Feldman BarrettL. (1999). Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology, 76, 805–819.
51.
SauterD. A. (2017). The nonverbal communication of positive emotions: An emotion family approach. Emotion Review, 9, 222–234. https://doi.org/10.1177/1754073916667236
52.
SchererK. R. (1994). Affect bursts. In GoozenS. H. M.van de PollN. E.SergeantJ. A. (Eds.), Emotions: Essays on emotion theory (pp. 161–193). Lawrence Erlbaum.
SchererK. R. (2013). Vocal markers of emotion: Comparing induction and acting elicitation. Computer Speech and Language, 27, 40–58. https://doi.org/10.1016/j.csl.2011.11.003
55.
SchererK. R.BänzigerT. (2010). On the use of actor portrayals in research on emotional expression. In SchererK. R.BänzigerT.RoeschE. B. (Eds.) Blueprint for affective computing: A sourcebook (pp. 271–294). Oxford University Press.
SchlosbergH. (1952). The description of facial expressions in terms of two dimensions. Journal of Experimental Psychology, 44, 229–237. https://doi.org/10.1037/h0055778
58.
Schmid MastM.HallJ. A. (2018). The impact of interpersonal accuracy on behavioral outcomes. Current Directions in Psychological Science, 27, 309–314.
59.
ShochiT.RilliardA.AubergéV.EricksonD. (2009). Intercultural perception of English, French and Japanese social affective prosody. In HancilS. (Ed.), The role of prosody in affective speech (pp. 31–60). Peter Lang.
60.
Simon-ThomasE. R.KeltnerD. J.SauterD.Sinicropi-YaoL.AbramsonA. (2009). The voice conveys specific emotions: Evidence from vocal burst displays. Emotion, 9, 838–846. https://doi.org/10.1037/a0017810
61.
StanislavskiC. (1936). An actor prepares. Theatre Arts Books/Methuen.
62.
TodorovA. (2017). Face value: The irresistible influence of first impressions. Princeton University Press.
63.
TullisT.WoodL. (2004). How many users are enough for a card-sorting study? [Conference session]. Proceedings of the Usability Professionals Association (UPA) 2004 Conference, Minneapolis, MN, June7–11, 2004.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
