Three experiments examine multimodal integration of tone pitch (high/low), facial expression stimuli (happy/angry), and responses (happy/angry) in a compatibility paradigm. When the participants’ task is to imitate facial expressions (Experiment 1), smiles are facilitated by high tones whereas frowns are facilitated by low tones. Experiments 2 and 3 further analyse this effect and show that there is both integration between the tone stimulus and the facial stimulus and between the tone stimulus and the facial response. Results suggest that pitch height is associated with emotion. An interpretation in terms of an embodied cognition approach that emphasizes an interweavement of perception and action is discussed.
BarghJ. A.ChenM.BurrowsL. (1996). Automaticity of social behaviour: Direct effects of trait constructs and stereotype activation on action.Journal of Personality and Social Psychology71, 230–244.
2.
BarsalouL. (2008). Grounded cognition.Annual Review of Psychology59, 617–645.
3.
BowerG. (1981). Mood and memory.American Psychologist36, 129–148.
4.
BroadbentD. E.GregoryM. (1962). Donders’ B- and C-reactions and S–R compatibility.Journal of Experimental Psychology63, 575–578.
5.
ClarkM. S.IsenA. M. (1981). Towards understanding the relationship between feeling states and social behavior.Cognitive social psychology. HastorfA.IsenA. M.Amsterdam: Elsevier/North-Holland73–108.
6.
de GelderB.VroomenJ. (2000). The perception of emotions by ear and by eye.Cognition & Emotion14, 289–311.
7.
DimbergU. (1982). Facial reactions to facial expressions.Psychophysiology19, 643–647.
8.
DimbergU.ThunbergM.GrunedalS. (2002). Facial reactions to emotional stimuli: Automatically controlled emotional responses.Cognition and Emotion16, 449–471.
9.
DrahotaA.CostallA.ReddyV. (2008). The vocal communication of different kinds of smile.Speech Communication51, 278–287.
10.
FernaldA. (1993). Approval and disapproval: Infant responsiveness to vocal affect in familiar and unfamiliar languages.Developmental Psychology64, 657–674.
11.
FischerM. H.ZwaanR. A. (2008). Embodied language: A review of the role of the motor system in language comprehension.The Quarterly Journal of Experimental Psychology61, 825–850.
12.
GlenbergA. M.KaschakM. P. (2003). The body's contribution to language.The psychology of learning and motivation. RossB.New York: Academic Press43, 93–126.
13.
HennenlotterA.SchroederU.ErhardP.CastropF.HaslingerB.StoeckerD. (2005). A common neural basis for receptive and expressive communication of pleasant facial affect.NeuroImage26, 581–591.
14.
HessU.AdamsR. B.KleckR. E. (2007). When two do the same, it might not mean the same: The perception of emotional expressions shown by men and women.Group dynamics and emotional expression. HessU.PhilippotP.New York: Cambridge University Press33–50.
15.
HevnerK. (1937). The affective value of pitch and tempo in music.American Journal of Psychology49, 621–630.
16.
HietanenJ. K.LeppänenJ. M. (2008). Judgment of other people's facial expressions of emotions is influenced by their concurrent affective hand movements.Scandinavian Journal of Psychology49, 221–230.
17.
HietanenJ. K.LeppänenJ. M.IlliM.SurakkaV. (2004). Evidence for the integration of audiovisual emotional information at the perceptual level of processing.European Journal of Cognitive Psychology16, 769–790.
18.
HuronD.KinneyD.PrecodaK. (2006). Influence of pitch height on the perception of submissiveness and threat in musical passages.Empirical Musicology Review1, 170–177.
19.
JuslinP.LaukkaP. (2003). Communication of emotions in vocal expression and music performance: Different channels, same code?.Psychological Bulletin129, 770–814.
20.
KornblumS.HasbroucqT.OsmanA. (1990). Dimensional overlap: Cognitive basis for stimulus response compatibility—a model and taxonomy.Psychological Review97, 253–270.
21.
LakoffG.JohnsonM. (1980). Metaphors we live byChicago: University of Chicago Press.
22.
MassaroD. W.EganP. B. (1996). Perceiving affect from the voice and the face.Psychonomic Bulletin & Review3, 215–221.
23.
MatsumotoD.EkmanP. (1988). Japanese and Caucasian facial expressions of emotion (JACFEE) [CD-ROM]San Francisco: Intercultural and Emotion Research Laboratory.
24.
MeierB. P.RobinsonM. D. (2004). Why the sunny side is up: Associations between affect and vertical position.Psychological Science15, 243–247.
25.
MeltzoffA. N.MooreM. K. (1977). Imitation of facial and manual gestures by human neonates.Science198, 75–78.
26.
MortonE. (1977). On the occurrence and significance of motivation-structural rules in some birds and mammal sounds.The American Naturalist111, 855–869.
27.
MortonE. (1994). Sound symbolism and its role in non-human vertebrate communication.Sound symbolism. HintonL.NicholsJ.OhalaJ.Cambridge, UK: Cambridge University Press348–365.
28.
NiedenthalP. M. (2007). Embodying emotion.Science316, 1002–1005.
29.
NiedenthalP. M.BarsalouL. W.WinkielmanP.Krauth-GruberS.RicF. (2005). Embodiment in attitudes, social perception, and emotion.Personality and Social Psychology Review9, 184–211.
30.
NiedenthalP. M.BrauerM.HalberstadtJ. B.Innes-KerA. H. (2001). When did her smile drop?.Cognition and Emotion15, 853–864.
31.
ObermanL. M.WinkielmanP.RamachandranV. S. (2007). Social Neuroscience2, 167–178.
32.
PourtoisG.de GelderB.VroomenJ.RossionB.CrommelinckM. (2000). The time-course of intermodal binding between seeing and hearing affective information.Neuroreport11, 1329–1333.
33.
SchererK. R.BanseR.WallbottH. G.GoldbeckT. (1991). Vocal cues in emotion encoding and decoding.Motivation and Emotion15, 123–148.
34.
Van den StockJ.RighartR.de GelderB. (2007). Body expressions influence recognition of emotions in the face and voice.Emotion7, 487–494.
35.
WalkerP.SmithS. (1984). Stroop interference based on the synaesthetic qualities of auditory pitch.Perception13, 75–81.
36.
WallbottH. G. (1991). Recognition of emotion from facial expression via imitation? Some indirect evidence for an old theory.British Journal of Social Psychology30, 207–219.
37.
WarrenJ. G.SauterD. A.EisnerF.WilandJ.DresnerM. A.WiseR. J. S. (2006). Positive emotions preferentially engage an auditory-motor “mirror” system.The Journal of Neuroscience26, 13067–13975.
38.
WolffP. H. (1987). The development of behavioural states and the expression of emotions in early infancyChicago: University of Chicago Press.
