Research showing that emotions can be recognized in point-light displays of human dyadic interactions was extended in the current study by investigating the impact of music on the perception of normal and exaggerated expressions of happiness, contentedness, sadness and anger in such visual stimuli. Sixteen musically untrained participants viewed short video clips of these emotion portrayals, each presented with emotionally compatible (e.g., happy music accompanies a happy interaction) and emotionally incompatible piano music (e.g., sad music accompanies a happy interaction). It was hypothesized that music will increase the accuracy of emotion judgements in displays where auditory and visual information is compatible relative to displays with incompatible audio-visual information. A two-dimensional emotion space was used to record participants' judgements of emotion in only the visual stimuli. Results indicated that music affected the accuracy of emotion judgements. Happiness and sadness were perceived more accurately in compatible than in incompatible conditions, while the opposite was the case for contentedness. Anger was perceived accurately in all conditions. Exaggerated expressions of sadness, which were evaluated more accurately than normal expressions of sadness, were also found to be resistant to the music. These findings can be interpreted in the light of previous research demonstrating that music's cross-modal impact depends on the degree of emotional ambiguity in the visual display. More generally, the results demonstrate that the perception of emotions in biological motion can be affected by music.
AtkinsonA. P.DittrichW. H.GemmellA. J.YoungA. W. (2004). Emotion perception from dynamic and static body expressions in point-light and full-light displays. Perception, 33 (6), 717–746.
2.
AtkinsonA. P.TunstallM. L.DittrichW. H. (2007). Evidence for distinct contributions of form and motion information to the recognition of emotions from body gestures. Cognition, 104, 59–72.
3.
BolivarV. J.CohenA. J.FentressJ. C. (1994). Semantic and formal congruency in music and motion pictures: Effects on the interpretation of visual action. Psychomusicology, 13, 28–59.
4.
ChouchourelouA.MatsukaT.HarberK.ShiffrarM. (2006). The visual analysis of emotional actions. Social Neuroscience, 1 (1), 63–74.
5.
ClarkeE. F. (2004). Empirical methods in the study of performance. In ClarkeE. F.CookN. (Eds.), Empirical musicology: Aims, methods, prospects (pp. 77–102). New York: Oxford University Press.
6.
ClarkeT. J.BradshawM. F.FieldD. T.HampsonS. E.RoseD. (2005). The perception of emotion from body movement in point-light displays of interpersonal dialogue. Perception, 34 (10), 1171–1180.
7.
CohenA. J. (1993). Associationism and musical soundtrack phenomena. Contemporary Music Review, 9 (1), 163–178.
8.
CohenA. J. (2005). How music influences the interpretation of film and video: Approaches from experimental psychology. In KendallR.SavageR. (Eds.), Selected reports in ethnomusicology: Perspectives in systematic musicology (Vol. 12, pp. 15–36). Los Angeles, CA: Ethnomusicology Publications, UCLA.
9.
DahlS.FribergA. (2007). Visual perception of expressiveness in musician's body movements. Music Perception, 24 (5), 433–454.
10.
DavidsonJ. W. (1993). Visual perception of performance manner in the movements of solo musicians. Psychology of Music, 21, 103–113.
11.
DavidsonJ. W.CorreiaJ. S. (2002). Body movement. In ParncuttR.McPhersonG. E. (Eds.), The science and psychology of music performance. Creative strategies for teaching and learning (pp. 237–250). Oxford: Oxford University Press.
12.
DittrichW. H. (1993). Action categories and the perception of biological motion. Perception, 22 (1), 15–22.
13.
DittrichW. H.TrosciankoT.LeaS. E. G.MorganD. (1996). Perception of emotion from dynamic point-light displays represented in dance. Perception, 25 (6), 727–738.
14.
DissanayakeE. (2006). Ritual and ritualization: Musical means of conveying and shaping emotion in humans and other animals. In BrownS.VolgstenU. (Eds.), Music and manipulation (pp. 31–56). Oxford and New York: Berghahn Books.
15.
EkmanP. (1993). Facial expression and emotion. American Psychological Association, 48 (4). 376–379.
16.
EkmanP. (1999). Basic emotions. In DalgleishT.PowerM. (Eds.), Handbook of cognition and emotion (pp. 45–60). Sussex: John Wiley.
17.
GabrielssonA.JuslinP. N. (1996). Emotional expression in music performance: Between the performer's intention and the listener's experience. Psychology of Music, 24, 68–91.
18.
GabrielssonA.JuslinP. N. (2003). Emotional expression in music. In DavidsonR. J.SchererK. R.GoldsmithH. H. (Eds.), Handbook of affective sciences (pp. 503–534). New York and Oxford: Oxford University Press.
19.
GabrielssonA.LindströmE. (2001). The influence of musical structure on emotional expression. In JuslinP. N.SlobodaJ. A. (Eds.), Music and Emotion: Theory and research (pp. 223–248). Oxford: Oxford University Press.
20.
JuslinP. N. (2001). Communicating emotion in music performance: A review and a theoretical framework. In JuslinP. N.SlobodaJ. A. (Eds.), Music and emotion: Theory and research (pp. 309–337). Oxford: Oxford University Press.
21.
JuslinP. N.LaukkaP. (2003). Communication of emotions in vocal expression and music performance: Different channels, same code?. Psychological Bulletin, 129 (5), 770–814.
22.
JuslinP. N.TimmersR. (2010). Expression and communication of emotion in music performance. In JuslinP. N.SlobodaJ. A. (Eds.), Handbook of music and emotion: Theory, research, and applications (pp. 453–489). Oxford: Oxford University Press.
23.
LipscombS. D.TolchinskyD. E. (2005). The role of music communication in cinema. In MiellD.MacDonaldR.HargreavesD. (Eds.), Musical Communication (pp. 383–404). Oxford: Oxford University Press.
24.
LivingstoneS. R.ThompsonW. F.RussoF. A. (2009). Facial expressions and emotional singing: A study of perception and production with motion capture and electro-myography. Music Perception, 26 (5), 475–488.
25.
MadisonG. (2000). Properties of expressive variability patterns in music performances. Journal of New Music Research, 29 (4), 335–356.
26.
PollickF. E.PatersonH. M.BruderlinA.SanfordA. J. (2001). Perceiving affect from arm movement. Cognition, 82 (2), B51–B61.
27.
RoseD.ClarkeT. J. (2009). Look who's talking: Visual detection of speech from whole-body biological motion cues during emotive interpersonal conversation. Perception, 38, 153–156.
28.
RunesonS.FrykholmG. (1983). Kinematic specification of dynamics as an informational basis for person-and-action perception: Expectation, gender recognition, and deceptive intention. Journal of Experimental Psychology: General, 112 (4), 585–615.
29.
RussellJ. A.WeissA.MendelsohnG. A. (1989). Affect Grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57 (3), 493–502.
30.
RussellJ. A. (1994). Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. Psychological Bulletin, 125 (1), 102–141.
31.
SchubertE. (1996). Continuous response to music using a two-dimensional emotion space. In Proceedings of the 4th International Conference of Music Perception and Cognition (pp. 263–268). Montreal, Canada: ICMPC.
32.
SchubertE. (2004). Modeling perceived emotion with continuous musical features. Music Perception, 21 (4), 561–585.
33.
SlobodaJ. A.JuslinP. N. (2001). Psychological perspectives on music and emotion. In JuslinP. N.SlobodaJ. A. (Eds.), Music and emotion: Theory and research (pp. 71–104). Oxford: Oxford University Press.
34.
ThompsonW. F.RussoF. A.QuintoL. (2008). Audio-visual integration of emotional cues in song. Cognition & Emotion, 22 (8), 1457–1470.
35.
TrojeN. F.WesthoffC.LavrovM. (2005). Person identification from biological motion: Effects of structural and kinematic cues. Perception & Psychophysics, 67 (4), 667–675.
36.
Van den StockJ.PeretzI.GrèzesJ.De GelderB. (2009). Instrumental music influences recognition of emotional body language. Brain Topography, 21 (3–4), 216–220.
37.
VinesB. W.KrumhanslC. L.WanderleyM. M.LevitinD. J. (2006). Cross-modal interactions in the perception of musical performance. Cognition, 101, 80–113.
38.
WallbottH. G. (1998). Bodily expression of emotion. European Journal of Social Psychology, 28 (6), 879–896.
39.
ZehnderS. M.LipscombS. D. (2006). The role of music in video games. In VordererP.BryantJ. (Eds.), Playing videos games: Motives, responses, and consequences (pp. 241–257). Mahwah, NJ: Lawrence Erlbaum Associates.
