We investigated through electrophysiological recordings how music-induced emotions are recognized and combined with the emotional content of written sentences. Twenty-four sad, joyful, and frightening musical tracks were presented to 16 participants reading 270 short sentences conveying a sad, joyful, or frightening emotional meaning. Audiovisual stimuli could be emotionally congruent or incongruent with each other; participants were asked to pay attention and respond to filler sentences containing cities’ names, while ignoring the rest. The amplitude values of event-related potentials (ERPs) were subjected to repeated measures ANOVAs. Distinct electrophysiological markers were identified for the processing of stimuli inducing fear (N450, either linguistic or musical), for language-induced sadness (P300) and for joyful music (positive P2 and LP potentials). The music/language emotional discordance elicited a large N400 mismatch response (p = .032). Its stronger intracranial source was the right superior temporal gyrus (STG) devoted to multisensory integration of emotions. The results suggest that music can communicate emotional meaning as distinctively as language.
AltenmüllerE.SiggelS.MohammadiB.SamiiA.MünteT. F. (2014). Play it again, Sam: Brain correlates of emotional music recognition. Frontiers in Psychology, 5, Article 114. doi:10.3389/fpsyg.2014.00114
2.
Angulo-PerkinsA.AubéW.PeretzI.BarriosF. A.ArmonyJ. L.ConchaL. (2014). Music listening engages specific cortical regions within the temporal lobes: Differences between musicians and non-musicians. Cortex, 59, 126–137. doi:10.1016/j.cortex.2014.07.013
3.
BaumgartnerT.LutzK.SchmidtC. F.JänckeL. (2006). The emotional power of music: How music enhances the feeling of affective pictures. Brain Research, 1075, 151–164. doi:10.1016/j.brainres.2005.12.065
4.
BeaucousinV.LacheretA.TurbelinM. R.MorelM.MazoyerB.Tzourio-MazoyerN. (2007). fMRI study of emotional speech comprehension. Cerebral Cortex, 17, 339–352. doi:10.1093/cercor/bhj151
5.
BelinP.FecteauS.CharestI.NicastroN.HauserM. D.ArmonyJ. L. (2008). Human cerebral response to animal affective vocalizations. Proceedings of the Royal Society B: Biological Sciences, 275, 473–481. doi:10.1098/rspb.2007.1460
6.
BernatE.BunceS.ShevrinH. (2001). Event-related brain potentials differentiate positive and negative mood adjectives during both supraliminal and subliminal visual processing. International Journal of Psychophysiology, 42, 11–34. doi:10.1016/s0167-8760(01)00133-7
7.
BradleyM. M.LangP. J. (1994). Measuring emotion: The Self-Assessment Manikin and the Semantic Differential. Journal of Behavior Therapy and Experimental Psychiatry, 25, 49–59. doi:10.1016/0005-7916(94)90063-9
8.
CoutinhoE.DibbenN. (2013). Psychoacoustic cues to emotion in speech prosody and music. Cognition and Emotion, 27, 658–684. doi:10.1080/02699931.2012.732559
9.
CuthbertB. N.SchuppH. T.BradleyM. M.BirbaumerN.LangP. J. (2000). Brain potentials in affective picture processing: Covariation with autonomic arousal and affective report. Biological Psychology, 52, 95–111. doi:10.1016/s0301-0511(99)00044-7
10.
GoerlichK. S.WittemanJ.SchillerN. O.Van HeuvenV. J.AlemanA.MartensS. (2012). The nature of affective priming in music and speech. Journal of Cognitive Neuroscience, 24, 1725–1741. doi:10.1162/jocn_a_00213
11.
GrossmannT.StrianoT.FriedericiA. D. (2005). Infants’ electric brain responses to emotional prosody. NeuroReport, 16, 1825–1828. doi:10.1097/01.wnr.0000185964.34336.b1
12.
JanataP. (2009). The neural architecture of music-evoked autobiographical memories. Cerebral Cortex, 19, 2579–2594. doi:10.1093/cercor/bhp008
13.
JanataP.BirkJ.Van HornJ. D.LemanM.TillmannB.BharuchaJ. J. (2002). The cortical topography of tonal structures underlying Western music. Science, 298, 2167–2170. doi:10.1126/science.1076262
14.
JeongJ. W.DiwadkarV. A.ChuganiC. D.SinsoongsudP.MuzikO.BehenM. E.ChuganiD. C. (2011). Congruence of happy and sad emotion in music and faces modifies cortical audiovisual activation. NeuroImage, 54, 2973–2982. doi:10.1016/j.neuroimage.2010.11.017
JuslinP. N. (2013). From everyday emotions to aesthetic emotions: Towards a unified theory of musical emotions. Physics of Life Reviews, 10, 235–266. doi:10.1016/j.plrev.2013.05.008
17.
JuslinP. N.LiljeströmS.VästfjällD.BarradasG.SilvaA. (2008). An experience sampling study of emotional reactions to music: Listener, music, and situation. Emotion, 8, 668–683. doi:10.1037/a0013505
18.
KanskeP.KotzS. A. (2007). Concreteness in emotional words: ERP evidence from a hemifield study. Brain Research, 1148, 138–148. doi:10.1016/j.brainres.2007.02.044
19.
KhalfaS.SchonD.AntonJ.Liégeois-ChauvelC. (2005). Brain regions involved in the recognition of happiness and sadness in music. Neuroreport, 16, 1981–1984. doi:10.1097/00001756-200512190-00002
20.
KhalfaS.GuyeM.PeretzI.ChaponF.GirardN.ChauvelP.Liégeois-ChauvelC. (2008). Evidence of lateralized anteromedial temporal structures involvement in musical emotion processing. Neuropsychologia, 46, 2485–2493. doi:10.1016/j.neuropsychologia.2008.04.009
21.
KoelschS. (2011). Toward a neural basis of music perception—A review and updated model. Frontiers in Psychology, 2, Article 110. doi:10.3389/fpsyg.2011.00110
22.
KoelschS.CramonD. Y. V.MüllerK.FriedericiA. D. (2006). Investigating emotion with music: An fMRI study. Human Brain Mapp, 27, 239–250. doi:10.1002/hbm.20180
23.
KoelschS.SkourasS.FritzT.HerreraP.BonhageC.KüssnerM. B.JacobsA. M. (2013). The roles of superficial amygdala and auditory cortex in music-evoked fear and joy. NeuroImage, 81, 49–60. doi:10.1016/j.neuroimage.2013.05.008
24.
KotzS. A.PaulmannS. (2011). Emotion, language and the brain. Language and Linguist Compass, 5, 108–125. doi:10.1111/j.1749-818X.2010.00267.x
25.
KutasM.FedermeierK. D. (2000). Electrophysiology reveals semantic memory use in language comprehension. Trends in Cognitive Science, 4, 463–470. doi:10.1016/S1364-6613(00)01560-6
26.
LauE. F.PhillipsC.PoeppelD. (2008). A cortical network for semantics: (De)constructing the N400. Nature Review Neuroscience, 9, 920–933. doi:10.1038/nrn2532
27.
MaurerU.ZevinJ. D.McCandlissB. D. (2008). Left-lateralized N170 effects of visual expertise in reading: Evidence from Japanese syllabic and logographic scripts. Journal of Cognitive Neuroscience, 20, 878–891. doi:10.1162/jocn.2008.20125
28.
MorrisD. J.SteinmetzgerK.TønderingJ. (2016). Auditory event-related responses to diphthongs in different attention conditions. Neuroscience Letters, 626, 158–163. doi:10.1016/j.neulet.2016.05.002
29.
MortonJ. B.TrehubS.E. (2001). Children’s understanding of emotion in speech. Child Development, 72, 834–843. doi:10.1111/1467-8624.00318
30.
PankseppJ. (1994). The basics of emotion. In EkmanP.DavidsonR. J. (Eds.), The nature of emotions: Fundamental questions (pp. 20–24). Oxford, UK: Oxford University Press.
31.
PaquetteS.TakerkartS.SagetS.PeretzI.BelinP. (2018). Cross-classification of musical and vocal emotions in the auditory cortex. Annuals of the National Academy of Sciences, 1423, 329–337. doi:10.1111/nyas.13666
32.
ParkesL.PerryC.GoodinP. (2016). Examining the N400m in affectively negative sentences: A magnetoencephalography study. Psychophysiology, 53, 689–704. doi:10.1111/psyp.12601
33.
PaulmannS.BleichnerM.KotzS. A. (2013). Valence, arousal, and task effects in emotional prosody processing. Frontiers in Psychology, 4, Article 345. doi:10.3389/fpsyg.2013.00345
34.
PaulmannS.KotzS. A. (2008). Early emotional prosody perception based on different speaker. NeuroReport, 19, 209–213. doi:10.1097/WNR.0b013e3282f454db
35.
PehrsC.DesernoL.BakelsJ. H.SchlochtermeierL. H.KappelhoffH.JacobsA. M.KuchinkeL. (2014). How music alters a kiss: Superior temporal gyrus controls fusiform-amygdalar effective connectivity. Social Cognitive Affective Neuroscience, 9, 1770–1778. doi:10.1093/scan/nst169
36.
PolkT.StallcupM.AguirreG.AlsopD.D’EspositoM.DetreA. (2002). Neural specialization for letter recognition. Journal of Cognitive Neuroscience, 14, 145–159. doi:10.1162/089892902317236803
37.
ProverbioA. M.CamporealeE.BrusaA. (2020). Multimodal recognition of emotions in music and facial expressions. Frontiers in Human Neuroscience, 14, Article 32. doi:10.3389/fnhum.2020.00032
38.
ProverbioA. M.De BenedettoF. (2018). Auditory enhancement of visual memory encoding is driven by emotional content of the auditory material and mediated by superior frontal cortex. Biological Psychology, 132, 164–175. doi:10.1016/j.biopsycho.2017.12.003
39.
ProverbioA. M.De BenedettoF.GuazzoneM. (2020). Shared neural mechanisms for processing emotions in music and vocalizations. European Journal of Neuroscience, 51, 1987–2007. doi:10.1111/ejn.14650
40.
ProverbioA. M.ManfrinL.ArcariL. A.De BenedettoF.GazzolaM.GuardamagnaM.ZaniA. (2015). Non-expert listeners show decreased heart rate and increased blood pressure (fear bradycardia) in response to atonal music. Frontiers in Psychology, 6, Article 1646. doi:10.3389/fpsyg.2015.01646
41.
ProverbioA. M.SantoniS.Adorni. (2020). ERP makers of valence coding in emotional speech processing. iScience, 23, 100933. doi:10.1016/j.isci.2020.100933
42.
ProverbioA. M.ZaniA.AdorniR. (2008). The left fusiform area is affected by written frequency of words. Neuropsychologia, 46, 2292–2299. doi:10.1016/j.neuropsychologia.2008.03.024
43.
RodwayP.SchepmanA. (2007). Valence specific laterality effects in prosody: Expectancy account and the effects of morphed prosody and stimulus lead. Brain & Cognition, 63, 31–41. doi:10.1016/j.bandc.2006.07.008
44.
RossE. D.ThompsonR. D.YenkoskyJ. (1997). Lateralization of affective prosody in brain and the callosal integration of hemispheric language functions. Brain and Language, 56, 27–54. doi:10.1006/brln.1997.1731
45.
RussellJ. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161–1178. doi:10.1037/h0077714
46.
SalmiJ.KoistinenO. P.GlereanE.JylänkiP.VehtariA.JääskeläinenI. P.. . . SamsM. (2017). Distributed neural signatures of natural audiovisual speech and music in the human auditory cortex. NeuroImage, 157, 108–117. doi:10.1016/j.neuroimage.2016.12.005
47.
SchirmerA.KotzS. A.FriedericiA. D. (2002). Sex differentiates the role of emotional prosody during word processing. Cognitive Brain Research, 14, 228–233. doi:10.1016/s0926-6410(02)00108-8
48.
SchmidtL. A.TrainorL. J. (2001). Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cognition and Emotion, 15, 487–500. doi:10.1016/j.bandc.2007.02.008
49.
SchuppH. T.ÖhmanA.JunghöferM.WeikeA. I.StockburgerJ.HammA. O. (2004). The facilitated processing of threatening faces: An ERP analysis. Emotion, 4, 189–200. doi:10.1037/1528-3542.4.2.189
50.
SpreckelmeyerK. N.KutasM.UrbachT. P.AltenmüllerE.MünteT. F. (2006). Combined perception of emotion in pictures and musical sounds. Brain Research, 1070, 160–170. doi:10.1016/j.brainres.2005.11.075
51.
TrostW.EthoferT.ZentnerM.VuilleumierP. (2012). Mapping aesthetic musical emotions in the brain. Cerebral Cortex, 22, 2769–2783. doi:10.1093/cercor/bhr353
52.
VidasD.DingleG. A.NelsonN. L. (2018). Children’s recognition of emotion in music and speech. Music and Science, 1, 1–10. doi:10.1177/2059204318762650
53.
VieillardS.PeretzI.GosselinN.KhalfaS.GagnonL.BouchardB. (2008). Happy, sad, scary and peaceful musical excerpts for research on emotions. Cognition and Emotion, 22, 720–752. doi:10.1080/02699930701503567
54.
VuilleumierP.TrostW. (2015). Music and emotions: From enchantment to entrainment. Annuals of the National Academy of Sciences, 1337, 212–222. doi:10.1111/nyas.12676
55.
ZaccarellaE.MeyerL.MakuuchiM.FriedericiA. D. (2017). Building by syntax: The neural basis of minimal linguistic structures. Cerebral Cortex, 27, 411–421. doi:10.1093/cercor/bhv234
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
