The integration of music sounds with concurrent visual scenes or objects is a occurrence in our daily lives, attracting the attention of researchers investigating how music influences our perceptions of simultaneous visuals. This study specifically investigates the role of musical pitch height in shaping our judgments of visual brightness during concurrent music–visual events. Participants were presented with pitch-modified versions of a wide range of emotional music pieces alongside various visual stimuli. The results demonstrate that lower-pitched music tends to elicit darker judgments of visual objects than higher-pitched music, when the actual visual brightness level remains constant. These findings suggest the influence of musical pitch height on introducing biases in our evaluations of visual brightness within the context of concurrent music-visual experiences, contributing to the advancement of our theoretical understanding of the complex audiovisual integration involving music in our everyday lives.
AnsaniA.MariniM.D’ErricoF.PoggiI. (2020). How soundtracks shape what we see: Analyzing the influence of music on visual scenes through self-assessment, eye tracking, and pupillometry. Frontiers in Psychology, 11, Article 2242.
2.
BaumgartnerT.LutzK.SchmidtC. F.JänckeL. (2006). The emotional power of music: How music enhances the feeling of affective pictures. Brain Research, 1075(1), 151–164.
3.
BhattacharyaJ.LindsenJ. P. (2016). Music for a brighter world: Brightness judgment bias by musical emotion. PLOS ONE, 11(2), Article e0148959. https://doi.org/10.1371/journal.pone.0148959
4.
BondB.StevensS. S. (1969). Cross-modality matching of brightness to loudness by 5-year-olds. Perception & Psychophysics, 6(6), 337–339. https://doi.org/10.3758/BF03212787
5.
Braun JanzenT.de OliveiraB.Ventorim FerreiraG.SatoJ. R.Feitosa-SantanaC.VanzellaP. (2023). The effect of background music on the aesthetic experience of a visual artwork in a naturalistic environment. Psychology of Music, 51(1), 16–32.
6.
BurnsB.SheppB. E. (1988). Dimensional interactions and the structure of psychological space: The representation of hue, saturation, and brightness. Perception & Psychophysics, 43(5), 494–507.
7.
ChenY. C.SpenceC. (2017). Assessing the role of the “unity assumption” on multisensory integration: A review. Frontiers in Psychology, 8, Article 445. https://doi.org/10.3389/fpsyg.2017.00445
8.
ChiouR.RichA. N. (2012). Cross-modality correspondence between pitch and spatial location modulates attentional orienting. Perception, 41(3), 339–353. https://doi.org/10.1068/p7161
9.
ChoiI.LeeJ. Y.LeeS. H. (2018). Bottom-up and top-down modulation of multisensory integration. Current Opinion in Neurobiology, 52, 115–122. https://doi.org/10.1016/j.conb.2018.05.002
10.
CollierW. G.HubbardT. L. (1998). Judgments of happiness, brightness, speed and tempo change of auditory stimuli varying in pitch and tempo. Psychomusicology: A Journal of Research in Music Cognition, 17(1–2), 36–55. https://doi.org/10.1037/h0094060
11.
EitanZ.TimmersR. (2010). Beethoven’s last piano sonata and those who follow crocodiles: Cross-domain mappings of auditory pitch in a musical context. Cognition, 114(3), 405–422. https://doi.org/10.1016/j.cognition.2009.10.013
12.
FraserC.BradfordJ. A. (2013). Music to your brain: Background music changes are processed first, reducing ad message recall. Psychology & Marketing, 30(1), 62–75.
13.
HagtvedtH.BraselS. A. (2016). Cross-modal communication: Sound frequency influences consumer responses to color lightness. Journal of Marketing Research, 53(4), 551–562. https://doi.org/10.1509/jmr.14.0414
14.
HanserW. E.MarkR. E.ZijlstraW. P.VingerhoetsA. J. (2015). The effects of background music on the evaluation of crying faces. Psychology of Music, 43(1), 75–85.
15.
HergetA. K.AlbrechtJ. (2022). Soundtrack for reality? How to use music effectively in non-fictional media formats. Psychology of Music, 50(2), 508–529.
16.
HolmS. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6, 65–70. http://www.jstor.org/stable/4615733
17.
HongY. J.ChoiB.LeeK. (2023). When background music in audiovisual advertisements can boost the perceived competence of the advertised brands—An empirical study from South Korea. Asia Pacific Journal of Marketing and Logistics, 35, 1991–2011.
18.
HuangX.LabrooA. A. (2020). Cueing morality: The effect of high-pitched music on healthy choice. Journal of Marketing, 84(6), 130–143. https://doi.org/10.1177/0022242918813577
19.
HubbardT. L. (1996). Synesthesia-like mappings of lightness, pitch, and melodic interval. The American Journal of Psychology, 109(2), 219–238. https://doi.org/10.2307/1423274
20.
International Organization for Standardization. (2003). Acoustics: Normal equal-loudness-level contours (ISO 226:2003). https://www.iso.org/standard/34222.html
21.
LaengB.EndestadT. (2012). Bright illusions reduce the eye’s pupil. Proceedings of the National Academy of Sciences of the United States of America, 109(6), 2162–2167. https://doi.org/10.1073/pnas.1118298109
22.
LogeswaranN.BhattacharyaJ. (2009). Crossmodal transfer of emotion by music. Neuroscience Letters, 455(2), 129–133.
23.
LoweM. L.HawsK. L. (2017). Sounds big: The effects of acoustic pitch on product perceptions. Journal of Marketing Research, 54(2), 331–346. https://doi.org/10.1509/jmr.14.0300
24.
LudwigV. U.AdachiI.MatsuzawaT. (2011). Visuoauditory mappings between high luminance and high pitch are shared by chimpanzees (Pan troglodytes) and humans. Proceedings of the National Academy of Sciences of the United States of America, 108(51), 20661–20665. https://doi.org/10.1073/pnas.1112605108
MarksL. E. (1974). On associations of light and sound: The mediation of brightness, pitch, and loudness. The American Journal of Psychology, 87(1–2), 173–188. https://doi.org/10.2307/1422011
27.
MarksL. E. (1987). On cross-modal similarity: Auditory–visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 384–394. https://doi.org/10.1037/0096-1523.13.3.384
28.
MarksL. E. (1989). On cross-modal similarity: The perceptual structure of pitch, loudness, and brightness. Journal of Experimental Psychology: Human Perception and Performance, 15(3), 586–602. https://doi.org/10.1037/0096-1523.15.3.586
29.
MarksL. E.HammealR. J.BornsteinM. H.SmithL. B. (1987). Perceiving similarity and comprehending metaphor. Monographs of the Society for Research in Child Development, 52(1), i–100. https://doi.org/10.2307/1166084
30.
MathôtS.GraingerJ.StrijkersK. (2017). Pupillary responses to words that convey a sense of brightness or darkness. Psychological Science, 28(8), 1116–1124. https://doi.org/10.1177/0956797617702699
31.
MeierB. P.RobinsonM. D.CrawfordL. E.AhlversW. J. (2007). When “light” and “dark” thoughts become light and dark responses: Affect biases brightness judgments. Emotion, 7(2), 366–376.
32.
MelaraR. D. (1989a). Dimensional interaction between color and pitch. Journal of Experimental Psychology: Human Perception and Performance, 15(1), 69–79.
33.
MelaraR. D. (1989b). Similarity relations among synesthetic stimuli and their attributes. Journal of Experimental Psychology: Human Perception and Performance, 15(2), 212–231.
34.
MishraJ.MartinezA.SejnowskiT. J.HillyardS. A. (2007). Early cross-modal interactions in auditory and visual cortex underlie a sound-induced visual illusion. The Journal of Neuroscience, 27(15), 4120–4131.
35.
MondlochC. J.MaurerD. (2004). Do small white balls squeak? Pitch-object correspondences in young children. Cognitive, Affective, & Behavioral Neuroscience, 4(2), 133–136.
36.
NosalA. P.KeenanE. A.HastingsP. A.GneezyA. (2016). The effect of background music in shark documentaries on viewers’ perceptions of sharks. PLOS ONE, 11(8), Article e0159279. https://doi.org/10.1371/journal.pone.0159279
37.
PlackC. J.OxenhamA. J.FayR. R. (2006). Pitch: Neural coding and perception. Springer.
SpenceC. (2020). Assessing the role of emotional mediation in explaining crossmodal correspondences involving musical stimuli. Multisensory Research, 33(1), 1–29. https://doi.org/10.1163/22134808-20191469
42.
SteffensJ. (2020). The influence of film music on moral judgments of movie scenes and felt emotions. Psychology of Music, 48(1), 3–17.
43.
SunagaT. (2018). How the sound frequency of background music influences consumers’ perceptions and decision making. Psychology & Marketing, 35(4), 253–267. https://doi.org/10.1002/mar.21084
VatakisA.SpenceC. (2007). Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli. Perception & Psychophysics, 69(5), 744–756. https://doi.org/10.3758/BF03193776
48.
VieillardS.PeretzI.GosselinN.KhalfaS.GagnonL.BouchardB. (2008). Happy, sad, scary and peaceful musical excerpts for research on emotions. Cognition and Emotion, 22(4), 720–752. https://doi.org/10.1080/02699930701503567
49.
WatanabeK.ShimojoS. (2001). When sound affects vision: Effects of auditory grouping on visual motion perception. Psychological Science, 12(2), 109–116.
50.
WeissgerberT. L.MilicN. M.WinhamS. J.GarovicV. D. (2015). Beyond bar and line graphs: Time for a new data presentation paradigm. PLOS Biology, 13(4), Article e1002128. https://doi.org/10.1371/journal.pbio.1002128
51.
WickerF. W. (1968). Mapping the intersensory regions of perceptual space. The American Journal of Psychology, 81(2), 178–188.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
