Previous studies concluded that first-time film viewers often had difficulty integrating shots into a coherent representation of the depicted events in the absence of a familiar action through the film cuts or a salient eye-gazing of a character in the film. In this study, we investigated whether diegetic sound (i.e., sound that seems to originate from the depicted cinematic space) could effectively bridge shots for first-time viewers. Across a range of films, both dialog, and salient environmental sound (e.g., barking dogs) helped first-time viewers connect shots. However, sound was not always successful in supporting first-time viewers’ interpretations. While experienced viewers were able to understand less-familiar linking sounds and environments, first-time viewers found this difficult. Overall, a range of diegetic sounds helped first-time viewers understand spatiotemporal relations between shots, but these viewers still had difficulty integrating views of unfamiliar environments.
AbelmanR. (1989). From here to eternity: Children’s acquisition of understanding of projective size on television. Human Communication Research, 15, 463–481. doi: 10.1111/j.1468-2958.1989.tb00194.x.
2.
AndersonD. R.LorchE. P.FieldD. E.SandersJ. (1981). The effects of TV program comprehensibility on preschool children’s visual attention to television. Child Development, 52, 151–157.
3.
AndersonD. R.PempekT. A. (2005). Television and very young children. American Behavioral Scientist, 48, 505–52. doi: 10.2307/1129224.
4.
AndersonJ. D. (1996) The reality of illusion: An ecological approach to cognitive film theory, Carbondale: Southern Illinois University Press.
5.
BakerL. J.LevinD. T. (2015). The role of spatial triggers in event updating. Cognition, 136, 14–29. doi: 10.1016/j.cognition.2014.11.030.
6.
BarrR.ZackE.GarciaA.MuentenerP. (2008). Infants’ attention and responsiveness to television increases with prior exposure and parental interaction. Infancy, 13, 30–56. doi: 10.1080/15250000701779378.
7.
BatemanJ. A.WildfeuerJ. (2014). A multimodal discourse theory of visual narrative. Journal of Pragmatics, 74, 180–208. doi: 10.1016/j.pragma.2014.10.001.
8.
BremnerJ. G.SlaterA. M.JohnsonS. P.MasonU. C.SpringJ. (2012). The effects of auditory information on 4-month-old infants’ perception of trajectory continuity. Child Development, 83, 954–964. doi: 10.1111/j.1467-8624.2012.01739.x.
9.
CalvertS. L.HustonA. C.WatkinsB. A.WrightJ. C. (1982). The relation between selective attention to television forms and children’s comprehension of content. Child Development, 53, 601–610. doi: 10.2307/1129371.
10.
ChionM. (1994) Audio-vision: Sound on screen, New York, NY: Columbia University Press.
11.
CoeB. (1992) Muybridge and chronophotographers, London, UK: Museum of the Moving Image.
12.
CohnN. (2013) The visual language of comics: Introduction to the structure and cognition of sequential images, NY: Bloomsbury Academic.
13.
Cohn, N. (Ed.). (2016). The visual narrative reader. NY: Bloomsbury Publishing.
14.
CrawleyA. M.AndersonD. R.WilderA.WilliamsM.SantomeroA. (1999). Effects of repeated exposures to a single episode of the television program Blue’s Clues on the viewing behaviors and comprehension of preschool children. Journal of Educational Psychology, 91, 630. doi: 10.1037/0022-0663.91.4.630.
15.
FrithU.RobsonJ. E. (1975). Perceiving the language of films. Perception, 4, 97–103. doi: 10.1068/p040097.
16.
Goldin-MeadowS. (1999). The role of gesture in communication and thinking. Trends in Cognitive Sciences, 3, 419–429. doi: 10.1016/S1364-6613(99)01397-2.
17.
GunningT. (2001). Doing for the eye what phonograph does for the ear. In: AbelR.AltmanR. (eds) The sounds of early cinema, Bloomington: Indiana University Press.
18.
HayneH.HerbertJ.SimcockG. (2003). Imitation from television by 24- and 30-month-olds. Developmental Science, 6, 254–261. doi: 10.1111/1467-7687.00281.
19.
HobbsR.FrostR.DavisA.StaufferJ. (1988). How first time viewers comprehend editing conventions. Journal of Communication, 38, 50–60. doi: 10.1111/j.1460-2466.1988.tb02069.x.
20.
IldirarS.SchwanS. (2015). First-time viewers’ comprehension of films: Bridging shot transitions. British Journal of Psychology, 106, 133–151. doi: 10.1111/bjop.12069.
21.
IordanescuL.GraboweckyM.FranconeriS.TheeuwesJ.SuzukiS. (2010). Characteristic sounds make you look at target objects more quickly. Attention, Perception, & Psychophysics, 72, 1736–1741. doi: 10.3758/APP.72.7.1736.
22.
KintschW. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological Review, 95, 163. doi: 10.1037/0033-295X.95.2.163.
23.
LangA. (1995). Defining audio/video redundancy from a limited-capacity information processing perspective. Communication Research, 22, 86–115. doi: 10.1177/009365095022001004.
24.
LemishD. (1987). Viewers in diapers: The early development of television viewing. In: LindlofT. R. (ed.) Natural audiences: Qualitative research of media uses and effects, Norwood, NJ: Ablex, pp. 33–57.
25.
LevinD. T.BakerL. J. (2017). Bridging views in cinema: A review of the art and science of view integration. Wiley Interdisciplinary Reviews: Cognitive Science, 8, 1–18. doi: 10.1002/wcs.1436.
26.
LevinD. T.SimonsD. J. (2000). Perceiving Stability in a Changing World: Combining Shots and Intergrating Views in Motion Pictures and the Real World. Media Psychology, 2, 357–380. doi: 10.1207/S1532785XMEP0204_03.
27.
LevinS. R.AndersonD. R. (1976). The development of attention. Journal of Communication, 26, 126–135. doi: 10.1111/j.1460-2466.1976.tb01391.x.
28.
LoweP. J.DurkinK. (1999). The effect of flashback on children’s understanding of television crime content. Journal of Broadcasting & Electronic Media, 43, 83–97. doi: 10.1080/08838159909364476.
29.
McGurkH.MacDonaldJ. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.
MeyerhoffH. S.HuffM. (2016). Semantic congruency but not temporal synchrony enhances long-term memory performance for audio-visual scenes. Memory & Cognition, 44, 390–402. doi: 10.3758/s13421-015-0575-6.
32.
MunkC.ReyG. D.DiergartenA. K.NiedingG.SchneiderW.OhlerP. (2012). Cognitive processing of film cuts among 4- to 8-year-old children: An eye tracker experiment. European Psychologist, 17, 257. doi: 10.1027/1016-9040/a000098.
33.
MurchW. (2003). Touch of silence. In: SiderL.SiderJ.FreemanD. (eds) Soundscape: The school of sound lectures 1998–2001, New York, NY: Wallflower Press.
34.
OsinaM. A.SaylorM. M.GaneaP. A. (2013). When familiar is not better: 12-month-old infants respond to talk about absent objects. Developmental Psychology, 49, 138. doi: 10.1037/a0027903.
35.
ParkinsonD. (1996) History of film, New York, NY: Thames and Hudson.
36.
PempekT. A.KirkorianH. L.RichardsJ. E.AndersonD. R.LundA. F.StevensM. (2010). Video comprehensibility and attention in very young children. Developmental Psychology, 46, 1283. doi: 10.1037/a0020614.
37.
RaynaudI. (2001). Dialogues in early silent screenplays: What actors really said. In: AbelR.AltmanR. R. (eds) The sounds of early cinema, Bloomington: Indiana University Press.
38.
RichardsJ. E.CroniseK. (2000). Extended visual fixation in the early preschool years: Look duration, heart rate changes, and attentional inertia. Child Development, 71, 602–620. doi: 10.1111/1467-8624.00170.
39.
RichardsJ. E.GibsonT. L. (1997). Extended visual fixation in young infants: Look distributions, heart rate changes, and attention. Child Development, 68, 1041–1056. doi: 10.1111/j.1467-8624.1997.tb01983.x.
40.
RoderoE. (2010). See it on a radio story: Sound effects and shots to evoked imagery and attention on audio fiction. Communication Research, 39, 458–479. doi: 10.1177/0093650210386947.
41.
SchwanS.IldirarS. (2010). Watching film for the first time: How adult viewers interpret perceptual discontinuities in film. Psychological Science, 21, 107–113. doi: 10.1177/0956797610372632.
42.
ShamsL.KamitaniY.ShimojoS. (2000). Illusions: What you see is what you hear. Nature, 408, 788.
43.
ShimojoS.ShamsL. (2001). Sensory modalities are not separate modalities: Plasticity and interactions. Current Opinion in Neurobiology, 11, 505–509. doi: 10.1016/S0959-4388(00)00241-5.
44.
SmithR.AndersonD.FischerC. (1985). Young children’s comprehension of montage. Child Development, 56, 962–971. doi: 10.2307/1130108.
45.
SmithT. J.LevinD. T.CuttingJ. E. (2012). A window on reality: Perceiving edited moving images. Current Directions in Psychological Science, 21, 107–113. doi: 10.1177/0963721412437407.
46.
SmithT. J.Martin-Portugues SantacreuJ. Y. (2016). Match-action: The role of motion and audio in creating global change blindness in film. Media Psychology, 20, 317–348. ISSN 1521-3269. doi: 10.1080/15213269.2016.1160789.
47.
Somanader, M., Garcia, A., Miller, N., & Barr, R. (2005). Effects of sound effects and music on imitation from television during infancy. Paper presented at the Society for Research on Child Development, 7–10 April, Atlanta, GA.
48.
StandishI. (2006) A new history of Japanese cinema: A century of narrative film, New York, NY: Continuum.
49.
TibusM.HeierA.SchwanS. (2013). Do films make you learn? Inference processes in expository film comprehension. Journal of Educational Psychology, 105, 329–340. doi: 10.1037/a0030818.
50.
TrosethG. L.DeLoacheJ. S. (1998). The medium can obscure the message: Young children’s understanding of video. Child Development, 69, 950–965. doi. 10.1111/j.1467-8624.1998.tb06153.x.
51.
Van der BurgE.OliversC. N. L.BronkhorstA. W.TheeuwesJ. (2008). Pip and pop: Nonspatial auditory signals improve spatial visual search. Journal of Experimental Psychology: Human Perception and Performance, 34, 1053–1065. doi: 10.1037/0096-1523.34.5.1053.
52.
WillemsR. M.HagoortP. (2007). Neural evidence for the interplay between language, gesture, and action: A review. Brain and Language, 101, 278–289. doi: 10.1016/j.bandl.2007.03.004.
53.
ZacksJ. M.SpeerN. K.SwallowK. M.BraverT. S.ReynoldsJ. R. (2007). Event perception: A mind-brain perspective. Psychological Bulletin, 133, 273–293. doi: 10.1037/0033-2909.133.2.273.
