Narrative media music, music used for narrative purposes in multimedia such as film, television or computer games, is becoming one of the largest sources of musical experience in our daily lives. Though typically experienced on an unconscious and unreflected level, this kind of music actively contributes narrative meaning in multimodal interplay with image, speech and sound effects. Often, what we (think we) see is to a large degree determined by what we hear. Using Halliday’s (1978) metafunctions of communication as a starting point, two short film scenes (from Jaws and The Secret of My Success) are examined, with a focus on the intermodal relationships of music and image. The examples illustrate how musical and visual expressions combine to form multimodal statements where the whole is certainly different than the sum of the parts.
Barker, M. & Brooks, K. ( 1998). Knowing Audiences: Judge Dredd, Its Friends, Fans and Foes. Luton: University of Luton Press.
2.
Bouzereau, L. (Writer/Director/Producer). (2000). ‘The Making of Jaws’ [DVD Documentary]. On Jaws: Anniversary Collector’s Edition [DVD Motion Picture]. United States: Universal Studios Home Video.
3.
Chatman, S. ( 1980). Story and Discourse: Narrative Structure in Fiction and Film. Ithaka, NY: Cornell University Press.
4.
Chion, M. ( 1994). Audio-Vision: Sound on Screen. New York: Columbia University Press.
5.
Cook, N. ( 1998). Analysing Musical Multimedia. Oxford : Oxford University Press.
6.
Cooke, D. ( 1959). The Language of Music. Oxford : Oxford University Press.
7.
Cope, B. & Kalantzis, M. ( 2000). ‘Designs for Social Futures’, in B. Cope & M. Kalantzis (eds.) Multiliteracies: Literacy Learning and the Design of Social Futures, pp. 203-234. London: Routledge.
8.
Donnelly, K.J. ( 2005). The Spectre of Sound: Music in Film and Television . London: BFI Publishing.
9.
Gorbman, C. ( 1987). Unheard Melodies: Narrative Film Music. Bloomington, Indiana: Indiana University Press.
10.
Halliday, M.A.K. ( 1978). Language as a Social Semiotic. London : Edward Arnold.
11.
Hanslick, Eduard ( 1955). Om det sköna i musiken [On the Musically Beautiful; orig. Vom Musikalisch-Schönen, 1854]. Uppsala: Schismens skriftserie1.
12.
Hickman, R. ( 2006). Reel Music: Exploring 100 Years of Film Music . New York: Norton.
13.
Hodge, R. & Kress, G. ( 1988). Social Semiotics. Cambridge , UK: Polity Press.
14.
Hodge, R. & Tripp, D. ( 1986). Children and Television: A Semiotic Approach. Cambridge: Polity Press.
15.
Jewitt, C. ( 2006). Technology, Literacy and Learning: A Multimodal Approach . London: Routledge.
16.
Juslin, P.N. ( 2001). ‘Communicating Emotion in Music Performance: A Review and a Theoretical Framework’, in P.N. Juslin and J.A. Sloboda (eds.) Music and Emotion: Theory and Research, pp. 309-337. Oxford : Oxford University Press.
17.
Jägerskogh, A. ( 2002). Visst ska det höras vad dom säger och sjunger? [Of course what they say and sing should be heard?]. Unpublished manuscript , Royal Institute of Technology, Stockholm.
18.
Kress, G. ( 2003). Literacy in the New Media Age. London : Routledge.
19.
Kress, G. & Van Leeuwen, T. ( 2006). Reading Images: The Grammar of Visual Design, (2nd edition). London: Routledge.
20.
Lindstrand, F. ( 2006). Att göra skillnad: Representation, identitet och lärande i ungdomars arbete och berättande med film [Making difference: Representation, identity and learning in teenagers’ work and communication with film] (Doctoral dissertation). Stockholm: HLS Förlag.
21.
Lipscomb, S.D. & Tolchinsky, D.E. ( 2005). ‘The Role of Music Communication in Cinema’ , in D. Miell, R. MacDonald & D.J. Hargreaves (eds.) Musical Communication, pp. 383-404. Oxford : Oxford University Press.
22.
Martinec, R. ( 2000). ‘Rhythm in Multimodal Texts’. Leonardo, Vol. 33, No. 4, pp. 289-297.
23.
Meyer, L.B. ( 1956). Emotion and Meaning in Music. Chicago : University of Chicago Press.
24.
Palmer, C. ( 1990). The Composer in Hollywood. London : Marion Boyars Publishers.
25.
Temperley, D. ( 2001). The Cognition of Basic Musical Structures. Cambridge: The MIT Press.
26.
Van Leeuwen, T. ( 1985). ‘Rhythmic Structure of the Film Text’ , in T.A. Van Dilk (ed.), Discourse and Communication: New Approaches to the Analysis of Mass Media Discourse and Communication, pp. 216-232. Berlin: De Gruyter.
27.
Van Leeuwen, T. ( 1999). Speech, Sound and Music. London : Macmillan.
28.
Van Leeuwen, T. ( 2005). Introducing Social Semiotics. London : Routledge.
29.
Van Leeuwen, T. & Jewitt, C. (eds.) (2001). Handbook of Visual Analysis . London: Sage Publications.
30.
Wingstedt, J. ( 2004). ‘Narrative Functions of Film Music in a Relational Perspective’, paper presented at ISME - Sound Worlds to Discover, Santa Cruz, Tenerife, 14-16 July.
31.
Wingstedt, J. ( 2005). Narrative Music: Towards an Understanding of Musical Narrative Functions in Multimedia (Licentiate thesis). Luleå: School of Music, Luleå University of Technology, Sweden. (Available at http://pure.ltu.se/ws/fbspretrieve/199292)
32.
Wingstedt, J. ( 2008). Making Music Mean: On Functions of, and Knowledge about, Narrative Music in Multimedia. (Doctoral dissertation). Luleå University of Technology, Dept. of Music and Media.
33.
Wingstedt, J., Brändström, S. & Berg, J. ( 2008). ‘Young Adolescents’ Usage of Narrative Functions of Media Music by Manipulation of Musical Expression’. Psychology of Music, Vol. 36 (No. 2): pp. 193-214.
