Stronger together: Moving towards a combined multimodal and intermedial model

Abstract

This article explores the complementary potential of intermedial and multimodal semiotic analysis. Both multimodal and intermedial research explore the multifaceted nature of communication. However, since research fields have different foci and analytical methods, they are less connected than they could be. We approach intermediality and multimodality as complementary frameworks and argue that there is much to gain in drawing on the analytical strengths of both. To this end, underlying differences in method and theoretical assumptions need to be made explicit. Drawing on John A. Bateman and Lars Elleström’s previous explorations of the common ground between the frameworks, we map an arena where multimodal and intermedial analysis can work together. We demonstrate how a combined multimodal and intermedial perspective can function by zooming in and out between the perspectives as we explore the role of “Ride of the Valkyries” in Wagner’s opera Die Walküre (1870), in a Nazi newsreel, and in Francis Ford Coppola’s movie Apocalypse Now (1979). The multimodal perspective provides us with nuanced language for transcribing and discussing how different semiotic resources work together, and the intermedial perspective allows us to discuss the chain of media transformation, where each instance increases and transforms the meaning potential of the “Ride of the Valkyries”.

Introduction

Multimodal semiotic theories and intermedial research have many points of contact. They both explore the mix of modes and media in human communication, yet, at the same time, they are surprisingly unconnected in their methods and approaches. Intermedial and multimodal researchers often analyse similar media products like comics, children’s books, film and computer games, and there is a growing awareness of the intersections between the two adjacent fields. Specifically, intermedial scholar Lars Elleström (2010, 2021) and multimodal scholar John Bateman (2017) have reached out to connect with the other field. Furthermore, scholars increasingly aim for joint explorations and studying “multimodal and intermedial interaction (Alarauhio et al., 2022, 2, see also Thurlow, 2015; Hallet, 2015; Giessen et al., 2019). However, while researchers in both fields increasingly acknowledge that communication is formed by “intermedial relations among multimodal media” (Elleström, ed. 2021: v), and that the complexity of digital representation calls for increased collaborations between the two fields (Fedrová and Jedličková, 2024: 417), the interdisciplinary dialogue between the fields remains challenging, impeded by different research interests and overlapping but dissimilar terminology (Alarauhio et al., 2022: 3–10; Fedrová and Jedličková, 2024: 416). As a consequence, “discourses on media and modalities tend to be either separated or mixed up” (Elleström, 2021: 43). Without an explicit dialogue about the overlaps and differences, intermedial and multimodal scholars cannot fully engage with the other field’s methods. However, this divergence can be turned into collaboration and provide the ground for a fruitful combination of each of these fields’ analytical strengths.

This article aims to contribute to an interdisciplinary dialogue and a better understanding of how intermediality and multimodality (particularly the subfield multimodal semiotics) can draw on each other as complementary frameworks. We first present the approaches in relation to each other to discuss differences in theoretical perspectives, terminology and analytical methods. We then explore this complementarity by analysing the use of Wagner’s “Ride of the Valkyries” in film. We take our cue from an incident in 2018, when “Ride of the Valkyries” blasted from the loudspeakers of a police car shortly before the eviction of environmental activists in the German Hambach forest. ¹ While a helicopter attack in Coppola’s film Apocalypse Now (1979) about the Vietnam war immediately springs to the mind, the previous use of “Ride of the Valkyries”, for instance, in Nazi propaganda, renders the choice of this powerful and confident music historically problematic. In fact, “Valkyries” time and again appears in contexts of conflict, and the Hambach Forest incident connects to a chain of transmediations across myth, opera and film—an intermedial perspective. And in every instance, “Valkyries” integrates with a different audiovisual, socio-cultural and political context—a multimodal perspective. Both the theoretical discussion of overlaps and differences and the joint multimodal-intermedial analysis in this paper provide steps towards a holistic media approach that is able to expand the analysis of the mix of modes and media in human communication through theoretical and methodological pluralism.

Multimodality and intermediality: Different approaches towards media analysis

Many introductions to multimodality (e.g., Jewitt, 2014; Jewitt et al., 2016; Serafini, 2022) and intermediality (e.g., Fedrová and Jedličková, 2024; Grishakova, 2024; Rippl, 2015) already provide overviews of the variety of approaches and important developments within each field. Therefore, the following comparative presentation doesn’t attempt to repeat these efforts but focuses instead on presenting the two fields in relation to each other, highlighting both divergences and overlaps, in order to facilitate a dialogue between the two fields.

Multimodal wholes versus tracing media relations

The word ‘Multi-modality’ implies a focus on the integration of multiple semiotic modes or resources (such as gesture, image, language and music) in communication and social interaction. Even before the concept was coined, scholars studied the integration of text and images (e.g., advertisements), music and images (e.g. film music) and music and text (e.g. songs) (see also Bateman, 2022). However, The concept ‘multimodality’ first started to gain academic ground with work using (critical) linguistics, semiotics, and discourse theory. Hodge and Kress (1988) is foundational, and Halliday’s social semiotic theories have been particularly influential (see e.g., Jewitt et al., 2016; Kress and Van Leeuwen, 2021; O’Halloran et al., 2015). ² Furthermore, most multimodal approaches see communication and interaction as socially constituted, emphasising social interaction and context. In short, multimodality is focused on the communicative use of semiotic systems or cultural practice in a social context, highlighting that when people use semiotic resources or entire media products like “Ride of the Valkyries”, they are making a semiotic choice grounded in conscious or unconscious intentions, availability of semiotic material and social conventions or habits. Multimodality can be applied to everything in the social sphere that is designed, such as everyday communication, popular culture and the arts (for examples, see Jewitt, 2014; Moschini and Sindoni, 2022; Serafini, 2022; Tønnessen and Forsgren, 2018).

Multimodal semiotic models analyse meaning potential by transcribing, or otherwise accounting for, the individual modes in a media product and relating these to a semiotic theory (for approaches to transcription in multimodal analysis, see, e.g., Bateman and Schmidt, 2012; Bezemer and Mavers, 2011; Cowan, 2014; O’Halloran et al., 2011). An important point in social semiotics and similar multimodal theories is that meaning potential is context-dependent and will vary between people from different semiotic communities (see e.g., Hodge and Kress, 1988: 21–23).

The term intermediality points towards the relationships between different media types and contexts. The dynamic interplay of media can be described as a kind of intertextuality transgressing media borders (Lehtonen, 2001: 76) or as “a bridge between medial differences that is founded on medial similarities” (Elleström, 2021: 5) that conveys “a sense of transformation and renewal of practices, perceptions and meanings” (Grishakova, 2024: 14). With roots in fields such as aesthetics, semiotics, comparative literature, media studies and interart studies (Elleström, 2021: 9), intermediality developed around the 2000s and partly as a response to how the digitalisation drew attention to the shaping force of media in communication. Intermedial research in the humanities often focuses on aesthetic aspects and artistic media use, such as in children’s books and comics, theatre and opera, novel-to-film adaptions, ekphrasis or media representations and imitations (see Bruhn and Schirrmacher, 2022; Rippl, 2015), where the blurring of boundaries and the crossing of conventional media borders are performed explicitly (Jensen, 2016: 2; Ljungberg, 2010: 94). However, in recent years there has been a notable shift from the study of artworks towards exploring the intermedial networks of the contemporary media ecology (Bruhn et al. eds., 2023: 1033–1283; Engberg et al., 2023; Linuo, 2022; Virginás, 2022; Jensen et al., 2022; Johansson, 2021).

The intermedial relationships ‘media combinations,’ ‘transmediations’ and ‘media representations’ are studied using slightly different terminology (see e.g. Clüver, 2007; Elleström, 2021; Rajewsky, 2005; Wolf, 1999, 2018). The distinction between a synchronic perspective (of combinations and integrations) and a diachronic perspective with a focus on transformation (in transmediations or representations) (Elleström, 2021: 73–75) highlights how different intermedial perspectives apply to the same media product (Bruhn and Schirrmacher, 2022: 105). Intermediality applies a broad and multilayered concept of media and connects objects (such as books or smartphones), semiotic configurations (such as texts or images) and genres of communication (such as literature, news, theatre) (see, e.g., Jensen, 2016: 1; Elleström, 2021: 54–66; Grishakova, 2024: 14). Elleström (2010, 2021) identifies these three levels of mediation as technical media of display, basic and qualified media types that together provide the basis for how we interact with each media product, such as a specific novel, a film or theatre performance. The communicative use of a media product in a specific social context, which is important to a multimodal perspective, tends to be less central. Instead, for the intermedial analysis, the assessment of cultural history or the history of ideas provides the relevant context (Bruhn, 2010: 232) to trace changes of meaning potential over time and across different media in a hermeneutical movement between analysis of objects and interpretation in a specific context.

This introductory comparison already highlights how multimodality and intermediality approach the heterogeneity of human communication from different directions and with different priorities. Multimodality studies the semiotic processes and the use of different semiotic resources or modes in specific social contexts and how/whether these semiotic processes can be generalized across media. Intermediality explores medial relationships between and within different kinds of media types with a focus on how concepts, narratives, and ideas are transmediated and transformed. While both approaches draw on semiotic and communication theories, this is not always made explicit in intermedial interpretations of aesthetic processes. Due to these different research interests, both approaches tend to downplay aspects that are central or foundational for the other approach. As a consequence, scholars of both fields easily tend to consider analysis within the other field to be beside the point, somehow close to the familiar concepts but slightly out of focus.

Overlaps

While there are differences in their theoretical foundations, both fields approach the phenomena of combinations and transformations of modes and media. As a consequence, scholars in both fields might consider both these phenomena to be sufficiently considered within their own field. However, discussing the overlaps brings forth conceptual differences and also helps to identify the areas where the approaches can function as complementary.

The ‘transmediation’ of concepts or narratives across media types is in multimodality termed ‘resemiotisation’ (e.g., Iedema, 2001; O’Halloran, 2011), ³ ‘transmodal semiosis’ (Newfield, 2017) or ‘transduction’ (Kress, 2010; Poulsen, 2017). Albeit an established practice, the transmedia aspect tends to be afforded a secondary role in many mappings and anthologies in the field (e.g., in Bateman et al., 2017; Jewitt, 2014; Jewitt et al., 2016; Serafini, 2022; Wildfeuer et al., 2019). Multimodal approaches to transmedia phenomena tend to focus on the change of modal affordances and how meaning content can be ‘translated’ from one mode to another. One might say that multimodal studies of transduction zoom in on how different modal affordances can serve different interests, such as how non-verbal modes can help students to express complex ideas (Jakobson et al., 2018), whereas intermediality is more focused on how the interpretation of the transformation process helps to make sense of the media products involved. Multimodal interests in this area thus diverge somewhat from intermediality’s interest in the study of media products as the result of transformation (Fedrová and Jedličková, 2024: 418), which approaches media adaptations as artistic productions in their own right (e.g., Bruhn et al., 2022).

The synchronic aspect of communication is central to multimodal theory but is also conceptualised as ‘media combination’ in intermediality. The intermedial combination perspective, also conceptualised as “plurimediality” (Wolf, 2018), tends to focus more on the combination of “conventionally distinct media” (Rajewksy, 2002: 19) and how media types merge and boundaries blur in specific moments of dynamic interplay that transgress and transform conventional expectations (Arvidson et al., 2022: 106; Rajewsky, 2002: 22). Intermediality tends to distinguish between degrees of more separable combinations and deep integrations (Clüver, 2007: 19) that can be understood on more of a “floating scale” (Elleström, 2021: 75) or differ between production and perception (Bruhn and Schirrmacher, 2022: 103).

Intermedial theory increasingly considers an intrinsic multimodality of media as a “fundamental condition of mediality” (Elleström, 2021: 73), which means that media are always “mixed” (Mitchell, 2005) or “heteromedial” (Bruhn, 2010), and perceived in multisensory and embodied ways (Engberg et al., 2023). The “multimodal conception of intermediality” (Johansson, 2021: 256) acknowledges the complex interplay with material, multisensory, embodied and intersemiotic processes that Elleström (2010, 2021) conceptualised in the ‘material’, ‘spatiotemporal’, ‘sensorial’ and ‘semiotic media modalities’. Thus, an intermedial perspective to the intrinsic multimodality of media often focuses on the transformative aspects of the interplay of different semiotic systems (Alarauhio 2022; Calzati 2014; Gibbons 2010; Korhonen and Lehtola 2022),

Communication across the fields: Elleström and Bateman

In the previous sections, we addressed the key concerns of multimodality and intermediality as broad fields. We now narrow our focus to two specific approaches within these fields, those represented by Lars Elleström in intermediality and by John Bateman and affiliates in multimodal semiotics. We find that the approaches developed by Elleström (2010, 2021) and Bateman et al. (2017) provide a good starting point for enhancing the communication between these fields, as they both actualise core questions and analytical levels of the other field and indicate possible complementary connections. Furthermore, both frameworks share an adherence to systematicity, detail and making analysis transparent and therefore offer potential complementary connections. Exploring these connections as we aim to do now can be used to establish a transdisciplinary language that combines an elaborated multimodal semiotic anchoring with the ability to trace relationships between different types of artistic and communicative expressions in media analysis.

Two of Elleström’s important contributions to intermediality have been to address the inconsistencies that easily arise in the broad use of the word ‘medium’ and to construct a detailed media model enabling analysists to distinguish between and compare different types of media. His framework of media types and media modalities, based on the semiotic approach of C.S Peirce, was developed to deal with the multilayered complexity of mediation which involves the interaction with material objects or physical phenomena with specific spatial and temporal qualities that we perceive with our senses and that we interpret as a mixture of iconic, indexical, and symbolical signs. Taken together, the ‘material, spatiotemporal, sensorial, and semiotic media modalities’ map a framework of “necessary categories of media traits” (2021: 47) and specific media products can be described, analysed and compared with reference to the specific set of ‘media modes’ within these modalities. While the media modalities enable intermedial analyses to describe how specific material, sensorial and space-time-relations inform semiosis, the ‘technical devices of display’, ‘basic’ and ‘qualified media types’ make it possible to analyse how a specific media product relates to conventional understandings and uses of certain media types. For instance, these tools help explain how audiobooks challenge and expand the conventional ideas of a qualified media type ‘literature’ that for a long time have been formed by the technical device of printed books and the basic media type of written text (Have and Pedersen 2023).

Elleström’s media modalities have opened up increased dialogue between the fields (see Elleström, ed. 2021), although it needs to be stressed that Elleström does not integrate multimodal theory. From a multimodal perspective, the media modalities defined by Elleström do not explicitly consider the social and socio-political embeddedness of communication (O’Halloran, forthcoming). Still, Heather Lotherington suggests that the intermedial perspective and the media modalities can be valuable resources for multimodal scholars to develop a definition of modes that is not only based on “cultural interpretation and exemplification” (2021: 218). John Bateman similarly stresses the complementary potential between Elleström’s intermedial and his own multimodal approach (2017). Bateman sees intermediality as apt for addressing aspects of “transmediality” related to “media technologies and networks, media ecologies […] cultural, institutional and technological forces”, and multimodality as providing “fine-grained descriptions and explanations of how meaning-making can co-deploy quite distinct expressive forms” (Bateman, 2017: 172). He suggests that in order to understand transmedial relations, we need to relate the sensorial configurations of media, which Elleström’s model accounts for, with a more detailed, multimodal model of semiotic modes employed in these media.

Bateman has built on work in social semiotic multimodality, linguistics and semiotics to develop a detailed semiotic model of how modes, materiality and social context relate. In Bateman’s model, a semiotic mode consists of three strata: (1) the material foundation, (2) the ‘technical features’, i.e., the organising principles, and (3) discourse semantics, which accounts for “contextual interpretations” (Bateman, 2017: 167). Technical features refer to the conventionalised composition of semiotic modes (e.g., grammar in language) (Bateman et al., 2017: 115). The discourse semantics refers to overarching conventions and communicative purposes which provide a guide for how to, within a specific community, utilise or interpret the semiotic mode (Bateman et al., 2017: 131). In the case of music, Jensen has argued that discourse semantics includes what can be understood as a “relational logic”, determining how musical resources gain meaning through different types of relations to other resources, for example, via contrast or enhancement (Jensen, 2021: 125–126).

Moving towards a combined multimodal and intermedial approach

While Elleström’s and Bateman’s approaches contribute to establishing an understanding of how multimodal and intermedial aspects of communication can be combined, there are still some challenges in regards to terminology and analytical focus which need to be addressed in order to better facilitate a mutual exchange of research and concepts between the two fields.

When considering terminology, it is a challenge to deal with the differences in the uses of the terms ‘modality’, ‘mode’, and ‘media’. When intermedial research refers to Elleström’s concepts ‘modalities’ and ‘modes’ this can lead to interdisciplinary misunderstandings. In Bateman’s multimodal semiotics, the concept of ‘modality’ is primarily used in the sense of ‘multimodality’, i.e. the combination of modes, and not in its linguistic meaning, where it refers to markers of the sender’s position (as can be found in other multimodal writings). Mode is an even more debated term ⁴ which in Bateman’s theory is used to refer to a socially developed semiotic system or practice, as discussed above. Elleström’s ‘media modalities’, on the other hand, are concerned with defining and stratifying the concept of medium, and he uses ‘media mode’ to characterise different properties of media. A consistent use of the term ‘media modalities’ helps to distinguish the intermedial from related multimodal concepts and here we distinguish between the ‘semiotic modes’ of multimodal analysis and the ‘media mode’ in intermedial discussions.

There is an overlap between Bateman’s definition of ‘semiotic mode’ and Elleström’s concept of the ‘basic medium’. They can be compared (Bateman 2017: 168) but also include a different set of dimensions. As intermedial analysis is interested in cross-media comparisons, Elleström’s basic media type of, for example, ‘written text’ is a neutral abstract unit that is first made specific and gains social-cultural dimensions when the media modalities characterising written text (i.e., two-dimensionality, visuality, sequentiality symbolic signs) are qualified by context and conventions and recognised as, for example, ‘journalism’, ‘social media post’ or ‘poetry’. In contrast, ‘written text’ as a semiotic mode can be broken down into individual signs and the structures/discourse semantics that organise these signs and make them meaningful, and the social component in the development and use is central.

In addition, multimodal researchers often consider the term 'medium' to refer to a type of material carrier of semiotic content, what Elleström would refer to as ‘technical medium of display’ (Elleström 2021: 34). Bateman considers the definition of medium as a material carrier insufficient, and he instead defines it as a conventionalised combination of semiotic modes within a specific material, called a canvas. Bateman’s concept of the medium is thus closer to Elleström’s ‘qualified media type’ than to the technical medium of display (Bateman, 2017: 168). However, Bateman’s focus on conventional modal combinations differs somewhat from the aesthetic and communicative institutionalisation and conventions that are understood as ‘qualified’ in Elleström’s concept. In this article, we work with Jensen’s understanding of medium (2021: 69–70) that incorporates both the combination of semiotic modes important to Bateman’s multimodal semiotics and the aesthetic and communicative conventions important to Elleström’s intermedial model.

The terminological overlaps accounted for above connect directly with fundamental differences between general interests in either the integration of modes or the relations across media. Elleström’s distinction between basic and qualified media types can highlight underlying similarities across different media, for instance, how the different qualified media types such as opera and cinema employ the same basic media types of bodies, auditory texts and organised sounds, which highlights how some conventions of cinema have developed from opera. Elleström’s model can point to these types of genealogies between media, and it offers cultural historical perspectives on the way meaning-potentials build up over time and across media.

However, having the ‘medium’ as the basic unit of analysis leaves open the area that Bateman’s multimodal semiotics specialises in, the area of detailed and systematic analysis of semiotic processes. Because intermediality is focused on analysing how various basic and qualified media transgress and challenge existing conventions and expectations, it is easy to lose sight of the complex interplay of smaller signifying units such as gesture, setting, lighting, body language and camera perspective. These are all integrated systems that work according to their own logic as well as to the logic of their medium, such as a film. While all of these aspects could very well be touched upon in an intermedial close reading, multimodal semiotics provides analytical models and vocabulary to address the combination of semiotic resources in a systematic, coherent manner.

In order to illustrate how intermedial and multimodal semiotic concepts can relate to each other, we will use a simple simile and compare a medium to a multilayered birthday cake with a printed image, of, for instance, Elsa from Frozen, on top. Imagine the cake is cut, not equally but more like a jigsaw puzzle. The semiotic mode can be compared to the individual puzzle pieces of the cake and a multimodal semiotic analysis is well-suited to look at how these different pieces together form a cake that produce a meaningful image of Elsa on top, how the jigsaw pieces and image are supported by the materiality of the cake (the canvas) and how the cake functions within the social context of a birthday party. The intermedial perspective asks instead how the Elsa cake relates to other cakes and Elleström’s concept of media mode can be seen to specify the different layers of the cake. Elleström’s media framework provides a differentiation of the kind of layers and ingredients that can be expected in cakes, which facilitates a more fine-grained comparison of the Elsa cake to other pastries in one or several bakeries.

This distinction between the ‘layers’ of the medium and the modal ‘puzzle pieces,’ as well as the fact that the different approaches tend to cut the cake into either pieces or layers, leads to terminological tension and diverging perspectives, but it also highlights an interesting area for discussions of complementarity and possibilities for more multifaceted analytical models that appreciate this complementarity. We explore this complementary potential of the integration of modes and the transformation of meaning-potential between media by analysing different uses of “Ride of the Valkyries”, zooming in and out between an analysis of semiotic processes and transformations across media.

In our multimodal musical analysis, we distinguish between modal systems, semiotic mode and semiotic resource. We consider a semiotic resource to be anything functioning as a sign, and we follow Bateman in his definition of a semiotic mode. We vary slightly from other multimodal scholars of music (Machin, 2010; Van Leeuwen, 1999), as we consider music to be a modal system combining a number of semiotic modes, such as volume, instrumentation and melodic shape (see Jensen, 2021: 68–69, 120–124). ⁵ To analyse musical meaning potential, we use Phillip Tagg’s semiotic concepts of auditive, tactile, kinetic, spatial and social “anaphones”—musical similarity with paramusical sounds, touch, movement, space or social relationship—and “styleflags”, connotations derived from the genre or style of the music (Tagg, 2012: 485–486). The concepts of anaphones and styleflags bear similarities with Van Leeuwen’s use of the terms “experiential meaning potential” and “provenance” (see, e.g., Van Leeuwen, 2021: 42–47), but provide a terminology rooted in musical semiotics and empirical studies of audiences’ interpretations of musical features (see Clarida and Tagg, 2003). Anaphones and style flags are means of describing musical meaning potential and interpretive practices, and they are governed by conventions in different genres and styles. Thus, they belong to and inform the discourse semantic level of musical modes, but they also inform the organisation of musical resources, as we will show.

In our intermedial analysis, we utilise Elleström’s ‘media modalities’ to connect the results of the multimodal intersemiotic analysis to the intermedial process of media transformation between myth, opera, newsreel and motion picture. Elleström’s concept of ‘media transformation’ (2014) involves the interplay of ‘media representations’ embodying “the characteristics of another medium” and the ‘transmediation’ of concepts and narratives previously mediated in other media types (15). This two-tiered concept enables us to track the dynamics between transfer and transformation, between the transmediation of the sounds of “Ride of the Valkyries”, the auditory representation of Wagner’s opera and the transmediation of the opera’s conflicts and concepts. By doing so, we can shed light on how the transmediation of concepts and narratives always incorporates elements from previous mediations, which in turn are transformed to align with the conventions of a new media type.

A multimodal-intermedial approach to “Ride of the Valkyries” in conflict and propaganda

The “Ride of The Valkyries” is one of Wagner’s most well-known pieces, and it has been widely used in soundtracks for various media products. We first analyse how Wagner’s music drama Die Walküre (1870) shapes the concept of Valkyries and how this concept relates to the analysis of musical modes of the “Ride of The Valkyries”. Then we analyse the integration of musical modes into new audiovisual contexts and the intermedial transformation process between Wagner’s music drama, a Nazi propaganda newsreel from 1941 (Deutsche Wochenschau, 1941) and Francis Ford Coppola’s Apocalypse Now (1979).

The scope of this paper does not allow for an in-depth discussion of the socio-cultural, historical and aesthetic contexts of the media products, and thus we assume a certain level of shared multimodal and intermedial literacy within the Western adult communities. As such, this analysis is necessarily done with a very generalised producer and Western adult receiver in mind.

Valkyries on the operatic stage

An intermedial perspective that approaches the recurring use of “Ride of the Valkyries” in representations of conflict and war starts out by exploring the connection between Valkyries and conflict. In Norse mythology, valkyrja (“chooser of the slain”) is a female death spirit who executes lethal fate in battle, and they are said to guide the fallen heroes to a continuous feast in the afterlife. Their position in between heaven and earth has been transmediated in different ways. They are depicted as female, armed figures who meet warriors with drinking horns in the Viking age. In literature and poetry, they also appear as lovers of heroes with supernatural gifts (Davidson, 1990: 61–65).

In Richard Wagner’s music drama Die Walküre (1870), Valkyries enter the operatic stage shaped by Romantic Nationalism ideals, and their role is transformed by the demands for a dramatic plot. As the third part of Wagner’s Ring des Nibelungen-cycle, Die Walküre expands and transforms Norse myth and medieval legends into a tragic mythological family conflict between gods and men, with Valkyrie Brünnhilde as a central character. Brünnhilde rebels against the orders of her father Wotan to protect a loving couple and is punished with mortality. In this adaptation process, Wagner favors the Norse versions, specifically the Völsunga saga from the late-thirteenth century, that introduces the legendary character Brynhilde as a Valkyrie. Wagner also expands on why the Valkyrie was punished with mortality for disobedience. As a consequence of this expanding adaptation, two conflicting concepts about Valkyries are present on stage: a mythological concept of executors of fate and a dramatic, tragic character Brünnhilde, a protector of lovers who rebels against authority.

“Ride of the Valkyries” is placed in the middle of the conflict escalation between Brünnhilde and Wotan. Act II ends when a furious Wotan sets out to pursue his disobedient daughter. During the prelude to Act III, “Ride of the Valkyries”, the Valkyries gather, called by Brünnhilde for support. Thus, both the mythological role of executors of fate, represented by the gathering Valkyries, and the dramatic rebellion against authority, represented by Brünnhilde, are transmediated by the action on stage while the prelude is heard.

This intermedially informed analysis of two Valkyrie concepts sets the context for our multimodal analysis. As opera is a performance art, we will focus on the stable elements of the score and the dramatic context that is evident from the libretto. The prelude is musically characterised by three short melodies, here referred to as the background theme, the gallop theme, and the Valkyrie theme. As the space of this article does not allow for a complete analysis, the musical modes of instrumentation, ⁶ volume and melodic shape ⁷ have been selected as focus points because we consider them particularly salient and generative of important meaning potential in this piece.

When we analyse the organising principle (cf. Bateman’s semiotic model) of the instrumentation ⁸ in “Ride of the Valkyries”, we see an antiphonal distribution of timbral qualities and textures, relating to kinetic, tactile and social anaphones, as the instrumentation builds gradually and attaches to different themes. Volume is also organised to set the three themes apart and define the relationships between them, working through kinetic anaphones. Melodic shape is sometimes highlighted for its emotive meaning potential (Machin, 2010: 101–106; Van Leeuwen, 1999: 94–97), but it can also be organised in accordance with anaphones. In the three themes in the “Ride of the Valkyries”, kinetic and social anaphones are used to make the melody take on properties of different elements in the narrative.

The background theme consists of an upward movement in the violins, followed by a tremolo in the woodwinds and then downward-moving chord progressions, also in strings. Both the chord progressions and the tremolo are played rapidly. The melodic movement going up and down covers a broad tonal area, and the theme gradually rises in volume. This high level of activity in strings and woodwinds creates a densely woven, tension-filled texture (kinetic, tactile and social anaphones), and the up- and downward movement (kinetic anaphone) is similar to the movement of rolling waves. In the context of the opera, these semiotic resources of the prelude link with the audience’s knowledge of the dramatic and multimodal context, which is constructed by the semiotic resources of the libretto and the theatrical setting. Act II ends with a conflict between Brünnhilde and Wotan, who, in his rage, disappears into the sky with thunder and lightning (see Wagner n.d., p.: 407f). Thus, the background theme can, in this context, be interpreted as an icon of a violent sky with rolling clouds.

The gallop motif is characterised by an upward-moving wavelike contour starting with an octave jump (kinetic anaphone), played in horns, bassoon and cello. The large octave jumps sound like they are jumping across the dense texture of the background. With each repetition, the theme is played louder, while the volume of the background theme is lowered, so the balance between these themes begins to shift. At this point in the opera, the curtain is still down, so the musical resources have no direct visual resources to link to. However, the opera audiences are again, based on the libretto and theatrical setting of the first two acts, knowledgeable of the Valkyries as riders. Some audiences might also have programme notes, clarifying that the Valkyries will gather at the beginning of the act (see Wagner, n.d.: 429f). In this case, and since the gallop theme in the musical score has a kinetic similarity with a gallop, it could be interpreted as an icon of Valkyries’ horses approaching from the sky. Musically painting a picture like this would also align with general expectations of a Wagnerian opera prelude (or ‘Vorspiel’), which should be “linked closely with the musical and dramatic events of the opera” (Tilmouth, 2001), and of programme music, Western art music which is “narrative or descriptive” (Scruton, 2001), that form part of the discourse semantics in the case of Western opera.

The Valkyrie theme is based on the same rhythm as the gallop theme and begins in bar 13, played forte in horns and a bass trumpet, while the curtain is still down. The meaning potential here comes both from sonic and kinetic anaphones derived from the volume and the strong and dark timbre and from style flags. Brass is often associated with heroism and battle in Western culture (see e.g. the interpretation in Kreitner et al., 2001 para: 4) because of its use in military music, which creates a style flag that is part of the discourse semantic context of Western art music. The meaning potential also comes from the contrast created between this theme and the background theme caused by the interaction of instrumentation and volume. This contrast can be interpreted as a social anaphone or as a relational discourse logic, as it places the Valkyrie theme in the foreground and thus constructs it as more powerful and dominant. In the context of the opera, this could be interpreted as an even stronger icon of riding Valkyries.

Consequently, although music is polysemic, certain meanings are attached to “Ride of the Valkyries” through this interplay of musical modes and the multimodal dramatic context. The musical modes of “Ride of the Valkyries” provide meaning potential relating to power, dominance and tension that here becomes linked with the Valkyries, but which can carry other associations depending on context. The music could connect to an idea of “elitism”, due to it belonging to the high art tradition, or with associations of German nationalism and antisemitism, due to the role that the music of Wagner has played in Nazi propaganda as well as Wagner’s antisemitic writings (see Cohen, 2008).

The intermedial perspective of media transformation can be used to further support the interpretation of the iconic meaning-potential of the themes suggested above. The interplay of the three themes not only connect to but also transmediate dynamic aspects of stage directions in the libretto that describe the setting of a beginning thunderstorm: “clouds rush past as if driven by a storm” and Valkyries appear in lightning and on horseback with “slain warriors” hanging across the saddle (Wagner, n.d.: p. 429f.). The dominance of the Valkyrie theme, which overrides the background texture, can therefore be heard as an auditory representation of Valkyries galloping through a stormy sky and the themes transmediate established visual representations of Valkyries in nineteenth-century paintings as armed women on horseback similarly flying across a stormy sky. However, the interplay between the three themes can also be heard as an auditory representation of the chase of Wotan that takes place between Acts II and III, transmediating Wotan’s anger and pursuit of Brünnhilde. The prelude thus transmediates the superhuman dynamic qualities of Valkyries but also parts of the dramatic conflict. This tension and ambiguity of dominance from the analysis of musical modes, together with conceptual ambiguity between the execution of fate and rebellion foregrounded in the intermedial analysis, reappear even in our following examples, though in different ways.

Fate transforms failure: “Ride of the Valkyries” in propaganda

The German newsreel from May 31, 1941, is one example of the use of Wagner’s music in Nazi propaganda (Deutsche Wochenschau 1941). The newsreel is in black and white with spoken words, moving images, ⁹ and music. The film shows footage from the German airborne invasion of Crete, and a speaker comments on the events shown and reports about the defeat of British and Greek forces. “Ride of the Valkyries” provides the soundtrack to the flight and attack. In our multimodal analysis, we focus on the choice of visual elements and how they interact with the music, as we see a clear pattern of signification here.

The content of the images is framed by the context of German WW2 propaganda film, and the video seems to be guided by a desire to provide a one-sided, positive account and showcase the power associated with weapons and airplane technology. The camera perspective and zoom are used to direct attention to these features. The few shots we get of soldiers show them smiling. The images are presented as if they were a window into the pre-existing world, in accordance with a ‘reality code’ (Bateman and Schmidt, 2012: 142), or a ‘naturalistic coding orientation’ (Kress and Van Leeuwen, 2006: 165). “Ride of the Valkyries is here used for ‘scoring’ the German troops and likely intended to provide a sense of invisible power to the attack, similar to techniques in fictional American films at this time. ¹⁰ “Ride of the Valkyries” starts as a plane takes off and continues to play as the speaker talks, while the video cuts between different shots of the plane. The onset of the background theme is close in time to the visual cuts, giving the impression that the intention could have been for the music and the editing to be synched. The visuals are focused on peaceful water, smiling soldiers and features of the airplanes; the restless background theme and the power relation between the themes have no parallels in these images. The plane and its passengers are rather linked with the confidence of the Valkyrie theme; however, this link is structurally weak, without any temporal synchronisation to reinforce the association. Only when the planes reach Crete does the conflict manifest: we see blurred images of explosions, and we hear the sounds of bombs. ¹¹ To a viewer in 1942, this lack of temporal synchronisation might not have been noticed at all. To a present-day viewer, this lack of synchronisation between the music and the cutting of the images could be problematic, breaking the viewer’s immersion or at least creating a smaller distancing effect, as technological innovations and aesthetic conventions today would dictate a higher level of synchronisation, conventions that are part of the discourse semantic level of film music.

The intermedial perspective explores this ‘scoring’ process as the result of media transformation between opera and newsreel and the interplay between music, images and the narrative they construct. Using a continuous soundtrack of pre-composed music was standard for a newsreel film in the 1940s, and Nazi propaganda frequently used Wagner’s music to transmediate the heroic and monumental as well as bolster claims of political and racial superiority (Schirrmacher 2012: 52–57; Cicora 1993). However, from an intermedial perspective, music can be used to add a fictionalising frame to reports on factual events (Knust 2021; Velasco-Pufleau, 2014), and specifically the sound of Wagner’s music drama frames the current events as part of a mythological narrative based on the fulfilment of a mythological fate rather than as a result of political decisions. Using Elleström’s media modalities, we can trace how the sensorial audiovisual link between the prominent Valkyrie theme and the aircraft that the multimodal analysis drew attention to, together with the spatial superiority of being high up in the air, create an iconic parallel between the attacking German forces and the Valkyries flying over battlefields. The sound of the prelude associated with mythological representations of fate thus aligns with Nazi propaganda claims of Germans as superior human beings, as Übermenschen.

From the intermedial perspective, the lack of visual reinforcement of the auditory tension and dynamic shares some similarities with the role of the prelude to the music drama. Once again, the auditory intensity of the prelude transmediates a sensorial dynamic and semiotic meaning difficult to represent visually in the specific setting. As the prelude is designed to fill in the gaps in an audiovisual narrative, it appears to be a suitable soundtrack to footage from the airborne transport to the battlefield that cannot convey much action, although the continuous switching between different angles constructs some form of visual dynamic. However, the auditory tension and the ambiguities concerning dominance and power stemming from the opera’s interplay between drama and myth are also transmediated and have unintended side effects beyond the multimodal audiovisual discrepancy. The images of passive, waiting soldiers whom we later see parachuting to an uncertain fate happen to transmediate the mythological role of Valkyries as carriers of fallen heroes. And although the music sounds powerful, it was selected to assert dominance in a context where claims of domination are under siege, as the attack on Crete was not the triumph the soundtrack aims to convey: strong resistance by British and Greek forces resulted in the death of more than 3000 German paratroopers (Niemetz 2021).

Thus, although Wagner’s music was probably chosen for its auditory dominance and to transmediate mythological claims of superiority, the ambiguities concerning dominance and power stemming from the opera’s interplay between drama and myth are also transmediated.

A soundtrack of coherence in a conflicting context: “Ride of the Valkyries” in a war film

In our final example, we discuss the helicopter attack from Apocalypse Now (Coppola, 1979). ¹² In this Vietnam war movie, Captain Willard is on a mission to assassinate the renegade Colonel Kurtz. In the sequence analysed here, a helicopter squad escorts Williard through a Viet-Cong-held area and raids a village while playing “Ride of the Valkyries” from loudspeakers.

In the mode of visual elements, the film scene bears traces of inspiration from the propaganda newsreel discussed above, and we see similar images of aircrafts, weapons and water. However, the visual elements in this segment are much more ambivalent, and the focus is less one-sided. The biggest structural difference from the propaganda video is that the editing of the images is now perfectly synched with the music, which is also represented as diegetic.

The different cuts of the video are synched with the onset of the background theme, shifting between different zoom and angles of the helicopter, the soldiers and the weapons and creating a link between the tension-filled theme and these visual elements. The onset of the Valkyrie theme is synched with an exterior shot slightly from below showing the escadrille of helicopters, structurally aligning the escadrille with the Valkyries dominating the sky. The camera angle reenforces a sense of power and dominance on the part of the American helicopters, which is juxtaposed with the nervous looks shown directly after on the faces of the soldiers. A pilot, Col. Bill Kilgore, is moving to the music with a smile on his face, seemingly confident and riled up by the music. A contrast is thus constructed between the helicopters/pilot/Valkyrie theme and the worried-looking private soldiers. The soldiers do not look strong or heroic; they are passive, and they are not riding across the sky but carried across the sky to attack.

While the multimodal analysis draws attention to the careful audiovisual synchronisation and provides evidence for how this scene engages with the Nazi newsreel, the intermedial perspective focuses on how these parallels transform the scene within a complex network of media transformation between opera, film and literature. Like the newsreel discussed above, the film scene combines “Ride of the Valkyries” with the moving images of an attack, exploits the auditory dominance of the Valkyrie theme and transmediates the mythological idea of superhuman executors of fate. However, there are also important differences. The increased audiovisual synchronisation that is possible in a feature film from 1979 enhances the expression of dominance and superiority, which at the same time is undermined by the intermedial interplay with the plot, context and additional layers of media transformation. As the U.S. soldiers are shown to use Wagner’s music on a cassette tape, the diegetic statement of dominance in this scene is questioned by the way the scene transmediates several media products that place this action in the context of Nazi and US racist ideology. In addition to how the camera angles and editing in this scene refer to the Nazi newsreel, the helicopter attack on the Vietnamese village ironically inverts the plot of D.W. Griffith’s The Birth of a Nation (1915), where the Ku Klux Klan rides to rescue white villagers from African American attackers, which is set to the same piece of music. Transmediating Joseph Conrad’s novella The Heart of Darkness (1899), Apocalypse Now both voices but also performatively questions the certainties of racist and colonial ideologies and places the role of the US in the Vietnam War in a tradition of racism and white supremacy beliefs. There is thus, once again, a tension between the narrative and historical context and the audiovisual statement of dominance and mythological claims of fate connected to the Valkyries' theme.

In addition, the sense of entrainment is a result of the multimodal audiovisual synchronisation but is also part of an intermedial conflict between perception and plot. The powerful audiovisual synchronisation keeps the audience locked in the attacker’s perspective while the images also show the victim’s perspective. The narrative plot only provides an extremely weak motivation for the attack. This ambiguity tends to provoke cognitive dissonance and ambivalence (Schirrmacher, 2019).

Furthermore, the audiovisual synchronisation dissolves as soon as the helicopters touch the ground, and the sound continues as anempathetic to what the images show (Chion, 1994: 7–8). Whereas this famous film scene draws attention to the medial affordances of music as a tool of auditory violence in conflict and war (Fast and Pegley, 2012; Johnson and Cloonan, 2008), the effect of this impact is short and replaced by confusion. Diegetically, the music provides coherence and a sense of purpose, meaning and power. Yet, what is meant as a sign of power becomes a “symptom of standing on the wrong side of history” (Salazar, 2017).

Intersemiosis and media transformation: The caution of Valkyries in conflict

In this paper, we started out by making explicit how and why multimodal and intermedial scholars approach similar phenomena from different angles. With an awareness of how multimodal and intermedial scholars speak with different terminological dialects and follow different analytical aims, the joint analysis of how “Ride of the Valkyries” communicates in conflict and war highlights their complementary potential.

The multimodal semiotic analysis, based on Bateman’s concepts, shows how different musical modes, namely instrumentation, volume and melodic shape, are organised and combined to create meaning potential. In “Ride of the Valkyries”, these modes are used to create meaning potential relating to power, dominance and tension. This meaning potential is repeated in each of the transmediations of the piece, but it is also flexible and able to connect with different narratives and images, involving different discourse semantics and social contexts. These meanings largely support the idea of dominance of whatever the Valkyrie theme might be synchronised with and undermine that same idea of dominance when synchronisation is lacking.

The media transformation perspective based on Elleström’s media modalities and media transformation perspective highlights how the transmediation of “Ride of the Valkyries” is used as mythological support and as a license to decide on life and death. While conflict parties use this music to perform dominance, “Ride of the Valkyries” also transmediates the dramatic conflict in a peculiar way. Fallen heroes and rebellion against domination are not only part of the opera scene but also appear in the context of its transmediation and reveal a recurring underlying weakness of the attacking party.

The “Ride of the Valkyries,” dominant as it might sound, tends to be heard in contexts when hegemony is under attack. The multimodal semiotic analysis of the integration of modes can point out moments of crossmodal cohesions and collisions that construct dominance and tension. The intermedial analysis anchors the synchronic multimodal interplay in a network of media products that can support claims of salience and point out dynamic destabilisation and transformation of meaning potential. By switching between the approaches and building on the results of the other method, we have made a case for how multimodality and intermediality can complement each other. Thus, even though our analysis here is based on Bateman and Elleström’s models respectively, the analysis also shows in a more general way how the two perspectives can be used as different lenses through which to see the same media product, resulting in different perspectives and outcomes. When combined, they might cross-pollenate each other.

Such an understanding of the analytical and terminological differences between these two perspectives does not necessarily have to lead to switching between the approaches, as we have done here. However, we hope to encourage scholars to consider the results of the other’s discipline and use them to support and expand their own analysis.