Centering multimodality: A personal and disciplinary journey

Abstract

While museums have always been multimodal in and of themselves, and while museum studies have long engaged with different modalities in the museum (exhibition design, visitor movement, etc.), a comprehensive understanding of museums as ‘multimodal constellations’ has been missing. This was certainly the case in the 1990s, particularly for linguistics as a discipline, in which language was likely to be viewed as an isolated, and primary, form of communication. Yet in a museum, every label is connected to an exhibit, an exhibition, and the museum itself, and communication needs to be understood as a holistic, multimodal enterprise. For me as a linguist involved with the Australian Museum, Australia’s foremost museum of natural history, the multimodal environment of language was critical in our project to improve communication with visitors through better label writing. This article explains the personal and disciplinary developments resulting in the centering of multimodality as a research agenda. An important feature of this approach is the metafunctional theory of Michael Halliday as a core tenet. I explain how this theory of meaning operates across different levels of the institution, in relation to different modes, and in relation to subsequent developments in multimodal research.

Museums and the challenge of communication

Museums have always been multimodal: at the very least, because they are collections of objects, housed somewhere, and presented in some particular way, and this has long been recognised in museum studies. Yet, to fully appreciate museums as ‘multimodal constellations’ (Kress, 2005), particularly in terms of their communicative practices and potential, multimodality needs to be placed at the centre of the research agenda. The current special issue does just that, and in this article I outline two trajectories – one personal, one disciplinary – in terms of how they point to the central role of multimodality in understanding museum communication. I give a personal reflection on my own initial language-focused engagements with museum communication, and how this was disrupted by both the multimodal nature of the museum itself as well as by disciplinary developments, especially a more expansive understanding of ‘text’ and the explicit inclusion of multimodality. I explain how such developments hinged on the particular perspectives afforded by social semiotic approaches to communication, and draw on examples from previously published work to illustrate the issues at stake. Ultimately, while primarily looking back on the historical trajectory of these ideas, I argue that the full complexities of multimodal communication in museums cannot be accounted for without engaging with museums as institutions in terms of their overall organizational semiotics, that is, the many ways in which they are constituted semiotically as organizations.

In this article, the term ‘museum’ is used broadly to encompass museums of natural history, science, or fashion; galleries of art; heritage centres, and so on. The differences between these can be vast, but for now the focus is on the shared feature of being an ‘institution in the service of society that researches, collects, conserves, interprets and exhibits tangible and intangible heritage’ (ICOM, 2022: n.p.). While no definition of such complex institutions can ever be complete, I would emphasize that they are institutions which communicate to and with their publics; about what they are and what they do; and in a variety of different ways. Everything a museum says and does is a way of communicating about its priorities, values, and purposes. This is not confined only to the explicit forms of communication – such as labels, websites, and publications – but encompasses the role and nature of their collections, how these are displayed, the building a museum is housed in, and the events provided for the public. Such an expansive view of communication might be seen to be problematic (cf. Drotner et al., 2019), but it is fundamental to my and others’ approach (see e.g., Blunden, 2020; Diamantopoulou and Christidou, 2019; Insulander and Thorsén, 2023; Kress and Selander, 2012; McMurtrie, 2022; Osmond, 2022). It certainly constitutes an extraordinary challenge to understanding, analysing, and critiquing museums as organizations, as well as assisting them to meet their social objectives.

Beginnings: A personal journey through disciplinary developments

In the 1980s and 1990s, museum studies already had well-established investigations of the many ways in which exhibitions communicated to visitors, especially through the visual and spatial design of the exhibition (see e.g., Dean, 1994; Falk, 1993; MacDonald, 1998). In particular, there was a strong focus on exhibitions as pedagogic instruments defining the museum experience for visitors (Belcher, 1991; Bennett, 1995; (Hooper-Greenhill, 1991, 1994, 1995, 1999, 2000); Pearce, 1995; Witcomb, 2003), as well as an emerging focus on the role of language in being central to communication with visitors (Blais, 1995; Coxall, 1991, 1996, 1999; McManus, 1989; Screven, 1995; Serrell, 1996). Prompted by this interest in language, the then Director of Education at the Australian Museum, Dr Carolyn MacLulich, invited me to work with the Chief Education Officer, Linda Ferguson, and other museum staff, on a project to improve their overall written communication with the public, particularly via the explanatory labels written to accompany exhibits. As a university academic, my disciplinary background was in linguistics, specifically Systemic Functional Linguistics (SFL), which had already been used extensively in primary and secondary schools to support improvements in academic writing, as well as in tertiary contexts (Christie and Martin, 2007; Martin, 2012; Ravelli and Ellis, 2004). I was also experienced in delivering workshops on language and linguistics to school teachers, so I was familiar with translational pedagogies.

As part of this project, we gathered examples of label writing from current exhibitions, and analysed them for various grammatical features, devising language guidelines for ongoing use at the museum (Ferguson et al., 1995), as well as delivering writing workshops for staff from across the education, design and curatorial departments. MacLulich’s own research (1991, 1994) had already demonstrated the negative impact of poor writing on visitors’ comprehension, for example where the absence of a consistent text structure, or the presence of complex and confusing definitions, impeded the clarity of scientific content. Our aim was to help staff produce written texts that were more accessible to the general public, while also conveying the desired and scientifically accurate meanings.

Rather than focusing on simplistic and often misleading advice about ‘good’ writing (e.g., to ‘avoid the passive voice’, ‘write for a reader aged 12 years old’, use ‘clear and simple’ language), we used the strongly theorised framework of SFL to identify a range of linguistic features which contributed to clearer communication. For example, to address issues of museum texts being overly complex, we explained where perceptions of complexity come from (through different understandings of spoken vs written communication). We also focused on how complexity can be actually identified (through measures referred to as grammatical intricacy and lexical density). Further, we identified the resources which contribute to a specific level of complexity, such as the inclusion or removal of nominalisation (loosely explained as turning nouns into verbs, e.g., degrade/degradation, Ferguson et al., 1995: 32–36). We addressed how texts can be structured to flow smoothly with logical connections between different components, through explaining the grammatical resources which achieve this within a sentence (referred to as grammatical Theme). We also explored how thematic patterns build up across a paragraph and between paragraphs (Ferguson et al., 1995: 15–24). Another focus was the actual scientific integrity of the written content, differentiating effective versus problematic ways of defining technical terms, and explaining how definitions need to be managed and integrated across a text to create a clear scientific picture, referred to as a taxonomy (Ferguson et al., 1995: 40–47) (Ravelli, 1996).

Immediately, however, it was necessary to consider labels in relation to the exhibits and exhibitions of which they were a part. To many, this may seem obvious, but to expert scientists writing for an exhibition, often separately from the design process, this was not at all obvious. It is an important tenet of SFL that language always occurs within a social and cultural context, and that the relation of text to context is mutually constitutive. That is, context influences the language used, but the language used construes the context. For example, in the context of an exhibit, it might be anticipated that its focus (say, an animal such as ‘common bandicoots’) would determine the focus of the exhibit label (i.e. ‘common bandicoots’). Often, however, the initial focus of labels is some other kind of information (e.g. ‘Like the common brushtail possum, common bandicoots …’), creating a mismatch with the exhibit itself. Thus, the guidelines included advice such as: ‘it [is] important that the starting point of the text and the starting point on display are the same’ (Ferguson et al., 1995: 17). Again, this was not something obvious to the writers of these texts, until it was pointed out, and it was the linguistic theory of text structure (specifically, grammatical Theme, Halliday, 1994; Eggins, 2004), which enabled us to make this connection.

Also in the 1990s, the discipline of linguistics, especially SFL, was undergoing a ‘visual’ turn, which quickly became a ‘multimodal’ turn. There was a growing understanding that language, while fundamental to communication, was not the sole source of meaning. Early innovators here included Robert Hodge, Gunther Kress, and Theo van Leeuwen, who, in their seminal books Social Semiotics (Hodge and Kress, 1988); Reading Images: The grammar of Visual Design (Kress and Van Leeuwen, 2021) and Multimodal Communication: The modes and media of contemporary discourse (Kress and Van Leeuwen, 2001), set the stage for a more expansive view. Kress and van Leeuwen wrote (2001: 110):

a past (and still existent) common sense [belief] to the effect that meaning resides in language alone – or, in other versions of this, that language is the central means of representing and communicating … – is simply no longer tenable … it never really was, and certainly is not now.

As Van Leeuwen (2010) defined it, multimodality is ‘a field of study investigating the common as well as the distinct properties of the different modes in the multimodal mix and the way they integrate in multimodal texts and communicative events’ (p. 668). An important step in this development was to recognise not only that language operated in conjunction with other modes (e.g. that ‘written’ language also always has visuality), but that a wide range of other resources could be understood as communicative modes. This led to more intensive exploration of diverse modes such as sound (Van Leeuwen, 1999) and touch (Jewitt, 2017), and in turn, this opened the focus of exploration to other phenomena in terms of their multimodal features, such as toys (Caldas-Coulthard and Van Leeuwen, 2002), hairstyles (McMurtrie, 2010) and computer games (Machin and Van Leeuwen, 2005).

Such burgeoning research enabled museums and their exhibitions to also be theorised as communicative texts. In my own book arising from this project (Ravelli, 2006), I argued that ‘communication is more than just language, and that all sorts of other practices – of architecture, of design, of display, of policy – are part of the meaning package that is a contemporary cultural institution’ (p. 148). Language remained critical but also only a partial contributor to the communicative impact of an institution. I identified the need ‘to explore an expanded notion of ‘text’, that is, of the museum itself as a kind of text’ (p. 119), and that ‘it is of course the institution as a whole which is the ultimate source of meaning making’ (p. 139). Others, including Pang (2004), Purser (2000), and Stenglin (2004, 2009) developed similar approaches at this time.

The consistent companion: Halliday’s metafunctional theory

As already noted, museum studies saw significant engagement with multimodality within museums, even if that term as such was not used. There was specific engagement with individual modes, not just the language of written texts as already mentioned, but also the visuality of such texts (their presentation, e.g. Jacobi and Poli, 1995); the nature of audio guides (Bath, 2000; Lamarche, 1995), and the role of layout in spatial design (e.g. Dean, 1994). There was also attention to the exhibition as a whole: how it came together as one integrated text, with specific narrative and ideological effects (e.g. McDonald, 1998; Witcomb, 2003). However, the explicit link between material resources and communicative potential was not always evident in these studies. For example, different patterns of exhibition pathways (linear, circular etc.) might be identified, without linking this to the meaningful effects such differences might make. Or, specific aspects of meaning might be identified (such as ‘write clearly’), without explicating the resources needed to achieve this.

In order to highlight such links, I drew upon Halliday’s social semiotic approach to language (1978, 1994), itself informative of the subsequent developments in visual and multimodal communication. In this framework, communication systems (be they linguistic, visual, spatial etc.) are understood as resources, not as a list of rules or simply formal properties. As a collection of resources, communication thus gives rise to meaning potential: a whole range of endlessly creative possibilities. Communication is thus understood as being appropriate and effective (or not) in relation to context and purpose. As already noted, communication is shaped by social context, but also, context shapes communication: the two are mutually constitutive (Ravelli, 2006: 5ff ).

In the Hallidayan model, meaning is based on the notion of systemic choice: wherever there is an option to communicate in one way rather than another, meaning arises. A passive sentence means differently to an active sentence; exhibits arranged along a linear pathway have a different meaning to the same exhibits arranged in a circle; an artwork placed up high contributes to meaning in a different way from the same one placed down low. Meaning is also connected to the explicit use of communicative resources; that is, there must be some evidential basis for interpretive claims: what are the explicit resources which make a written text seem confusing, as opposed to clear? What are the explicit resources which give an authoritative and impersonal impression to an exhibition, as opposed to one which seems friendly and accessible? It is important in this model to not just identify such different effects, but also to identify how they arise: that is, through which communicative resources.

Crucially in the Hallidayan model, meaning is seen to be complex, not only because of its multimodal expression, or the fact that all communication is positioned interdiscursively and hence open to varied interpretations, but more specifically because it is complex in its functions, serving several purposes at once (Ravelli, 2006: 5). The argument is that all forms of communication simultaneously create meanings about ‘what’ is being represented (representational meaning), ‘who’ is communicating (interactional meaning), and ‘how’ that communication is taking place (organisational meaning). ¹

Representational meaning (termed ‘ideational’ in Halliday’s model) is closest to the common-sense understanding of ‘content’, that is, to what the communication is about, or what is being represented. In relation to language, representational meaning differentiates topics (‘dinosaurs’ vs ‘trains’), as well as how those topics are construed: ‘[T]hat which is communicated is not a passive transmission of ‘reality’, but an active construction of it’ (Ravelli, 2006: 108). Thus, a museum text which says ‘The body of Diplodocus was designed to bear enormous weight’ implies, through the passive voice construction (i.e. was designed), some invisible agent who is responsible for this feature. Beyond isolated sentences, choices in representational meaning create a specific focus for an exhibit or an exhibition: an exhibition on dinosaurs, for example, could be presented in terms of their scientific classification, for example, or in terms of the impact of evolutionary processes upon them, and so on (Ravelli, 2006: 113). In this way, the overall meanings about dinosaurs are construed through the choices which are made.

Interactional meaning (‘interpersonal’ in Halliday’s model) occurs alongside and simultaneous with representational meaning. It refers to the function of communication to ‘take up roles, construct relations and convey attitudes’ (Ravelli, 2006: 7), and it ‘goes beyond conventional and technical senses of interactivity, to encompass the ways in which interlocutors engage with each other in the communication process’ (Ravelli, 2006: 70). For example, visitors can be positioned to take up different roles via statements versus questions: statements position visitors to receive and accept information, questions invite them to respond, perhaps challenging them to provide a response. A statement such as ‘There are five types of cockroach living and breeding in this display’ could be followed by a question such as ‘How many can you see?’, inviting the visitor to look in the display and see for themselves (Ferguson et al., 1995: 30). Again, beyond isolated sentences, such choices also have an impact at the level of the whole institution. The 80s and 90s saw dramatic shifts in museums’ general relations with visitors, moving away from an impersonal, authoritative, and expert voice, which saw visitors as an undifferentiated mass, to one which addressed visitors explicitly, inviting them to engage and respond to content, and possibly with different points of view (Ravelli, 2006: 72).

Organisational meaning (‘textual’ in Halliday’s model) ² constitutes the third, simultaneous type of meaning, and arises because all communication must be conveyed in some way. The mode of delivery (written vs spoken, for example) necessarily impacts the nature of the communication. At the same time, however, the nature of the communication itself can create different impressions of the mode of delivery: a text which is written might seem to come from a densely written textbook, or could seem to be conversational in style, because of the way it is written. It is the choices made in the language which construct this. Organisation ‘is an intrinsic meaning-making resource in its own right’ (Ravelli, 2006: 46): it is not only something which can break down (disorganised texts lose meaning), but the particular organisational focus can emphasise some meanings over others. A sentence such as ‘The fishing industry was given a boost by new technology in 1979’ could be rewritten to have a different starting point, such as ‘In 1979, the fishing industry was given a boost by new technology’, or ‘New technology gave the fishing industry a boost in 1979’ or ‘A boost was given to the fishing industry by new technology in 1979’ (Ravelli, 2006: 36). Beyond isolated sentences, organisational choices at the level of the exhibit, exhibition and institution combine to create focus and cohesion in the institution.

While these different types of meanings (also called ‘metafunctions’) can be analysed separately, and while specific communicative resources are associated with each (Agency with representational meaning, for example; speech function with interactional meaning; and grammatical Theme with organisational meaning; Halliday, 1994), it is important to note that the metafunctions are always integrated. That is, each instance of communication functions in each of these three ways at the same time: every text is about something, creates a stance with its interlocutors, and takes a particular form. Additionally, specific resources can have impacts across the metafunctions, even if they might be more typically analysed in relation to one only. For example, using a passive voice (Dinosaurs were designed….), has specific implications for representational meanings (implying an Agent); interactional meanings (creating an impersonal effect); and organisational meanings (placing the emphasis on dinosaurs).

Developing a multimodal lens on museums

As already noted, a multimodal perspective is not just about seeing how language works in relation to other modes, but it is also about understanding a wide range of resources as communicative modes. With the project at the Australian Museum, Ferguson, MacLulich and I (1995) already understood that the language of the exhibition was intimately related to the design of the exhibition itself, as noted above in relation to aligning the grammatical Theme of a written text with the focal point of an exhibit. Overall, we addressed the complementary role that written texts typically played in exhibitions, noting that they can ‘provide information about the specific objects and images on display, and interpret why they are significant. They can instruct visitors on what to look for in a display, or on how to use an interactive [exhibit]’ (p. 8).

But Halliday’s metafunctional framework offered another opportunity, to take a more expansive view of communication, as Hodge, Kress and van Leeuwen had done, and to use this lens to ‘see’ museums and their exhibitions in different ways. At this time, other multimodal scholars were developing this potential. For example, Pang (2004) examined the Singapore History Museum, and the chronological and linear circulation path of one exhibition, ‘From colony to nation’. Pang argued that the sequencing of exhibits in relation to each other contributed to a particular narrative about Singapore’s history – one which evolved from the chaos of the communist era to the stability, unity and diversity of Singapore’s contemporary era. Stenglin (2004, 2009) explored the particular interactional resources of museum exhibitions (and other spatial environments) and how they can position a visitor’s emotional response in terms of security and belonging.

In 2006, I extended a social semiotic understanding of the language of exhibition texts to that of ‘museums as texts’ (Ravelli, 2006: 121):

‘Museums as texts’ encompasses a number of different levels, or different perspectives on this notion. It can mean the way in which a whole institution makes meaning – from the sense of authority it constructs, to the way it validates an approach to knowledge, to the way in which it functions as a unified whole. These meanings are made through semiotic resources which are realised both physically – through the design and layout of the building, for instance – and discursively – through policies which dictate appropriate institutional goals, for instance. At the same time, ‘museums as texts’ might refer just to the level of exhibitions: the ways in which one exhibition can facilitate particular forms of visitor interaction, can prioritise some meanings in the exhibition rather than others, and can construct a picture of what the subject matter ‘is’. The texts which arise at the levels of exhibition and institution are multi-modal texts, that is, texts which make their meanings by drawing on a variety of semiotic resources’.

I argued that multiple resources contributed to meaning-making in the museum, both at the level of the institution as a whole and the level of the exhibitions and activities which make up the institution. These resources included those of language, but also of design, from the nature of pathways, to the ceiling height of an entrance, to the selection of colours on the walls. While websites were in their infancy at this time, these too are clearly part of the ‘multimodal constellation’ that is a museum (Kress, 2005). I departed from the metafunctional frameworks of Halliday’s model of language, using these as a tripartite lens through which to examine a wide range of multimodal practices within museums. While all of the metafunctions work in consort, my approach is to freeze-frame the analysis three times, to focus in detail on one of the metafunctions, and then to bring them together to see how they interact.

This can be illustrated with one resource of significance to museums, that of pathways. Pathways in museums are of particular interest, because they facilitate visitor movement in terms of their circulation in and around the institution and its exhibitions, and because it is that very movement which Bennett (1995) identified as a key museological device in the historical development of museums, a form of ‘organised walking’. This is typically seen in classical museums of the 19th and early 20^th centuries, where ‘an intended message is communicated in the form of a (more or less) directed itinerary’ (p. 6). Today’s museums explore other alternatives (e.g. Witcomb, 2003), but some kind of directionality for visitors remains integral to the experience. Formal aspects of visitor circulation, such as different basic types of pathways, are well understood in the museum literature: pathways may be circular, linear, branched, open, and so on (Belcher, 1991; Dean, 1994; Falk, 1993, 2009, 2022; Falk and Dierking, 1992; Falk et al., 2008). But how do they work to make meaning, and what kinds of meaning do they make? First it is necessary to identify the resources which can be drawn upon to create a pathway, including vectors (actual paths on the floor, corridors, lighting, the provision of empty space between objects, escalators etc), arrangement of objects (to guide the visitors’ movement) or the repetition of design practices (continuity between components in terms of colour, material or objects, termed ‘framing’ in Kress and Van Leeuwen, 2021) to create a sequence. Such meaning is again metafunctional: any one pathway contributes to each of the metafunctions at the same time. Organisational meanings are made through the placement of different elements in relation to each other, attributing different values to elements depending on the nature of the sequence and the placement of elements within that sequence. Interpersonally, different kinds of pathways can create different relations between institutions and their visitors: directing them explicitly, allowing visitors a more open choice of direction, or denying them access to certain areas, for example. Representationally, different kinds of paths, combined with what is placed upon them, say what the nature of a museum visit is, and how it can or should be experienced.

An example which illustrates this tri-functional analysis was the then-contemporary renovation of the public entrance to the Earth Galleries at the Museum of Natural History, London (discussed in Ravelli, 2006). Formerly named the Geology Gallery, the public entrance – a vast, triple volume height hall – was redesigned to accommodate an enormous escalator which whisked visitors from the bottom floor to the third floor, from which the exhibitions began (and visitors then walk downwards through the exhibitions to the ground floor). As illustrated in Figure 1, the escalator, and visitors upon it, pass through the centre of an oversized globe, symbolically representing the earth (Bloomfield, 2002). Such a design makes meaning through multiple resources. Not only the pathway itself (the escalator transitioning between the entry point to the museum and the beginning of the Earth Galleries exhibition), but also many other features such as the vast size of the hall and globe, the linearity of the escalator (it only goes up), the use of lighting to place focus on the globe, the decoration of the globe – all contribute to the overall effect. Together, these resources contribute to all of the metafunctions.

Figure 1.

Entrance to the Earth Galleries, The Natural History Museum, London (Photo by author).

Organisationally, the design positions visitors in a particular starting place. Following Kress and Van Leeuwen (2021), I argued that such placement attributed it with a particular value, that of being ‘understood’ or ‘Given’ (such as, the visitors’ interest, and/or lack of knowledge about Earth and its geology, as the starting point) whereas the placement of the exhibitions as the arrival point at the end attribute that placement with a value of ‘New’ (such as, previously unknown details about the Earth’s geology, or a promise of something exciting to be explored). The issue is not whether those starting and arrival points are actually ‘given’ or ‘new’ (someone who is an expert in geology can also traverse the escalator), but that the placement confers such values. ‘Placement is ideologically laden: it is not a question of whether any component actually is or is not known to the visitor, but rather, it is a question of how the information is presented, that is, as if it has that value’ (Ravelli, 2006: 125). While there has been criticism of the empirical validity of claims regarding resources such as placement and their values (e.g. Bateman, 2021; Thomas, 2014), there is no doubt that placement, pathways and sequencing are meaning-making resources, as attested by the many vigorous debates about them (e.g. Pang, 2004; Witcomb, 2003).

Crucially, it is the linearity of the escalator connecting the endpoints, and visitors’ traversal of this through the centre of the earth, which marks the transition between those endpoints. Interactionally, the journey can be read as either an invitation or a directive; in either case, visitors must make an active commitment to commence this journey. The size of the hall and globe confer authority and power to the institution, but as visitors travel through the center of the globe, this simultaneously invites intimacy. Representationally, the effect is symbolic: the journey is a metaphor for the visitors’ experience to come, that is, that they will be able to understand the earth from within, and that visitors are part of this experience (Ravelli, 2006: 138).

One further consideration in the analysis of any multimodal text, is how to get from a localised observation of one resource and its use, to an understanding of how that contributes to meaning at the level of the whole institution. The answer to this is in an additional theoretical step: intersemiosis. Intersemiosis is ‘the co-ordination of semiosis across different sign systems’ and it ‘arises from the patterning of patterns’ (Ravelli, 2006: 151). Patterns made at one, first order, level (in the details of language, e.g., or in the layout of a room) are re-patterned when multiple systems work together, creating second-order meanings, ‘the level of symbolic articulation’ (Ravelli 2006: 151, drawing on Hasan, 1989). At this level, potentially disparate patterns are integrated through foregrounded features (Halliday, 1973), creating a consistency of patterning. For example, an entrance policy which makes it free for all visitors to enter, might be complemented by exhibitions which enable active exploration by the visitor, and by patterns in the language of exhibition texts which invite responses from visitors. The symbolic articulation of a feature such as ‘openness’ is thus manifested across and by a range of resources. ³

A journey ongoing: Multimodal extensions

The multimodal analyses presented in Museum Texts (Ravelli, 2006) encompassed broad descriptions of a range of resources operating at the levels of exhibit, exhibition and institution. Since then, these analyses have been developed as a more detailed theory of Spatial Discourse Analysis (SpDA, Ravelli and McMurtrie, 2016), applicable but not confined to museums. SpDA makes use of the social semiotic approach inspired by Halliday’s metafunctional frameworks, to describe multimodal resources drawn upon in the built environment. Applied to the Portuguese Language Museum of Brazil, SpDA reveals how a changing design focus across the multiple levels of this museum provides a diverse and engaging representation of language and its relation to history, society and culture in Brazil (Ravelli and Heberle, 2016). In relation to a contemporary art museum in China, SpDA explores how multimodal resources, from the design of the building to the layout of the exhibitions, construe this art museum as being both part of the global world, as well as being intrinsically linked to local and national culture (Ravelli and Wu, 2022). And while not strictly concerning a museum, SpDA is also used to investigate the complex cultural implications of a Disney theme park ride, based on the 2009 film Avatar (Ravelli, 2019). Further analytical tools which account for how visitors occupy space, how they negotiate their attention, and how their movement might align or disalign with a curatorial vision are developed in Robert McMurtrie’s 2017 book, The Semiotics of Movement in Space: A user’s perspective, providing a more specific account of users’ movement in spatial environments.

One particular challenge for multimodal analysis arises from more recent work of van Leeuwen (e.g. 2021a), which examines the contribution of multimodal resources to more intangible aspects of meaning, such as style and identity. This approach identifies parametric systems, such as texture, touch, and colour, which ‘not only make meaning in a different way, they also make different kinds of meaning’ (p. 3, original emphasis), in contrast to the functional systems identified in Halliday’s work on language and Kress and van Leeuwen’s work on images. A parametric approach would complement the above analysis of pathways, for example, by accounting for the materiality of the pathway. A pathway made from smooth vinyl versus one made from rough gravel creates a different effect, a different ‘identity’, its meanings deriving from a combination of provenance (associations of that material in other contexts) and experience (what it feels like: smooth vs rough; Van Leeuwen, 2021a; and see Hellwig, this volume, building on Van Leeuwen 2021b, for a parametric approach to movement in museums).

The multimodal agenda is much broader than that which has been described here in terms of my own focus on the design of spatial environments, as other contributions to this special issue will attest, and there is infinite scope to continue this journey, theoretically and practically. Yet a particular challenge for such research will be to understand museums and their multimodal practices ‘from within’, that is, as organizations, with their many pressures, many competing agendas and needs, and invariably, finite resources. As stated in the introduction, museums – as organizations – are semiotic entities, complexes of meaning-making practices. These meaning-making practices cannot be understood unless the museum as an organization is also understood: ‘multimodal social semiotic studies need to account for the semiotic nature of organizations as socially critical sites of our contemporary world’ (Ravelli et al., 2023: 2-3). Integrating a social semiotic approach to multimodal communication ‘with established insights from organizational research’ results in an approach called ‘organizational semiotics’ (Ravelli et al., 2023: 12). This draws upon the insights of organization studies to learn from ‘its detailed and sociologically grounded accounts of the meaning-making activities of organizations, such as managing, innovating, controlling, or marketing and of the relations between its participants – its hierarchies, leadership models, team dynamics, etc.’ (Ravelli et al., 2023: 15).

Returning to my personal starting point in the 1990s, I was immensely privileged at that time to be included ‘on the inside’ with the Australian Museum. This meant that my analyses were fully contextualised within the concerns of the museum, and especially its educational agenda. It meant that materials and workshops had to be prepared with sensitivity to the staff involved: they were all already experts in their own areas of curatorship and design, but perhaps not as expert in communicating with the public. It also meant that, even while the museum’s initial request required a focus on language, multimodality could not be ignored. This included the complementarity of language with other resources (e.g. that the visual focus of an exhibit and the starting point of a written text be the same), but also the way in which diverse communicative resources functioned together as text. It is the adaptability of Halliday’s social semiotic theory of language which afforded metafunctionally complex analyses of the diverse linguistic, visual, spatial and embodied resources of exhibits and institutions. Multimodality as a phenomenon is not new: it is obvious that museums have always been multimodal. However, multimodality is a relatively new point of focus for specific disciplines, especially in terms of developing both deep understandings of specific resources other than language, and in terms of developing deep understandings of how multiple resources come together to form complex texts. Communication in museums must be understood as something much more wide-ranging than the language used in exhibition labels or other resources, as important as that is in and of itself. Multimodality must be at the centre of our agenda, connected to a deep understanding of museums as organisations with deep responsibilities as meaning-makers for and with visitors.