Not burning,but rebuilding: The ABCS of anthropology using multimodal methodologies and DEAF values

Abstract

In 2019, Ryan Cecil Jobson made the case for ‘letting anthropology burn’, arguing that: “the field of anthropology cannot presume a coherent human subject as its point of departure but must adopt a radical humanism as its political horizon” (2019:328). In this article, Robinson upholds Jobson’s mandate to adopt a radical humanism, though not by burning, but methodologically rebuilding anthropology using multimodal ethnographic praxis, drawing on learnings from time spent working and learning from DEAF spaces. She charts her intellectual and affective journey from working with deaf artists and educators, through studies in anthropology, and towards transformative encounters with multimodal social semiotics. Through learning to listen and evaluate in DEAF ways, she found that non-deaf, majority-centred ways of conducting research naturally receded, thereby re-scaffolding ethnographic methods, proffering a ready-made approach to inclusive anthropological knowledge-making and a potential means for democratising anthropology from the ground up. Robinson argues that approaching ethnographic praxis in a way that both de-prioritises English-text and prioritises participant-led approaches by default reshapes what constitutes valuable anthropological knowledge, facilitating broader participant inclusion and legibility, while also rendering more apparent entrenched power hierarchies and epistemic imbalances.

Preamble: Situatedness matters in research design

While factions in anthropology have long been focused on ‘person-centred’ inclusive practices (e.g., utilising Participant Action Research (Freire, 1996) and Citizen Science approaches to centre participant ‘voice’ via direct contributions (Krick, 2022), many colleagues within anthropology view these activities as ‘outside of’ or ‘additions to’ the proper work of analysis. Though focus has been placed on interpretive (Laidlaw, 2018), affective (Navaro-Yashin, 2009), and ontological turns (Holbraad and Axel Pedersen, 2017), as well as the essential reckoning and recalibration taken forward by decolonising movements (Nyamnjoh, 2020; Bei and Knowler, 2022), this piece argues that anthropology can only truly rebuild its analytical surety by addressing the hierarchies inherent in its methodological processes in the name of voicing the lives of others. I am certainly not the first to address this issue (see Hui, 2023; Pillen and Matthews, 2022, Pink, 2022, Ingold, 2020). However, my 10-year learning trajectory using multimodality as a key methodological approach amidst DEAF colleagues and interlocutors has provided me with an ongoing case study of how our methods can rebuild the means with which we produce anthropological knowledge.

Late last year, I finally completed an article I’d taken far too long in writing. It was a piece called “Knowing by DEAF ¹ listening” for the journal American Anthropologist. My primary objective was to somehow convey the debt I owe to my deaf colleagues for patiently – and sometimes less patiently – guiding me away from my persistent non-deaf understanding of world, turning instead toward one which thinks and knows differently, one which thinks and knows ‘DEAF’-ly. By capitalising ‘DEAF’ as one would when quoting a sign language word, the article signals not simply a move toward documenting sign language or tenets of Deaf culture (Humphries, 2008). Rather DEAF refers to authority which is entirely apart from (rather than in binary opposition to) sensorial or cultural hHearing. It commands acknowledgement of DEAF as a way of being in the world. I argue in that piece that shifting knowledge to be DEAF affords value I hadn’t realised was missing until I was shown what it does to concepts of ‘the/a world’. In this essay I wish to add further articulacy of exactly what had heretofore been missing: multimodal listening, thinking, analysing as a way to ‘make strange’ and engenders a more robust anthropology through true reflexivity.

I am an anthropologist whose research uses various theoretical frameworks to understand the empirical points related to stereotypy and definitional disparities that lead to social and communicative breakdowns including: superdiversity; linguistics; sensory anthropology; epistemic injustice; and affordance theory/semiotics. It cannot be overstated how much the lessons I learned during my early postgraduate studies and in the field with my Deaf colleagues, interlocutors and friends transformed the way that I crafted my approach to ethnographic study, taking the tenets of multimodality as a practical means to conducting field research and also as a way to open up the analytical potential of such work.

I began my postgraduate study from a standing start; previously I had worked in development and consultancy for 10 years following my undergraduate studies in English Literature and Theatre. I was studying ‘at home’ in the UK, having been made a citizen only 2 years prior.

While every ethnographer has an unshedable skin, at that time mine felt thicker, and more unshedable than most. Not only did I hold the outsider status common on some level to all anthropological endeavours (I was then/am now an American by birth, an English speaker and Hearing, though that latter was only made visible to me by virtue of my role as interloper in the Deaf Arts community in London), I was also older than my study cohort, a married person and relatively new mother to a 1 year and 3 years old when I began my studies, 12 years out of undergraduate studies and with a decade-long first career in business development and consultancy under my belt. Though I had a well-formed research project idea even during my masters, and though I was somewhat conversant in British Sign Language (BSL) having by that time been working with my interlocutors in a professional Deaf theatre company called Deafinitely Theatre for more than 2 years, I was well aware that language was not the only gap I had to bridge.

My early research aimed to understand how the vast minority population of British deaf people – roughly 11 million of about 60 million total population (Jabb, 2021) – listened and were (or weren’t) listened to. I investigated why sign language and other visual-tactile dominant communicative modes were often either reified or else ignored depending upon the environment and audience involved in the communicative exchange. I set out to document DEAF (Gulliver, 2009) listening away from hearing-dominated paradigms which dictate by multi-sensory hierarchies, backgrounds, and discordant hegemonic rigidities, mapping British Sign Language-in-use, as well as non-signing use of DEAF modalities even in instances deaf people do not or cannot use sign language. I geared my fieldwork specifically toward institutional reception/resistance to deaf-centred social practices, and related mental health impacts within two key groups:

1. Deaf professional artists working in theatre, dance, film, television, and fine art; and

2. Deaf social care advisors and their clients in their encounters when claiming support

Even before I began fieldwork, I felt myself to be at odds with the university world in which I suddenly found myself, with a jargon-laden, canonically centred discipline to learn from scratch, two toddlers in day-care, as a well as a partner bemused by the sudden shift in my life-path. It was serendipitous therefore that I was encouraged to take the Gender and Language module offered in my first term, taught by a scholar whose work is very much grounded in linguistic anthropology and multimodal thinking (Pillen, 2016, 2017) with particular focus on communication ethnography (Rampton, 2006). Her teaching was unlike any I had encountered before, lectures centred on the soft power of language regimes and related cultural domination, from antiquity through to contemporary conflict. In her seminars, she played audio recordings of spoken Kurdish or – as described in Charles Briggs work (1992) – Warao women’s ritual wailing (without transcription) leaving in silent and unsilent pauses and as heralded as all important in Tedlock’s treatise on the value of caesuras (2009). The soundscapes in the seminars were sometimes presented alongside animated maps, demarking the places in which colonialism had blotted out local language in a wave of English, French, Dutch and Spanish langue & parole, photographs, and scattered glyphs from languages I had not yet encountered. It was she who encouraged me to begin mining the burgeoning discourses on multimodality for a way of engaging with the entropy I was experiencing and feeling as I sought to become ‘anthropologist’/’adult’; the two were symbiotic. In my life and in my fieldsite I was generating “a representation of the friction between incommensurable worlds” (Pillen, 2017).

Anthropological praxis

My primary methodological approach throughout my doctoral field research was, as is typical of anthropology, participant observation, what Geertz (1998) coined as “deep hanging out” and which meant spending intensive, prolonged periods of time amongst groups or individuals who were deaf and/or who used BSL as their preferred language. I knew from my time working amongst deaf-centric theatre-makers that British Sign Language (BSL) is paramount to understanding “Deaf worldview” (Bechter, 2009; Bauman, 2008; Bechter and Bauman, 2008; Robinson, 2019a, 2019b, 2022), or more accurately ‘DEAF world’, a life-way grounded in DEAF experience. In the face of this requirement, I confronted a keen awareness of unknowability, requiring a rejection of “mono-realism: the idea that there is one, and only one, reality that our thought is or can be connected to […] From it emerges the possibility that what we call “reality” is merely a dominant reality, and that there are always minor realities in which we are equally enmeshed” (Hage, 2011: 7).

I understood enough to know that any analysis of this visual-tactile language must include not only the signs themselves, but also the context of this language: its history, component parts, and actors. As I did so, I attempted to maintain a particular awareness around not only my linguistic position as an English speaker, but also my sensory biases, and thus became aware of my socially-assigned position as a ‘hearer’. In order to maintain awareness of my sensorial perspective throughout my observations, I began to think with and incorporate into my study the ideas of Social Semiotic Multi-modal Analysis Theory (SSMAT) (Bezemer and Kress, 2008; Kress, 2010; Jewitt, 2007, 2009) an approach to communicative

entities in which meaning and form appear as an integrated whole, a sign […] always newly made according to the interest of sign-makers in specific social environments […] made in and for the conditions of its use. (Kress, 2010: 48)

This conceptualisation assisted in my development of my two-fold analytical approach: firstly, it addressed the ways that environment and other contextual factors or ‘affordances’ (Gibson, 1977; Keane, 2018. See also Silverstein, 2016 for sociocultural affordances) are essential to creation and reception of communicative content. Secondly, back then (and it has continued to ever since) this way of working provided me with a framework through which I can better understand why intentionally constructed, context-specific content can (and does) ‘fail’, certainly between people without shared communication resources, but more interestingly to me between people who have language in common. In each case the message is often misconstrued or ‘fails’ because it is not received or receivable by the listener due to their contrasting/divergent affordances. Positioning persons via their self-specific affordances enables one to look not only at linguistic differences but also at other equally important communication components including differing historic, cultural, sensorial, educational, affective, and modal concerns. Through mapping the ways that communicative exchange is structured – taking into account proxemics (the nearness of participants to one another in space see Hall et al., 1968), social cues/mores, education levels, language resources and so on – not only knowledge-making, but social and power hierarchies are also dragged in-frame. For this reason, my research approach (from which my analysis would subsequently develop), centred on the ways that people not only author their social realities in self-specific ways, but also on how these realities are or are not understood by their audiences.

Using this understanding I recognised that one could include differences in production-listening practices; all the modes contained within the actions of creating a message, as well as how participants paid attention and listened to that message containing language, space, gaze, place, history (personal, community, spiritual), body placement, image, sound – anything that informs communicative exchange. Multimodality used in this way becomes invaluable to the anthropologist-ethnographer. It is not only a sort of checklist of things to attend to when conducting observations, but also a means of formulating one’s methodological approach, particularly valuable because it portends inclusion of all the cogs in communicative exchange, not only vision, tactility, sonic, olfactory, but upbringing, accent, ethos, environment. ‘Modality’ becomes a gloss for the sensate or affective container, all that reverberates in flesh and bone or impacts upon various perceptual channels; it recognises that the message’s referent is the body and experiences of the listener as much as it also the embodied-expertise and intent of the message-maker. Through witnessing the influences of, in, and through that body, the witness gains an awareness of an unendingness of that body – its continual reconstitution in and of society, a lemniscated, dynamic puzzle.

The potential for communication to be the friction between incommensurable worlds to use Pillen’s phrase became apparent to me first in arts settings, when the artists and professionals with whom I worked in our office required support in the form of, for example, paid-for BSL interpretation (during business hours) but were refused by UK funding schemes such as Access to Work ² which imposed limits on hours of interpreted communication. In these spaces, I witnessed the ways that my colleagues, accomplished and lauded across the sector and much more senior than me in professional terms, were subjected to a systemic, synecdochic minimisation: reduced to the decibels of their audial apparatus. Their accomplishments, their art, their formidability as parents, partners, advisors, these were all erased in favour of focus on a majority-perceived ‘fault’ in hearing – or conversely, not being faulty enough to merit public support. These colleagues would therefore consistently labour outside of working hours to contend with the bureaucracy that enabled them to do their actual paid-for work during the day.

I later saw far worse situations in advisory client scenarios working with deaf people for whom deafness was part of a panoply of complex health needs, and during which the mode of telling, even if it seemed to be conducting passive information, was in fact hugely important. Through attending to the nuances of communicative exchange, I began to see the value someone/I might be able to bring into these spaces as a communication support worker (CSW), but equally I saw the even greater possibility of my role in getting the whole message wrong. The role of the intermediary is always one of flattening (Robinson, 2019a, 2019b), a faded photocopy as my colleague once told me. It is at best always a partial erasure of the person who first expressed the message because the conduit – the mediator – takes over the ‘indexical centerpoint’ (Silverstein, 2013: 3), thus making the message a contingent thing, a creature born of both the original teller and the mediator simultaneously, thus making the power hierarchies inherent in the telling a crucial part of the meaning made. Every message-making exchange intrinsically holds this tension, a tether tightly taut between two or more poles in their own distinct moorings, each of whom brings an agenda, and the potential (likelihood) of taking away a fundamentally different understanding of what has been said. Those exchanges which add to already varied frames of reference distinctly different modes being employed in telling and listening, each communicative event will be laden with even greater power differentials crucial to not only communication of a message, but to ‘the social’ as a broader project. So it is that multimodal witnessing in the field yields methodological issues which are also analytical issues.

Applied multimodal thinking in anthropology

Linguist Blommaert (2007) writes of language analysis that “[a]n ever-present concern in this exercise is the difference we ourselves can (should?) make: that of specifying detail in discourse, of demonstrating the precise and minute ways in which this symbolic commodity works and generates or articulates power and inequality.” Buscholz (2000)’s case study exemplifies this via the transcription of an African-American man’s account of the Rodney King verdict in the media written down in English text but subtracting accent and inflections as well as the dynamics and cadences of speaking, shows a prejudiced treatment of the testifying man’s language. Just as Buscholz shows the representation was adjusted by the journalist to avoid appearing too “ethnic” for the press, so signed utterances have been transformed to be more hearing in the non-signed, textual transduction of the original message. It is transduction rather than transcription because a textual interpretation in written notation manipulates not just language but modality in order to create the desired/required representation. This “naturalized transcription,” that is, “one in which the process of transcription is made less visible through literacization, the privileging of written over oral discourse features… runs the risk of failing to call enough attention to linguistic form and its transformation from speech [replace with sign] to writing” (Buscholz, 2000: 1461 [interruption mine]).

Replacing the word speech with sign, renders apparent the similitude between the King verdict response and the near-constant occurrence in sign or indeed, any transcription in linguistic spaces. Stokoe’s “Sign Language Structure: An Outline of the Visual Communication Systems of the American Deaf” was primarily concerned “with observed behavior corresponding with phonetic behavior in spoken languages” (2005:16), that is, presenting signed and spoken languages as similar, that being crucial to the eventual acceptance of American Sign Language (ASL) as truly linguistic. Sign language in Stoke’s work was valuable primarily because it could be mobilised to further-illustrate the innatist arguments of popular Chomskyian discourse. However, in order for this logic to hold, Stokoe had to ignore the ways in which the two languages are not similar, that there are “five meaningful dimensions of contrast—three of space, one of time, and one of facial expression (representative, evaluative, and addressive)” (Bechter, 2009: 256), and that these are delivered through different and differently-valued communicative modes. In documenting sign language, it is crucial to understand that stripping of one element of the composite utterance (Enfield, 2009; Kendon, 2012, 2013) -- that is, the full communicative construct which can include speech, gesture, facial expression, referents, spatial placement -- may seem like a small elimination, but the removal of that one element fundamentally shifts the semiotics and thus the intention and meaning for/of/by the signing Deaf person. Moreover, “use of words of a spoken language to transcribe sign language data may result in a researcher (unconsciously) ‘transferring’ characteristics of the [spoken] words to the signs” (Vermeerbergen, 2006 see Figure 1) rendering it not a transcription but an interpretation.

Figure 1.

Simultaneous construction in Flemish sign - “Person approaching car” - the face conveys the person’s emotion, the flat hand shows the stationary vehicle, the finger, in motion, shows the person on approach (Vemeerbergen, 2006).

Multimodality as a necessary practical and analytical concern in ‘the social’

Anthropologists are in a constant battle to accurately document this complex inter-connection between the ‘interior’ capacity, the physical limitations of a particular body’s receptive/comprehension capabilities, combined with the ability/desire/practice of expressing such capabilities.

Take for example, Julia Sauma’s account (forthcoming 2024) of negotiating the onset of deafness during her fieldwork meant that she experienced a modal shift in the ways in which she attended to and processed knowledge which had a considerable impact on how she herself constructed knowledge in the field. However, her shifts were also productive of new relations with her interlocutors. She writes that her experiences of ‘not-hearing’ particular animal or environmental sounds, opened up different ways of investigating and discussing with her participants the ways that they were experiencing the sonic landscape. Her new sensory self was generative of entirely different ways of knowing the social forms and spaces around her. I’ve reflected elsewhere (Robinson and Carroll, 2023) about anthropologist Ghassan Hage’s consideration of his experiences of adult-deafness; though the habitus he had cultivated around listening aurally was no longer viable, his curiosity to learn through casual information gathering, his knowing-disposition, remained undiminished. Though capacity to hear was fundamentally altered within both Sauma’s and Hage’s bodies and their research and interpretive practices were impacted accordingly, they were afforded means of evaluation and thus new value.

Reflecting on this in 2013 and ever since, I understand that anything I attempted to decipher as I conducted field research and later as I wrote about what I had witnessed would be filtered through my own physical/mental capacities and dispositions. Each documentation was my interpretation of a communicative moment rather than an accurate representation of that moment telling as much about me as it did about the people who I was attempting to represent. I recognised that my Hearing, non-native, non-fluent understanding of the role of BSL and DEAF ways of being in the world would always be at best a poor facsimile of DEAF lifeways, and at worst could be appropriative and fundamentally incorrect, potentially missing the key importance of BSL to DEAF people. This flummoxed me and to some extent froze my research progress. However, as anthropologist, Michael Jackson, writes, “when action on the world around us proves impossible, we have recourse to action on our own emotions and thoughts, thereby transforming the way we experience the world” (2004: 49).

That “perception is not an ‘inside-the head’ operation, performed upon the raw material of sensation, but takes place in circuits that cross-cut the boundaries between brain, body and world” (Ingold, 2000: 244) drives this point home. It is the witness who interprets phenomena via their sensing/experiencing body, then tells the story and thus that story delivers meaning contingent on their own receptive biases. It is through their reckoning with their own body as a ‘horizon’, a precipice from which the person connects to objects, to others, that each listener generates the extended concept of self in world (Hanks, 2000: 514). The process of engaging in sensing that is critical to the ways that a given society articulates their schematisations of ‘world’. This is what ethnomusicologist, Steven Feld, has referred to as ‘ethnoaesthetic negotiation’. For him in his field site listening was a process of recognising his own embedded predispositions, and then “trying to work with Bosavi people to understand how they listened, how they heard the dimensionality of forest sound, how they would balance a mix of birds, water, cicadas, voices, and so forth” (Feld and Brenneis, 2004: 467; Feld, 1984). His account of the Kaluli people in Papua New Guinea depicts a society in which sensing and listening is not only how the world is experienced but offers insight into foundations of the structural framework on which the entire Kaluli world hangs and the modalities to which one must attend in order to be able to approach understanding of that world. Throughout his field research, Feld revealed to himself not only the importance of listening to his interlocutors, but also how their brand of listening alerted him to his own textual biases (Feld, 1990).

‘Worlds’ are drawn from, in Marcel Mauss’ words, biological, sociological, and psychological elements (1973:73), elements uniquely attuned to and by each person’s body combined with their social inputs and experiences. And as I realised over time, through engaging in the role senses and perception played in anthropologists’ ethnographic endeavours and their outputs, innovation can arise through contending with one’s own boundedness, ultimately self-determining perhaps not what local worlds ‘are’, but rather what they ‘do’ – the ways through which they exert witnessable shapes, sounds, smells, and tastes. What if, rather than trying to guess other people’s entirely opaque (to me) interiors whilst ignoring my own (an impossibility), I instead tuned in to the ways that DEAF predicated communication practices shapes ‘world’ in certain ways? With this in mind I began to shift my line of thinking toward this latter framing, noticing that if I paid attention not just to linguistic forms but to all sorts of elements that feed communication, I would be in much better stead to advance a much more successful argument grounded in my experiences and interpretations of the social worlds I was researching. I also began to reassess the ways in which my own research might be exclusionary and how I could address this.

In this regard, I took practical cues from Deaf sociologist, O’Brien (2015), and from American Sign Language linguistic anthropologist, Bechter (2008). O’Brien acknowledges that, though he is deaf, his linguistic capabilities were often at odds with those of his research participants in the field, some of whom used BSL, some spoken English, some a combination, and all at varying levels of fluency. He recognised from his doctoral research as well as from the lived experiences of being a deaf person in the UK that written and spoken use of English had a complicated and often contentious role in the lives of his adolescent interlocutors, and that its use would have excluded many participants who were less fluent or less confident in the language.

O’Brien also expresses his feeling that using English as the primary interviewing language gave spoken/textual language too much power (corroborated in the work of Baker-Shenk, 1986, 1990, 1991; and in work on linguistic regimes, particularly Kroskrity (2015). He instead encouraged participants to engage in descriptions of photos, and even asked some participants to take photos of the things they most valued, allowing these to guide his investigation and thereby empowering his young interlocutors to explore ideas via their preferred communicative channels. This approach in some sense mirrors ‘everyday talk’ approaches (Ochs, 1993; Schieffelin and Ochs, 1986;), and Rampton (2006, 2007, 2009), observing communication for both what is expressed, and the context or deictic field in which the talking (signing) was occurring (see also Enfield, 2009: 519–520). I used social media posts and photos in a similar way. I observed BSL videos, photos, and written commentaries, contacted the commentator to discuss and request permission to cite, and used the concerns highlighted in these arenas to help to guide the questions I used in my field interviews and later, in my analysis.

I was also very inspired by American anthropologist, Frank Bechter’s field approach and analysis (2009. See also Carroll, 1996). In seeking to establish generic American Sign Language storytelling conventions across a number of different narrative forms including videos and static images, he used a combination of existing video and photographic archives of story-telling performances with fresh footage/photos that he captured during his time in the field. This enabled him to treat the archived performances with same weight as he would live interlocutors. In this way, Bechter to some extent mitigated the privileging of English born of traditional transcription methods. During my PhD field research and analysis, I did this as well, documenting performances on film myself, but also using online video postings – particularly from Vimeo and YouTube – to bolster my explorations and mitigate exclusion of topics I may have otherwise unintentionally eliminated through editing, unrecognised biases, or just by not paying close enough attention. If there was something I had witnessed, particularly within the confidential advisory domain where film was not permitted, I would also try to see if it was something that was discussed amongst my colleagues at the centre, interlocutors online, or topically covered in deaf-generated art.; According to Paddy Ladd, the National Association for the Deaf (NAD) in America “pioneered the use of film in 1913, recording a number of Deaf speeches to ensure that sign language was preserved, and several of these filmed lectures are still available, containing themes which themselves indicate a new depth to self-perception” (2002: 127. See also MacDougall, 2022). He illustrates this point using the following quote, signed and captured on film, from Veditz (1913):

as long as we have Deaf people on earth we will have signs… It is my hope that we all will love and guard our beautiful sign language as the noblest gift God has given to Deaf people. (Veditz in Padden and Humphries, 1988: 35–6 via Ladd, 2002: 127)

Pursuant to my desire to chronicle DEAF ways of knowledge-making that were in play around me throughout my field research period, amongst my artist interlocutors, I took ‘film notes’, shooting hundreds of hours of candid film footage. As this documentation was not possible within the social support advisory settings, I explored more creative multimodal approaches to make this reduction visible to non-signing readers of my work. In one instance, for the purpose of a detailed case study which had been documented by me entirely in text from the original BSL, I engaged an actor, a camera-man, and an animator (all deaf, sign-language users) to help me to interpret the signed testimonial that a client had performed and which I described in my English textual translation in order to further illustrate the pitfalls of flattening sign into textual public sector forms via a process of (impossible) inverse-re-dimensionalisation of my interlocutor, Tuck. What was paramount was to open up a space to make visible the lingual and epistemic hierarchies that were constantly in-frame and to find ways to not just reveal but attempt to address their limitations within my work.

Applied social semiotics and multimodality

The same elements that made multimodal thinking a brilliant practical means of examining the shapes and systems that arise through ways of paying attention using socially-specific, normatised sensory capacities (Lane, 1989), benefit many/any studies in which there are significant power imbalances, whether between researchers and participants, or in the wider field as a whole. It was for this reason that Kress’ Social Semiotic Multimodal Analysis Theory (SSMAT) used in combination with (Sperber and Wilson, 1987) Theory seemed the most comprehensive framework within which one could documentation of BSL and consequentially of DEAF epistemologies.

Michael Halliday’s Language as Social Semiotic frames language and learning are societal: “rather than the individual is at the centre of the picture, and the individual’s language potential is interpreted as the means whereby the various social relationships into which he enters are established, developed and maintained.” (1978: 16). Kress’s and Jewitt’s version of social-semiotics go further in their focus on the combinatory power of multiple ‘modes’ of communicative exchange, rather than the exclusively socio-linguistic approach taken by Halliday. This requires moving beyond language to pay “attention to other modes which are in play: gaze, facial expression, gesture, spatial positioning” (Kress, 2010: 54, 58. Havilland, 2000), which are likewise key components of sign languages (Woll and Sutton-Spence, 1999).

Kress’ situational understanding of semiotics uses visual, audial, and temporal examples of mode to emphasise both the historic trajectories of message-making as well as the ways in which the ubiquity of technology has fundamentally changed communication in the contemporary world with “far-reaching changes in the domain of meaning: in representation and in ‘semiotic production’; in dissemination and distribution of messages and meaning; in mediation and communication” (2010: 5–6). In contrast to Chomsky’s theory that there are “innate mental functions that provide for the acquisition of [linguistic] competence” (Chomsky, 1967: 3), these multiple modes align with “shared social, semiotic, communicational principles and dispositions…” and “(1) signs are motivated conjunctions of form and meaning; that the conjunction is based on (2) the interest of the sign-maker; using (3) culturally available resources” (Kress, 2010: 10).

Jewitt’s seminal volume Routledge Handbook of Multi-modal Analysis lends further credence to the relevance of multimodal analysis couched within both a time of great social change and also transformed the potential of what multimodality might be useful for by marking “the visual turn” (p3), not as a new phenomenon but made possible via new forms uses of visual modes whilst also understanding “multimodality as a commentary on new ways of making meaning and marking a historical turning point, [acknowledging] that to live in any culture is to live in a multimodal culture” (Jewitt, 2009: 4). This addition to the discussion marked a move away from “easy binary models,” opening the possibility that multimodal analysis not only situated the material and sensory modalities in the same ways they are witnessed, but also allowed admission of those affordances that are contiguous with sociality but may be less easily witnessed: heritage, history, bodily capacities, upbringing, faith and a myriad of other factors which colour not only what one experiences in the communicative moment, but the means by which that experience, message and response is determined in each exchange.

This rhetoric of non-binary approaches mirrors the calls from Deaf anthropologists and Deaf-led sign language scholars to focus more concretely on the differential visual, tactile, aural, temporal shapes of sign languages and sign language using people not in relief to spoken-textual languages but rather going beyond them (Friedner and Kusters, 2020; Robinson, 2022), signalling the unique and important value of DEAF knowledge-making processes. Compounded further by the immediacy of information-availability via smart phones and web applications, leading to dramatic changes in the way that all humans interact with one another, but with deaf sign language users at the vanguard of such changes. That this alteration required a reorientation of communication frameworks, reflecting the fact that “[electronic] forms of communication can now make aspects of any specific ‘where’ into features encountered everywhere, with an unspoken and urgent requirement for it to be made sense of ‘there’ (Kress, 2010: 7), is for me utterly convincing.

Despite this, as I witnessed throughout more than a decade working alongside and for deaf colleagues, friends and with deaf participants: language and listening rigidities are so embedded into the architecture of British social behaviours and public systems (Robinson, 2022), that their knowledge and means of constructing knowledge and listening are consistently undervalued by the non-signing majority in the UK. Over the course of my post-graduate studies, many of my deaf interlocutors discussed with me the ways they could see unacknowledged hierarchies of knowledge operating in their lives, “fundamental power differentials that serve political agendas while also relegating particular epistemologies to a position of being lesser than others, transforming instances of epistemic dissonance into injustice” (sensu Fricker, 2007 in Robinson, 2022). Their lived knowledge was too often undervalued by those in positions of power, whether teachers, medical professionals, government officials or policymakers. In the latter portion of this essay, I mobilise the learnings I outlined in part one of this article to show the ways that thinking using DEAF world values and priorities has transformed my anthropological praxis. The Key outcome is that these approaches are valuable within not just DEAF spaces nor only within anthropology. The bottom line is that using multimodality as a methodological praxis and as part of one’s analytical approach facilitates inclusion of lived-expertise and thus fosters a move toward democratisation of knowledge.

DEAF signatures, values and echoes: Multimodal shifts in listening tp transform knowledge-making

DEAF-anthropologically-grounded, multimodal ethnographic research practices have shown me how to develop a means to address power hierarchies and epistemic value imbalances across my work in varied cultural, social, and modal contexts in the UK. Since 2021, I have investigated UK-based relational ‘faultlines’ and resulting exclusionary knowledge-making by examining the limits of connection between individuals and institutions with particular attention paid to how adults and children experience barriers to inclusion in the UK’s ‘national conversation’. Experiences of second-class economic citizenship, non-status in cases of recent arrival to the UK or other experiences of poverty/precarity can be traced in/on bodies (Lazar, 2013; Wilkinson and Pickett, 2019), both in the flesh they carry but also in terms of the modes on which they relie. As I have moved within this broader context in my work – one shaped by nationhood and exclusion, rather than the socio-cultural DEAF forms which were the focus of my earlier work – I nonetheless continue to reflect on the practical, methodological and analytical value that has come out of Deaf Anthropology, Disability Anthropology, Deaf/Disability Studies in combination with Multimodal discourses over the recent years in terms of addressing epistemic hierarchies and systems of value by changing the shape of the research conversations via the shape of our outputs.

A clear example of the epistemic seam that runs between my previous work with DEAF social forms and my current emphasis on non-resolution of listening limitations have led me to think about enduring inequalities amongst deaf people as compared to hearing people in the national space. For instance, Morisod and colleagues have examined the ways that non-DEAF-centred healthcare means that “d/Deaf people suffer from inequitable access to healthcare and health information. This results in worse health literacy and poorer mental and physical health compared to hearing populations” (2022: 548). Their systemic review of 1507 papers on deaf health inequalities found that use of different modalities in listening and communicating with deaf people improved their health and wellbeing; “study highlights reduced health in the Deaf signing population, compared to the general population […] health initiatives focused on BSL users, aiming to increase physical and mental health, are needed to address this gap” (Shields et al., 2020: 547). These included using sign language during care, healthcare provider Deaf Awareness training programmes, development of adapted healthcare facilities to meet the needs of deaf patients (e.g., digital captioned signage rather than reception announcements, video sign language interpretation in clinical spaces), the offer of online interventions with captioning or interpreting, signed filmic material for public health campaigns) were all means to support parity in information availability for deaf patients (2022: 549). What we see here is that multimodality is not simply a lens onto fieldsites and particular life-ways, but rather it facilitates the ability better to understand the differentials in listening and knowledge production with deaf people to reconfigure these in a DEAF-way. This necessitates greater representation of DEAF-led research and DEAF-centric outputs, leading to solutions which themselves are DEAF, with leading to innovations in how one might reshape space, place, communication, and the values cultivated by each.

This innovation seems especially germane to the context of the UK in the present moment; following more than a decade of UK austerity measures, Covid, and the cost-of-living-crisis health inequalities – particularly mental health – have significantly worsened for young people with disabilities and other young people in marginal positions, whether because they are people with disabilities or are of unsettled or refugee status, because of trauma, or other reasons why they may not be legible or listened to by those who make decisions on their behalf without their input. Inequalities impinge on future potential because these create what anthropologist Stefan Wellgraf refers to as “incarnated hierarchies of worth” (2022. See also Perera, 2020), a blame game in which one views one’s own body through a lens of systemic, socially-instantiated value judgements.

The UN Convention on the Rights of the Child (1989) asserts that all students should have an active role in making known their needs, taking part in decisions about them and directing changes where needed (De Leeuw et al., 2023; Lundy, 2007). ‘Pupil Voice’, the agenda which aims to represent young voices in decision-making about them, has been advanced to make this viable. Though Ofsted recently reconfirmed the importance of Pupil Voice representation, evidence shows that children from minority groups are under/un-represented. For children with special educational needs, those who have been excluded from mainstream education, or are of unsettled/refugee status, and as the ghost children crisis highlights, enduring educational inequalities mean England is developing different kinds of young people with different levels of related voice concerning their health and educational needs. Providing young people with surveys or student councils in order to ‘speak up’ without embedding the skills/disposition required to do so, can result in children falling through the cracks. Special focus needs to be placed on under-represented young people, supporting Pupil Voice as a built disposition. It is here that multimodality is essential not only to research investigative measures but also to solutions to the challenges these young people confront.

Using tenets of DEAF listening alongside multimodal thinking and methods within my work has helped me to chart these manifestations of bodily-borne emotional impacts of stress, trauma, mental health, and confidence (O’Connell, 2023). The programme I am currently running, called Anthropology By Children/Communities or ‘ABC’, upskills less-listened-to people in multimodal auto ethnographic methodologies (literally meaning ‘self-life-charting’) to generate mostly non-textual auto-ethnographic data of the participants’ values and their priorities for systemic social transformation. This approach necessarily moves away from the primacy of words and thinks instead with tones, visuals, and spatial dynamics (see also Clark, 2011). The ABC participants anonymously document autoethnographic work using creative narrative, self-reflections, social rules mapping, photovoice project outputs on physical/mental impacts and filmic reels featuring audio/textual/graphic/photo/filmic participant-driven data. This work aims to mobilise people’s verbal/nonverbal ‘signatures’ to expose how definitional disparities concerning worth are impacted by societally-driven communicative modes and judgements which affect their ability to make themselves understood, to cultivate social connections, and to be recognised for their contributions to knowledge-making as a relational process. ABC also facilitates a more multisensory listening experience, which in some sense re-dimensionalises the young people in a way that written text typically does not. Through film, drawing, mapping, photographing, sounding/recording etc, the young person is offered a different road into articulation and self-advocacy. In this way, ABC’s participants are not only the researchers, they are a burgeoning cohort of student and teacher citizen social scientists (noncitizen/unsettled status social scientists as well) who foreground the kinds of need they feel most acutely and thus directly address the kinds of support that can produce the greatest outcomes for unheard young people and adults alike, they are the stereotype-busters.

In her approach to ethnographic work, Regina Bendix aspires to pull assumptions about listening and being heard apart, focusing primarily on the value of sensory affect. She offers commentary particularly on the tremendous value of the limited aural sensory descriptions found in ethnographic writing, attempting “’ethnography of listening’ […] the process of turning the body into a site of study, grasping it as a site of cultural agency” (Bendix, 2000: 34). The textures and emotion of the varied multimodal signatures conveyed via citizen social science are critical to facilitating the listening to and learning from the value of what less-listened-to stories convey about Britain today through the telling of their unique experiences. The work I do now would not be possible were it not for my colleagues, friends and research participants who taught me, despite my embodied limitations, how to listen and learn DEAF-ly. Following the argument of medical anthropologist Mkhwanazi (2016), by opening up forms that facilitate different means of data capture, shifting the makers of knowledge from elite institutions and fostering instead the legibility of less-listened to people’s stories, data and message-making has the opportunity to break free from a singular, external, majority-driven narrative. Instead, the potential of multimodal citizen (or ideally non-citizen) science work is that it might more systematically admit individual lived expertise via the communicative modalities that work best for research participants but that are also receivable by their audiences, facilitating meaningful shift in the balance of knowledge-making power. If successful, this could lead to the democratisation of data via citizen science of otherwise less-known lives, offering counterpoint “to resist the tendency to use [the single story’s] components as scaffolding” (Mkhwanazi, 2016), opening up the multitude of dimensions rendered apparent via multimodal thinking and documentation, and reflecting DEAF values.