A diffractive approach to multimodal transcription: Materialising entanglements between humans and non-humans

Abstract

Posthumanism is a transdisciplinary paradigm which challenges our anthropocentric gaze when carrying out research. In this article, I argue that multimodality aligns with the posthuman paradigm, and that multimodal analysis can be a useful tool when completing empirical posthuman research. I present empirical data from a study on pre-schoolers’ creativity with tablet computers which combined multimodal methods and a posthuman theoretical lens. Data were analysed using multimodal (intra) action analysis. Multimodal (inter) action analysis focuses on social action and interaction between humans, whereas multimodal (intra) action analysis focuses on the entanglement between human and non-human. Drawing upon Haraway and Barad’s “diffraction”, a methodological alternative to reflexivity, multiple different multimodal transcripts of the same audio-visual data were created and then read through one another to see what “differences made a difference”. Repeatedly engaging with the data in different formats allowed me to slow down and pay attention to (intra) activity between humans and non-humans. The unique contribution of this paper is in my transfer of methods and concepts between multimodality and posthumanism, particularly the use of a diffractive approach to reading multimodal transcripts.

Keywords

Multimodality posthuman transcription diffraction new materialism

Introduction

Posthuman and more-than-human ontologies are increasingly being used to re-conceptualise the social sciences, arts and humanities. The ‘post’ within posthumanism is not to signify ‘after the human’, but instead to de-centre the human (Kuby and Rowsell, 2017). It involves challenging traditional humanist research which positions humans as the central point of analysis (Spyrou, 2018). In this conceptualisation, humans are not centred when discussing agency, non-humans and materials are also recognised as having agency. Posthumanism places emphasis on the entangled relationships between concepts, humans, place, and materials (MacRae et al., 2018).

Exploring methods to empirically ground posthumanism is vital, as without empirical grounding it can have no real-world impact and therefore creates a theory/practice divide (Lenz Taguchi, 2010). Nonetheless, researchers have encountered difficulties in the translation of posthuman theories into empirical research methods (Fox and Alldred, 2022). Indeed, some academics argue against the use of methods (St. Pierre, 2024) as conventional humanist methods are incompatible with posthumanism, and it requires “a different empiricism that is not grounded in the human subject” (St. Pierre, 2016: 29). In this article, rather than reject method, I “open up the empirical to reworkings” (Barad, 2011: 148). I re-think the existing use of multimodal transcription in posthuman research (Marsh, 2017), considering how Barad’s (2007) diffraction methodology might make this practice less anthropocentric and more in line with a relational ontology. I explore how diffracting multimodal transcripts could help to empirically ground posthumanism and undo the theory/practice binary. My intent is not to be methodologically prescriptive, but to “place the understandings that are generated from different interdisciplinary practices in conversation with one another” (Barad, 2007: 92-93). It is fundamental that this diffractive approach to transcription is not viewed as an authoritative tool but, as a dynamic thinking pattern (Rautio, 2021), a practice to bring forth the new. Additionally, it should not be applied to a study which began with a traditional humanist research design —posthuman and/or more-than-human theories should shape the entire inquiry.

Posthumanism and multimodality are diverse fields which encompass a wide range of theories and approaches, and it would be impossible to explore them both fully within one article¹. Within posthumanism, I expound on diffraction and within multimodality I focus on multimodal transcription. My rationale to centre on diffraction and multimodal transcripts is because they provide a much-needed tool to carry out empirical posthuman research. Applying diffraction to multimodal transcription is significant as these methodological approaches have not been linked together in this way before. I begin by discussing existing research that combines the broader fields of multimodality and posthumanism. After that, I define the new materialist term diffraction. Next, I outline a small-scale study on pre-schoolers’ creativity with tablet computers which used multimodal methods, illustrating how the diffractive analysis of multimodal transcripts aids in empirically grounding posthuman theories. Finally, I discuss the ways in which diffractive analysis aligns with multimodal research methods and how linking these two methodological approaches contributes to knowledge.

There are multiple ways in which multimodality and the broader field of posthumanism intersect, which enables the transfer of methods and concepts between them. Indeed, several researchers have already begun to make links between multimodality and posthumanism (Hackett and Somerville, 2017; Marsh, 2017; Sakr, 2022; Wohlwend et al., 2017; Wohlwend and Thiel, 2019). For example, in her study on connected play, Marsh (2017) employed a tabular transcription method whereby the visuals, audio, action and proximity of both the human and non-organic matter were transcribed, coining this Multimodal (Intra) action Analysis (re-framing Norris's (2011) Multimodal Interaction Analysis). Intra-action is a new materialist term which contends that separate bodies do not exist before interactions but come into being in a process of intra-action. When bodies intra-act they do so in co-constitutive ways and are entangled in the process of meaning making (Barad, 2007). Marsh (2017) clearly demonstrated that multimodal transcripts are a useful method for analysing the entangled relationships between humans and non-humans.

Nevertheless, while previous links have been made between posthumanism and multimodality, there is yet to be an article which applies a diffractive approach to several multimodal transcripts of the same event. Furthermore, most empirical articles on diffractive analysis read texts or data together, rather than focusing on methods (Fox and Alldred, 2023). To address these gaps, I will concentrate on the potentials of applying a diffractive analysis to multimodal transcripts. Exploring a diffractive approach to multimodal transcripts is important as it can: provide an understanding that the reality enacted through research is partial, be used to de-centre the human and, support in empirically grounding posthumanism. It is important to note that while I have narrowed in on a diffractive approach to multimodal transcription this is not a complete representation of a diffractive analysis which involves reading a range of materials through one another to provide novel philosophical insights (De Freitas, 2017; Fox and Alldred, 2023).

[D]iffraction can be a metaphor for another kind of critical consciousness... one committed to making a difference and not to repeating the Sacred Image of the Same (Haraway, 1997: 16)

The root of diffraction is from dis, “apart,” and frangere, “to break”, it derives from physics, referring to the phenomenon when waves spread outwards as they move through or bend around objects (Barad, 2014). When I use the term diffraction, I am alluding to the new materialist framing of the term (Barad, 2007; Haraway, 1997). When new materialists diffract data, they break, cut or fracture it into multiple parts to show the different components, and re-construct them anew (Hickey-Moody and Willcox, 2019). During this process they explore the messiness and multiplicity of the data, investigating what new patterns unfold, examining how differences are made and considering what is excluded through their chosen research apparatus (Murris and Bozalek, 2019).

Challenging reflexivity: Diffracting the self

“There is no ‘I’ that exists outside of the diffraction pattern, observing it, telling its story… I am neither outside nor inside; ‘I’ am of the diffraction pattern. Or rather, this ‘I’ that is not ‘me’ alone and never was, that is always already multiply dispersed and diffracted throughout spacetime (mattering)” (Barad, 2007: 181).

One of diffractions crucial distinctions from traditional qualitative multimodal research methodology is its departure from reflexivity. The aim of reflexivity is to make qualitative research more scientific and objective through self-reflection on our biases (Chee, 2022). Diffraction challenges this centring of the human researcher, suggesting it is impossible to represent representation, asking the question: how can one objectively represent their own biases (Knafo, 2016)? The risk in reflexivity is creating a mirror of a mirror and that the bias of the researcher influences how they identify their bias. Additionally, if a researcher is in a constant state of becoming, how can they cut up their subject positions and know which aspects are relevant during the dissemination of their research² (Chee, 2022)?

Diffraction does not claim that the researcher’s biases do not have an influence on the research process. Instead, it positions the researcher as one part of the research apparatus, without centring the self (Larsen and Schwennesen, 2024). Diffraction involves “an embodied engagement with the materiality of research data: a becoming-with the data as researcher” (Taguchi, 2012: 265). This “becoming with” approach moves away from representation which places the researcher in a hierarchy with their participants and acknowledges that a researcher’s position in relationship to their research is not static (Vagg, 2022). A diffractive reading is not simply a reading of a transcript as it is, but reading with the transcript, considering your affective and embodied encounter with it. In this process, both the researcher and their thinking are affected and being affected by each other which creates new lines of inquiry (Hultman and Lenz Taguchi, 2010).

Data materialisation

To illustrate the application of diffraction, I will discuss empirical data from a study which explored pre-schoolers’ creativity with tablet computers in the home. My study extended over an 18-month period, involving a sample of eight families with children aged 3-5. The aim was to investigate if a multimodal approach could be effectively used to carry out empirical posthuman research on children’s creativity with tablets in the home. I used a posthuman paradigm and a focused ethnography. I employed diffraction throughout my entire study. This diffractive approach included reading source materials (texts, theory, data, transcripts, research diaries) through each other whilst being open and playful. The methods of data collection included observations and video of 3–5-year-olds intra-actions with tablets and their family members, interviews with parents and informal discussions with children. While these are traditionally humanist methods, my aim in using diffraction was to work towards de-centring the human participants and researcher. Each family was asked to use their mobile phone to record intra-actions between the child, the tablet and family members providing access to interactions that would otherwise be inaccessible to a researcher (Given et al., 2016). Using audio visual technologies slowed down my readings of the data (Cowan, 2014) allowing me to see relational entanglements between humans, materials and space (Taylor, 2020).

Precise recording of audio-visual data was needed to be able to use multimodal analysis and to think-with posthuman theories. To have an equal focus on the tablet and the humans, I needed to not only film the humans but also to do a simultaneous screen recording of the tablet. Including the screen capture allowed me to see the tablet screen clearly no matter the angle chosen in the video. Without the addition of the screen capture, I would have had a limited view of the screen which would have impacted on my ability to reflect on the role of the tablet. The focus would therefore be on the human and not conducive to a posthuman analysis. The screen captures of the tablet provided me with a detailed account of the tablet interface, but with only this data source I would not be able to see the children’s body position or how they are physically interacting with the interface which would prevent me from being able to analyse multimodal communication between the humans. One form of data is human-centred, and the other is machine centred, neither presenting the full picture. The two images put together give us insight into the entanglement between human and machine and the communication modes of both the tablet and humans are visible (gaze, gesture, touch, proximity, graphics). I acknowledge separating data into human-centred and machine-centred performs an agential cut and materialises a fixed boundary between human/non-human. To address this, I remained accountable and attentive to this divide while creating and diffracting multimodal transcripts.

Ethical approval was gained from the university ethics board. Parents filled in consent forms for themselves and their children, and children’s assent was gained verbally using props, simple language and a picture book. Pseudonyms were used and images of participants were transformed into comic strip representations, enabling the reader to see multimodal interaction modes while maintaining participants anonymity.

Diffraction in practice: Reading multimodal transcripts through one another

In this section, I explore the potential of diffractively analysing multimodal transcripts. I outline three different multimodal transcription methods used on the same video and screen recorded data, including exemplars of each transcription approach (Figures 1 –3). The transcripts were read through one another (diffractively) to see what differences made a difference. Reading-through involves understanding that boundary production is a material-discursive practice which configures and performs reality. Rather than reflecting on individual transcripts, comparing them against one another and creating a hierarchy, reading-through accounts for how the differences between specific practices matter (Barad, 2007).

Figure 1.

Tabular transcript separated by entities.

Figure 2.

Tabular transcript separated by mode.

Figure 3.

Comic strip transcript.

Whereas traditional transcription involves turning speech into writing (Bezemer and Mavers, 2011), multimodal transcription explores methods to transcribe multiple modes of communication. In my study, multimodal transcription was initially completed using ELAN software, creating annotation tiers for both the humans and the tablet. Annotating gaze, gesture, speech, and action allowed me to analyse embodiment (Price and Jewitt, 2013). Embodiment helps to empirically ground posthumanism (Braidotti, 2019). Creating tiers for the tablet (vocalization, colour, movement) provided an opportunity to focus on the non-human and to think-with posthuman theories. A limitation of the ELAN transcript was that it did not allow for the simultaneous transcription of the screen recording alongside the parent recorded video. This was fine for this video in which the parent has filmed the screen throughout but, in some of the recordings, children moved around which made the screen not fully visible in parts.

During the process of annotation, I came to the realisation that this separation of the two data sets was not appropriate to explore the entangled nature of the intra-actions between child, family member, and tablet. When considering other software to complete the analysis, I was wary of choosing another that might restrict the way I wanted to transcribe and analyse my data through a posthuman lens. So, while there is software available which allows two images to be annotated simultaneously, I chose to complete this process manually using a word document. This method was time consuming but allowed for more flexibility to explore different ways to annotate without relying on the software’s restrictions and to ensure a focus on the entangled elements of play with the tablet. This meant that I did not encounter another issue such as that experienced while using my initial choice: ELAN. Creating my own transcripts manually, compelled me to think carefully about what I was choosing to include and exclude.

Applying a diffractive analysis involved creating a range of different transcripts and then reading these through one another to see what they include and exclude. While previous papers have compared a range of multimodal transcription methods (Bezemer and Mavers, 2011; Cowan, 2014) or read multimodal transcripts through theory (De Freitas and Sinclair, 2014), no researcher has read multiple transcripts of the same event through one another. In my analysis, I did not pit one transcript against another, instead diffraction provided detailed understandings of a range of viewpoints and how they built upon each other to create new understandings. Indeed, the very process of compiling transcripts of the same data in a range of formats is posthuman, as it forces me to diffract my lens and see how difference can make a difference.

Using the same data presented in different ways provided me with an opportunity to read each of the methods through one another. The process of diffraction and repeated engagement with the same data in different formats clearly revealed what different transcription methods include and exclude and which ones are suitable to think-with posthuman theories. Having different methods of data transcription “can be understood as an obstacle or an interference that overlaps with my embodied affective and theoretical thinking as researcher, causing me to read diffractively one through the other” (Hultman and Lenz Taguchi, 2010: 536). In the act of writing and viewing things in multiple formats, my relationship with the data changed both in affective and embodied ways.

Exploring how differences are made differently

Below, I outline three multimodal transcription methods, whereas in my wider study the analytic process was ongoing and dynamic, and I explored a much wider range of transcripts. I have chosen to focus on these three transcripts as they are distinct enough that when read together, they reveal differences. When transcribing this short interaction in different ways, my relationship with the data changed. I realised that one method alone cannot accurately represent reality and it is in this exploration of multiplicities that we gain insight into the gains and losses that come from our methodological choices. Diffracting multiple transcripts of the same event through one another reveals that data is not pre-determined prior to analysis, that it emerges and is materialised through the analytic process.

The tabular method of transcription in Figure 1 builds upon a design by Cowan (2018) originally used to transcribe pretend play, whereby columns were included for each human participant alongside two visual representations of the interaction. My transcript includes four columns: the first column depicts still images from the video and screen-recording, the second a written description of the audio and action from the tablet, the third and fourth a written description of the speech and action of the child and their family member. In my transcript, there is an individual column which describes in text both the actions and audio of each entity/agent (the child, their family member and the tablet). The columns which record the human participants feature written descriptions of their speech, gaze, gesture and action which allowed me to analyse embodiment, helping to ground posthumanism empirically. Including transcription of the audio and the screen-recording of the tablet alongside the child and their family member allowed focus to be placed on the tablet as much as it does on the humans. In this way, the tablet is positioned equally as a participant in the intra-action, and not in a hierarchal position below the humans. This enabled me to focus on the role of the tablet in the intra-action and to consider its agency. It allowed me to see the complex ways in which human and non-human entities intra-act, co-construct reality and make meaning together.

Figure 1 is multimodal in two ways: it shows multiple modes of interaction (gaze, gesture, touch, sound) and it uses multiple modes in the presentation of the transcript (image, writing, layout). The multimodal process of both describing and including the images may appear redundant but, the process of taking time to look at and transduct (remake meaning across modes) (Kress, 1997) the image into words diffracted my lens, changed my relationship with the data and the effort put into this furthered my posthuman focus. Including the still image of the screen-recording and video, forced me to slow down and notice how the visuals on the tablet screen are communicating with the humans and how they intra-act frame by frame, enabling me to de-centre the human. Methods enact realities (Law, 2004) and during the process of re-configuring and transducing modes, new meaning and understanding was produced (Mavers, 2012).

Figure 1 does not have time stamps, but follows a temporal sequence, recording both the simultaneous and sequential action and audio between tablet, child and family member. Separating and including a column for each entity was a clear way to re-present the role of the different agents, but its limitation is that it presents them as independent entities, rather than entangled. Barad (2007: ix) states that:

“To be entangled is not simply to be intertwined with another, as in the joining of separate entities, but to lack an independent, self-contained existence. Existence is not an individual affair. Individuals do not pre-exist their interactions; rather, individuals emerge through and as part of their entangled intra-relating.”

So, while it was helpful to break this down into components, this presentation of my data is not truly an entangled representation of the event. Barad (2007: 389) suggests presentation of data are not “(more or less faithful) pictures of what is, but productive evocations, provocations, and generative material articulations or re-configurings of what is and what is not possible.” Multiple transcripts would therefore allow for multiple provocations to think differently with the data. I decided, therefore, to further diffract my lens and design a multimodal transcript which included the human and the non-human but rather than separate agents to separate out modes.

Figure 2 develops upon a transcription design by Marsh (2017) which she used to analyse intra-actions. Marsh’s (2017) transcript included columns for vocalisation/speech, gaze, gesture, facial expression, action and posture/proximity and in-organic matter. In my transcript, I have removed the columns for gaze and in-organic matter to create an entangled transcript which does not separate out individual entities. Instead, it only has columns separting out multiple modes (audio, action, posture/proximity) and humans and non-humans are written about together.

This transcript uses one mode of representation, written language. In the place of an image or a description of the image on the screen, I described the movement of the image on the screen. This involved transducing (Kress, 1997) embodied and visual modes into a linguistic format. As a detailed description of the screen recording or an image was not provided, there were analytic losses. During the process of diffraction, I experimented with including a description of the image in this table, however, I removed it as my priority was to make entanglement visible and to analyse how the human and non-human entities emerged through their intra-action. Overall, this method of transcription was time-consuming, and it was impossible to present all the modes of intra-action on a table given the restrictions of page size, and thus agential cuts needed to be made. An agential cut is a new materialist term which refers to the enactment of boundaries within an entangled reality (Taylor, 2019). When referring to a research apparatus, this means that only part of a phenomenon can be made visible at a time and that the methodological choices or ‘cuts’ we make as researchers construct partial realities (Arlander, 2017). An example of an agential cut is when researchers chose to transcribe some communication modes and exclude others. Each transcript in this article performs a cutting-together apart (Barad, 2014), a temporary stabilisation in an open and unfolding world. A diffractive reading involves performing multiple agential cuts which enables us to: see more than one enactment of reality, understand the differences our agential cuts make and reflect upon the power-laden hierarchies that these choices produce.

Figure 2 supported thinking-with posthuman theories such as intra-action, as the human and non human are written about together, which presents the entanglement between them. While it could be argued that this is a more “entangled” representation of the intra-action than Figure 1, it involves a separation of modes which is an agential cut. Further exploration of how to transcribe entanglement is something which could be improved in future research, perhaps by including diagrammatic representations, line drawings or writing about modes as entangled (De Freitas and Sinclair, 2014). A key issue I experienced with Figure 2 was its layout, which is inaccessible to my participants, particularly my child participants. Therefore, in my third transcript I explored more accessible ways of presenting the data as in a diffractive methodology, the researcher should make efforts to not position themselves in a hierarchy to their participants (Vagg, 2022).

Figure 3 is a comic strip transcript presenting still images from the screen capture and the video side by side. Speech bubbles are included to show the children talking and there are annotations on the screen capture to denote both the action of the child towards the tablet and the movement on the screen. Including stills from the screen capture side by side with the images from the video puts emphasis on the non-human and places it in a non-hierarchical position with the humans. As the images are not frame-by-frame and there is no additional written description it does not count the embodied modes of communication and interaction that occurred between the stills. Choosing stills therefore becomes an agential act and pushes the researcher to slow down and identify key areas of concern (Plowman and Stephen, 2008). This slowing down reminded me to focus on what is important in a posthuman reading and pay attention to entanglement. Nonetheless, this analytic act creates a partial representation of the event, which reduces reality. To address this lack, I annotated on the image of the screen capture. Annotations recorded the interaction of the child with the screen (swipes, presses, scrolls) and the movement on the screen. Overall, this is the easiest transcript to read, but it is not as detailed and precise and there are many analytic losses and exclusions in attempting to make it more readable. Within Figure 3, for example, the gesture, gaze, posture or proximity of the humans are excluded in between the selected stills. Paying attention to minute gestures is an important part of an embodied reading. This is acknowledged by Braidotti (2019) as one of the primary methods in researching posthuman intra-actions empirically. Another loss is that I did not transduct the image into text, which, when completed in Figures 1 and 2, enabled me to “become with” the data and provided me with deeper insights into the intra-actions. These exclusions mean that alone, this transcription method is not suitable to make entanglements visible. However, if included as part of a diffractive reading, reading this transcript through multiple other methods and reflecting on differences can provide a much more nuanced and in-depth analysis.

Multimodal transcription is a semiotic act, and researchers should choose different transcription formats to make it readable to their specific audience (Kress, 2010). I acknowledge the different audiences that my research can impact (parents, practitioners and children), and so, my diffractive readings do not immediately prioritise academics. This performs the act of decentring myself, as an adult and an academic and I gain insights from diverse viewpoints on the same event. Making this transcript accessible allowed for my participants to “become with” and be transformed by the data that they helped to collect. I used the comic strip transcripts to elicit responses during interviews and their additional insights following this enhanced my diffractive reading.

The differences between transcripts and the difference this makes

“Envisioning difference differently – i.e. theorizing a different difference – leads to a thought-practice in which concepts are not abstraction from the world, but an active force of this world – and thus always/ already implicated in and concerned with world(ing): practicing and envisioning specific practices for this world” (Thiele, 2018: 203).

Reading theories/methods/data through one another and accounting for the relations of difference is a key part of a diffractive reading. In Figure 1, the differences include: placing image and text description together; and separating out the agents and positioning the non-human as an equal participant. The difference this makes is that it gets us to reflect on the agential cuts made during transduction and clearly positions the non-human on an equal playing field as the humans. Additionally, the process of scrutinising the image, thinking through it and describing it in words changed my relationship with the data as a researcher and helped me to de-centre my anthropocentric gaze. In Figure 2, the difference is that it separates out communication and interactional modes but positions all participants together. This transcript presents a more entangled view of the event; entities are not presented as being self-contained. In Figure 3, the difference is that it is a more accessible presentation of the data, meaning that the participants can experience and become with the data and diffract their analysis. This transcript does not position the researcher’s reading of the transcript above the participants and is non-hierarchical. In short, Figures 1 –3 are different, and those differences provide an insight into the multiple realities present within the same data set. In a diffractive reading, we are looking at differences not to separate and compare opposing states, but instead to understand more about the connectivity, multiplicity, relationality and continuity of events. A diffractive analytic process is both creative and transformational (Vagg, 2022) disclosing a hidden reality that exists amongst the multiple realities being enacted (Lenz Taguchi and Palmer, 2013).

Reading transcripts through one another enabled me to read with methods, understanding that they were acting upon me as I read and created them (Lenz Taguchi and Palmer, 2013). Diffracting my lens involved recognising how, as a researcher, I am entangled with the data and that as I act upon the data the data acts upon me. Viewing the data in different formats allowed me to think differently. As I “experienced with” the data, I “became with” it — this generated new understanding. This was most evident when I was separating the data first by entity and then by mode and transducing images into words. In both cases, the process of re-making something anew, of transformation, forced me to think in a new way. Pushing myself to think in new ways and create multiple alternative methods to present the data made me attend to the numerous ways in which an event can be observed and illustrated. It compelled me to reflect on what I privilege in an observation, why this might be, and what this agential act reproduces and makes visible/invisible.

For instance, initially when watching my video data, I privileged social interaction between humans, and speech. If I consider why this might be, as an early childhood educator, I am trained in completing narrative observations of children which concentrate on the development and learning outcomes of human children. Narrative observations centre on speech and do not record multimodal interaction. The agential act of using narrative observation reproduces developmental perspectives on childhood. In focusing on humans, these observations re-establish the binary divide between the human subject and the non-human object. In a narrative observation, children are the subjects of observation, and the tablet is being acted upon, causing entanglements with non-humans to become invisible. Furthermore, the focus on speech makes multimodal interaction and embodied cognition invisible. In this article, I argue for the use of diffractive analysis on multimodal transcripts. This process is valuable as multimodal transcription makes multiple modes of interaction observable, and a diffractive analysis enabled me to give form to entanglements with non-humans, making visible things which are typically ignored in Early Childhood Studies. Therefore, applying a diffractive analysis to multimodal transcripts allowed me to think differently with the data, rather than duplicating accepted arguments within my field.

Applying a diffractive analysis to multimodal transcripts provided me with tools to work across (and re-conceptualise) the disciplines of multimodality and posthumanism. It enabled me to think-with a range of theoretical frames, cross disciplinary boundaries and to think-through disciplines. Diffraction does not place one discipline or theory in a hierarchy with another. Instead, it provokes new thinking through blurring disciplinary boundaries and reading theories against one other. Diffraction remains “attentive to important details of specialized arguments within a given field” (Barad, 2007: 25) without erasing them, comparing them or creating dualisms. This is valuable as specialised arguments and terms are frequently diluted down and weakened when they cross disciplinary boundaries.

Given the gap identified in the literature, I have focused here on applying diffraction to methods. However, diffractive analysis should also be used to read theory, data, texts and the researcher’s affective responses through one other. For instance, Hultman and Lenz Taguchi (2010) carried out a diffractive analysis of a photograph of a child in a sandbox. They considered the same photograph through a developmental lens, the sociology of childhood and finally through the posthuman theory of intra-action. They read these theories through one another coming to a more nuanced understanding of the agency of non-humans and how the sand acts upon the child as much as the child acts on the sand. In my study, I performed a similar diffractive analysis reading both socio-cultural and posthuman theories through one another, but rather than analysing a singular photograph, I also diffracted my methods of transcription. Creating a range of multimodal transcripts from the same data further diffracted my lens.

When reviewing different types of transcription methods to use in my study, I noticed that multimodal methods cannot, in their current form, materialise entanglement between non-humans, as they either centre human or non-human or place them in a hierarchy. Nevertheless, re-conceptualising these approaches offers exciting possibilities for capturing and transcribing intra-actions between humans and non-humans. Multimodal Inter-action Analysis (MIA) focuses on how space and artefacts mediate interaction between social actors (Jewitt, 2011). MIA places social actors (humans) in a hierarchal position to mediated tools (non-humans). Adopting key concepts and methods from MIA and altering it, placing social actors and mediated tools on a flat plane, provides a methodological approach to study intra-actions between humans and non-humans (Marsh, 2017). Being playful with form is an important part of re-working transcription so that it can materialise entanglements. For instance, De Freitas and Sinclair (2014) ‘playfully overturn’ written transcription by bracketing speech (rather than actions) and presenting these in sentences entangled with other modes.

How diffraction and multimodality align conceptually

I contend that multimodality and diffraction align conceptually in three significant ways. The first way which I argue that multimodal research and diffraction converge is in their claim that language has been placed in a hierarchy above other matter and/or communication modes. Multimodality challenges the idea that language (speech and writing) is central to interaction, asserting that both communication and representation encompass multiple modes (gaze, gesture, touch, image, proximity) (Dicks, 2019). Correspondingly, Barad (2003: 801), one of the key theorists behind diffraction, suggests that “language has been given too much power” and is positioned as more trustworthy than matter (Barla, 2023). This conceptual alignment was clearly illustrated in my study. Multimodal transcripts enabled me to de-centre human language and focus on the entanglements between humans and non-humans, empirically grounding posthuman research.

The second way in which multimodal research and diffraction overlap is through their recognition that the methodological choices a researcher makes enacts a partial representation of reality. Diffraction posits that we cannot discover a pre-defined truth about the world and that knowledge production is an agential act (Jenkins et al., 2021). Research is a product of the researchers chosen apparatus and, even studying the same phenomenon, results would be different if they used an alternative method (Uprichard and Dawney, 2019). In a similar vein, multimodal research recognises that multimodal transcripts are partial representations of an event and that they foreground specific elements from the data (Mavers, 2012). Any transcript is limited temporally, spatially and by the modes in which it is presented (Bezemer and Mavers, 2011) and different modes perform different communicative and representational functions (Jewitt, 2011). Through the acknowledgement of partial representation and agential cuts, multimodal and diffractive researchers both acknowledge that their methodological choices reduce accounts of reality (Flewitt et al., 2009; Haraway, 2004) and they take accountability for what is included and excluded in their research apparatuses (Bezemer and Jewitt, 2010; Bezemer and Mavers, 2011; Chee, 2022). In my research, I acknowledged that each multimodal transcript was only a partial representation of reality. Creating multiple multimodal transcripts and reading them diffractively through one another allowed me to see the multiplicities and hidden realities in my data that would have been lost from using just one method alone. Each transcript offered different insights into the event, making my overall analysis more nuanced. For instance, Figure 1 flattened out the hierarchy between human and non-human; Figure 2 revealed how all entities are entangled and act upon one another; and Figure 3 facilitated my participants to “become with” the data. This acknowledgement of the multiplicity of realities, challenges assumptions that human researchers can step back, separate themselves and describe a singular autonomous reality (Spyrou et al., 2018).

The third way in which diffractive analysis aligns with multimodal research is that they both involve repeated engagement with the same data during analysis. Multimodal analysis involves exploring a range of transcription methods (Cowan, 2014) and “different stages of analysis and presentation will require multiple transcriptions” (Goodwin, 2000: 161). Multimodal researchers engage in this repetitive process to analyse the data and to delineate the most appropriate method (and mode) of transcription. My process of diffracting multimodal transcripts expands upon this. A diffractive researcher reads disciplines through one another as a means of “luxuriating in the data, creating a space for analytical thinking and a launch pad for further development of ideas” (Mac Lure, 2013 in Sakr, 2022: 441). With diffraction, repeated engagement with theory, data and methods is not about comparing disciplines or getting to a “truth” but instead to explore differences, consider what is excluded and the significance of those exclusions (Larsen and Schwennesen, 2024). In particular, it considers the exclusion of non-humans, more-than-humans, and those that do not fit into the dominant political category of human (those who are adult, white, middle class, male, heterosexual and able-bodied) (Murris, 2016). This repeated engagement with my data during analysis helped to avoid simply reproducing dominant discourse in my field. It forced me to slow down and re-think the agential hierarches that exist between adult, child and non-human in contemporary Early Childhood discourse.

While there are conceptual alignments between multimodality and posthumanism they come from different ontological positions. In its current conceptualisation, multimodality studies pre-existing fixed entities that interact with one another. Within multimodality, the human researcher is positioned as distinct from the data which they collect and then analyse. In contrast with this, posthumanism derives from a relational ontology. Relational ontologies emphasise interdependence, claiming that individual entities do not pre-exist and that they materialise through intra-actions (Spyrou, 2018). Within posthumanism, the researcher is not seen as separate from the data, they are enmeshed, a part of the research assemblage (Mazzei, 2014). Thinking-with diffraction shifts the creation and reading of multimodal transcripts towards a relational ontology. My aim in diffracting multimodal transcripts was not to capture and represent pre-existing fixed entities interacting but to perform what is and what is not possible. I used diffraction to (re)consider the ways reality is materialised, and how boundaries are fixed through the process of transcription. Being responsive to how research gives the world material form is aligned with a relational ontology. The details that researchers choose to include and exclude from transcripts matter significantly for research participants and for how the world is (re)configured. This diffractive approach enables a researcher that uses multimodal transcription to acknowledge the interdependency of entities, take responsibility for their agential cuts and recognise they are part of the research assemblage. “That is, the diffractive methodology that I use in thinking insights from different disciplines (and interdisciplinary approaches) through one another is attentive to the relational ontology that is at the core of agential realism” (Barad, 2007: 93).

Conclusion

There are four key findings from this paper. Firstly, a multimodal approach can support the transcription and analysis of entanglements between humans and non-humans. Multimodal transcription provides researchers with a method to slow down their focus and transcribe the actions of non-humans. Secondly, diffraction and multimodality converge through (1) de-centring written and spoken language (2) recognising that the methodological choices a researcher makes enacts a partial representation of reality (3) repeated engagement with data. Thirdly, diffracting multimodal transcripts discloses hidden realities in the data. Through re-making transcripts and transducing modes, new meaning and understanding is produced. One transcript reduces reality, whereas reading multiple transcripts through one another and reflecting on differences can provide a much more nuanced and in-depth analysis. Finally, applying diffractive analysis to multimodal transcripts provides tools to work across (and re-conceptualise) the disciplines of multimodality and posthumanism, without watering down the specialised language or arguments from either field. Together, these findings demonstrate that a diffractive analysis of multimodal transcripts can provide researchers will a tool to de-centre humans and empirically ground posthuman research, which is this papers main contribution to knowledge.

A limitation of this article is that it presents only one case study out of eight. In my wider study, I noticed that alternative data sets produced different differences and thus made me reflect on different exclusions. For example, in the transcripts presented in this article there are very few sounds coming from the tablet application. This means that the visual had priority in the enactment of non-human participants, creating a potential imbalance with the modes recorded between human and non-human. Therefore, while these multimodal transcripts were suitable for presenting intra-actions in my specific data, I am in no means suggesting that these transcripts are immediately transferable, and it is important for researchers to “experience with” their own data and create their own transcripts. Indeed, it is through this process of creation that they can be affected by and “become with” their data.

As this was an exploration of what is possible, I did encounter a few challenges. Whilst completing a diffractive analysis. I found “reading through” transcription methods difficult. It was extremely challenging to not revert to “reading against”, critiquing and making comparisons. Certainly, there are gains and losses of each transcription method – some are more suited to re-configuring entanglements and thinking-through posthuman theories. However, my aim in applying a diffractive analysis to multimodal transcripts was not to find the most appropriate method but to see what difference each transcription method provides, what reading them through each other affords and how this process supports the empirical grounding of posthuman theories.

Another obstacle which I encountered when completing this process was in de-centring humans. I was cognisant of focusing my transcript on the humans and made an explicit effort not to do this. Nevertheless, my unconscious and automatic anthropocentric focus was a constant concern. This is an issue which other posthuman researchers have encountered when completing empirical posthuman research (Hackett and Somerville, 2017; Wargo, 2018). To address this, after completing each multimodal transcript, I reflected on the agential cuts I was making and how my choices were creating a partial, humanistic and linguistically centred representation of the data. I carefully questioned the variables of what each type of transcript might include or exclude and how this might support (or hinder) thinking-through a posthuman theoretical frame. This involved zooming in on the different modes of communication of both human and non-human.

Going forward, my recommendation is that researchers explore this diffractive process of analysis themselves, which can only be achieved by engaging with their own data set. Initially, video data collection needs to be precise in how it records all modes of intra-action between humans and non-humans, and a small excerpt should be chosen. Following this, a range of transcription methods can be explored. When exploring transcription methods, I recommend replicating Figure 1 as this will provide a starting point and de-centre an anthropocentric gaze. After completing this, I suggest reflecting on the gains and losses from this transcript. Then, a range of transcription methods can be created, using a combination of writing, layout, typography and image. As each new transcript is created, I advise reading these through each other, considering the differences they make, whether they materialise intra-actions and what they make invisible/visible. These transcripts can then be used to think-with posthuman theories and empirically ground posthumanism.

Footnotes

Acknowledgements

Thank you to the ESRC for the funding of this work. I would like to thank my supervisors Sara Price and Natalia Kucirkova for their continued guidance and support. I am extremely grateful to the families that participated in this project.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was completed at UCL supported by the Economic Social Research Council; grant number 2229350.

Ethical statement

ORCID iD

Fiona Byrne

Data availability statement

Participants have not consented for the video data to be made available.

Notes

Author biography

Fiona Byrne is a senior lecturer in Early Childhood Studies at the University of East London. She previously worked as a nursery teacher. This article derives from her ESRC-funded PhD at University College London which focuses on pre-schoolers' creativity with tablet computers. Her research interests are posthumanism, multimodality, digital technologies and children's rights.

References

Arlander

(2017) Agential cuts and performance as research. In: Performance as Research. UK: Routledge, 133–151.

Barad

(2003) Posthumanist performativity: toward an understanding of how matter comes to matter. Signs: Journal of Women in Culture and Society 28(3): 801–831.

Barad

(2007) Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Duke University Press. Available at: https://www.jstor.org/stable/j.ctv12101zq (accessed 19 July 2021).

Barad

(2011) Nature’s Queer Performativity. Qui Parle 19(2): 121–158.

Barad

(2014) Diffracting diffraction: cutting together-apart. Parallax 20(3): 168–187.

Barla

(2023) Beyond reflexivity and representation: diffraction as a methodological sensitivity in science studies. Distinktion: Journal of Social Theory 24(3): 444–466.

Bezemer

Jewitt

(2010) Multimodal analysis: key issues. In: Research Methods in Linguistics. London: Continuum, 180–197.

Bezemer

Mavers

(2011) Multimodal transcription as academic practice: a social semiotic perspective. International Journal of Social Research Methodology 14(3): 191–206.

Braidotti

(2019) A theoretical framework for the critical posthumanities. Theory, Culture & Society 36(6): 31–61.

10.

Chee

(2022) Being of use: diffraction and an ethics of truth-telling in post-cartesian IR. Global Studies Quarterly 2(3): ksac049.

11.

Cowan

(2014) Multimodal transcription of video: examining interaction in Early Years classrooms. Classroom Discourse 5(1): 6–21.

12.

Cowan

(2018) Visualising young children’s play: exploring multimodal transcription of video-recorded interaction. UCL (University College London). UCL Discovery Doctoral. Available at: https://discovery.ucl.ac.uk/id/eprint/10048404/ (accessed 30 November 2023).

13.

De Freitas

(2017) Karen Barad’s Quantum Ontology and Posthuman Ethics: Rethinking the Concept of Relationality. Qualitative Inquiry 23(9): 741–748.

14.

De Freitas

Sinclair

(2014) Mathematics and the body: material entanglements in the classroom. Mathematics and the Body: Material Entanglements in the Classroom. New York, NY, US: Cambridge University Press.

15.

Dicks

(2019) Multimodal Analysis In: Atkinson

Delamont

Cernat

, (eds). SAGE Research Methods Foundations. London: Sage (accessed 13 May 2024).

16.

Flewitt

Hampel

Hauck

, et al. (2009) What are multimodal data and transcription? In: Jewitt

(ed) The Routledge Handbook of Multimodal Analysis. 1st edition. London: Routledge, 40–53.

17.

Fox

Alldred

(2022) Doing new materialist data analysis: a Spinozo-Deleuzian ethological toolkit. International Journal of Social Research Methodology 25(5): 625–638.

18.

Fox

Alldred

(2023) Applied research, diffractive methodology, and the research-assemblage: challenges and opportunities. Sociological Research Online 28(1): 93–109.

19.

Given

Cantrell Winkler

Willson

, et al. (2016) Parents as coresearchers at home: using an observational method to document young children’s use of technology. International Journal of Qualitative Methods 15(1).

20.

Goodwin

(2000) Practices of seeing: visual analysis: an ethnomethodological approach. In: Handbook of Visual Analysis. London: Sage Publications, 157–182.

21.

Hackett

Somerville

(2017) Posthuman literacies: young children moving in time, place and more-than-human worlds. Journal of Early Childhood Literacy 17(3): 374–391.

22.

Haraway

(1997) Modest_Witness@Second_Millennium. FemaleMan_Meets_OncoMouse: Feminism. New York: Routledge.

23.

Haraway

(2004) The Haraway Reader. United Kingdom: Psychology Press.

24.

Hickey-Moody

Willcox

(2019) Entanglements of difference as community togetherness: faith, art and feminism. Social Sciences 8(9): 264.

25.

Hultman

Lenz Taguchi

(2010) Challenging anthropocentric analysis of visual data: a relational materialist methodological approach to educational research. International Journal of Qualitative Studies in Education 23(5): 525–542.

26.

Jenkins

Ritchie

Quinn

(2021) From reflection to diffraction: exploring the use of vignettes within post-humanist and multi-species research. Qualitative Research 21(6): 975–989.

27.

Jewitt

(2011) The Routledge Handbook of Multimodal Analysis. 1st edition. Oxon: Routledge. Available at: https://www.routledge.com/The-Routledge-Handbook-of-Multimodal-Analysis/Jewitt/p/book/9780415667777 (accessed 13 May 2024).

28.

Knafo

(2016) Bourdieu and the dead end of reflexivity: on the impossible task of locating the subject. Review of International Studies 42(1): 25–47.

29.

Kress

(1997) Before Writing: Rethinking the Paths to Literacy. London, New York: Routledge. Available at: https://library.bathspa.ac.uk/items/75757, https://www.vlebooks.com/vleweb/product/openreader?id=BathSpa&isbn=9780203992692 (accessed 15 May 2024).

30.

Kress

(2010) Multimodality: A Social Semiotic Approach to Contemporary Communication Routledge. London: Routledge.

31.

Kuby

Rowsell

(2017) Early literacy and the posthuman: pedagogies and methodologies. Journal of Early Childhood Literacy 17(3): 285–296.

32.

Larsen

Schwennesen

(2024) Beyond the insider/outsider debate in “at-home” ethnographies: diffractive methodology and the onto-epistemic entanglement of knowledge production. Nursing Inquiry 31(1): e12611.

33.

Law

(2004) After Method: Mess in Social Science Research. London: Routledge. Available at: https://www.routledge.com/books/details/9780415341752/ (accessed 15 May 2024).

34.

Lenz Taguchi

(2010) Going beyond the Theory/Practice Divide in Early Childhood Education: Introducing an Intra-active Pedagogy. New York: Routledge.

35.

Lenz Taguchi

Palmer

(2013) A more ‘livable’ school? A diffractive analysis of the performative enactments of girls’ ill-/well-being with(in) school environments. Gender and Education 25(6): 671–687.

36.

Mac Lure

(2013) Classification or Wonder? Coding as an Analytic Practice in Qualitative Research. In: Coleman

Ringrose

(eds). Deleuze and Research Methodologies. Edinburgh: Edinburgh University Press, 164–183.

37.

MacRae

Hackett

Holmes

, et al. (2018) Vibrancy, repetition and movement: posthuman theories for reconceptualising young children in museums. Children’s Geographies 16(5): 503–515.

38.

Marsh

(2017) The internet of toys: a posthuman and multimodal analysis of connected play. Teachers College Record 119(12): 1–30.

39.

Mavers

(2012) Multimodal transcription. In: using digital data for multimodal research. In: Introductory Training Day. London: UCL Knowledge Lab.

40.

Mazzei

(2014) Beyond an easy sense: a diffractive analysis. Qualitative Inquiry 20(6): 742–746.

41.

Murris

(2016) The posthuman child: educational transformation through philosophy with picturebooks. In: Contesting Early Childhood. London: Routledge.

42.

Murris

Bozalek

(2019) Diffracting diffractive readings of texts as methodology: some propositions. Educational Philosophy and Theory 51(14): 1504–1517.

43.

Norris

(2011) Identity in (Inter)action: Introducing Multimodal (Inter)action Analysis. Germany: De Gruyter Mouton.

44.

Pillow

(2015) Reflexivity as interpretation and genealogy in research. Cultural Studies ↔ Critical Methodologies 15(6): 419–434.

45.

Plowman

Stephen

(2008) The big picture? Video and the representation of interaction. British Educational Research Journal 34(4): 541–565.

46.

Price

Jewitt

(2013) A multimodal approach to examining ‘embodiment’ in tangible learning environments. In: Seventh International Conference on Tangible, Embedded and Embodied Interaction, Barcelona, Spain, 7 February 2013, 43–50, ACM. Available at: https://dl.acm.org/citation.cfm?id=2460632&dl=ACM&coll=DL&CFT (accessed 15 November 2021).

47.

Rautio

(2021) Post-qualitative inquiry: four balancing acts in crafting alternative stories to live by. Qualitative Inquiry 27(2): 228–230.

48.

Sakr

(2022) Turning towards discomfort in postdevelopmental approaches to childhood art: the potentials of multimodal mediated discourse Analysis. Multimodality & Society 2(4): 434–450.

49.

Serra Undurraga

JKA

(2023) What if reflexivity and diffraction intra-act? International Journal of Qualitative Studies in Education 36(6): 1109–1122.

50.

Spyrou

(2018) Disclosing Childhoods: Research and Knowledge Production for a Critical Childhood Studies. 1st edition. New York, NY: Palgrave Macmillan.

51.

Spyrou

Rosen

Cook

(2018) Reimagining Childhood Studies. London: Bloomsbury Academic.

52.

St Pierre

(2016) Rethinking the empirical in the posthuman. In: Taylor

Hughes

(eds) Posthuman Research Practices in Education. London: Palgrave Macmillan UK, 25–36. (accessed 29 August 2024).

53.

St Pierre

(2024) A primer for post qualitative inquiry. Qualitative Research in Psychology 1–25, In press.

54.

Taguchi

(2012) A diffractive and Deleuzian approach to analysing interview data. Feminist Theory 13(3): 265–281.

55.

Taylor

(2019) Diffracting the curriculum: putting ‘new’ material feminism to work to reconfigure knowledge-making practices in undergraduate higher education Theory and Method in Higher Education Research. 5th edition. Leeds: Emerald Group Publishing Ltd, 37–52.

56.

Taylor

(2020) Knowledge matters | 2 | Five propositions concerning the reconceptuali. In: Navigating the Postqualitative, New Materialist and Critical Posthumanist Terrain across Disciplines: An Introductory Guide. London: Routledge, 22–43. Available at: https://www.taylorfrancis.com/chapters/edit/10.4324/9781003041177-2/knowledge-matters-carol-taylor (accessed 15 May 2024).

57.

Thiele

(2018) Ethos of diffraction: new paradigms for a (Post)humanist ethics Diffracted Worlds - Diffractive Readings. London: Routledge.

58.

Uprichard

Dawney

(2019) Data diffraction: challenging data integration in mixed methods research. Journal of Mixed Methods Research 13(1): 19–32.

59.

Vagg

(2022) Experiencing-with data: exploring posthuman creativity through rhizomatic empathy. Qualitative Inquiry 28(5): 541–551.

60.

Wargo

(2018) Writing with wearables? Young children’s intra-active authoring and the sounds of emplaced invention. Journal of Literacy Research 50(4): 502–523.

61.

Wohlwend

Thiel

(2019) From cutting out to cutting with. In: Kucirkova

Rowsell

Falloon

(eds) The Routledge International Handbook of Learning with Technology in Early Childhood. 1st edition. Routledge, 170–181. Available at: https://www.taylorfrancis.com/books/9781351389860/chapters/10.4324/9781315143040-12 (accessed 28 November 2023).

62.

Wohlwend

Peppler

Keune

, et al. (2017) Making sense and nonsense: comparing mediated discourse and agential realist approaches to materiality in a preschool makerspace. Journal of Early Childhood Literacy 17(3): 444–462.