Sensoria: An exploratory interdisciplinary framework for researching multimodal & sensory experiences

Abstract

This paper describes the development and salience of an original and innovative interdisciplinary approach, Sensoria, that combines methods and techniques from social science and performance to address the methodological challenges of researching sensory/multimodal experiences. It sets out the core components and methodological principles that underpin the approach and uses an illustrative example to show how it can facilitate research on hard to access sensorial experiences, to access, understand and analyse people’s experiences and perspectives of touch, a highly tacit sensory mode. The paper discusses the methodological contribution and challenges of this approach to sensory research for social science and artistic practice and ‘more-than-representational’ research more generally. It concludes by making a case for more critical research spaces at the intersection of these disciplines to foster multi-dimensional research dialogues and to advance the exploration and understanding of the relationship between the sensory, social and the digital.

Introduction

This paper describes the development and salience of Sensoria, an exploratory interdisciplinary framework that combines methods and techniques from social science and artistic performance to address the methodological challenges of researching sensory/multimodal experiences. The Sensoria framework was developed through a 2-year interdisciplinary research collaboration between a social scientist, a digital performance artist, and an electronic sound composer, and is a part of InTouch, a 5-year project researching the sociality of touch and digital touch. Sensoria consists of two core components and three methodological principles, all of which are interconnected. The first component is the generation of sustained and substantial interdisciplinary dialogue and exploration which emphasises collaborative ‘doing/making’ focused on the development of materials and resources. The second component is the design of an interactive performance experiment. Three methodological principles thread through these two components these are to: first, research sensory communication through the sensory; second, bring sensory materials and resources to the research; and third, provide a range of multimodal/multisensory ways to document the sensory.

The research collaboration at the centre of this paper led to the development of Thresholds, a case study and an Interactive Performance Experiment performed in the United Kingdom in January 2020. In this paper we use Thresholds to illustrative how the Sensoria framework can facilitate research on hard to access sensorial experiences, to access, understand and analyse people’s experiences and perspectives – It focuses on the highly tacit sensory mode of touch. Through the Thresholds case study and performance experiment we ‘flesh out’ the character of the collaboration at the heart of Sensoria and the strategies that can be used to provoke tactile dialogues across disciplinary difference. We describe how it was designed to explore and experience touch through touching, to bring new materials and materialities to the creation of a sensory environment tailored to researching touch, and the multimodal and multisensory resources that it used to enter into and document participant touch experiences.

Our focus is on the Sensoria framework and how it was employed in Thresholds to access participants touch experiences, dispositions and perceptions to enable ‘non-verbal’ and tacit sensory experiences perceptible to participants and researchers, through a process of enhancing participants’ touch awareness, vocabularies, and creating a new space for their broader exploration of touch. The paper discusses the methodological contribution and challenges of the Sensoria framework for social science, artistic practice and ‘more-than-representational’ research more generally. It concludes by making a case for more collaborative research dialogues and critical research spaces at the intersection of these disciplines to foster multi-dimensional research dialogues and to advance the exploration and understanding of the relationship between the multimodal, sensory, social and the digital.

Background: touching dialogues across methodological terrains

Touch matters. It is the first sense through which people apprehend their environment and it is central to their development (Field, 2003). Touch provides significant information and experience of the world, it is crucial for tool use (Fulkerson, 2014), and touch is central to communication, ‘Just as we ‘do things with words’ so, too, we act through touches’ (Finnegan, 2014: 208). Touch is significant for developing and maintaining personal relationships, from ritualised greetings, to communicating emotion or intimacy (McLinden and McCall, 2002). Touch improves information flow and compliance, and is an effective means of influencing attitudes, and creating bonds between people, places or objects (Field, 2003). In short, knowing how to infer meaning from touch is considered the very basis of social being (Dunbar, 1996).

Qualitative research and touch

It is paradoxical given its significance that touch is perhaps the most neglected of the senses within qualitative research. Although there are a few qualitative studies that prioritise the tracing of socio-cultural histories and/or the variation of touch practices and rituals (Finnegan, 2014), changing touch cultures and epochs (Classen, 2005, 2012), and the critique of the universality of touch and its categorization (Howes and Classen, 2014), qualitative research seldom brings touch into focus – touch tends to be filtered out of qualitative descriptions (Barker and Jewitt, 2021). The same is true for sensory ethnography, touch is rarely attended to with the exception of a few studies including touch in the context of laundry (Pink, 2005) and touching mobile media (Pink et al., 2016). Similarly multimodality – which sets out to understand the social dimensions of meaning, the resources and processes of meaning-making, and how these shape individuals and societies (Kress, 2010), with a few exceptions (e.g. Jewitt, 2017; Jewitt et al., 2020; Cranny-Francis, 2013), has not touched on touch.

Art and touch

The visual sense has been given priority within art (Jones, 2007) and touch is frequently associated with irrationality, lack of sophistication, and ‘primitivism’: a devaluing of touch that is echoed in its positioning within the hierarchy of the senses as the most primal or base of the senses, a history tied to persistent colonial and classed relations (Classen, 2012). Nonetheless, in comparison with the social sciences, art as a domain is more comfortable with the tacit, the unspoken, and the sensory. The Thresholds collaboration was informed and inspired by three significant artists whose works explored and engaged with how tactility influences the artistic process, choices, and aesthetics and historical traces of which M brought into her practice. This included Marinetti, a futurist who created the ‘Manifesto of Tactilism’ (1921 (1909)) and made tactile boards which were passed around an audience in the belief that through tactility they could attain ‘true sincerity’. Installation artist Lygia Clark (1920-1988) a member of the Brazilian Constructivist movement who developed a conception of art as an activity focused on the modification of the spectator through the use of relational-objects designed to stimulate imagination flows through touch (Butler and Perez-Oramas, 2014). A third key influence is avant-garde theatre designer, artist and filmmaker, Svankmeiyer (2014) who promoted ‘tactile sensibility’ as an unlikely political weapon that slipped under the radar of the Czech state and sought to demonstrate that tactile experience can restore access to irrational thought, emotion and perception. While touch is often subsumed beneath the visual senses in aesthetic artistic processes, it is centre stage within some theatre, scenography and performance art practices where the border between performer and audience has long been contested; bringing the participant and/or performer into touch in manifold ways (e.g. van der Vlugt, 2015); Salter (2010).

Methodological dissatisfactions, turnings and touch

The tactile neglect outlined above is one part of a broader research landscape; one that has generated social turns to the multimodal and sensory in response to an increasing ‘restlessness or dissatisfaction among researchers’ at the limits of social science methods (Mason and Davies, 2009: 588). These turns accompany calls for methods able to better capture people’s lived practices and experiences. A call that has heralded ‘messing methods’ (Law, 2014), ‘non-representational theory’ (NRT) theory and methods (Vannini, 2015), and ‘more-than-representational’ approaches (Pink, 2015). These approaches re-orientate to data and knowledge by ‘enacting multiple and diverse potentials of what knowledge can become’ and propose a kind of ‘witnessing’, a stance that is orientated towards being ‘in tune to the vitality of the world as it unfolds’ (Vannini, 2015: 15).

Thrift’s seven ‘tenets’ of the ‘non-representational project’ resonate with our work on touch: a concern with capturing the ‘ordinary, everyday actions’ (p. 142) and the ‘on-flow of everyday life’ (Thrift, 2008: 5); a focus on practice, action and performance; an interest in materiality; attention to the importance of ‘bodies’ in environments; and an experimental stance against traditional method and theory. NRT is increasingly considered a matter of style rather than a particular method, epitomised by a ‘fight against timid and formulaic research’ and a call for more creative and imaginative methods that disrupt research habits (Vannini, 2015: 12). Indeed, non-representational/more-than-representational has been described as ‘an umbrella term for diverse work that seeks to better cope with our self-evidently more-than-human, more-than-textual, multi-sensual worlds’ (Lorimer, 2008: 551).

The call for diverse multimodal/multisensorial methods speaks to the challenges of researching touch. Touch is where words often fail: people tend to have low awareness of their touch and to find it difficult to articulate their touch experiences (Jewitt et al., 2020); vocabulary for discussing touch including tactile metaphors is limited (Obrist et al., 2013). As a result, talk-based methods (e.g. interviews) generally provide limited insight on touch experiences. Touch also exposes the limitations of observational methods as the meanings of touch are hard (perhaps impossible) to understand when they are dislocated from the felt, sensorial or affective dimensions of touch.

The search for new methods to engage with the complexity of lived experiences has intensified social science interest in mobilising artistic research methods (Jewitt et al., 2017). Exploiting methodological synergies across this particular disciplinary frontier can open up social science to different perspectives, generate imaginative research questions and make available a wider range of methodological tools for creative use (Crow et al., 2011) in ways that are particularly suited to addressing contemporary, often ‘unfixable’, challenges (Dunne and Raby, 2013: 5).

The Sensoria framework

The Sensoria framework was developed through an interdisciplinary research collaboration between the authors of this paper – a qualitative social scientist (C) specialising in multimodal and multisensory research, a digital performance artist (M), and an electronic sound composer (F). It was initiated following our meeting at a conference on touch in 2018 and became a case-study within InTouch, a 5-year research project which examines the sociality of touch and digital touch and seeks to advance methods for the study of touch. We sought to interrogated the methodological potentials and challenges of researching touch at the intersection of the arts and social science, specifically to explore ways to attune to touch in an ocular and text-centric society, to develop resources (vocabularies, metaphors, materials), processes and methods to talk, think and feel touch in order to engage with the sociality and aesthetics of touch/digital touch. Thresholds resulted from our research collaboration. It developed as an affective, intimate, challenging and playful exploratory research environment to explore the sociality and sensorial aspects of touch communication and its digital-mediated futures. Thresholds was ‘performed’ (on four occassions, identified as performances A, B, C, D) in a central London Theatre Studio at the end of January 2020, a few weeks before public awareness of Covid-19 started to build in the United Kingdom.

Sensoria consists of on two component processes – one centred on interdisciplinary dialogue and the other on interactive performance, and three methodological principles. Each of these elements and their relationships to one another is outlined below.

Component 1: interdisciplinary multimodal/multisensory dialogue

Sustained and substantial interdisciplinary multimodal/multisensory dialogue which emphasises exploratory playful collaborative doing and making, moving and sensing is at the core of the Sensoria framework. This could include a variety of strategies to provoke sensory interaction, for example, reflective, playful, exploratory use of one’s own body through sensory experiments or artistic propositions; the creation of mini-sensory inventories (e.g. of textures); the re-enactment of sensory routines/practices; creating collective opportunities to respond to (everyday/unusual) objects or sensory experiences; shared sensory walks (face to face or digitally mediated); seeking out points of connection through film references, imagery; annotating or writing into shared documents. Such activities need to be designed to provoke and embedded in regular reflective discussion to bring each other’s disciplines and experiences to bear in challenging and critical ways. The purpose of this dialogic exploration is twofold. First, it provides a foundation for the collaborative development of research materials, resources, vocabulary and processes to inform component 2 – the interactive performance experiment. Second, it serves to sharpen and situate the three methodological principles that underpin the Sensoria framework (see below) and to establish a shared understanding and conceptualisation of component 2.

Documenting of the process and the content of these dialogues is key to Sensoria. This involves drawing on and trying-out the range of documentation practices used by the collaborators and the establishment of a collective space (e.g. a digital platform) for sharing content and ideas.

Interdisciplinary dialogue is necessarily filtered through the different disciplinary training and practices, expectations and experiences of the researcher/collaborators, making the formation of the collaborative team a vital consideration in the Sensoria framework. Alongside seeking a range of disciplinary practices, dialoguing requires experimenting with disciplinary boundary crossing, the sharing (through doing) of each other’s practices, collective interrogation of terminologies and concepts. While the dialogue is ongoing throughout Sensoria, it is essential that significant exploratory interaction precedes component 2 as this informs the development of the interactive performance experiment. Depending on the timeline of a project, the dialogue at the heart of component 1 can be in the form of an extended unfolding dialogue or a shorter more intensive period of time. (In Thresholds, the example we present in this paper, the dialogue took place over 1 year.)

Methodological principles

Three interconnected methodological principles underpin the Sensoria framework, these were developed through the collaborative case study and articulate multimodal/sensory sensitivities from both social science and artistic practice/research (Jewitt and Leder Mackley, 2019). Through interdisciplinary dialogue (component 1) these principles are fine-tuned to speak to the focus and context of the research collaboration and to inform the conceptualisation and design (e.g. its structure, content, and activities) of the interactive performance experiment (component 2). Each principle is briefly outlined below.

Principle 1: research multimodal/sensory communication through the multimodal/sensory

The Sensoria framework is underpinned by the co-opting of the sensing body to research to realise an intensive and immersive engagement with the multimodal/sensory to achieve a depth of collective attunement. Through Thresholds, for example, participants were offered possibilities to collectively and individually engage in embodied ways with themselves, one another, objects and the environment.

Principle 2: bring sensory materials and resources to the research

Sensoria makes the case for the benefits of researching multimodal and sensory experiences by fostering engagement with a wide range of materials and resources – including unusual or strange sensory experiences and sensory conflicts/surprises. We argue that this helps to prompt and support sensory exploration and reflection, and offers new methodological starting points and routes into the multimodal/sensory. In the case of Thresholds a wide range of touch resources were made and used to position social, sensorial and aesthetic aspects of touch centre stage: drawing out touchy temporalities, sensorial connections between touch and sound, touch vocabularies and metaphors, and creating future facing critical spaces for collective speculation on touch anchored to the past and present.

Principle 3: provide a range of multimodal/multisensory ways to document the sensory

The collaborative dialogue between arts and social sciences at the centre of the Sensoria framework is designed to open up methodological potentials for generating and documenting participants’ sensory experiences. Through Thresholds, for example, we developed multimodal and multisensorial possibilities to collectively and individually engage with the material, embodied and tactile practices of mark-making and meaning making to capture and document and through which to further explore touch experiences. Each principle is fleshed out through the illustrative example of Thresholds presented in this paper.

Component 2: interactive performance experiment

The second component of the framework is the development and production of an interactive performance experiment. This emerges through and is informed by sustained interdisciplinary dialogue and embodied multimodal and multisensory experimenting (component 1) and the methodological principles outlined in the previous section. These inform the dramaturgy and composition of the environment, experiences and reflections of the interactive performance experiment.

This component combines an exploratory interactive performance (which may also be a piece of artistic research in its own right) and workshop. The focus is on opening up and posing ‘conditions’ for experience rather than posing direct, explicit questions or asking for explanations. While the aesthetics and form of the interactive performance experiment may vary, the research environment created needs to provide:

• An immersive embodied multimodal/sensory participant experience/journey

• Space for both shared/collective and individual interaction

• Features that are affective, aesthetic, immersive, interactive, novel (e.g. making the familiar strange), challenging, exploratory, critical and playful

• Varied materials, resources, activities, exercises to provoke sensations and meanings

• Means to express and capture experiences of the performance and the multimodal/sensory more generally.

Given the interdisciplinary character of the Sensoria framework, while the two parts of the interactive performance experiment build on different disciplinary resources, methods and expectations, there needs to be a strong sense of connection between and across them, to create a coherence that offers more than the sum of its parts. Its length can vary (Thresholds was 2 hrs long) but needs to be long enough to achieve a sense of immersion: we suggest that the two parts of the experiment be kept in equal balance.

Thresholds: an illustrative example

In the remainder of this paper, we draw on Thresholds as an illustrative example to flesh out the Sensoria framework.

An interdisciplinary multimodal/multisensory dialogue on touch (component 1)

In the case of Thresholds (discussed in this paper) the research collaborators dialogue was conducted online, face-to-face and included intensive research-visits to each other’s research lab (London), conservatoire (Utrecht), and artist Atelier (Amstel) and collective work in a theatre studio (London). Our encounters were documented using written ethnographic field notes, short audio or video recordings, artistic responses, and photographs. These were shared, written into and discussed to capture our different perspectives, terminologies and concepts in order to generate reflection on ‘key’ moments (e.g. of difference or confusion) and new directions.

We used a variety of strategies to provoke tactile dialogues and bring touch into our research process in new felt, visual, and aural ways. We exchanged ideas, significant texts including research publications, images, videos, technologies and artistic artefacts to inform our work on touch (Figure 1).

Figure 1.

Examples of the texts and experiences exchanged and shared.

Our bodies proved to be a significant starting point for touch explorations.

I feel that some questions can be researched starting with our three very different bodies. Doing embodied research together, sharing. (M, Fieldnote Jan., 2019)

We collectively interrogated touch rituals, touch practices, the social and cultural norms of touch, bodily boundaries of touch, tactile memories and imaginations, tactile making practices, as well as the tactility and acoustic qualities of touching everyday materials and objects, the self and the other. A process illustrated by two (collectively produced) short vignettes below:

Visit to HKU Conservatoire (Collaborative, April 2019): The studio was strewn with materials that M & F had been experimenting with. We spent the day touching – squeezing, stretching, stroking, and tapping them. M has spent time working with capacitive E-wire in a variety of knitted shapes, textures and colours. We felt them, their weight, and the sounds they made as they fell to the floor. Knitting the e-wire had bruised her knees- which she showed us. We shared stories of how knitting with electric wire might connect to touch to fear by virtue of the dread for touching anything with live current; and intimacy in relation to the domestic, familial, and female.

Visit to M’s Atelier (Collaborative, Nov. 2019): M had laid out the objects (‘propositions’ or ‘provocations’) she had made on the studio table (Figure 2) and we spent the afternoon ‘thinking with and through touching’.

Figure 2.

The artefacts (‘propositions’ or ‘provocations’) on the studio table.

We pressed the objects, felt their temperature, picked them up to weigh them. Touch exploration was indispensable for mobilising our imaginations. Together (frequently in silence) we felt and tried to describe the tactile and sensory qualities and imprints that these objects left. Our vocabulary was limited but expanding. M retrieved the casts of the objects, and she re-enacted their making – showed us the choreography of her fingers, hands and body at work. We retrieved social and cultural associations and tactile experiences that this evoked for us. C mapped these descriptions using multimodal concepts (e.g. materiality, agency, provenance) traces of labour and skills, degrees of malleability – from flexible to rigid, responsive to resistant and so on to create an inventory of tactile qualities. Alongside us, F worked on the sound creation and sonic exploration. He has collected everyday materials and objects – natural (leaves, stones) and manufactured (spoons, plastic) to create ‘tactile sounds’ (Figure 3): he arrived with sound samples and a bag of material samples. He played the sound recordings, tinkering and composing, we acted out the sounds, we touched and ‘played’ objects – scraped a twig across a kitchen tile, scrunched leaves in our hands to investigate the tactility of F’s sound scape, attune, and name them (the sound files needed names), this stretched our vocabularies of touchy sounds, and we translated words between our first languages (Dutch, English and German) to explore their etymology: stuttering push, glide and stuck, scratching, rubbing and so on. Sound was getting us to the ‘liveness of materiality and the unknowingness of human interaction’.

Figure 3.

F used different materials and types of touch to create and record sounds.

Through multimodal and multisensory dialogue across our disciplinary boundaries we generated a wide variety of touch experiences: felt and tested by us and trialled with other audiences (e.g. conservatoire students, and our colleagues). Through these felt experiments, we explored ways to design an environment to attune tactile acuity, create space for new touchy narratives and experiences, and feel towards imagined future landscapes of touch. In this way, we sought to both sensitise ourselves and others to touch and to ‘decondition’ everyday touch routines.

Dialoguing in this way required us to move in and out of our roles as researcher, artist, performer/assistant, and participant. To a large extent, our roles were/are structured through our professional biographies, although at times the complexity of stepping into and out of the flow of artistic practice to research and reflect enabled a productive blurring of roles. Reflection on our roles held in disciplinary tensions (e.g. between research and artistic outputs) informed the design of Thresholds where the methods and demeanour of performance and social science research bled into and influenced one another. This enabled us to think through performing and exploring touch together, and led to the materials selected and developed for the performance experiment.

An interactive performance experiment (component 2)

Thresholds consists of two interconnected parts (each 50 minutes long). Part one consists of an exploratory interactive performance (an artistic-research output in its own right), rooted in artistic performance and embodied, participatory research. This provides the conceptual and material starting point for part 2, a participatory exploratory workshop which engaged participants with a series of activities and artefacts to support the processing and expression of their experiences of the performance and touch more generally. While the two parts are built on different disciplinary resources, methods and expectations, as we discuss there are lines of connection threaded between and across them, at times blurring the disciplinary boundaries between us. Thresholds was ‘performed’ four times (data from each identified as PA, PB, PC, PD) in a central London Theatre Studio at the end of January 2020, a few weeks before public awareness of Covid-19 started to build in the United Kingdom.

The performance experiment emerged through a year of multimodal and multisensory dialogue on touch. That dialogue informed the dramaturgy and composition of “touches” evoked through “tactile” performances built through artistic reflections on touch including the enactment of (self) care, the inner experience of physical movement, and notions of liveness. Using the analogy of a gravity chamber used to prepare astronauts for the journey into space, Thresholds, was conceived it as a sensitising “tactile preparation chamber” to attune researchers’ and participants’ bodies to touch, enhance their tactile acuity, and offer them novel resources through which to explore, express and document their touch experiences. A variety of techniques were used to attune to touch through touch: doing and making were brought to the fore to interrogate micro-touch activities, we drew on our own and others touch experiences, evoked touch memories, and explored tactile sensations and meanings through a wide range of materials, environments, and natural elements to support an opening up to touch. This filtered into a series of exercises rooted in ritualistic and everyday touching of the other, self, objects and engaging with tactile, aural, visual, and written prompts alongside bodily interaction and the purposeful use of artistic practice to make “strange” everyday touch and the use of responsive materials to “Open up and interrogate this space’, “disrupt it”, “stretch it” to “insert new touch qualities that keep people in the experience” (Field note excerpt M, Nov 2019).

Participants were recruited via digital notices placed in multiple London and national free event listings, the InTouch project website and twitter. These notices informed potential participants of the experimental, participatory and potentially challenging (e.g. touching between strangers) character of Thresholds. To ensure the intimacy of the experience a maximum of 14 participants per experiment was set. A total of 56 people participated in the four performance experiments, ranging from 17 years old to 60+ (due to issues of consent an age limit of 16 + years was applied). Roughly one-third of participants were from the arts (choreographers, dancers, designers, artists, musicians), one-third from social sciences and humanities (academics and students), and one-third from other occupations including the community and charity sectors. Participation was voluntary and participants were informed that they could leave the performance (withdraw) at any point. Following university ethical procedures, participants received a detailed information sheet and consent form in advance of the experiment. Participants were provided with permission options for the documentation of the experiment (e.g. re photographs and video recording). Thresholds is described in more detail below through the elaboration of the three methodological principles.

Researching touch through touch (principle 1)

Thresholds involved considering how to research touch through touch. In this section, we describe the process by which it attuned the participants (and researchers) to touch by providing tools to feel/think/talk about touch. Thresholds started, and the improvised soundscape created by F filled the space as participants entered the studio. He manipulated a large sheet of knitted conductive wire which was linked to the series of sounds composed from touching everyday materials (e.g. walking through dried leaves). His touch interaction was projected onto a large screen in the studio. Participants were initially seated in a semi-circle (Figure 4). C and L (a member of the InTouch team) took the role of assistant performers to M.

Figure 4.

The Threshold performance environment.

One by one M or C approached each participant took their hands and walked to the Hand-washing ‘station’: an intimate, personal, and performative invitation (Figure 5):

May I wash your hands?

Sensitising your hands

Waking up your sensorium

These are clean, soft towels

Let me dry your hands

Figure 5.

Threshold’s began with a hand washing ritual.

We slowly poured warm water on their hands, a ‘pump’ of hand-wash, we washed and dried their hands with individual fluffy towels. L then led each participant to be seated, slowly moving the chairs to create a circle. During the handwashing, participants were watched by those seated: they became performers. The rhythmic ritual created by waiting, observing, washing and drying created a reflective space that M described as ‘a space in-between, a feedback loop’. This provoked a self-conscious awareness of participants’ own body and that of others, focused on and through touch. Participants later commented on watching and knowing they were being watched: ‘I enjoyed seeing how other people interpreted touch, which ultimately made me think about touch in new ways’ (participant PB). This rhythmic alternation between public-participant-performer, toucher and touched, continued throughout the experience.

Participants described having their hands washed as comforting, warm, caring, protected, safe. The felt experience evoked spontaneous stories of hygiene, germs, health and learning the ‘proper’ way to wash their hands at school, home or work, and stories of childhood hand washing. For some, the offer of handwashing was problematic, one participant did not want to wash hands, she had very dry skin and eczema, she was ‘sorry’, and asked ‘is that okay’, she was reassured and offered a hypo-allergenic hand-cream which she accepted. One participant was heard to say to a friend (while sitting in circle) ‘I’m not doing that!’, but later willingly took L’s hand, laughed and moved in ‘small swaggering movements’ as he followed her to the ‘handwashing station’. Resisting rituals. Positive or negative, the ritual offered a one-to-one touch experience for all participants that set the tone and sense of connection for the experiment. It also connected the researchers in unexpected ways:

Field note on handwashing (C, Jan 2020): I was surprised I enjoyed it. I thought I might find it repulsive to wash strangers’ hand. It felt intimate and I felt ‘caring’ and ‘good’. The experiment gave us/me permission to touch. Through my actions I felt a sense of connection with professions with an intimate caring touch that accesses the secrets and lives of people in a moment of touch interaction: hairdressing assistants, manicurist, care-givers, and nurses.

Once in the circle, M invited the participants to touch one another: shake hands; to hold each other’s wrists: to feel for each other’s pulses (Figure 6).

Please hold each other’s wrists, just above the hand

Hold it for a little while

Try to squeeze a little

Experience how long it takes for your flesh, tissue, muscles to bounce back

Try to turn your wrists around

Go slow

Can you locate the other’s heartbeat with your fingers?

It might be tucked away, hiding

We observed the hesitancy, awkwardness, and taboos of touching a stranger and the discomfort of gendered and sexuality taboos felt by a pair of men holding hands. These touch taboos felt palpable as the performance ruptured and opened the social norms and regulation of touch.

Figure 6.

Threshold participants are invited to hold and feel each other’s wrists.

M invited and instructed participants to touch their voice box, and the large arteries on the side of the neck (Figure 7):

Place your right finger in the little hollow in the base of your neck

Scratch it a little

Draw a line with your fingernail, upward until you reach the small voice box underneath your skin

It moves up and down when you swallow

They are attuning to their body through touch. Engaging with the felt sensation of sound as vibration. F’s continuous activities invited the participants to feel the sound.

Participants were touched in the hand washing, they touched others, and touched themselves.

Figure 7.

Threshold participants are invited to touch and feel their necks.

M then asked the participants to place their hands in their lap, palms facing-up (Figure 8). C and L entered the circle and placed an object in the palm of each participant.

Figure 8.

Participants place their hands palm up on their lap to receive an object.

The object had three layers each of which had different material and physical sensations, and respond differently to touch. The first outer layer, was made of silk, dyed presenting a tangible pattern of Bāndhanī dots an Indian tie-dyeing technique. The second layer was a disfigured ‘ball’ made using memory foam cast in a round mould. The third inner-layer was a hard plaster-object – cast from a hole within the ‘ball’, and manually stitched to hold it inside the ‘ball’ (Figure 9). Each object had the same three layers but was unique due to the hand-made process of its making. M invited the participants to close their eyes and to feel the outer layer, ‘the skin’, of the object and to explore its inside:

Hands caressing, Hands wipping tears

Hands touching fabric

Cotton, Angora, Satin, Brocade . . .

Did you read the tiny knots with your fingers?

How will you know what is inside?

M asked the participants to unwrap their object, to explore it and finally, to squeeze out the inside ‘core’: to break the stitching; to break the attachment.

Figure 9.

Participants unwrap and feel the object they received.

Squeeze carefully, Experience how long it takes to bounce back Did you give it an ‘equal’ touch?

A comforting one?

The object is touching you back

It will tell what’s inside

Slowly squeeze the inside out

As the participants unwrapped their objects, some reacted with repulsion, ‘Urgh. . . it looks like POO’ or ‘I’m not touching that!’, others with surprise, and some with laughter. In the moment of squeezing out the centre, participants’ touching varied. Some delicately tried to unpick the thread holding the core in place and gently pulled at the thread. Others tugged at the thread, tried to break it, and roughly pushed the core out. All participants were intensely engaged with the objects and interacted with them as if organic or alive. Some participants later commented that breaking the thread felt too ‘violent’: ‘I couldn’t do it, I couldn’t break the thread to take out the inside. I wanted to protect it’.

Through these scenes, the performance gradually built a sense of bodily-touchy awareness and bound the elements of the performance together to create a felt sense of tactile-reciprocity, the responsiveness of the body, of others and objects – with the intention of connecting participants to how touch sets us in the world. One participant commented:

‘From the outside, it is hard to realise how channelled and funnelled the experience of participation was, moving from engaging with another person and being aware of the group, to yourself, to really focusing in on just you and the object’.

Others echoed this sense of an ‘intensifying’ or ‘narrowing’ of their focus through the performance to ‘decode sensory layers little explored in our daily lives, due to lack of time’.

Bringing touch materials and resources to the research (principle 2)

Researching multimodal and sensory experiences is enhanced by fostering engagement with a range of materials tailored to the work of sensory attuning. Part two of Thresholds consisted of an exploratory workshop designed to extend participant engagement with, and reflection on, tactile materials and touch practices with the purpose of supporting participants to process their touch-based experiences evoked through the performance and beyond.

The workshop offered participants a range of multimodal, social and sensory resources arranged at six ‘Reflective Stations’. These transformed the material residues of part one of the performance experiment to bring them to life in new ways and this served to infuse the workshop atmosphere with participants’ performative experience. The ritual hand-washing element of the performance, for example, was maintained in an altered form to ‘extend’ into the workshop. The bowls, jug and hand-soap remained at the Reflective Hand-wash Station (Figure 10), however, the water and towels were removed, and the performers were not present, and a looped-compilation video of different handwashing rituals/routines from religious and health organisations displayed on a laptop was added to the station. F attended the Reflective Sound Station where participants could interact with him and the sound curtain (Figure 11, Bottom Left); M attended the Reflective ‘Gift’ Station with the unwrapped artefacts similar to those gifted in the performance alongside a video documenting the tactile processes involved in their production (Figure 11, Top Left).

Figure 10.

(a) The workshop reconfigured wash station; (b) Still from the washing video.

Figure 11.

Examples of materials available in the workshop.

Alongside these residues, other Reflective Stations introduced new touch materials and resources including touch metaphors and vocabulary, video recordings of touch practices, and a range of non-linguistic processes of engaging with and documenting touch (e.g. tactile mood-boards) as well as drawing tools, paper, and video cameras (Figure 11).

In the first part of the workshop, participants were invited to explore and document their felt experiences of the performance (e.g. sensations, tactile surprises, memories evoked, challenges, pleasures and repulsions). This was a deliberately open invitation in-line-with the focus of the Sensoria framework on posing ‘conditions’ for experience rather than direct questions or explanations.

The last third of the workshop centred on an activity on ‘The future of digital touch’ that built on the participants’ processing of their experiences of touch to bridge to considering the digital futures of touch. The performative character of Thresholds carried through into this activity which was designed to explore participants’ tactile imagination of digital touch communication: they were invited to consider how future touch experiences might be mediated and shaped by the digital and communicative possibilities of touch. A short (5 min) silent video compilation of adverts for digital touch devices for remote couples (Figure 12) was projected onto a large screen.

Figure 12.

Screen shots from the compilation video shown in activity 2.

In an effort to disrupt the stereotypical utopian or dystopian narratives of exploring digital touch, we ‘punctured’ the video’s smooth hyperbolic commercial imagery by reading aloud of critical quotes on the social and historical trajectory of touch technologies. These quotes from Descartes, Marx, Keats, Merleau-Ponty and others, were printed on postcards, and read aloud by facilitators and participants (Figure 13). The quotes foregrounded questions of culture, technologies, and society which served as counterpoints to the video and primed reflection on the importance of touch in human history and the different roles it has occupied as epochal and technological tides change. They provided a critical overlay for the video, and this helped to create a platform from which participants could journey beyond the extremes of utopian and dystopia that futures so often generate.

Figure 13.

Participants reading quotes aloud.

Participants were given a blank post-card with an invitation to: ‘Write a speculative quote to represent what you imagine (hope, desire, dream, fear) for future digital touch in the year ‘X’, and to date the year’. These quotes provide insight on the participants’ hopes and fears and sociotechnical imaginaries of digital touch’ and collectively contributed to a timeline of digital touch imaginaries. Participants’ postcards were collected and an anonymised selection was read aloud. The workshop closed with a short (5 min) de-brief space for participant comments. As participants left the studio they received a written touch-related quote and a sweet or orange (a ‘mouth-texture’ experience).

Providing a range of multimodal/multisensory ways to document touch (principle 3)

We used field-notes to capture the activity and visible responses of participants across the performances, including verbatim comments and participant interactions, as well as to record and reflect on our perspectives as researchers/artists. Alongside these we used video recording and photography to document Thresholds using a mix of fixed and roaming cameras to capture the detail of participant interactions with one another and/or objects.

In addition to the broad range of multimodal/multisensory materials and forms to interrogate and document their sensory experiences provided by the Reflective Stations, each participant received a notebook and pen as they entered the workshop. The label on the notebook invited them to:

Use this notebook, your body and a pen to observe and reflect on the touches that you witness or experience. Can you feel the ‘echo’ of touch? Find ways to document these: use your body to mirror the touch, act-it-out, sketch, write, jot.

The small (A5) size of the notebook and the aesthetic of its plain brown cover and pages are significant in that it was chosen to be held and carried by participants, to become personal, reminiscent of a small diary or pocket notebook, and to sit comfortably between an artistic and a social science resource. A total of 56 notebooks were collected from participants as they left workshop and many participants photographed pages of their notebook before returning it. Participants produced written stories, poetry, notes, lists, diagrams, drawings, rubbings and tracings in their notebooks. Supplementary materials included the speculative quotes written on postcards in response to the imagined future for digital touch activity, drawings or rubbings made on larger pieces of paper, and two short videos made by participants in response to the invitation to process their experience. A week after the performance an open email-request for feedback was circulated to all participants – nearly a third of the participants (29%, 16/56) responded.

Participants documented touch experiences related to Thresholds, their past and present touch memories/experiences, as well as imaginary touch futures. To do this they drew on a wide range of resources and modes of expression including bodily re-enactments, movement, visual forms, printing and imprinting, and a variety of writing genres (Figure 14). While forms of expression and meanings are always intimately connected, here the connection was palpable, with the processes of enacting and touching revealing felt memories to participants in ways that they commented on as surprising or strange and leading them to documentation. The exchange below is typical:

Field note excerpt (C, Performance A workshop) In the workshop a participant told me that as she watched me and M washing the hands of others she had felt ‘sceptical, wary of the experience’, but found it ‘really comforting’. She reflected on her ‘strange’ mix of ‘wariness and comfort’. In the course of this, she remembered one of her earliest memories of having her hands washed, which she shared with me, and she decided to draw a picture of this in her notebook.

Figure 14.

A selection of pages from participant note books.

The materials provided at the Reflective Stations opened a space of conscious feeling through newly felt encounters with everyday objects (e.g. dish-washing sponges, tea bags, toilet roll) and generated new touch re-enactments and discussions of the everyday, with several participants talking of having a newly felt relationship to such objects. Participants reacted strongly to some of these materials and tactile sensations (e.g. the feathers on the tactile mood-boards produced both repulsion and delight).

The performance heightened participants’ sensitivity and awareness and reflection of touch and the exploratory activities supported by the workshop provided a reflexive space for participants to hone in on touch. For instance, one participant wrote a short reflective narrative on the performance in their notebook of the moment she felt the hard ‘gift’ inside the ‘squishy ball’ and then felt the hand of the person next to them – asking herself, ‘Did I recalibrate? Can you calibrate touch?’. Others asked how they might indicate ‘preferred touch interaction without using words/voice’. For some it opened an imaginative space:

When I focused on my sense of touch, every touch is exaggerated, I couldn’t help but imagine and visualize the feelings, everything else became silent and faded away. . .when we held each other’s hands, I could feel there’re thousands of little sensors beneath my skin, being triggered by every tiny move and touches. (Participant comment in notebook PB)

Methodologically, the performance both exercised participants’ vocabularies of touch and expanded them. This illustrative example from one workshop is typical, a participant (for whom English was a second language) asked other participants what the difference is between ‘pat’ and ‘patter’ which led to conversations and physical demonstrations of pattering on one another’s bodies and the different surfaces, and the sharing of metaphors such as the ‘pitter-patter of rain, or of tiny feet’. Through physically engaging with the sound curtain and F, participants reflected on the concepts, features and dimensions of touch through the performance experiment, including the sensorial connections, for example, sound and touch. Touching also led participants to explore sensorial conflicts, for example, two participants squeezed the ball objects as they watched a video of them being made and commented ‘It has the texture of a chocolate brownie, but looks like porcelain!’ leading them to discuss the dissonance between the visual and the felt more generally.

The participants’ notebooks offer up expressions of the temporality and spatiality of touch, through the audio or visual rhythm of drawings and writing (e.g. of repeated words – rub, rub, rub). The aesthetics of touch were also brought to the fore, with participants’ visualising material qualities of touch, and exploring the process of documenting touch as itself a process of creating, performing or feeling. Throughout the performance experiment, the politics, and sociality of touch constantly emerged through participants’ explorations (including the breaking) of the norms of touch, notably in relation to gender, for example, in commentaries on the size of hands, and the avoidance or discomfort of cross-gender as well as sexuality and same-sex touching.

Participants documented specific scenes or touch experiences from the performance experiment, pointing to cultural and social norms of touch, as well as their personal histories, memories, narratives and imaginations of touch. Having a variety of materials available was central to the documentation of these experiences. Many participants made rubbings with charcoal and other materials to capture the textural qualities of objects, including the ball used in the performance – and some labelled these with associated memories, sensations, sensorial preferences or emotions (Figure 15, Row 1, Left). Using their own word or selecting words from the ‘Touchy Vocabulary’ (i.e. touch word labels) provided, participants listed, classified, or made continuums of touch (e.g. from ‘caress to handshake’, or ‘connection to imposing’). Some linked words to touch experiences bringing the whole body and environment into the realm of touch – for example, the page displayed (Figure 15, Row 2, Left) expands on RUB as shoes, and the incidental touching of hands or shoulders on public transport – something they positioned as ‘contact’ which the easing of the Covid-19 Pandemic lockdown brings starkly into view. The awkwardness of touch came through in the experiment – in observations captured in field notes, in the video data, and in participant notebooks in many ways, indicating social norms as well as discomfort, strong reactions and touch-aversions. Several participants, for instance, noted feeling ‘unbalanced’, sick or expressed a desire to be ‘untouchable’ (Figure 15, Row 2, Right).

Figure 15.

Examples of participant quote cards.

Participants used affective abstracted drawings to convey their emotional responses to touch – e.g., an almost electric looping and hopping of energy lines between people, or a flowing cascade of water touch their hands. Some combined drawings and writing to create narratives, poetic expressions and imaginative reflections to make sense of or explore their touch experiences (Figure 15, Row 3). In one notebook (Figure 15, Row 4), for example, a participant reflects on a sense of touch as a distraction from the visual sense, and re-imagines the object from the performance as a living creature, with the small plaster object hidden inside as its heart. While another participant tries to make sense of the ‘feeling sound’, drawing the hairs on her arm standing up, and the traces of sound waves in the air.

An analytical network was achieved through the layering of different activities, methods of documenting, types of data collected, actors (i.e. researchers and audience), and the disciplinary analytical views this brought to bear: a kind of ‘interdisciplinary triangulation’. The vignette below illustrates how the two parts of Thresholds, and the resources that these provided participants with were brought together to explore the ‘politics of touch’.

Figure 16.

Left, a page from participant’s notebook; Right, video still of her action and C’s mirroring of the action.

Mud vignette (C, performance C)

A participant had been standing at the ‘Residue Hand Washing’ Reflective Station for some time and I went over to see what she was doing. She was smiling and slowly rubbing her hands together. She started laughing when I arrived. Her hands were covered in hand soap lather and she was squishing it between her fingers and making bubbles. As we talked I mirrored her actions (Figure 16). The suds were grey as she had been making charcoal rubbings earlier. She said, ‘I feel naughty, like a kid’. She said the feeling reminded her of being a kid playing with mud on the beach. She told me a childhood story, that she liked to ‘get dirty’ which got her in trouble because she was a girl, but that her brother was not told off. She pushed her hand on her notebook page and made a hand print. Another gesture from childhood? She said ‘I liked to break the rules’. We were both laughing. She was very messing by this point. I offered to bring some water and wash her hands: I could have shown her where the toilets were – I wanted to wash her hands – was I entering her re-enacting of the naughty child sequence? She said ‘yes please’. As I poured the warm water over her hands she moved her hands under the water – we washed her hands together and she said, ‘When you washed my hands,[in the performance] I felt so protected’. I appreciated her freedom at getting them dirty without wondering how they would get clean. Later this bodily imprinting and re-enacted exploration appeared in her notebook as a personal narrative and memories of touch: a gendered tale of mud, dirt, breaking the rules and still getting in trouble years later for ‘being too tactile’.

Touching, getting dirty, and washing brought forth many other participant stories. This example demonstrates how working across the research materials – here the participant notebook, the researcher field note, and the video recordings, helped us to make sense of participant touch experiences.

Conclusion

Touch and the sensory more generally, is increasingly valued and understood as central to interaction, however, people generally find it hard to access and express these experiences, a difficulty compounded for touch by a paucity of tactile vocabularies and metaphors, and the limitations of visual and talk based qualitative research methods to bring touch to the fore. In this paper, we described the development and illustrated the use of Sensoria, an innovative interdisciplinary framework and showed the dynamics and potentials of interdisciplinary collaboration between the arts and social sciences to support multimodal/sensory research.

The Sensoria framework seeks to facilitate and operationalised what Vannini (2015) has called the “impossible” desire of more-than-representational approaches to “make us feel something powerful, to give us a sense of the ephemeral, the fleeting and the not-quite-graspable . . . without necessarily having to resort to spoken commentary, to extended captions, and to research informant’s transcribed accounts and illustrating narrations ..’.. (Vannini, 2015: 6). It does this by grounding the idealized impossibility of more-than-representational approaches in the form of interdisciplinary dialogue and interactive performance at the intersection of multimodal and multisensorial social science methods and artistic research/practice. While the arts and social sciences circulate within different spheres, we position the artistic dimensions of the case study as a form of “experimental humanistic science” occupying a position similar to that of experiments in social science (Elo, 2009: 22). The Sensoria framework is thus engaged in a “critical encounter or confrontation” (Elo, 2009) which frames “research into and through art” (Scrivener, 1999: 71) in the context of social science, to avoid the binary “trap” of either leading or subsuming the other, to provide a methodological hybrid that combines experimental and established practices to amplify sensory, bodily and affective aspects of touch.

We have shown the potential of the Sensoria framework to make a methodological contribution through the creation of imaginative research environments for sensory research that foster entry into difficult to access experiences. In Thresholds, the framework provided researchers and participants novel ways to get in touch with touch, it helped them to attune to touch, offered a platform and vocabulary for exploring touch, and generated insights on the felt, sensorial, and affective dimensions of touch. This helped to bring touch out from beneath the other senses in the aesthetic process, and to hold it up to critical reflective address and exploration. The Sensoria framework has also contributed to artistic practice: for example, M, is engaged with the Threshold materials to explore notions of affect through her ‘felt responses to them’ towards the construction of a ‘living archive’ – a reservoir of objects, questions and materials designed to be enacted and to evoke a variety of responses, and informing the development of a new artistic work. We suggest this framework could potentially be extended to collaboration between the arts and other disciplines (e.g. engineering, computer science).

The Sensoria framework raised challenges for sensory research across (and at the intersection of) social science and artistic practice, including the challenge of engaging with novel research materials, working across disciplinary difference, the need to balance the value of art and social science, and the management of collaborator roles and identities.

Artistic and social science research circulate in spheres with different ontological and epistemological perspectives. The Sensoria framework fosters critical research encounters (Scrivener, 1999) that seek to interrogate and value disciplinary difference as a route to bring the values of art and social science into balance and to avoid social science reduction of artistic work to a mere research tool, prompt or means of dissemination, or conversely social science as wholly reductive. The methodological differences revealed through our collaboration frequently touched on the values of art and social science including differences related to research design, research questions and parameters, requirements for research outputs, as well as levels of (dis)comfort with disciplinary porousness and exchange.

The Thresholds case study and performance experiment brought social science and artistic research into balance through the blurring and exchange of videos, practices, and the materials that we engaged with throughout the collaboration. The residue of artistic performance seeped into the exploratory workshop and became a core part of it. The workshop activities and materials responded to the performative sense of Thresholds, for example, the looped videos we made, the reading aloud of quotes, and the invitation to participants to write their own quotes, stood somewhere outside of the usual design and experience of a social science workshop. This methodological boundary crossing led to challenges, for instance, the extent or desire to which Thresholds could prepare participants to think through touch without over-directing. This brought social science desires for clear research questions and informed consent into new conversation with artistic research preference for ‘conditions’ which are set up to create aesthetic experiences shaped through the introduction of speculation rather than being tied to a desire to find an answer to a predefined question or through the realm of authority of parameters such as predictability.

The recognition of disciplinary differences impacted on the management of our roles and identities within Thresholds, and required us to move in and out of roles as researcher, artist, performer/assistant, and participant and to foster a degree of productive blurring of roles at times, and at others to maintain the boundaries of our critical differences. This highlights the need for those who use this framework to embody a degree of research flexibility and to work through moments of disciplinary discomfort and confusion. Through dialogic collaboration, Sensoria establishes permission for speculation and recognition that we cannot begin to think about sensory instabilities through stable research positions and methods of research.

To conclude, the creation of exploratory interactive critical research spaces that can open participants and researchers up to, and immerse them in, the messiness of our sensorial worlds is central to the value of the Sensoria framework. This paper argues for the generation of more such critical spaces between the arts and social sciences (and beyond) to foster multi-dimensional research dialogues and open up pathways to the sensory and the digital to generate imaginative experimental research to advance the exploration and understanding of the relationship between the sensory, the body, environment and the technology.