Editorial: Towards sensory studies of digital health

Introduction

My call for papers for the special issue invited contributions from a sociocultural perspective that address the entanglements of human senses with digital sensors in the portrayal, conceptualisation and use of digital health technologies. A major impetus for this special issue was my growing awareness that representations of digital health paradoxically often leave out the flesh-and-blood, visceralities and sensory experiences of bodies. Digital health technologies are all about bodies: monitoring, measuring, visualising and treating various bodily states and experiences. Yet public discourses on digital health and expert portrayals in the medical and public health literature tend to take a curiously dispassionate and disembodied perspective. And in some ways, digital technologies tend to engage with human bodies in ways that reduce their fleshly attributes to flat, two-dimensional images that privilege the visual over other sensory responses.

Recent theoretical and applied research in the social sciences has begun to highlight the sensory and affective dimensions of human and nonhuman assemblages. These include sensory ethnography, ¹ material anthropology, ² the sociology of the senses, ³ new materialism, ⁴ haptic media studies, ⁵ emotional geographies ⁶ and non-representational theory. ⁷ All of these approaches address the materiality of the ways in which the human senses engage with nonhuman actors, or what Vannini ^8(p322) describes as ‘sensuous scholarship’: ‘research about the senses, through the senses, and for the senses’. They all have implications for understanding and theorising the relationships of people with digital technologies.

Little research thus far, however, has devoted attention specifically to digital health technologies. This lacuna was the inspiration for this special issue. Humans' engagements with digital health technologies may involve different sensory capacities and sensations from accepted customs and norms. The senses may be enhanced, displaced or subdued as they are mediated and remediated with and through digital technologies. We know little about how it feels to be a patient who is engaging in a video medical appointment with a healthcare practitioner, or what it is like to be a healthcare practitioner who is attempting to assess patients' bodies at a distance. Few researchers have investigated how medical trainees use digital objects such as simulations, virtual reality or 3D printed patient anatomical replicas to learn about human bodies. We do not yet have a sophisticated understanding of how people negotiate the data about their senses that are captured by digital sensors, and incorporate these data into the mundane embodied routines of their everyday lives. These experiences, and many more, require further detailed investigation and theorising.

Some reflections on health experiences and the more-than-digital world

A few days before I began writing this editorial, I ran a discussion group with some people who were attending an outpatient cardiovascular rehabilitation programme at a hospital in my home city, Canberra. The purpose of the discussion was to discover what sources of information and support people who had recently received hospital treatment for a serious heart condition were using and found valuable. As one of my major areas of research is the social and cultural dimensions of digital health (see, for example, my recent book on the subject ⁹ ), I was particularly interested in the digital media and devices they may be using.

I began with general questions about what sources of information the participants had found useful in learning about their heart condition and rehabilitation following their diagnosis and surgery at this hospital. The group members told me that the hospital cardiac rehabilitation sessions were very important to them, not only as a way to learn about recovery and preventive actions they could take to improve their coronary health, but also as an opportunity to interact with other people who had gone through similar experiences. They explained that, together with the sessions they attended as part of this programme, the print material (pamphlets and a book) about cardiac rehabilitation that had been given to them by the hospital had been the major contributors to learning about their disease and recovery. They commented that they had been able to discuss aspects of these materials during face-to-face encounters with healthcare staff if they needed to ask questions or receive clarification on any of the information within. Some people had also attended pre-admission group information sessions after their cardiac condition had been diagnosed, which their partner was also encouraged to attend. Others had had no opportunity for this kind of preparation, as they had experienced a sudden heart attack and found themselves in the emergency department receiving medical attention with little warning.

A dominant theme that emerged from the participants' accounts was their desire to share insights from their experiences about the mysterious and unexpected nature of heart disease or heart failure. Several participants recounted their stories of how they had been diagnosed with heart disease or suffered a heart attack without realising that there was any problem with their hearts. For these people, the best way to share the insights they had gained from their own experiences was to tell their friends and family about it, as a form of warning. For some, friends or family members who had already experienced a heart condition were a source of information. They had listened to these other people recount their experiences and learnt about the symptoms and treatment.

When I moved onto the topic of digital technologies, it was clear that these were not important to most people in this rehabilitation programme. Only about half of them even owned a smartphone. Several people said that they used at-home blood pressure and pulse-rate monitoring devices as a way of tracking their heart health. They had purchased these from pharmacies, on their own initiative, rather than being encouraged to do so by their doctors. They printed out the data from a spreadsheet they maintained, or recorded their details with pen-and-paper, and showed this information to their doctors on follow-up appointments. None of the participants used a digital device like a smartphone app or wearable device for monitoring their blood pressure. A small number did use these devices for tracking other body metrics, such as physical activity levels. They were all in the younger age group (aged below 60). In terms of online sources of information, very few of the group had searched prior to their diagnosis for information related to any symptoms they may have experienced. About a third of the participants did go online after their diagnosis or surgery to seek information. However, none of the participants had ever used a patient support online forum or social media community for their health condition. When asked what they will do once they have finished the six weeks of the cardiac rehabilitation programme, some mentioned that they would join one of their local gyms to continue their exercise routines. None was interested in joining an online patient support group at that point.

Reflecting on this focus group discussion as I write this editorial has highlighted some of the key issues I envisaged the issue exploring. While my initial focus was digital health, these responses proved enlightening to me in their very de-emphasis and backgrounding of the digital. They provide a compelling counter to the techno-utopian visions that are often put forward by advocates of digital health technologies and the ideal of the ‘digitally engaged patient’ ¹⁰ that has become so dominant in the technological, medical and public health literature.

Profound affective and sensory aspects of living as a cardiac disease survivor were expressed in the participants’ accounts. For them, a key issue in how information about cardiovascular disease is communicated and shared was finding some way to let others know about the diverse symptoms that are not always recognised as signalling a heart problem. They reflected that they themselves in many cases had not recognised the symptoms when they were living through the experience. The discussion group provided a forum for people to tell stories of hidden illness striking suddenly and catastrophically. They emphasised the uncertainty of not knowing what the physical sensations they were experiencing were, and whether they should be concerned and seek immediate medical attention. Listening to their heart disease stories, and reading over them later as transcripts, I was reminded of Arthur Frank's influential book The Wounded Storyteller, ¹¹ in which he discusses how people's illness and physical suffering are expressed as narratives. Frank describes the wounded storyteller as ‘anyone who has suffered and lived to tell the tale … a guide and companion, a truth teller and trickster. She or he is a fragile human body and a witness to what endures’. ^11(pxi-xii)

In the face of this uncertainty and experiencing life-threatening illness, major surgery, and then lengthy recovery, the medical care and continuing support provided to the patients were vital to their sense of security and confidence in the integrity of their bodies. The participants’ positive feelings towards the rehabilitation programme and what it offered them were obvious in their accounts. While the space and people were unfamiliar to me, I could perceive that the group members felt at ease coming to this space to which they were now habituated through their twice-weekly visits, and with staff who knew them and spoke to them kindly, and the other cardiac disease survivors in the group they had come to know. Compared with the strength of feeling about the face-to-face encounters they had in this programme, the support and information offered by digital technologies were very much in the background. They were simply not important in these people's everyday experiences of recovering from and managing their cardiac conditions.

These people's experiences as they recounted them were filled with sensation and affect: the intense and sudden pain they experienced when having a heart attack, the surprise they felt at being diagnosed with a heart condition, the relief of having survived a serious medical problem and, in many cases, major surgery, and the comfort and reassurance of being supported during their rehabilitation by hospital staff and other group members. These were people whose everyday routines and assumptions about their bodies had been thrown into disarray. They wanted to be able to convey these sensory and affective experiences to me, and to others to warn them and instruct them on how to interpret their bodily signs and symptoms.

For this group, comprised of people who in many cases were not highly digitally literate or regular users of digital devices, digital technologies were on the margins of their care and support, or simply non-existent in their lifeworlds. It was the health professionals at the hospital, the other group members, the space provided for them to which they had become accustomed, and the print material given to them at the hospital that were the important and trusted elements in lifeworlds in which they were moving and recovering their bodily integrity and confidence. The findings from the discussion group raise further questions about what further support should be offered to people once the six weeks of the rehabilitation programme are over, and whether this should be mediated via digital technologies or provided in other ways.

Contributions to this special issue

The seven contributions to this special issue on the senses and digital health, take up sensory studies from a number of directions. A range of disciplines are represented, including sociology, science and technology studies, consumer studies, anthropology, communication and media studies. The authors focus their analyses on several different digital health technologies: wearable devices and health and fitness smartphone apps, and technologies used for medical visualisation, telemedicine and anaesthesia.

My contribution to this special issue is in the form of an essay in which I describe the concept of affective atmospheres and how this can contribute to understanding the entanglements of affect, space and place with human bodies and digital technologies. Structured around the key question ‘How does digital health feel?’ I review some of the literature on affective atmospheres, including the studies that have investigated the enactments of digital devices with humans. This discussion serves to contextualise and frame the other contributions, in emphasising the importance of the senses and affect as they intertwine with and are generated from humans’ encounters with digital devices in specific spaces and places.

Autoethnographies and other modes of personal and creative writing can provide powerful insights into how bodily states are experienced in specific spaces and places. In her essay in this special issue, Vyshali Manivannan employs this approach in her attempt to put her sensory experiences of fibromyalgia into words. She describes her experiences of seeking medical treatment in a USA-based hospital for abdominal pain and being positioned as the object of digital medical imaging technologies. Manivannan observes that her condition is hidden from sight, and therefore others find it difficult to understand her pain. Her physical sensations cannot be readily detectable by standard digital visualising technologies. Manivannan demonstrates how medical visualising technologies can work to draw attention away from the fleshly body of the patient ‘in the rush to find visual proof’, thus dehumanising the individual who is feeling the sensations of pain. As Manivannan demonstrates, medical language renders the lived experiences of pain to an anodyne jargon that diminishes its intensity. In response, Manivannan calls for a ‘new poetics of pain’, in which words are used evocatively to better communicate how it feels to live with a chronic pain condition like fibromyalgia. She argues that such a poetics can work to challenge the ocularcentric tendencies of biomedicine, including the digitised imagery that stands in for the patient's embodied sensory perceptions of her condition.

In another analysis that focuses on discursive strategies, Martin Berg uses critical discourse analysis to investigate the ways in which the Finnish developers of the ŌURA sleep-tracking and Moodmetric emotion-tracking smart rings describe their sensory potential. As he shows, the developers seek to promote their wearable devices as ways of contributing to wearers' intimate embodied knowledge of themselves. The devices are represented explicitly as sensory pedagogical objects, offering insights to their users that can then be taken up to decrease their stress levels and improve their sleep and general states of health and wellbeing. In the developers' accounts, it is constantly emphasised that these smart devices offer superior sensory capacities than those possessed by human bodies. These devices, however, render complex sensory, affective and other physiological embodied states into simple biometric measures that their digital sensors are able to capture and visualise in metric formats.

Smart devices for biometric monitoring or medical visualisation digitally sense elements of human bodies but require users to use their sensory responses to interpret these data. I have developed the concept of data sense in the attempt to encapsulate the assemblages of human senses and digital sensors as they generate sense-making capacities. Human users work together with these digital sensors to produce data about their bodies and then must engage in sensory work to understand and learn from these data. They are called upon to decide how accurate the data generated by the digital sensors are, to what extent they reflect or marry with the information that their embodied sensations convey, and how to incorporate these digital data into their body activities and performances. Assemblages of bodies-devices-data-sensations are continually enacted as part of data sense.^{12, 13}

Sarah Maslen takes up this concept of data sense in her contribution, noting that medical practitioners are faced with similar sense-making work when interpreting and enacting digital data about patients. Practitioners employ their embodied senses to sense the data generated by digital sensors. Maslen also draws attention to the senses of sight, touch and hearing as they are employed in diagnostic procedures by the Australian anaesthetist and general practitioner she uses as case studies, as she explores how they engage in sensory work. This work involves the practitioners ‘sensing the sensors’, moving back and forth between making sensory observations of the patient's body and of the digital sensing data returned to them by the monitoring technologies they are using, and combining both sources of sensory information to make clinical judgements. Maslen's analysis reveals both what is lost and gained when human sensing capacities are mediated via digital sensing technologies, and how practitioners respond accordingly when making assessments of their patients’ embodiment. This article provides an interesting alternative perspective to Manivannan's account from the patient's position, and Berg's analysis of how technological developers articulate the capacities of their digital monitoring devices. Maslen demonstrates the highly complex and dynamic nature of sensory work from the practitioner perspective, a complexity that can be acknowledged from the perspectives of patients and technological entrepreneurs.

In their article, Pantzar and Ruckenstein also focus on data sense-making. They develop the concept of ‘situated objectivity’ to encapsulate the contexts in which people engage with their personal digital data generated by self-tracking technologies. They contrast this concept with that of ‘mechanical objectivity’, or the accuracy that is promised by the developers and promoters of digital automated self-tracking devices. Pantzar and Ruckenstein also refer to the ‘trained judgement’ that is required of people when they are engaging with information, involving awareness of the contextual and learnt nature of knowledge. Their article reports on their interviews with Finnish people who were recruited into a study in which they were asked to use digital devices to track their heart rate variability (as a marker of stress). The interviews show that situated objectivity involves interactions between felt embodiment, cultural understandings of ‘stress’ and the metric data generated by digital monitoring. Many of the participants were surprised and confronted by their digital data, finding it difficult to explain why their heart rate variability showed little stress at times when their bodily senses and experiences suggested high stress, and vice versa. They drew on their lived sensory perceptions and interpretations of the social contexts to determine how valid these data were. Pantzar and Ruckenstein argue that when people are making sense of their digital heart rate variability data, they combine elements of trained judgement and mechanical objectivity, resulting in situated objectivity. As part of responding to and evaluating the accuracy of the metrics, they were learning from them. Rather than ‘living by numbers’, therefore, they are ‘living with numbers’, actively working to engage in data sense-making rather than simply accepting the findings of the digital devices.

The studies reported by Schwennesen and Urban respectively focus on the experiences of specific groups who have been encouraged to use digital health technologies. Schwennesen reports on her research on Danish patients in a rehabilitation programme following hip replacement surgery, while Urban's study is on German elderly people's use of wearable devices, home-based monitoring systems and fitness apps. Both used interviews as well as observations of the technology in use in their projects.

The participants in Nete Schwennesen's project were provided with wearable digital sensors strapped to their stomach and each leg sending data to an app to monitor their home-based exercises as part of their rehabilitation. The intention is to provide a ‘virtual trainer’ that patients can use at home in the place of visits to see a physiotherapist. The patients are expected to review their monitoring data in real-time as they exercise, to determine how well they are doing. Monitoring of their progress is therefore delegated from the physiotherapist to the patients and their devices. Schwennesen found that some patients often became vigilant in performing their bodies according to the directions and feedback responses of the software, to the point that they ignored pain and instead relied on the digitised feedback. Others, however, resented the constant correction of the software, and began to skip training sessions or gave up their use of the devices. They did not find the digitised training programme to be very responsive to the complexity of their individual circumstances and bodies. They preferred the personalised guidance of human physiotherapists.

Monika Urban's interviews and observations with elderly people identify the ways in which health and fitness self-monitoring devices sometimes generated resentment, guilt and feelings of being controlled by the device in this group, but also, in some cases, reassurance, pride and sociality. There were concerns about privacy expressed too, with some people worrying that family members or healthcare providers might be monitoring them too closely by reviewing the personal digital data generated by their devices, or that unwanted alarms might be raised. Frustration and dismay were also elements in these participants’ accounts. While they may have taken up the use of these technologies to prevent disabilities or ill-health created by the ageing process, the technologies were not necessarily designed for older users, and therefore were difficult to use for some people. Urban found, like Schwenessen, that some of the participants tended to discount their own bodily sensations in favour of the metricised outputs of their self-monitoring devices, and had become reliant on these to provide them with reassurance about their state of health. Sensory judgements, therefore, had in some cases been delegated to the devices.

Directions for further research

The contributions published in this special issue emphasise the importance of looking beyond the digital and acknowledging the other people and things that are part of lifeworlds. They also call into question the extent to which digital health technologies can or should substitute for face-to-face encounters in offering diagnostic capabilities and care, and what they can offer to augment traditional modes of medical care or preventive medicine. This might be achieved by not placing digital technologies at the centre of research and relinquishing the assumption that they are more important than other objects, or indeed, people, in experiences of health and illness. Moores ¹⁴ has called for digital media research that recognises and investigates the more-than-media worlds in which digital technologies are experienced and enacted. He uses the term ‘non-media-centric media studies’ to encapsulate this idea. This approach offers important possibilities for future research into the sensory and affective dimensions of digital health.

Research methods that are able to identify and understand the multisensory dimensions of human and nonhuman assemblages are vital to this project. Most social research relies on verbal interpretation and description of people's sensory engagements. These engagements can be difficult to put into words, however. They are felt rather than articulated, sometimes at the subconscious level. Turning to methods such as the video re-enactments ¹⁵ can help devote more attention to the sensory dimensions and contexts of digital health of which people themselves may be unaware simply because they are so routine and taken-for-granted. Another promising method involves ‘interviewing objects’ ¹⁶ as a way of focusing on digital devices, rendering the affects and effects of these objects more visible. Encouraging research participants to engage in developing ‘soundscapes’ ¹⁷ or ‘smellscapes’ ¹⁸ can be evocative ways of understanding the sensory dimensions of digital health technologies beyond the visual. Memory work, in which people are invited to write or verbally tell accounts of memories using key words as triggers, can be successful in eliciting sensory and affective experiences. ¹⁹ Building on Pantzar and Ruckenstein's methods, using digital devices to collect information about people's sensory responses and using these data as stimulus material in discussions with them, can provoke insights into their data sense-making. Inviting people to engage in speculative design, ²⁰ speculative fiction writing ²¹ or world building activities ²² are other inventive methods in which the sensory and affective dimensions of digital health can be accessed, including imaginaries about how these technologies might enact embodiment in future worlds.