Layers of sense: the sensory work of diagnostic sensemaking in digital health

Abstract

Sensory judgements have always been a part of medical practice and this sensory work is often entangled with technologies, from the stethoscope to digitised devices for advanced life support. This article investigates this sensory work and its entanglements with technological sensors in diagnostic practice. Based on semi-structured interviews, it presents a close analysis of practitioners’ use of anaesthetic monitoring and telemedicine. It argues that senses and sensors are recursively combined in the moment towards understanding. In this, digital technologies do not present self-evident data, but rather the practitioner must learn to sense the sensors to interpret health and illness. Sensory work (of both the senses and sensors) is not dispensable or entirely delegable because it is intimately entwined with sensemaking. The significance of sensory work to sensemaking reinforces the importance of its consideration in digital health sociotechnical assemblages.

Introduction

The face of modern medicine is being radically transformed in response to developments in digital health technologies coupled with the broader epistemic shifts that surround them. Some digital technologies and their effects on healthcare have been rendered less visible through the passage of time, such as monitoring technologies in hospitals and their operating theatres that have been seamlessly integrated into medical and surgical work.^1–3 Digital health technologies are becoming increasingly portable and are leaving hospitals and healthcare practitioners into the hands of lay users for self-monitoring and self-care.^4,5 Models of health care such as telemedicine also make use of digital health technologies and raise a host of questions about medical expertise and care.^6–10 These digital health technologies tend to be surrounded by techno-utopian discourses. ⁴ However, critical scholarship encourages deeper inquiry into the ways in which they are integrated into medical work and possible limitations and unintended consequences.

Sensory judgements have always been a part of medical practice, as sensory studies scholars have emphasised.^11–16 The practitioner uses their human senses to gather information for diagnosis, patient monitoring and treatment, and the patient’s body is also interpreted through technological sensors.^{1–3,15,17,18} Both human sensing and technological sensors contribute to sensemaking about health and illness, as the practitioner creates meaning in their sensed experience. We can say that in a contemporary healthcare context there are, in effect, layers of ‘sense’. The notion of ‘data sense’ is a useful theoretical approach to examine the intersection between human senses, technological sensors and sensemaking. ¹⁹ To date, this concept has been used to highlight the ways in which digital data are sensory, in that sensors collect information ‘to know’ about the body and its performance, and this interacts with and can mediate human sensed experience towards a final judgement. ²⁰ The recursive nature of sense is vital here, as sensing and sensemaking can in turn influence sensory responses, which then generate more sensory data and so on. This is not to say that human sensing and technological sensors are equally relied on in all contexts for diagnostic sensemaking. Rather, the continuous interactions between layers of sense direct attention to their co-constitution.

This article presents accounts of the diagnostic processes of an anaesthetist and a general practitioner who uses telemedicine systems. The analysis is situated in socio-materialist accounts of diagnosis with specific attention to the literatures on anaesthetic work and telemedicine. The case of anaesthetic diagnosis focuses on the ways in which senses and sensors are recursively combined in the moment in order to make sense of a patient’s health. This analysis highlights that sociotechnical assemblages introduce their own sensory work, as practitioners sense the sensors. The case of telemedicine sheds light on the necessary learning to sense via the sensors in order that the practitioner can undertake their own sensory work essential to their diagnostic sensemaking. This example also captures the potential for missed information due to the challenge of proximity and the limits of the technology.

Senses and sensors in diagnosis

In recent decades the expertise literature has moved from conceptualising decisions as rational and based on logical knowledge and ability to intuitive competence. ²¹ Experts are responsive to contexts and situations; they take in their worlds and act accordingly. ²² This taking in of the world includes a sensing of the world, as has been a finding in areas as diverse as sea navigation, firefighting and medical practice.^18,23,24 At least for the Global North, digital technologies form part of everyday life to the extent that we are, as Haraway claimed, ‘hybrids of machine and organism’ with heightened connection to our tools. ²⁵ In this context, judgments are not only the product of conceptual ideas and embodied intuitions within the minds of individuals, but rather they are the result of delicate negotiations between the ‘intellectual and embodied, collaborative and distributed, and ever more deeply entangled with technologies’. ²⁶ One consequence of this entanglement is that perception is always embodied but it is also made ‘more’ through the ‘extrasomatic resources’ of technologies such as those in imaging, monitoring and communications that allow patients to be sensed and made sense of in additional ways. ²⁷ A second consequence is that diagnostic work requires a learning to engage with these technologies, ²⁸ which are ultimately developed with certain assumptions about the nature of the work that they support and in this may be more or less successful. ²⁶

These entanglements of human bodies and technologies are evident in sociologies of diagnosis which have shown how diagnosis is not the product of an isolated event of rational decision-making, but rather can be a continuous monitoring of the patient’s condition and adjustment to care.^1–3,18 In this diagnostic process there is often a back-and-forth between human sensing and a sensing of sensors. Goodwin captures how an anaesthetist continuously checks that a patient’s pulmonary fibrosis is controlled whilst under the anaesthetic via clinical observations and monitoring technologies, subtly adjusting practice as needed. ¹ Schubert similarly examines the continuous diagnostic practices of anaesthetists, for whom different tools become part of embodied perceptual habits that support their evaluation of patients’ depth and tolerance of anaesthesia. Patient monitoring occurs via sensed observation: ‘Red lips indicate sufficient blood oxygenation; blue lips indicate a lack of oxygen’. ³ These perceptions are also supported by digital sensors where ‘pulse oximeters measure and report the percentage of oxygen saturation in the blood’. ³ Schubert theorises these layers of sense not as independent, but as co-constitutive. In relation to the stethoscope, he makes the observation that:

One could argue that it is actually not so much the stethoscope itself, but the “trained ear” of the doctor which constitutes the diagnostic instrument, the stethoscope being a mere extension of the ear. But this would fall short of the delicate interrelations between the tool and the body. Instead, the physician and the stethoscope are mutually configured in the practice of mediated auscultation. ³

The same observation could be made of use of digital health technologies, a line of argument I will go on to present here. Attempts to separate digital sensors or human sensing (either of a patient or a sensor) from sensemaking are misguided, as diagnosis is not based on isolated pieces of evidence but on layers of sense that are emergent, contingent, continuing and recursive in nature.

These layers of sense are not without their politics. Schubert found that anaesthetists frame their monitoring choices in terms of levels of expertise, with more expert practitioners stating that they rely on their sensory skills such as the feel of the anaesthetic bag during manual ventilation in their assessment of patients. ³ This politics around the senses, technologies and diagnosis is also visible in translations of sensory clinical judgments into test-based evidence. Goodwin analyses a diagnosis of an abdominal aortic aneurysm via the senses. The patient’s body is ‘positioned for perception’, the abdomen is seen, it is felt (palpated), and this skilled touch reveals ‘what the eye cannot know’. ¹ However, this sensory diagnosis is inadequate on its own. Goodwin also finds that after this initial diagnostic event tests like ultrasound are used to decide a course of treatment. Gardner and Williams similarly find that in diagnosis ‘momentary affects’ are registered by the senses of the clinician: ‘A successful diagnosis depends upon … their ability to perceive and register contrasts that many other bodies would be insensitive to’. ¹⁷ While these momentary affects are an important component of the process of diagnosis, they must be captured and translated into a more ‘durable’ form such as that produced by digital sensors. This politics is significant because it highlights assumptions about medical work and the role of digital technologies. In particular, with an emphasis more on rational decision-making procedures and test-based evidence there is the potential to undervalue the embodied, sensory work of diagnosis. ¹⁵

This issue of the interdependence of sensing, sensors and sensemaking becomes particularly critical when we move towards models of healthcare such as telemedicine in which a full suite of sensed and sensored information is no longer available. Telemedicine covers a range of technologies that facilitate ‘virtual medical encounters’ over a network of distributed locations and actors, ⁷ and features ‘remote doctors’ and ‘absent patients’. ²⁹ Scholars have emphasised how this sociotechnical assemblage changes the nature of the healthcare experience and requires new responsibilities and competences.^6–10,30,31 Oudshoorn reveals that telemedicine relies on ‘invisible work’, including ‘articulation work’ (work that supports the more visible patient-doctor interaction), ‘inclusion work’ (work done to bring people into new telemedical systems and turn them into effective users) and ‘affective work’ (emotional support to create ‘intimacy at a distance’). ⁷ A fourth variety of invisible work that is hinted at in Oudshoorn’s analysis is that of sensory work. Where clinical skills are the ‘cornerstone’ of traditional modes of medical practice, these clinical skills are not available in telemedicine. ³² Oudshoorn observes that different forms of information then become important: in the absence of visual clues, the auditory becomes critical, as well as communicative skills and intuition. In this, diagnosis involves increased delegation of work: patients are required to ‘inspect’ their own bodies; nurses who may be physically present with the patient while the doctor is communicating remotely may also be required to engage in the kind of sensory assessments usually performed by doctors. She notes, ‘For those physical indications of poor health nurses cannot see, they will ask specific questions such as the colour of skin, whether patients feel tired, or whether they sweat. Or as the physician told us: “What I cannot see, I have to ask”’. ⁸ The analysis in this article adds that this new sensory work is also demanded of doctors themselves, as they work to sense the sensors.

The existing scholarship directs attention to the new forms of work and care that emerge in digital health as well as the entanglements of technologies and embodied ways of knowing. To build on this scholarship, this article fixes focus on the sensory work of diagnosis with close attention to the recursive processes of sensemaking and the new sensing that emerges to engage with digital sensors. It asks: How do digital sensors change the nature of medical work and what forms of work emerge as a result? In what ways is knowledge of health and illness via human senses and digital sensors recursive? Are there opportunities to better consider and facilitate sensing within digital health sociotechnical assemblages?

Methods

The cases examined in this article were selected because they include accounts of diagnostic processes that prominently feature forms of digital health, telemedicine and digitised monitoring devices, ³³ and sensory work in both human sensing of patients and sensing of sensors. The peculiarities of the two cases allows for analysis of aspects of the entanglements of sensing-sensor-sensemaking assemblages not perceivable through consideration of only one context and in this way facilitates deeper understanding. The diagnostic process of the anaesthetist reveals the recursive nature of sense as his long term use of monitoring technologies means that they are seamlessly integrated into practice. For the general practitioner, diagnosis via videoconference represents a fundamental change to his diagnostic process and is relatively recent (he has been seeing patients in this way for only a few years). As such, this second case furthers the analysis by shedding light on constitution of new sensory knowledges to make sense via the technologies. This second case also highlights the importance of technologies developed in response to practitioner ways of working, so that the layers of sense do not fail.

These cases are taken from a study on the senses in occupational contexts conducted over four years with ninety-two participants in Australia and New Zealand. Fifteen of these participants were doctors from different specialisations and all had at least twenty years of professional practice in addition to their years in training and residency. Potential participants were identified based on their role and level of experience. They were then sent a letter inviting them to participate and most participants were recruited this way. There was also an element of snowballing in the recruitment. Participants came from one major city and three regional centres across Australia and New Zealand. There were fourteen male participants and one female.

Examining the layers of sense in diagnosis raises the methodological challenge of making this tacit knowledge visible. The data that I engage with in this article is taken from semi-structured interviews. To elicit tacit knowledge through interview as opposed to observations of practice the study borrowed from Klein’s critical decision method. This interview technique has been developed to facilitate discussion of sources of knowledge and sensemaking processes that are unconscious or unspoken among experts in areas including healthcare. The technique focuses on situations that are challenging or non-routine and the given example is worked through multiple times to draw out the reasons for one course of action over another. ²⁴ In the study reported on here, participants were encouraged to share illustrative examples of their decision-making that involved their senses. Examples were worked over repeatedly, drawing out tacit assumptions and practices in each telling. The examples were also examined via probing and ‘reverse’ questions, such as ‘how did you know …’ or ‘if someone was not a cardiologist, what diagnostic mistakes could be made in this situation?’ Given the place of technology in contemporary practice, these discussions featured technological sensors as well as human senses.

These interviews took place most often in doctors’ consulting rooms as part of their workday, which was a practical strategy but is also significant from the perspective that the interviews took place while the doctors were in ‘work mode’. They lasted between 45 and 240 minutes, and were recorded and later transcribed with the consent of participants. Interviews were thematically analysed.

Recursive sense in the operating theatre

In an operating theatre, an anaesthetist’s role is to monitor the patient’s condition, not only in terms of their anaesthetic, but also in terms of pain management, airways, circulation and advanced life support. Previous sociological research into anaesthetists’ diagnostic practices discussed earlier captures that anaesthetic work involves both human sensing and technological sensors, which are conceptualised as co-constitutive,^1,3 though analysis focuses on choices to emphasise one sense channel over another in different contexts. The ways in which the senses and sensors interact in the process of sensemaking is not exhaustively explored. To contribute to these scholarly conversations this article presents another example that allows for analysis of the ways in which diagnosis is recursive. The analysis focuses on the diagnostic experience of Matthew, who qualified as an anaesthetist in 1971. He works in a regional Australian centre in a medium size private hospital. We see how sensed and sensored are not distinct modes that Matthew chooses from, but rather they work together seamlessly in his anaesthetic practice.

Matthew first meets his patient in a preadmission clinic or on the ward and will reassess a patient in the anaesthetic bay. He works quickly to take the patient history and conduct a physical examination. He needs to get a ‘general feeling’ for their health but focuses specifically on cardiorespiratory health as well as other factors such as patient anxiety that may impact on anaesthetic safety. Developing a sense of patient health at this point occurs via his human senses and ‘quizzing’ the patient. He reflects: ‘You will get a bit of an idea just by looking at them and what they say’. This human sensing is diagnostic, but he also works to put the patient at ease. He first verifies that he is with the correct patient by asking their name and their operation, and this conversation has the dual purpose of giving him an idea of ‘how you feel about it, and how much of the interview I need to dedicate to trying to allay your anxiety’. Assessing a patient’s anxiety is a matter of how they look, how they sound and how they feel. He pays attention to tone of voice: ‘They will either sound relaxed or very anxious. They may come out and say straight out that they are terrified’. He lays a hand on them, which the patient reads as being ‘kind and gentle’ but allows him to assess whether they feel anxious or calm. He goes on to ask them about their degree of physical fitness: ‘Can they walk up a hill without getting dreadfully breathless, or one flight of stairs, or many flights? What do you normally do in the way of exercise?’ In the same touch of the arm, he feels their pulse, the texture of their skin, he visually pays attention to skin colour: ‘You can start to assess things like whether they are probably anaemic or whether they have a normal haemoglobin, are they cold and sweaty or warm and dry, is their pulse strong, regular and a normal rate?… even a bit about their blood pressure and the state of their vessels. Are they a little blue and cyanosed or a little breathless? That lets you know if you have a significant respiratory problem to sort out’.

Matthew’s sensing of the patient continues with the aid of a stethoscope. He has ‘the most cursory listen’ to a fit person’s chest to confirm there is no abnormality, taking the time for finer detail if the patient has a history of heart and lung issues. Making sense of these auscultated sounds is a matter of embodiment, interaction, language with which to articulate and know certain sounds and this knowing is fundamentally entangled with the tool itself. ¹⁴

Matthew listens and ‘keeps an eye’ on the patient throughout the operation. This includes human sensing of the patient’s body. However, much of his perception of patients occurs through observation and listening to monitoring devices, and in this his sensing of the patient is a sensing of digital auditory and visual signals. The aural outputs from sensors are particularly useful because they fill space in a way that the visual does not, meaning that sensors can be sensed while he is doing other things. In both cases, there is a continuous, tacit collecting of information with a connection between sensing (both of the patient and of the sensors) and understanding. He explains:

There are normal sounds that are going all of the time. For example, you hear the pulse, whether it is the ECG or the gadget attached to the finger that measures oxygen concentration and the flow of blood through the finger. These things beat away and they become a subliminal sound that everything is alright …. So the presence or the absence of a sound is often a signal as to whether everything is okay or not. One of the most important things with an anaesthetic is making sure that the patient is oxygenated; that they are respirated and have adequate circulation to carry the oxygen around the body. The machines will give a tone that is graduated to the actual oxygen content of the body. When the oxygen is at 100% there is a certain pitch of tone and if the oxygen content falls the tone falls too. I remember having a very clever young student with me one day, and he was a musician as well, and he could tell the actual note on the musical scale that he was listening to. Equally, he could turn away from the machine and he could tell the oxygen saturation of the patient on the basis of the note that he could hear. I was very impressed. Not being a musician, I can tell you whether it is high or low, but I can’t tell you what percent it is. I would be pretty confident I could tell whether it was above or below 90 percent, and I can tell when it drops a percent or two by the change in pitch, but I couldn’t tell you if I was listening to a constant pitch whether it was 92 percent or 94 percent.

This account is revealing in multiple ways. First, Matthew’s aural engagement with the monitoring device is central to his continuous sensemaking regarding the patient’s level of oxygen saturation. Second, while Matthew’s awareness and sensemaking of the beeps is ‘subliminal’, if the sounds change or cease, they are bought to his attention. Matthew reflected that anaesthetists rely on noticing these changes and are trained to observe them. Third, the sensing of these sensors is interpretive and it is acquired. We are alerted to these aspects of the sensory work of using monitoring technologies through Matthew’s connection between sensing the sensors and the perceived benefits of musical training. Equally, Matthew’s own capacity to aurally understand oxygen saturation is the product of many years of work with these devices. Fourth, this sensing of sensors is but a first step in understanding the patient’s condition. As Matthew goes on to explain: ‘When you have an alarm on one of your monitoring systems it doesn’t necessarily tell you what has caused the alarm. The patient is hypoxic, but there could be many reasons for this. So you are looking for a pattern and a hierarchy of causes.’

To find the cause of the alarm, Matthew looks to outputs from the monitoring technology and uses his human senses to continuously and concurrently interpret both the physical patient and the patient’s data as produced by these monitoring technologies. Both work together to inform his course of action, as we see in the following example of assessing patient ventilation. A capacity to hear breath sounds in the lung through a stethoscope is critical, and can be challenging in some patients due to the thickness of their chest wall. Matthew is listening ‘to know whether a patient is breathing properly, or if we have taken over their breathing, whether we are adequately ventilating them’. Where there is a trouble with the latter, this can present a dramatic problem (for instance, where their endotracheal tube for ventilation is misplaced). Matthew explains:

It could have gone into their oesophagus instead of into their trachea, then you are ventilating their stomach instead of their lungs …. So you must check that the tube is in the right place …. Yet, it is not fool proof, and there are been many inquests and tragic deaths where an anaesthetist has thought that their tube was in place because he listened and thought he heard breath sounds, but in fact he hadn’t or couldn’t have, because the tube had subsequently been found to be misplaced …. Breath sounds are very faint in a large person with a thick chest wall, and so eliciting normal breath sounds in someone who may be becoming very cyanosed, very blue, and hypoxic, is not easy, and you don’t know whether you just can’t hear them or they are just not there because the tube is misplaced …. Indecision at a time like that has led to anaesthetic deaths, so it is one of the vital parts where your own perceptions may or may not be adequate at the time, whether that is due to problems with the perceiver or problems with the source of the sounds.

Given this risk, Matthew draws on his theoretical knowledge and his clinical skills extended through tools from the stethoscope to monitoring technologies which facilitate access to further evidence for his diagnostic sensemaking. In this, knowing what could possibly happen with the patient’s ventilation and the sensory cues of different scenarios is essential. Some cues are heard through a stethoscope, as we have already seen, and others are a matter of hearing and reading the CO₂ (carbon dioxide) trace:

We have ways of measuring oxygen in the blood, and if a patient remains hypoxic then you have to do something. If you are ventilating them with an endotracheal tube, you will see a change in their CO₂ levels, so one of the things that students are taught to do now is to look at that CO₂ trace, and if there is none there, then almost certainly the tube is not in the lungs, because you would be getting CO₂ if you blew some gas in and got some back. So the absence of the CO₂ most likely means that the tube is in the oesophagus, but it is not 100 percent certain. If they had had a can of Coke there could be some CO₂ coming from their stomach and you could be misled, but that is a rather silly example and unlikely, but of course there is always a silly exception. But the technology is a big help when you can’t tell with your own ears.

The technological sensors extend perception substantially. However, they are not foolproof, just as listening to a patient’s chest gives an incomplete picture. Equally, the embodied and the technological are not discreet ways of knowing: breath sounds are heard via a stethoscope, a technology taken for granted to the extent that this goes unspoken; engagement with the CO₂ trace is aural and visual. There is also conspicuous interpretive work occurring through engagement with the patient’s body and monitoring technologies, drawing attention to the application of expertise in assessing their meaning. Understanding is then recursive. Matthew senses the patient, the sensors sense, he senses the sensors, these feed into sensemaking about the possible meanings of these indicators, and this sensing, sensoring and sensemaking necessarily continues in this moment of diagnosis as it also does throughout the procedure.

Learnings and limits of sense in telemedicine

We now look at an example of diagnosis via telemedicine to further explore these layers of sense in digital health. Telemedicine represents a fundamentally different mode of healthcare, rather than simply being a replica of conventional healthcare provision at a distance.^6–8,30 However, as critical scholarship has highlighted, these fundamental differences are often overlooked in medical and healthcare delivery literatures and, in this, certain assumptions have been made about clinical sensemaking and the capacity for telemedicine systems to support healthcare provision. Previous scholarship has revealed the invisible work of telemedicine as well as the reshaping of care.^6–10,29 The sensory work of telemedicine has been an emergent theme,^7,8 and the example presented here aims to contribute to the existing analysis through focus on this sensory work in diagnosing via videoconference. Malcolm has been a general practice specialist since 1984 and currently works in a regional Australian centre in both private consulting rooms and in the public health care system as a specialist in drug and alcohol care. The case highlights the new forms of sensory work involved in diagnosis, as Malcolm is challenged with needing to develop new sensory knowledge to sense the sensors for diagnostic sensemaking. It also captures limitations of the technology for sensory work, as other scholars have drawn attention to.^6–8

Malcolm uses telemedicine in his role as a drug and alcohol specialist. In addition to providing health services to people in the town in which he is physically based, he also provides his services to patients in smaller towns in the surrounding region. Services to patients in smaller towns involve periodic ‘in the flesh’ consultations, but mostly patients are seen via videoconference, with a nurse at the patient’s end. When in the room with a patient, which, if local, can be in a hospital context, Malcolm’s consultation follows a ‘standard’ routine. He starts with a patient history before moving to a physical examination, of which he says he ‘starts from a distance and works in’. He continues: ‘You look, then you listen, touch and go from there. There are various ways to do it, like the side of the bed that you stand on, where you listen for heart sounds, where you listen for breath sounds, what you listen to when you listen to a stomach, if you are looking for arterial sounds’. The significance of his human senses in his sensemaking is evident in this process, as is the spatial. He approaches the patient physically in a particular way, down to where he stands. For local patients, assessing suitability for the drug and alcohol program and review clinics continue to be conducted face-to-face and Malcolm’s assessments often rely on his ‘intuitions’ about the interaction which, on deeper inquiry, are a matter of sensed pattern recognition.

For patients in other towns in the region, this assessment and review clinic are primarily conducted via videoconference. Malcolm starts by telling me about the challenges of these systems due to missing information. Because he has also seen these patients in the flesh, he knows how much information he misses via the videoconference. While the technology transmits a picture, it introduces changes to the visual experience in terms of level of detail and capacity for eye contact. Limited bandwidth makes video resolution less than ideal. Malcolm explains that, with the limited resolution, skin textures and colours cannot be perceived accurately. Equally, ‘the rate of change for facial expression is inadequate, and you can’t see pupillary response very well’. This change to visual information is problematic because ‘all of these subtle things that you don’t know you’re actually looking at [when diagnosing] can’t be accessed’. The interview works to unpack how this is managed. This discussion reveals that, since using telemedicine, Malcolm has needed to develop new sensory knowledge in order to sense the sensors. He is conscious that he has to work to see parts of the patient not transmitted, which he likens to DVD up-scaling. He says: ‘You try and upgrade to high definition by sort of filling in the gaps for this person, so when you’re talking to them, you can get an increased resolution of the person because you have seen them before.’ This is like talking to a friend on the telephone and being able to ‘see’ them. In this way, his human senses and sensemaking as engaged in the flesh feed into his sensing of the sensors for sensemaking via videoconference. This new sensory work to make the technology work represents an example of the reshaping of care. ¹⁰ Is this mode of care as appropriate for diagnosis of patients with drug and alcohol issues? For Malcolm, it is ‘better than nothing’. He cannot assume that his patient, like his friend, is the same as when he saw them last. And yet in the videoconference, subtle visual indicators for diagnosis need to be speculated on, rather than observed. That he needs to fill in the gaps says that he needs this information. However, this filling in raises questions about the accuracy of diagnostic sensemaking based on sensory information that is partially imagined.

In part, the limitation here is with the technology which will no doubt be improved over time. An aspect of the diagnosis challenged via telemedicine that cannot be addressed through the technology alone is with eye contact. Malcolm reflects that the doctor–patient relationship is essential to diagnosis and care, and in telemedicine patients ‘don’t relate to you’. This finding runs counter to previous studies that have found that relationships can be successfully developed between operator and patient, albeit in a different form.^9,10 In Malcolm’s experience, often the nurse and patient who are together in the same room ‘will have a conversation as if you’re not there, so you’re really not part of a three-way dialogue’. Eye contact would normally be part of his strategy in development of a doctor-patient relationship, but despite the transmission of a picture, he cannot ‘see’. He explains:

With what I do you need the subtleties. They can look at you on the screen, but you can’t get that, whatever the experience is. When someone looks at your eyes, they are actually looking at your iris, but if they are making eye contact, they are looking just behind, almost as if they are looking at your retina.

He envisions possible ways to negotiate this challenge and is conscious that ‘news readers look into the camera in a way that makes you feel like you’re looking at them’, but he is also conscious that it ‘is a learnt skill’. This news reader inspired mode of seeing through the camera may change how the patient relates to him, but the patient would also need to learn to look in this way to achieve the experience of eye contact.

The technology also compresses sound. This can mean that attributes of a patient’s voice like an ‘edge’ may not be perceivable. Malcolm describes a woman who uses amphetamines: ‘When you see her in the flesh, there are other tones, often higher tones, that don’t seem to come across in telehealth’. Patients’ vocal tone is identified as including information critical to diagnosis, meaning that a doctor can misjudge the health of a patient if this information is compromised. Equally, the correction of sound levels used within videoconferencing technology (which aims to ensure a consistent, easily perceivable audio output), also has the impact of excluding certain critical information. Malcolm suggests the current technology is unsuitable for the kind of diagnostic work he needs to do:

I don’t know how broad the spectrum is, but it is compressed and it knocks out some of the frequencies. Because of the sensitivity of the audio correction sound levels, you miss some little sighs, you miss other little sounds that again, you can pick up when they’re in the room. Again, it is better than nothing, but if someone was to say ‘you have made these decisions about whether someone was suitable to have take-away doses of methadone based on your feeling about how you interacted with them’ and then they go away and overdose, there would be a question about whether you should have relied on that assessment.

As with the visual, Malcolm claims that he cannot ‘listen’ despite transmission of sound. A perhaps more nuanced reading of this is to say that he can and does use his hearing to sense the patient via the audio output from the videoconference, this hearing is constituted in relation to his hearing of the patient in the flesh and these sensings and sensorings feed into his diagnostic sensemaking. However, his hearing and sensemaking is changed when compared to his practice in person. The physical presence of the nurse is important with these changes to sense and concerns over the accuracy of diagnoses. As Oudshoorn found, some sensory work is delegated. ⁷ He says, the nurse ‘can say when the patient is gone whether he was really twitchy, or looked really crook, or he was uncomfortable, stuff that I wouldn’t pick up on’. The idea here is that Malcolm still has ‘eyes in the room’, but he acknowledges that the nurse does not have his ‘eyes and ears’; they do not share the same sensory expertise.

He reinforces the significance of being able to exercise his senses (including via sensors) for sensemaking through an example of a patient admitted to the ward with delirium. At the time of presentation, the patient had also recently consumed alcohol and benzodiazepines and so his condition had been classified as a ‘drug and alcohol problem’. A specialised nurse and a resident in drug and alcohol went to assess the patient and could not reach a conclusion about the cause of the patient’s condition. Malcolm assesses the patient himself and on further investigation the patient is diagnosed with pneumonia. He says ‘We couldn’t say it was pneumonia, but… the pattern matching wasn’t quite right for us to say it was alcohol or benzodiazepine withdrawal’. Without his clinical assessment of the patient in which his senses and sensemaking are intimately intertwined, the patient could not be correctly diagnosed or treated.

As a result, he feels more positive about advancements to this form of digital health to better support his layers of sense over reliance on the assessment of other health practitioners on the ground. This perspective is important because it suggests that there is not an inherent issue in his mind with assessing patients via a sensing of sensors. Rather, the technology needs to be appropriate and, as previously pointed out, this sensing of sensors demands new forms of sensory knowledge. As Wears and Berg have argued, failures in healthcare information technology are most productively viewed as demonstrating ‘“not developing the right systems” due to widespread but misleading theories about both technology and clinical work’. ³⁴ There are good reasons why this healthcare mode is being embraced, such as the ability for services to be given when none would be available otherwise, but the implications of its use and possible strategies to increase its effectiveness deserve further attention.

Conclusion

Through the case studies in this article, we see the ways in which human senses and technological sensors interact and work together in sensemaking. A critical aspect of the sensing–sensor–sensemaking assemblage is its recursive nature. Making sense is a continuous process in which there is back and forth between the layers of understanding gleaned from direct human sensing and sensing of sensors. The sensemaking from sensing informs the sense that is made of future sensory experience and so on. Imagine the potential for alternate sensemakings about the sensory patterns for assessing ventilation. Knowing that it can be difficult to hear lung sounds on a patient with thick chest walls could result in an interpretation that the patient is ventilated and the sounds can simply not be heard due to the form of the patient’s body. Alternatively, a pre-existing knowledge that their endotracheal tube can be misplaced and this can also result in the absence of breath sounds leads to this potential sense of the situation. In engaging with outputs from technological sensors there is a similar back-and-forth between what is sensed and prior knowledge of the possibilities and their sensory and sensored presentation. There is not always a clear cut answer. Rather, judgments are arrived at through the lenses of prior sensed experience, knowledge of the patient and their life-world and technical knowledge of where human senses and technological sensors can be misleading. Through each encounter there is, ideally, an expansion of sensory knowledge and an expansion of possibilities for sensemaking.

New entanglements with digital technologies introduce ‘extrasomatic resources’ that can be woven into a practitioner’s perception of phenomena, as Latour has argued. ²⁷ However, this weaving is dependent on development of new forms of sensory knowledge and ways of sensemaking in order to sense and make sense of the sensors. We see this need for learning in the case of telemedicine, where ‘seeing’ a patient is not solved through transmission of a picture alone, but requires that the practitioner works to see through the technology. The necessary interpretation of this visual data draws on ‘in the flesh’ encounters and a good dose of imagination. Eye contact is understood as a needing to learn to look at the eye through the camera. In this, sensors do not present a self-evident reflection of reality but instead require sensed interpretive work and action.^35–37

Where previous scholars have captured practices of delegation in telemedicine where information is otherwise unavailable, this article has demonstrated the continuing importance of the practitioner’s own sensory work. This sensory work on the part of the practitioner is vital because sensing and sensors become part of sensemaking, and without this sensing and sensoring there is heightened potential for diagnostic errors. Digital technologies can radically transform care for the better to the extent many areas of medical practice are almost unimaginable without them. However, it is critical that these technologies are developed and implemented in conversation with practitioner ways of working. The manifestation of telemedicine as it is practiced by the drug and alcohol specialist currently falls short of this goal. This does not imply that sensing through sensors is inherently problematic for sensemaking. Rather, that the practice of sensory work in diagnosis needs greater consideration.