Abstract
Medical devices are essential tools for modern healthcare delivery. However, significant issues can arise if medical devices are designed for ‘work as imagined’ when this is misaligned with ‘work as done’. This problem can be compounded as the details of device design, in terms of usability and the way a device supports or changes working practices, often receives limited attention. The ways devices are designed and used affect patient safety and quality of care: inappropriate design can provoke user error, create system vulnerabilities and divert attention from other aspects of patient care. Current regulation involves a series of pre-market checks relating to device usability, but this assumes that devices are always used under the conditions and for the purposes intended (i.e. work as imagined); there are many reasons for devices being used in ways other than those assumed at development time. Greater attention needs to be paid to learning points in actual use and user experience (i.e. work as done). This needs to inform manufacturers’ designs, management procurement decisions and local decisions about how devices are used in practice to achieve co-adaptation; without these, we foster risks and inefficiencies in healthcare.
Introduction
There is a growing recognition of the importance of human factors in healthcare delivery, particularly since the publication of the Francis and Berwick Reports.1,2 Human factors take a broad approach to improving the performance of systems of people, processes, products and policies. Thanks to successes in aviation, Crew Resource Management (CRM) has been strongly associated with human factors, which has been highlighted for use in health care. 3 However, human factors cover more than training and teamwork. We aim to raise awareness of another significant aspect of human factors that has received less attention in health care: the design and use of interactive medical devices.
Devices are playing an increasingly important role in the delivery of care, whether in monitoring vital signs, medication administration, analgesia or delivering specialist treatments. However, many interactive medical devices are difficult to use, and the assumptions that were made at the time of design, and on which regulatory checks were based, often sit uncomfortably with actual clinical practice (i.e. work as imagined is different from work as done 4 ). In this article, we highlight the importance of going beyond evaluation of usability to ensuring that device design and use are aligned, so that systems are fit for purpose in practice. Our focus is on interactive devices that are used by people with a variety of backgrounds, such as infusion pumps and glucometers.
Usability: Between user error and device error
Like all tools, medical devices are typically ‘invisible’ except when something goes wrong, most notably in an untoward incident. When an incident involves a device, it is most commonly categorised as either user error or device error. The cause of an incident is normally classed as a device error if there is some fault in hardware or software. If the device performed as designed then user error is assumed: the problem must be something to do with training, inattention, incompetence or failure to read instructions. This simple attribution as either user error or device error fails to raise questions about the suitability of the design itself, and whether design and use are misaligned. Usability and fitness for purpose lie in the interaction between the user, the device and the context of use.
Many interactive medical devices pose usability challenges, and such difficulties can contribute to incidents involving serious harm. 5 For example, Denise Melanson died following an incident in which an infusion pump was programmed to deliver over 4 h medication that should have been delivered over four days. The report on her death 6 includes a brief summary (p. 63) of a subsequent usability test of the infusion pump involved in that incident. That usability test aimed to replicate the situation as closely as possible, and involved qualified nurses. Three of the five made programming errors, and all experienced difficulties interacting with the device. This was not the only factor contributing to the incident, but nevertheless was identified as being significant in this and several other ‘similar’ incidents. The design of interactive medical devices contributes to their ease of use, to the likelihood of people making mistakes with them, and hence to patient safety.
In 2001, a Panel on Transforming Healthcare report to the US President called for “research on user-interface hardware and software to promote the development of better solutions to the problem of human computer interaction in healthcare”. 7 Usability evaluation forms part of the human factors, or usability engineering, process outlined in international standard 62366. 8 Along with Food and Drug Administration (FDA) draft HF guidance 9 and design principles such as HE75, 10 they summarise best practice in “usability engineering”. However, usability engineering as part of pre-market approval is necessary but not sufficient for ensuring fitness for purpose in practice.
Usability in practice
Usability is not an abstract concept independent of context: a device may be poorly designed for the way it is being used. For example, frequently used functions may be difficult to access and less frequently used ones easier, typically because actual use is different to that envisaged by the manufacturer. In a study of the use of infusion devices in an intensive care unit (ICU), Rajkomar and Blandford 11 noted that volume reset was used frequently, but could only be accessed by navigating deep into the menu hierarchy: “Every hour, to record the hourly intake of a drug with the volumetric pump, the nurse needs to access a Status menu from the Main Menu of the pump interface, choose an Intermediate Parameters option, read the volume infused, and then reset the counter to zero. The operation takes 8 key presses”. Such a mismatch between local practices and the design (or configuration) of the medical device may not be inherently error-prone, but it adds to staff workload in an already stressful environment and draws attention away from the core task of patient care.
Another kind of difficulty emerges in the “unremarkable errors” observed by Furniss et al. 12 : these are little, every-day errors that clinicians barely regard as errors because they are minor and are quickly recovered from. However, these are commonly a source of frustration and could escalate if they are not caught; they draw attention away from other aspects of clinical work. For example, on some infusion devices, it is easy to insert the decimal point in the wrong place, which could result in an ‘out by ten’ error 13 : constant vigilance is needed while programming the device to detect and correct any such data entry errors. Each small correction takes little time, but aggregated over thousands of interactions they add up to significant time taken from direct patient care, and to a more significant risk of having errors pass undetected.
One important way in which poor fit becomes apparent is identifying workarounds in use. Workarounds have been widely reported in healthcare, including in the use of devices. For example, Koppel et al. 14 identified various workarounds in the use of a barcode medication administration system, such as placing barcodes on objects in the environment rather than attaching them to the patient, while Furniss et al. 15 noted an ‘official’ workaround that has been established in the use of a particular glucometer: namely the use of 2222 or 9999 as the patient identifier if someone has not yet been allocated a patient number. Workarounds typically arise because, in the here-and-now of work, they offer a timely solution to delivering care, and they are overcoming some mismatch between the design and the needs of use.
Sometimes, devices are used for purposes for which they were never designed. When people cannot find the right tool for a job, they may use another tool to achieve their purpose. When devices are appropriated for uses for which they were not designed, this introduces vulnerabilities into the system. This is illustrated by the case reported by Grissinger 16 of a neonatal baby who received milk intravenously. The baby was being fed through a nasogastric tube, using a syringe driver to regulate the flow of milk; this is not a use for which the device had been designed. Indeed, the baby was also receiving medication intravenously via an identical pump: a use for which it was designed. At some point, “a nurse mistakenly connected a syringe containing breast milk to the wrong line”. 16 Many factors may have contributed to this incident (workload, multi-tasking, availability of equipment, etc.), but the confusability of the pumps and their tubing, and the fact that the pump was being used for a purpose for which it was not designed, were factors that made the mistake more likely and less easy to detect, and the consequences could have been fatal. Consideration needs to be given to whether the tasks that need to be done can be achieved safely using the available tools.
As others 17 have noted, workarounds often become normalized in clinical practice, despite the fact that many of them introduce vulnerabilities into the system by eroding safety mechanisms (e.g. increasing the risk of patient misidentification or confusability of devices, or circumventing safety checks that were designed to catch errors). 18
As Debono et al. 17 note, senior managers are often unaware of workarounds; also, staff themselves typically do not question inefficient interactions and unremarkable errors. These work practices are semi-tacit: people are loosely aware of them if they are pointed out, but would not think to mention them if asked directly about them. Practices and ways of managing them need to be reviewed to ensure that they are not introducing unnecessary vulnerabilities into the system of use. The most effective way of identifying these practices is through observation by an ‘outsider’ for whom these practices are not taken-for-granted knowledge and who is experienced in studying clinical work and people’s interactions with technology.
Aligning device design and use
At development time, manufacturers have to make assumptions about how their devices will be used in order to construct the safety argument that the device is market-ready. However, as illustrated above, actual clinical practice diverges from the assumptions that are made at development time in many ways, and for various reasons. Thus, the basis for the safety argument that was made during pre-market checks may no longer be valid. There is a need to review actual clinical practice and the fitness for purpose of devices in supporting that practice, as a form of post-market evaluation. 15 This would be typical in other industries: for example using a safety management system and safety case documentation. The latter would involve periodic review to assure that safety goals are being achieved. 19
While basic usability can be tested away from the intended context of use (e.g. through the use of expert assessment and user trials), 20 fitness for purpose can only be assessed by understanding clinical needs and practices and checking the quality of fit between devices and practices. The quality of fit can be improved through careful selection of devices to match needs, or by working closely with clinical teams to ensure that practices evolve to match the assumptions embodied within design; typically, good fit may be achieved through a combination of these approaches.
The purchasing process for medical technology represents one opportunity for optimising this fit, by taking account of human factors for equipment and of current clinical practices so as to reduce risk. 21 Good human factors can save money 22 as well as improving safety. 23 However, the story does not end with purchasing. Further decisions are made about how devices are configured and the protocols for clinical use. These decisions should be made (and reviewed) with a view to optimising patient safety and patient care. This may include reviewing how devices are used; how information from devices is recorded in patient notes; under what circumstances devices are set to sound an audible alarm (so as to minimise alarm fatigue and unnecessary disruption to patients); how device maintenance is managed so as to maximise availability; how staff are authorised to use a device; the integration with wider health IT systems; etc.
Fitness should also be monitored as contexts and needs change. Recently issued guidance by the UK MHRA 24 contains the following checks: “were there any problems in using this device which should be noted and could be fixed in the future?”, and “Is the equipment still appropriate in the light of the patient or client’s changing needs?” These checks recognise the need to continually monitor usage, to check for safety and improve the fit to the circumstances in which equipment is being used.
In this article, we have drawn on examples of poor usability and poor fitness for purpose to illustrate the points being made. Each example might be dismissed as a one-off. However, such difficulties are pervasive in modern healthcare, and each compromises patient safety in a small way. By attending to and eliminating difficulties, many small gains that together yield substantial improvements in patient safety can be achieved.
In summary, human factors has an important role to play in device design and use, and usability testing as part of pre-market approval is necessary but not sufficient. The complexity of health service practice makes it impossible to anticipate every possible situation in which devices are used; further, funding limitations in healthcare may make it impossible to design explicitly for every eventuality. Improved safety requires co-adaption of technology, i.e. continual refinement and review, likely to involve those using equipment as well as those supplying it. Usability and fitness for purpose should be a focus during procurement, configuration, deployment and use; patient safety can be improved through regular review of needs, practices and designs to ensure that device design and use is aligned with clinical practice.
