Does Artificial Intelligence Change How We Design User Interfaces? A Case Study from Representatives in the Medical Device Industry

Abstract

Intravascular ultrasound (IVUS) is an imaging technique that allows interventional cardiologists (ICs) to capture and visually display images of blood vessels during Percutaneous Coronary Interventions (PCI). IVUS images allow ICs to: (i) evaluate the type and characteristics of lesions present, (ii) assess the appropriate intervention, (iii) determine the size of the stent or balloon to be used if they were needed, and (iv) examine if the intervention was successful. The size of the stent or balloon is determined by identifying, and then measuring, the diameters of vessel or lumen borders at the treatment locations. Identification of vessel borders is a manual time-consuming process that is rigorous to accomplish. Alternatively, a machine learning model that automatically identifies and measures the lumen and vessel borders of IVUS images was developed. This industry case study shows how the HFE team lead the UI design process to incorporate this automation into the existing UI.

Objectives

Percutaneous Coronary Interventions (PCI) are minimally invasive procedures performed to relieve the narrowing or occlusion of coronary arteries and usually include placing a stent using a catheter. PCIs provide a more efficient and less traumatic way to treat coronary artery diseases, while the medical and industrial communities keep striving for even better ways to improve procedures.

Intravascular ultrasound (IVUS) is an imaging technique that allows interventional cardiologists (ICs) to capture and visually display cross-sectional images of blood vessels during PCIs, which compliments the planar visualization of the vessels using X-ray imaging technology. To do so, the IC inserts a thin flexible catheter over a guidewire (typically in the groin) that has an ultrasonic transducer at the end. Next, they navigate the catheter to the target location and collect IVUS images while performing a catheter pullback either manually or automatically. These images allow ICs to: (i) evaluate the type and characteristics of lesions present, (ii) assess the appropriate intervention, (iii) determine the size of the stent or balloon to be used if they were needed, and (iv) examine if the intervention was successful. The size of the stent or balloon is determined by identifying, and then measuring, the diameters of vessel or lumen borders at the treatment locations.

Presently, the identification of a blood vessel border is a manually time-consuming process that is rigorous to accomplish. On one hand, learning how to interpret the IVUS images can be challenging. On the other hand, manually marking the vessel and/or lumen borders on IVUS requires multiple clicks and adjustments which involves lots of interactions between the IC in the procedure room and the console operator typically sitting in the observation room. These challenges have deterred some ICs from incorporating IVUS into their practice as they prefer to continue to use traditional X-ray imaging by itself.

As an alternative, a machine learning model (Matsumura et al., 2023) that automatically identifies and measures the lumen and vessel borders of IVUS images was developed. This automation eliminates the manual vessel marking process resulting in faster assessments of vessel sizes and provides a more simplified workflow. A human factors engineering team was tasked with leading the UI design process to incorporate this automation into the existing UI.

The objective of this industry case study is to show how the human factors engineering (HFE) team analyzed, researched, designed, and validated an expanded UI design to seamlessly incorporate an AI feature. This work was performed under challenging circumstances as the development timeline was tight, the COVID-19 pandemic restricted opportunity to perform in-person research, and a mandated regulatory process constrained how usability studies were conducted.

Approach

UI development was organized in three stages. In stage one, the team gained extensive understanding of the users and work domain. This was completed by studying the basics of PCIs, studying internal documentation, informally interviewing internal Subject Matter Experts (SMEs), and conducting remote semi-structured interviews with primary users.

During stage two, UI concepts were ideated, tested, and refined. Low fidelity concepts were informally reviewed with SMEs and cross-functional team members. Improved UI concepts were then evaluated in a remote usability study with multiple IVUS primary users from USA, China, and Japan. Participants were asked to determine stent sizing and vessel plaque morphology utilizing an interactive PowerPoint prototype.

In stage three, the task and use error analysis were finalized, and an in-person confirmatory formative evaluation was conducted with the further refined UI. This study included primary and secondary users in a simulated catheter lab environment performing tasks in various use scenarios that included the new AI features and other UI changes.

Finally, the summative evaluation was performed to validate that the UI design and the relevant risk control measures were safe and effective. In total there were 33 testing sessions completed including 18 sessions with interventional cardiologists (primary users) and 15 sessions with catheterization lab nurse/techs (secondary users) across the US. The study was performed in a simulated use environment in three US cities. The data was submitted to support regulatory approval across the geographies.

Findings

Learnings about the users and work domain were synthesized in workflow diagrams, task and user error analysis, and detailed breakdown of the information inputs and outputs used to support clinical decision making with IVUS.

Interviews with users and SMEs revealed an almost universal high-level workflow for both pre and post PCI IVUS runs. Typical IVUS run analysis starts with a qualitative assessment of images followed by optional measurements. Therefore, AI automatic measurements were recommended to only be provided upon user request. Also, participants reported situations where vessel borders were not visible. This resulted in a UI recommendation to indicate when the automatically drawn borders were identified and displayed with uncertainty.

The remote formative evaluation identified further improvements and showed that the proposed UI fit the users’ clinical workflows. If the users decide not to initiate the AI feature for automatic measurements, the system keeps supporting their current workflow. Subsequently, the confirmatory formative evaluation reinforced the acceptability of the user interface design and informed the summative study protocol and design.

The summative evaluation demonstrated that the final product is safe and effective for its intended users, uses, and use environments. A few use errors and use difficulties were identified in the study, but none of the root causes were user interface related.

The entire HFE process was documented as the final HFE report and submitted to the regulatory bodies. The product has been cleared by FDA, EU and Japan regulatory bodies late last year (2023).

Takeaways

This case study demonstrated a successful application of HFE principle and best practices in a medical device industry setting where various constraints were present. As a highly regulated industry, the medical device industry has adapted to understand, learn, and adopt HFE to ensure use-related risks are adequately addressed, controlled, and evaluated.

This case study followed an internal HFE process that was established to comply with FDA guidance and international standards. The effort and scope of the activities carried out were considerations of use-related risks, business and user needs, resources, budget, and timeline. The outcome of this case study shows that, even with the defined constraint challenges, the HFE methodology was applicable to tackle real-life problems and can be used strategically as a tool to deliver safer and more effective patient care. In particular, the fundamental HFE principle of deeply understanding the work domain and the users is still key to enable a successful design even when incorporating emerging technologies such as machine learning and AI. The HFE framework to strive for the best design remains the undeterred.