Lessons Learned from a Comparative High Fidelity Usability Evaluation of Anesthesia Information Systems

Abstract

Comparative high fidelity usability evaluations of health information systems can help inform the selection of products in a hospital's procurement process. This method is well-suited for predicting which systems best match user needs and workflow prior to implementation. Yet, these evaluations are challenging due to the complexity of health information systems, their broad functionality, the large amounts of data they aggregate, and the complex and multiple environments in which they exist. The evaluation of three Anesthesia Information Management Systems highlighted several important methodological issues with this type of assessment. Recommendations discussed for refining this approach, to allow for more targeted findings and efficient use of resources, include emphasizing vendor customization, altering the method of training provided, and limiting the number of participants.