What Did You Order? Developing Models to Measure the Impact of Usability on Emergency Physician Accuracy Using Computerized Provider Order Entry

Abstract

Computerized Provider Order Entry (CPOE) is a critical component of electronic health records (EHR). Although widely adopted, the usability of these systems is not well understood. A cross-sectional usability study was conducted at four healthcare systems using two commercially available EHRs. This research aims to predict the accuracy of an order placed in the EHR by emergency medicine physicians. Physician-EHR interactions were recorded during six simulated task scenarios in a test environment. The data from 222 interactions were used to predict the accuracy of the order placed. Multiple models were developed, and their performance was compared. The Random Forest model had the best performance with F1 score 0.82 and AUC of 0.89. These results demonstrate the utility of using keystroke, time and user demographics as variables in predicting accurate order placement. Applying such models to passively monitor EHR usage in real world clinical environments may help improve patient safety and care.