Usability Assessment of OpenEMR: Enhancing Healthcare Interactions

Abstract

Electronic medical record (EMR) systems are pivotal in modern healthcare for digitizing patient records and improving clinical workflows. However, persistent usability challenges hinder their effective adoption and user satisfaction. This study employs a comprehensive user-centered design approach to identify and address usability issues in one such EHR, an open-source system named openEMR. Our main objective was to assess openEMR’s usability through several usability metrics. In addition to a summative usability test with 10 participants, we employed usability inspection (i.e., a heuristic evaluation) to generate additional findings using multiple methods. The results from the summative usability test provide a comprehensive perspective on openEMR’s performance. In addition to task-specific observations, participants offered insights into openEMR’s overall usability through a debrief interview. Through quantitative surveys, usability testing, qualitative interviews, and a heuristic evaluation, this study identified critical usability issues, proposes targeted solutions, and discusses implications for enhancing EMR systems in healthcare settings.

Introduction

Electronic medical record (EMR), or electronic health record (EHR), systems revolutionize healthcare delivery by centralizing patient data and streamlining clinical operations. Despite their transformative potential, usability shortcomings often impede their integration into clinical workflows, affecting user satisfaction and system efficiency. Our objective was to assess, openEMR (OpenEMR Foundation, Inc., 2024), as a widely adopted open-source EMR solution, for potential usability challenges that impact its effectiveness in diverse healthcare documentation and ordering tasks.

To enhance the usability of OpenEMR and in turn potentially inform the design of other EMR systems, this study adopts a user-centered design (UCD) methodology. By focusing on the needs, behaviors, and preferences of healthcare professionals, UCD aims to optimize system usability through iterative design improvements. This approach ensures that user perspectives guide the enhancement process, thereby aligning technological advancements with real-world clinical requirements.

Methods

In this study, we investigated the impact of an EMR systems on healthcare provider efficiency and patient satisfaction. Our research was conducted under Institutional Review Board (IRB) approval (IRB # 23.0544), focusing on evaluating openEMR. This study employed a mixed-methods approach integrating quantitative surveys, usability testing, and qualitative interviews to comprehensively assess and improve openEMR’s usability. Each of these methods are described in detail:

1. Quantitative surveys: The Computer System Usability Questionnaire (CSUQ; Lewis, 1995) was administered to participants (N = 10) to assess overall satisfaction, system usefulness, information quality, and interface quality. Responses were analyzed using mean scores and standard deviations to gauge user perceptions of openEMR’s usability.

2. Usability testing: Usability testing sessions were conducted with the same participants (N = 10) to observe user interactions with openEMR. Tasks focused on common and diverse workflows such as patient data entry, prescription management, and clinical note documentation. Performance metrics and user feedback were collected to identify usability issues and areas for improvement.

3. Qualitative interviews: In-depth interviews were conducted with every participant to gather qualitative insights into usability challenges and user expectations. Thematic analysis of interview transcripts provided nuanced understanding of user experiences and informed targeted design interventions.

4. Heuristic evaluation: We employed usability inspection (i.e., a heuristic evaluation) to generate additional findings. We used Nielsen’s usability heuristics (Nielsen, 1994) to identify violations of basic usability design principles.

By triangulating quantitative data from surveys, observational insights from usability testing, qualitative feedback from interviews, and heuristic evaluation, this study offers a comprehensive assessment of openEMR’s usability landscape. Findings from each methodological approach converge to identify critical usability issues, propose evidence-based solutions, and advocate for user-centered design principles in EMR system development.

Results

The results from the summative usability test are shown in Tables 1–3. Findings from the qualitative interview and heuristic evaluation are summarized as well.

Table 1.

Satisfaction Scores from the Computer System Usability Questionnaire (CSUQ), as Rated by Participants (N = 10).

Score	Mean (SD)
Overall satisfaction (items 1–19)	4.7 (1.4)
System usefulness (items 1–8)	4.6 (1.5)
Information quality (items 9–15)	4.5 (1.3)
Interface quality (items 16–18)	5.1 (1.8)

Ratings are derived from 7-point Likert-type scales ranging from 1 = strongly disagree to 7 = strongly agree. Results are shown as Mean (Standard Deviation).

Table 2.

Average Time and Errors for Each Task (N = 10).

Tasks	Time to complete in min—Mean (SD)	Average errors
Task 1 (encounter)	5.1 (2.4)	1.1
Task 2 (allergies)	2.9 (1.6)	0.1
Task 3 (medical issues)	1.2 (0.9)	0.5
Task 4 (prescriptions)	3.8 (1.4)	0.9
Task 5 (patient reminders)	1.4 (1.4)	0
Task 6 (clinical reminders)	1.8 (1.0)	0.1
Task 7 (review of system checks)	5.3 (2.1)	1.8

Table 3.

Usability Issues Identified in OpenEMR Though Usability Testing.

Category	Usability issue	Severity level	Proposed solution
Navigation	Complex menu structure	High	Simplify menu hierarchy and improve labelling
Documentation	Cumbersome data entry	Medium	Implement smart forms and auto-population
Alerts	Ineffective alert notifications	High	Revise alert system with user customization
Integration	Poor interoperability with external systems	Low	Enhance API support and data exchange formats

Average Time Calculation

The average time for each task was calculated by initially converting the time duration from the minute:second format to seconds for each participant’s performance. This conversion ensured uniformity and consistency in numerical representation. After determining the average time in seconds for each task across all participants, the results were converted back to the minute format for readability. This approach accurately reflects the combined performance of all participants. Results are presented as Mean (Standard Deviation).

Error Categorization

Errors were systematically categorized to assess the accuracy of participants’ performance. A participant received 0 errors if they successfully completed a task without any mistakes. If they completed the task correctly but made errors during the process, they were assigned 1 error, indicating the presence of errors in an otherwise correct completion. Participants who required a hint to complete a task but entered the data correctly received 2 errors. However, if they still entered the data incorrectly despite receiving a hint, it resulted in 3 errors, signifying a more substantial level of difficulty or inaccuracy. In cases where a participant didn’t complete a task at all, they received 3 errors, indicating a failure to finish the task. This error categorization system provided a clear, consistent, and fair method for assessing and quantifying participants’ accuracy across various tasks.

Findings from the qualitative interviews are summarized below.

• Question 1: What were your overall impressions of open EMR?

Question 2: Was there any particular task or tasks that you found difficult to do?

• Many participants found the first task (selecting an encounter) challenging, possibly due to initial unfamiliarity with the system.

• Difficulties with tasks like (Review of System Checks and Reminders) suggest that these functionalities need clearer instructions or visual cues to guide users.

• The issues encountered with the prescription-related task imply that the UI needs refinement for better user understanding.

Question 3: Did you find the overall user interface of openEMR intuitive and easy to navigate?

• Participants’ consensus on the user interface being intuitive and easy to navigate is a strong point for openEMR’s usability.

• The appreciation of visual cues like colors and icons underlines the significance of excellent UI design in aiding user comprehension.

Question 4: How do you rate the clarity of task instructions provided during the usability test?

• High ratings for task instruction clarity indicate that participants generally found the instructions easy to follow.

Question 5: Did you encounter any unexpected behaviors or inconsistencies while performing the tasks?

• While few participants reported encountering unexpected behaviors or inconsistencies, the mention of a bug related to tabs (Encounter tab, mainly) not working correctly indicates the need for quality assurance and bug-fixing efforts.

• A lot of comments were made on the placement of the “Save” button at the top instead of the at the bottom (On the Prescription tab) which highlights the importance of consistent and user-friendly UI design in preventing user errors.

Question 6: How did you feel about the time that we gave for each task?

• Participants felt satisfied with the time allotted for each task.

Question 7: Are there any modifications or improvements you’d suggest to enhance the usability of openEMR for these tasks?

• Suggestions for adding more options to dropdown menus and incorporating a search feature reflect a user-driven desire for more efficient data entry processes, which can lead to increased productivity.

• Recommendations for visual enhancements like color-coding and improved clarity reinforce the significance of good UI design in healthcare software.

Question 8: Can you recall any moments during the test where you experienced frustration or difficulty?

• The reported frustration with the first task (Encounter) and the last task (Review of System Checks) indicate potential issue with the platform’s onboarding process or initial interface clarity.

The heuristic evaluation revealed several opportunities for improvement, including:

• UI Consistency and Bug Fixes: Addressing UI bugs, such as text fields turning orange with warning labels even after correct input and irregular placement of “Save” buttons in some tabs, to enhance overall aesthetic and user experience.

• Streamlining Search: Simplifying the patient search process by providing more flexibility and efficiency.

• Enhancing Appointment Feedback: Improving the visibility of the system status for appointments, ensuring users can easily determine whether an appointment has been saved.

• Billing Process Clarity: Making the billing process more understandable and usable, with clear documentation that avoids confusing terminology.

• Error Handling and Recovery: Implementing better error-handling mechanisms and providing solutions to help users recognize, diagnose, and recover from errors.

• Documentation Updates: Keeping documentation up-to-date and enhancing its clarity for users of varying levels of experience.

• User Control and Freedom: Ensuring consistent and intuitive navigation, particularly regarding the “back” button functionality to prevent unintended logouts.

Discussion

Our research methodology was structured and methodical. We developed a comprehensive test plan with specific usability metrics to guide our assessment of openEMR. In conducting the evaluation, we engaged 10 participants in sessions simulating typical tasks performed by physicians using the openEMR system. These sessions were facilitated using the Lookback usability testing platform, allowing us to record and analyze user interactions in detail. By employing a combination of qualitative and quantitative methods, including participant interviews, debrief sessions, and performance metrics analysis, we gained valuable insights into user experiences and system performance. Furthermore, we incorporated heuristic evaluation techniques to assess openEMR’s adherence to established usability principles and guidelines, providing a comprehensive understanding of its usability strengths and areas for improvement. This multi-method approach enabled us to explore the usability of openEMR from various perspectives and gather actionable insights to inform recommendations for enhancing its usability and user experience.

Our exploration into openEMR’s usability unveiled insights that are paramount for enhancing EHR systems. Participants’ experiences unearthed various challenges and highlighted critical areas for improvement, offering implications for the broader landscape of healthcare technology. Through careful analysis of task completion times, error rates, and user satisfaction ratings, we gained a comprehensive understanding of openEMR’s performance metrics. While participants generally found the system’s user interface intuitive, effective, and efficient, some challenges emerged. Notably, tasks such as the review of system checks, encounter, and prescription-related activities revealed areas for improvement in system functionality and user guidance.

In addition to task-specific observations, participants offered insights into openEMR’s overall usability through a debrief interview. Recommendations included enhancing button placement for consistency, adding a search feature for more efficient data access, providing some examples within any feature used, and emphasizing the importance of user training. Moreover, participants shared nuanced insights, revealing challenges in urgent information retrieval, and provided practical suggestions for system refinement.

Furthermore, our study delved into user preferences, revealing patterns that transcend conventional metrics. Participants expressed difficulties in navigating the system during critical scenarios, revealing where openEMR’s usability can be further improved. These findings contribute not only to academic knowledge but also provide actionable insights for enhancing healthcare systems’ efficiency and effectiveness. By identifying both pain points and user preferences, we lay a robust foundation for proposing personalized user interfaces that cater to the unique needs of healthcare professionals, ultimately leading to a substantial improvement in EHR systems’ overall efficiency and user satisfaction.

The findings underscore significant usability challenges in openEMR, affecting user experience and system performance. Navigation complexity and inadequate alert notifications emerged as critical issues requiring immediate attention to enhance system usability and efficiency. The proposed solutions aim to streamline workflows, reduce cognitive load, and improve overall user satisfaction through iterative design enhancements.

Conclusion

The study offers important lessons and implications for improving openEMR and advancing the field of EHRs. By addressing identified pain points and usability challenges based on participant feedback, actionable recommendations are provided to enhance openEMR’s usability and effectiveness. These recommendations aim to streamline workflows, improve data accessibility, and ultimately enhance the quality of healthcare delivery.

Furthermore, the findings emphasize the necessity for ongoing collaboration among usability researchers, healthcare practitioners, and system developers. Such collaboration is crucial for ensuring the design and implementation of highly usable EHR systems that align with healthcare professionals’ workflow requirements and ultimately contribute to improved patient care and outcomes.

In summary, the study’s takeaways underscore the importance of user-centered design in developing intuitive and efficient healthcare systems. By integrating user-centered design principles into EMR design and implementation processes, we can foster a more seamless and effective healthcare ecosystem.