Sage Journals: Discover world-class research

Abstract

The goal of this qualitative study was to assess physicians’ perceptions around features of key screens within a prominent commercial EHR, and to solicit end-user recommendations for improved retrieval of high-priority clinical information. We conducted a qualitative, descriptive study of 25 physicians in a medical ICU setting. at a tertiary academic medical center. An in-depth, semi-structured interview guide was developed to elicit physician perceptions on information retrieval as well as favorable and unfavorable features of specific EHR screens. Transcripts were independently coded in a qualitative software management tool by at least two trained coders using a common code book. We successfully obtained vendor permission to map physicians perception’s on full Epic© screenshots. Among the 25 physician participants (13 female; 5 attending physicians, 9 fellows, 11 residents), the majority of participants reported experiencing challenges finding clinical information in the EHR. We present the most favorable and unfavorable screen-level features for four central EHR screens: Flowsheet, Notes/Chart Review, Results Review, and Vital Signs. We also compiled participants’ recommendations for a comprehensive EHR dashboard screen to better support clinical workflow and information retrieval in the medical ICU through User-Centered Design. ICU physicians demonstrated a mix of positive and negative attitudes toward specific screen-level features in a major vendor-based EHR system. Physician perceptions of information overload emerged as a theme across multiple EHR screens. Our findings underscore the importance of qualitative research and end-user feedback in EHR software design and interface optimization at both the vendor and institutional level.

Keywords

electronic health records information retrieval user-centered design ICU

Background

Electronic health records (EHRs) have changed the way that medical providers care for their patients. Physicians have expressed growing frustration with the click-heavy, data-busy screens of existing EHRs, in addition to stringent documentation requirements.^1,2 Recent studies that stratified physicians by age showed that older physicians were more dissatisfied with EHRs than younger physicians.^3,4 Scrolling through pages of notes and screens hunting for information may be one source of frustration.^4–7

EHRs give physicians access to more patient data than ever before.⁸ Data representation within current EHRs often falls short of users’ needs, leading to difficulties in pattern recognition. It is reported that 38% of the identified patient safety issues were associated with inconsistency between the informational needs of users and the content displayed by the EHR.⁹ Data representation is a complex issue that requires input from end users in order to have a meaningful effect on EHR usability and decrease the incidence of patient harm related to information overload. Reduction of irrelevant data can prevent information overload and reduce the risk for medical errors.¹⁰

EHR vendors have notoriously been protective of publishing screenshots of their products.¹¹ Cultural and contractual gag-clauses hinder the dissemination of screen shots with academic research, as well as research around data standards in screen-level display across different EHR vendors.^12–14 Prior work has included publication of partial screenshots, such as order set boxes or allergy lists, to examine contextual variation – and the patient safety implications – of various EHR interface arrangements and displays.^15,16 In an unique action, Epic© Systems granted permission to the authors to publish full Epic screenshots to help the goal of this research of improving EHR interface design.

Although the importance of user-centered design is well-recognized, EHR usability research into screen-level factors across multiple high-traffic screens has been limited. One recent ethnographic study examined some system features, but these were mostly restricted to information seeking and clinical documentation.¹⁷ Another multi-specialty survey of 280 physicians demonstrated an association between some EHR design factors and burnout, but focused primarily on other challenges.¹⁸ Given the high prevalence of provider dissatisfaction with EHRs, there is a need to assess physician perceptions around specific features of key EHR screens in the context of comprehensive clinical workflows. Any insights to improve the provider-EHR experience would align with the broader initiative to put “patients over paperwork” which has been championed by federal payers.¹⁹

Objective

The goal of this study was to evaluate features of key screens within a prominent commercial EHR, and to investigate ICU physician recommendations for improved retrieval of high-priority clinical information. This study, for the first time, adds rigor to EHR assessment studies by mapping physicians’ perceptions onto full Epic© screenshots.

Methods

Study setting and participants

This qualitative, descriptive study was conducted at a tertiary academic medical center in the Southeast. Participants were recruited from the Department of Internal Medicine and the Division of Pulmonary and Critical Care Medicine, which provide staffing for a 30-bed medical intensive care unit (MICU). The medical center and MICU care team have used an institutionally-customized implementation of a certified EHR (Epic Systems, Madison, WI) since 2014. Institutional Review Board approval was obtained.

A purposeful sampling technique was used to achieve a breadth of representation of ICU physicians (residents, fellows, attendings).²⁰ Physicians engaged in this study were selected due to their extensive experience in completing EHR tasks in the ICU. ICU physicians were recruited through department emails, and 25 physicians and physician trainees volunteered to participate in the study. Only medical ICU physicians with current EHR experience were included in this study. Interviews were conducted in an EHR usability laboratory equipped with audio and video capturing capabilities technologies.

Data collection procedures

An in-depth, semi-structured interview guide was developed to elicit physician perceptions on locating information on the EHR, benefits and barriers of features to specifics screens within the EHR (Appendix A). The interview guide was informed by a literature review and domain expert feedback; interview questions were then refined with assistance from qualitative research experts at the study institution (Odum Institute for Research in Social Science). Interviews were conducted by one interviewer (SK) in person. SK is a male, PhD holder and at the time of the study was assistant professor with qualitative training on previous research projects. Each interview lasted approximately 10–15 min (range: 5–20 min), and was audio-recorded. Two research assistants were present during the interviews for note-taking. Interview participants were offered a $100 gift card as compensations for their time. Interviews were transcribed verbatim by two members (PF, SR) of the research team. Interview transcriptions were not returned to participants to comment or correct.

Materials

A list of seven semi-structured questions were developed by domain experts in critical care and health informatics to solicit feedback from participants (Table 1). The interview guide was tested in a pilot study including two medical residents at the ICU. There were three potential areas of interest: (1) information retrieval in the EHR, (2) perceived reactions toward key EHR screens: Flowsheet, Vital Signs, Notes/Chart Review, and Results/Lab Review, (3) possible solutions for better information retrieval. This approach to breadth over depth has been used before to describe complex provider and practice-level systems in qualitative research.²¹

Table 1.

Study participants characteristics.

	N	Average age	Average experience with EHR system
	N	Years (SD)	Years (SD)
Total	25	33.2 (6.1)	4.2 (1.3)
Clinical role
Resident	11	29.0 (1.4)	4.0 (0.4)
Fellow	9	32.7 (0.5)	5.7 (0.9)
Attending	5	44.0 (6.5)	3.8 (0.4)
Gender
Female	13	31.5 (3.1)	4.0 (1.0)
Male	12	34.9 (7.6)	4.3 (1.4)

We selected these specific EHR screens (Flowsheet, Vital Signs, Notes/Chart Review, and Results Review) as an area for focused qualitative evaluation after discussion with a clinical subject matter expert as well as our prior findings from a live observational study that these screens, with the exception of “Vital Signs,” are among the top four most frequently visited screens by ICU providers during morning pre-rounds.²²

Analytic approach

Analysis followed an inductive approach to qualitative research and used the individual interview participant as the unit of analysis. Interview transcripts were independently coded in Dedoose©, a qualitative software management tool. All interviews were independently coded by two trained coders using a common code book. The code book was developed based on the research questions and topics from the interview guide and initial readings of transcripts by coders and the research team. Coders (PF, SR) pilot-tested the initial code book by independently coding two provider transcripts and met to discuss and compare their results and fine-tune the code book. A few concept definitions and decision rules were revised, and the enhanced version of the codebook was applied to remaining transcripts. (Final codebook available as Appendix B) Coding discrepancies were reconciled by discussion and consensus.

Coded excerpts were reviewed to identify the co-occurrence of similarities and differences within and across physician responses for a given code. Based on this review, code reports for each code were generated that included code definitions, a description of the emergent themes and sub-themes, and illustrative quotes. Emergent theme identification is an accepted method for organizing and evaluating qualitative data on health systems.²³ Participants did not provide feedback on the findings.

We categorized the emerging themes in each EHR screen using the Nielsen’s Heuristic Evaluation principles.²⁴ The mapping characterizes the positive and negative features in each EHR screen based on widely accepted usability evaluation standards, which add more rigor and generalizability to the study findings.

We mapped the emerging themes from the interview analysis onto Epic® screen shots to visually represent the positive and negative perceptions of physicians around the interface design. We used alphabetical letters to indicate positive perceptions of features, and numerical values to indicate negative perceptions of features. Additionally, we used color coding to show negative perceptions in red, and positive perceptions in green. Approval to publish screenshots of the institutional Epic EHR was approved by Epic® headquarters.

Results

A total of 5 h and 19 min of interviews were audio recorded for this study. Of the 25 physician participants, 13 were females, the mean number of Epic years of experience was 4.2 years, and the estimated average number of hours spent in Epic were 32.6 h per week. The distribution of participants spanned multiple clinical roles: 11 residents, nine fellows, and five attending physicians. We present approved Epic© screenshots in Figures 1 to 3 to provide context around participants perceptions of screen-level features.

Figure 1.

Mapping of physician perceptions on to Epic© ICU Clinical Flowsheet screenshot (Permission to publish obtained).

Figure 2.

Mapping of physician perceptions on to Epic© Notes/Chart Review screenshot (Permission to publish obtained).

Figure 3.

Mapping of physician perceptions on to Epic© Results Review screenshot (Permission to publish obtained).

Flowsheet screen

The overwhelming consensus among interview participants (n = 24 [96%]) was the flowsheet is the most useful and most often accessed EHR screen for clinical workflows in the medical ICU. Favorable features of this screen include: integration of key clinical data elements into a compressive view (such as vital signs, ventilator settings, and intravenous drips) and a tabular interface design that supports visual assessment of trends. Some participants mentioned that due to the large amount of patient information, it is sometimes a burden to sift and scroll through all the information to find what they need. Participants also mentioned that a learning curve comes with utilizing the Flowsheet screen. Table 2 details the features within the Flowsheet screen that participants liked and disliked or wish they could change. Figure 1 maps the features like and disliked on a full Epic© Flowsheet screenshot to illustrate the areas needed of a redesign.

Table 2.

Positive and negative aspects of the ICU clinical flowsheet EHR screen.

Neilsen Heuristic evaluation	EHR usability feature	Features I like
Match between system and the real world	1. Integrates key clinical data	“I’m. . .interested in heart rate, blood pressure, [vasopressors]. . . mechanical ventilation (yes or no). Those are the first things I’m interested in, that tells me this guy’s really sick or not very sick.” – Male Attending“In the Flowsheet, I think the trend of multiple vital signs at the same time is much easier to see. . .so, I think the comprehensiveness of the Flowsheet is what makes it easier for me to use.” – Male Resident
Flexibility and efficiency of use	2. Supports visual assessments of clinical trends over time	“I like having the labs at the bottom of the Flowsheet, so that you could just follow the timing . . .with all the other intervention stuff. So, yes, it makes for a much longer scroll, but at least you could go, okay, 10am, they changed vent settings, and there’s 9am gas that went with that; well, a better example, so, 11am got a bolus of a liter of fluid. What impact did that have on. . .[kidney function] two hours later?” – Male Attending“All of the information is there and I can track it. . .visually I can see trends. . .The big example would be like vital signs: I can see a down trending blood pressure even if it’s a normal value right now. I can see a rising heart rate over the last couple days. I can see average heart rates, and how the current one relates to that.” – Male Resident
	Features I dislike
Aesthetic and minimalist design	A. Extraneous information	“It’s so messy, right? . . . I feel part of the thing is like your eyes are always going to all sorts of different things because there’s a lot of information there . . .” – Female Resident“There’s just some parts . . . that are . . . just excess information that I don’t really need.” – Male Fellow
Match between system and the real world	B. Discordant data; mismatch between EHR presentation and clinical heuristics	“There’s multiple [examples], the vent settings are always represented in multiple places and ways. . .And so, you get all of this extra stuff that I don’t use very frequently.” – Male Resident“The vent settings are not always accurate in there. Nor are they necessarily. . .displayed in the most easy-to-read manner. It would be nice if it was displayed in the manner that we actually see them in when we present them on rounds.” – Female Attending
Error prevention	C. Default settings can mask – or fail to capture – clinically relevant details	“. . .If there’s somebody who has a period of vital sign instability, it’s not always clear just by reviewing the flowsheet because if their heart rate was 80 and then 80 the next hour but in between they had a period. . . where their rate was 180, that’s not always captured in the flowsheet . . . So you do have to revert [to reading] the nursing notes as well to see if there were any issues. . .” – Female Attending“Data that’s pulled at a specific time, so I can’t necessarily see. . . my lowest or my highest values over an hour, and that can be difficult to understand what happened, so it might say that the O2 saturation is 90%, but it doesn’t actually report that the patient dipped to. . .60% at one point.” – Female Fellow
User control and freedom	D. Wish I could customize	“I wish I was able to. . .modify the things that I do want to see. For example, in patients who have frequent . . .neuro checks, sometimes that’s not always on there, you have to look at a separate screen. Patients who are requiring certain respiratory therapy maneuvers. . .I wish I could add more things in there.” – Male Resident“If I want to see multiple lab trends, I can’t do that. I have to look at them individually. For example, one time I wanted to see the creatinine, the [total bilirubin], and . . . the INR. And just those three things. And I wanted to see their trend together, but I couldn’t do that. . . it would pull up all [results] together, but that’s too much information. . . So, I wish you could preselect which labs you wanted to see in the Flowsheet.” – Female Resident

Notes/chart screen

Most participants (n = 18 [72%]) lamented the volume of notes that are generated in current clinical settings but noted the filtering/sorting functionality as the most favorable aspect of this screen. Unfavorable features of this screen included poor default filter settings, non-intuitive design and layout of the filter tool itself, the scrolling burden necessary to navigate this screen, and the lack of transparency around note authorship in some scenarios. We summarized participants’ attitudes toward favorable and unfavorable aspects of the notes/chart review screen, Table 3 and Figure 2.

Table 3.

Positive and negative aspects of the notes/chart review screen.

Nielsen heuristic evaluation	EHR usability feature	Features I like
Flexibility and efficiency of use	1. Filtering and searching	“It’s easy for me to filter the notes through there and see all of their Cardiology notes and all their ambulatory notes.” – Female Resident“They added in little check boxes at the top where you can throw in automatic filters, so I can filter by me or my division. . . It’s much easier to find the notes I’m looking for.” – Male Fellow
Consistency and standards	2. Warehouse	“Chart Review’s very helpful. . . It’s the only place I can reliably find labs in process.” – Male Fellow
	Features I dislike
Aesthetic & minimalist design	A. Poor default filter settings	“It’s confusing. To find what you want is I think really hard. . . There’s like 40 screens and it’s like respiratory therapy, nursing, I mean everybody has left a note; and to try to find things that you want . . .You want the consult notes, you want the H&P. Those are the things that matter.” – Male Attending“I don’t know why it automatically goes to incomplete notes when you start on it. Even if you don’t have any incomplete notes. That just seems silly.” – Male Resident
Flexibility and efficiency of use	B. Filter functionality: hard to use	“But the Chart Review is a hard screen, I think. It’s just like, there’s just a lot of stuff in there, and it’d be nice to filter out. I mean, I know there are the filters. It’d be nice to, I don’t know, to have that be a little more natural. To filter out some of the excess stuff in there.” – Male Resident“And so, it takes a long time to filter. . . If the filtering screen was easier and less slow I would probably use that more to kind of figure out who was saying what.” – Female Resident
Flexibility and efficiency of use	C. Scrolling and clicking burden	“I do not love the Chart Review screen. . . when there’s a lot of notes it can be super overwhelming. Granted you can filter, like I just want to see Endocrine notes. . . [but] it feels like an extra step a lot of the time, versus just. . . scroll, scroll, scroll, scroll, scroll, scroll, scroll to see if I can find [what I’m looking for]. It’s easy to miss stuff in there.” – Male ResidentI get annoyed having to click through, I mean, you know, when I do my chart review, I end up having to click through like, 6 screens, which is a little bit annoying. – Female Fellow
Match between system and real world	D. Authorship: not transparent	“I don’t like that . . . the original author of the note is not easily discernible if they didn’t happen to put a signature line down at the bottom.” – Female Attending“I mean, I think it has to be displayed in chronological order, but it would just be nice if it was more apparent upfront. What note, who it was from.” – Female Fellow
Error prevention	E. Lacks dedicated place to summarize overnight events	“So I think like, if there were some way to capture overnight events into one little screen, again, you know, ventilator changes, pressor changes, so like big medication changes that happened overnight. I think that’s something that also gets missed a lot is like, some things get changed.” – Female Resident“If there was some easy way rather than going through 8 million nursing notes. . . If there was some sort of summary like at 10 pm [the patient] de-saturated, at 10:30 they had to be intubated and just like a really quick, this is the timeline of events. . .because that’s what we get from them verbally.” – Female Fellow

Results review screen

Most participants (n = 21 [84%]) voiced a favorable opinion of the Results Review screen because it consolidates laboratory, imaging, pathology, and other test result data which are organized and accessible from a hierarchical tree system on one screen, Figure 3. Other favorable features of this screen included the presence of a “time-stamp” feature – to allow users to recognize new results – and a flexible layout that supports trend visualization (e.g. chronological or reverse chronological ordering at the discretion of the individual user), Table 4. Among the unfavorable features were the horizontal scrolling burden, the lack of visual clarity around pending laboratory studies, and the cumbersome nature of the embedded filter/graph tool.

Table 4.

Positive and negative aspects of the results review screen.

Neilson Heuristic evaluation	EHR usability feature	Features I like
Recognition rather than recall	1. Ability to time stamp	So, if [a lab test] was [collected] five days ago [but the result comes back today], and you don’t scroll backwards, you won’t see that new lab. . . So I’ve learned to time mark things, and if there’s a new result, then I’ll go to new result, and it actually filters out, so you see, by date, what the new results are. So that really is an important feature that I didn’t know existed before. – Female ResidentI like time marking it and then you can. . . get a result flag on your screen. I don’t know if everyone knows how to do that but it keeps you from going back again and again. – Female Resident
Match between system and real world	2. Supports trend assessment	But I like that you can, sort of, trend things. You can change. . . whether or not the new results are seen on the left side or the right side. . . At least in medicine and in the ICU, [we] are big on trending data. So, it makes it easy to do that. – Male Resident The Results Review screen is great. The reason is if, for patients that you’re first admitting you can see the labs that have been done, and then you can also look back for any other information that’s been in [their] history. . . So you can get a quick feel for the baseline. . . – Male Resident
Consistency & standards	3. Consolidates results of different tests	Yes, I do look at [Results Review] for the labs. . . then often you [can] then click over to look at the x-rays and things like that. . . It is very helpful in that sometimes they’ll have micro [biology] and some of the imaging [there also]. – Male Attending
	Features I dislike
Recognition rather than recall	A. Status of some lab orders can be unclear	I [wish it were] possible [for some lab tests] to show. . . when they’re in process and stuff like that. You can see, I think, maybe not just if they’re collected, but like, for other labs. . . maybe if on the Results Review you could see if something had been ordered or collected or not collected – Female ResidentThe problem with it, though, is it doesn’t always tell you what’s been ordered. . . and what’s pending too, which is really frustrating. So, then you [ask] the Residents, ‘Hey, I asked for this, why isn’t this here?’ And they’re like, it is [still in process] . . . – Male Attending
Flexibility and efficiency of use	B. Too much scrolling	You have to scroll too far across, you know, and these [ICU patients] get so may labs so often through the day that each different lab, every time there’s a lab that gets posted a different time, or ordered a different time, it creates a new column. So, you have to scroll across too many [columns]. – Male Attending
Aesthetic & minimalist design	C. Graphing tool is cumbersome	So, anything in Results Review you can graph, but it’s, the graphing tool is really cumbersome and it’s really hard to pull, [for instance, if] I want [only] the white blood cell count and the sodium. That’s almost impossible to get. They have to be next to each other for you to really graph them. And then there are scales. . . it forces them to the same scale. – Male Fellow

Vital signs screen

The majority of participants (n = 19 [76%]) reported the Vital Signs screen to be one they rarely use in the medical ICU, preferring to review a patient’s vital signs in the Flowsheet screen where other patient data are integrated to provide clinical context. Unfavorable features of the Vital signs screen: vertical layout made it difficult to trend patient data at a glance, and it is less comprehensive than the Flowsheet.

Preferred content and characteristics for a patient-level dashboard screen

All interview participants (n = 25 [100%]) agreed on the need for an information visualization dashboard. They described what an ideal EHR dashboard screen would include in the medical ICU, both in terms of information needed and its desired “look and feel.” A “top 5” list of clinical data elements emerged across users: vital signs, ventilator data, laboratory data, active medications, and microbiology data. Participants also emphasized the importance of customization at the user-level, and that only data from the last 24 h should be presented. Table 5 summarizes participants’ recommendations for a user-centered, patient-level dashboard screen in the medical ICU.

Table 5.

Preferred content and characteristics of a next generation patient-level dashboard screen in the EHR.

Nielsen Heuristic evaluation	Information	Illustrative quote
Flexibility and efficiency of use	Flowsheet view to Integrate Key Clinical Data:1. Vital signs2. Ventilator data3. Labs4. Medications5. Microbiology	The Flowsheet is most helpful. So, a lot of the stuff that’s on the Flowsheet. – Female ResidentYeah, I’d say you need the vital signs, including their fevers, you need their [in’s and out’s], you would also need their vent settings. . . also any anti-infectives or vasopressors that they’re on. – Male ResidentVentilator settings I would like to see and whether there’s been any changes in the settings. Blood gases I would like to see in there as well. Those are probably the main things. – Male ResidentMicrobiology would be helpful to have in your face. Cause I think that’s something that you often have to scroll back through to see if something new has resulted. . .it’s hard to know when a culture [turns] positive if you didn’t get the call [from the lab], or when. . .one of those gazillion labs that [the Infectious Disease consultant] had you order was positive. – Female Resident
Error prevention	Medication Administration Record	So. . .if there was a way to look at current infusion rates, at least for the sedatives and the opioids and other similar medicines. Cumulative doses, that’s so helpful to know but it’s not really concisely presented. . . and so that would actually be a helpful thing. Did they get. . . [sedative drugs] overnight? – Female FellowI think when medications were given in relation to certain vitals can be helpful. – Female ResidentActive drugs, so any [vasopressors] that the patient’s on or blood pressure medications, ins and outs, probably anti-microbials with date of, you know, how many days into the course they are. . . – Female Attending
Recognition rather than recall	OvernightEvents	If there were some way to capture overnight events into one little screen, again, you know, vent changes, [vasopressor] changes, big medication changes that happened overnight. . . – Female ResidentIf there was some sort of summary, like ‘at 10 pm they de-saturated, at 10:30 they had to be intubated’. . . and just like a really quick, ‘this is the timeline of events.’ – Female Fellow
Match between system and the real world	Primary and Consult Team Identifiers	One screen?. . . I would probably do a link to any consult notes in the last 24 h. That would be helpful. – Female FellowJust have a corner that says, ‘this is what all the consultants have [recommended]’, like, have access to their most recent notes that you can just quickly look at their plans. – Male ResidentI’m imagining [the patient’s] name. . . And then, under that would be like, the team information, people following: . . .the main team, the consultants. . . the pharmacist and the case manager. – Female Resident
Consistency & standards	Active problem list/organ-system based list	I would say active problems, although I don’t think the problem list is up to date in most patient charts. So I think it would [require] a culture shift to make that updated. . . – Female AttendingAt least in ICU I think about patients as an organ-based approach to their problems. . .You know, kind of like how we fashion the notes in the ICU as organ system based. A dashboard that pools together all the relevant information in organ system-based approach might be useful. – Male Fellow
Flexibility and efficiency of use	Advanced Visualization	But something that I think would be helpful in the ICU is building a graphing tool where it’ll graph your vital signs and pertinent lab results. – Male FellowAnd just having it maybe in some way that’s, you know, visually intuitive. – Male Resident
Match between system and the real world	Patient Preferences	[Including] advanced care directives makes sense. – Female Attending

Discussion

This study investigated ICU physicians’ experiences around key EHR screens and screen-level features within a prominent commercial EHR system, identifying information retrieval challenges and soliciting feedback from end-users for potential design solutions. To our knowledge, this is the first study to present and link full EPIC© screenshots of three prominent EHR screens (Flowsheet, Notes/Chart Review, and Results Review) to interface, layout, content, and navigation features that physicians identified as favorable and unfavorable. This study helps to distinguish which screens within a prominent vendor-based system display are reported by physicians to be the most useful and frequently visited, while also identifying specific elements of three particularly high-yield screens that make it easier to find information.

Generally, almost all participants recalled a time when they had challenges finding important patient information in the EHR due to confusing layout, design, or navigation issues. Multiple physicians noted challenges and inconsistencies with the layout of the medication administration record screen that made it difficult to identify current, previous, and future medications. Another design deficit identified by participants included the lack of quick views to show longitudinal trends in a patient’s weight across inpatient and outpatient encounters. Other patient information deemed difficult to locate included notes from other institutions or outside the EHR.

Recommendations for improved EHR screens

Study findings may vary in generalizability depending on the size, complexity, and choice of EHR in the ICU; however, there are main themes that are largely applicable. The main complaint about Flowsheets was that there was unnecessary information which increases the amount of scrolling required to find information. To improve this screen, developers should limit the overall information provided, making the screen more concise and only displaying critical data. Our participants identified key data to include: vital signs, weight, intake and output (including medications), ventilator settings, and lab values. The ability of an EHR to display clinically relevant data trends will be paramount to the success of any design.²⁵ Moreover, as users will interact with data in their own way due to role and experience, perhaps one recommendation for vendors would be to create a default Flowsheet but to allow users to easily “drag and drop” the data elements they prefer to see. Such enhanced customization functionality would best serve support a user-centered approach to patient data.²⁶

When asked about Results/Lab Review, a criticism brought up by multiple participants was the difficulty in identifying when a lab order was requested without specifically searching for them. A previous study has also demonstrated user difficulty related to this type of screen during their simulated study and subsequent discussion.²⁷ One solution at the vendor-level would be to create a screen or summary “widget” to display all pending studies and their current status (e.g. ordered, in process, not scheduled, etc.); in addition, vendors could re-tool current lab orders using a “pin” feature, “drag-and-drop” option, or “right-click; alert me when” feature to support users in maintaining situational awareness of pending studies. Users also identified too much scrolling as being problematic. This can be fixed by giving users the option to hand-pick which results are displayed in an intuitive fashion, perhaps with advanced visualization menus or quick check-box sorting functionality similar to that used on airfare sites.

Chart review screen was another screen that participants criticized heavily due to the amount of information that was displayed. The screen required too much scrolling and clicking and relied on an inefficient filter feature to sort through the notes. The number of screens on the top of the screen further contributed to the increased cognitive load of the user. Multiple steps are needed to fix this problem. We recommend eliminating the screens at the top of the screen as they are currently displayed and allow users to customize which screens they want to see. This ensures that no unneeded information is displayed. The filter feature can also be updated to allow users to more seamlessly and intuitively select a specific service and a date range by which to filter the documentation they view, the way that online hotel and e-travel websites present intuitive, interactive “slider” tools for swift filtering.

The vital signs screen was poorly reviewed by participants. A majority of the participants stated that they do not ever use the screen and view it as redundant given that Flowsheets carry the exact same information. One possible user-centered design change to emerge from this finding might be to remove the vital signs screen from the default physician view in the ICU setting. The more screens there are to filter through, the higher the cognitive load for the end user.

Strengths and limitations

The EHR used in this study, Epic©, is widely utilized in the US and commonly used internationally, suggesting that there is value in descriptive characterization. Other health systems utilizing Epic© may benefit from our analysis when considering their own customization or implementation strategies, especially given the relative dearth of screenshots previously available in the EHR usability literature. Study strengths included Epic© screenshots, a relatively large sample size for qualitative usability research; a diverse and representative sample; an excellent balance of physician roles, gender, and prior EHR experience. Lastly, we focused on four particular screens reported by physicians (and confirmed during live-observational study) to be most important for workflow.²²

Our study also has important limitations. First, it was conducted in single clinical setting at a single-site, and physicians were the only participants; as a result, we cannot exclude the possibility of selection bias. Future studies should include multiple hospitals and clinical settings. Second, the EHR screen layouts published in this paper are based on institutional customization, so the screenshots may not exactly match their respective counterparts at other institutions using the same EHR vendor due to differences in local customization or system version. Third, user interfaces and screen layouts often undergo subtle changes with system upgrades, so some aspects of the screen-level maps may become less representative in the future, though it remains important to archive such findings as the field continues to evolve.

Conclusion

This qualitative study of ICU physicians demonstrated a mix of positive and negative attitudes toward specific screen-level features in a major vendor-based EHR system. Physician perceptions of information overload emerged as a theme across multiple EHR screens. Our findings underscore the importance of qualitative research and end-user feedback in EHR software design and interface optimization at both the vendor and institutional level.

Supplemental Material

sj-docx-1-jhi-10.1177_1460458221997914 – Supplemental material for Physician experiences of screen-level features in a prominent electronic health record: Design recommendations from a qualitative study

Supplemental material, sj-docx-1-jhi-10.1177_1460458221997914 for Physician experiences of screen-level features in a prominent electronic health record: Design recommendations from a qualitative study by Saif Khairat, Cameron Coleman, Randall Teal, Salma Rezk, Victoria Rand, Thomas Bice and Shannon S Carson in Health Informatics Journal

Supplemental Material

sj-docx-2-jhi-10.1177_1460458221997914 – Supplemental material for Physician experiences of screen-level features in a prominent electronic health record: Design recommendations from a qualitative study

Supplemental material, sj-docx-2-jhi-10.1177_1460458221997914 for Physician experiences of screen-level features in a prominent electronic health record: Design recommendations from a qualitative study by Saif Khairat, Cameron Coleman, Randall Teal, Salma Rezk, Victoria Rand, Thomas Bice and Shannon S Carson in Health Informatics Journal

Footnotes

Acknowledgements

The authors would like to acknowledge the support of UNC ISD team and Epic©. As well as Thomas Newlin and Peter Fleming, MSN for their assistance in data collection and transcription.

Authors’ note

Thomas Bice is now affiliated with Novant Health, Monroe, NC. All authors had access to the data and a role in writing the manuscript.

Author contributions

Study concept and design: SK, CC, TB. Acquisition of data: SK, CC, VR, TN. Analysis or interpretation of data: PF, SR, RT, CC, SK. Drafting of the manuscript: SK, CC, RT, PF, SR. Critical revision of the manuscript for important intellectual content: SK. Thematic analysis: PF, RT, SR, SK. Administrative, technical, or material support: TB, SK. Study supervision: SK, SC. All authors have read and approved the manuscript.

Availability of data and material

All manuscript authors had access to study data.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported under grant number 1T15LM012500-01 by funding Dr. Cameron Coleman to assist with designing, conducting, analyzing this study. Also, the University of North Carolina at Chapel Hill Office of the Provost Junior Faculty Award was awarded to Dr. Saif Khairat to run a pilot study that assisted in the design of the study design.

Ethics approval

Ethics approval and consent to participate Institutional Review Board (IRB) approval was obtained from the University of North Carolina at Chapel Hill prior to the beginning of this study. All participants provided written consent prior to the study.

Consent for publication

Approval obtained from Epic© to publish Epic© screenshots.

ORCID iD

Saif Khairat

Supplemental material

Supplemental material for this article is available online.

References

Grabenbauer

Skinner

Windle

Electronic health record adoption – maybe it’s not about the money: physician super-users, electronic health records and patient care. Appl Clin Inform 2011; 2(4): 460–471.

Downing

Bates

Longhurst

CA.

Physician burnout in the electronic health record era: are we ignoring the real cause?

Ann Intern Med 2018; 169(1): 50–51.

Shanafelt

Hasan

Dyrbye

, et al. Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014. Mayo Clin Proc 2015; 90(12): 1600–1613.

Khairat

Burke

Archambault

, et al. Perceived burden of EHRs on physicians at different stages of their career. Appl Clin Inform 2018; 9(2): 336–347.

Gawande

Why doctors hate their computers. The New Yorker, 5 November 2018. https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers

Ratwani

Savage

Will

, et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc 2018; 25(9): 1197–1201.

Khairat

Coleman

Ottmar

, et al. Physicians’ gender and their use of electronic health records: findings from a mixed-methods usability study. J Am Med Inform Assoc 2019; 26: 1505–1514.

Manor-Shulman

Beyene

Frndova

, et al. Quantifying the volume of documented clinical information in critical illness. J Crit Care 2008; 23(2): 245–250.

Meeks

Smith

Taylor

, et al. An analysis of electronic health record-related patient safety concerns. J Am Med Inform Assoc 2014; 21(6): 1053–1059.

10.

Herasevich

Ellsworth

Hebl

, et al. Information needs for the OR and PACU electronic medical record. Appl Clin Inform 2014; 5: 630–641.

11.

Tahir

Doctors barred from discussing safety glitches in U.S.-funded software. Politico, 9 Novermber 2015. http://www.politico.com/story/2015/09/doctors-barred-from-discussing-safety-glitches-in-us-funded-software-213553

12.

Ratwani

Hettinger

Fairbanks

RJ.

Barriers to comparing the usability of electronic health records. J Am Med Inform Assoc 2017; 24(e1): e191–e193.

13.

Ratwani

Savage

Will

, et al. Identifying electronic health record usability and safety challenges in pediatric settings. Health Aff (Millwood) 2018; 37(11): 1752–1759.

14.

Khairat

Coleman

Russomagno

, et al. Assessing the Status Quo of EHR accessibility, usability, and knowledge dissemination. EGEMS (Wash DC) 2018; 6(1): 9.

15.

Kuziemsky

Schwartz

Airan-Javia

, et al. Context and meaning in EHR displays. Stud Health Technol Inform 2019; 265: 69–73.

16.

Hultman

Marquard

Arsoniadis

, et al. Usability testing of two ambulatory EHR navigators. Appl Clin Inform 2016; 7(2): 502–515.

17.

Rizvi

Marquard

Hultman

, et al. Usability evaluation of electronic health record system around clinical notes usage: an ethnographic study. Appl Clin Inform 2017; 8(4): 1095–1105.

18.

Kroth

Morioka-Douglas

Veres

, et al. Association of electronic health record design and use factors with clinician stress and burnout. JAMA Netw Open 2019; 2(8): e199609.

19.

CMS . Patients over paperwork, 2 June 2020. https://www.cms.gov/About-CMS/Story-Page/patients-over-paperwork

20.

Palinkas

Horwitz

Green

, et al. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health 2015; 42(5): 533–544.

21.

McAlearney

Reiter

Weiner

, et al. Challenges and facilitators of community clinical oncology program participation: a qualitative study. J Healthc Manag 2013; 58(1): 29–44; discussion 45–46.

22.

Coleman

Gotz

Eaker

, et al. Analysing EHR navigation patterns and digital workflows among physicians during ICU pre-rounds. Health Inf Manag. Epub ahead of print 1 June 2020. DOI: 10.1177/1833358320920589.

23.

Chapman

Hadfield

Chapman

CJ.

Qualitative research in healthcare: an introduction to grounded theory using thematic analysis. J R Coll Physicians Edinb 2015; 45(3): 201–205.

24.

Nielsen

Heuristic evaluation. In: Nielsen

(ed.) Usability inspection methods. New York, NY: John Wiley & Sons, 1994, pp.25–62.

25.

Wright

Borbolla

Waller

, et al. Critical care information display approaches and design frameworks: a systematic review and meta-analysis. J Biomed Inform X 2019; 3: 100041.

26.

Dziadzko

Herasevich

Sen

, et al. User perception and experience of the introduction of a novel critical care patient viewer in the ICU setting. Int J Med Inform 2016; 88: 86–91.

27.

Wright

Dunbar

Macpherson

, et al. Toward designing information display to support critical care. A qualitative contextual evaluation and visioning effort. Appl Clin Inform 2016; 7(4): 912–929.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

0.09 MB