Sources of variation in primary care clinical workflow: Implications for the design of cognitive support

Abstract

This article identifies sources of variation in clinical workflow and implications for the design and implementation of electronic clinical decision support. Sources of variation in workflow were identified via rapid ethnographic observation, focus groups, and interviews across a total of eight medical centers in both the Veterans Health Administration and academic medical centers nationally regarded as leaders in developing and using clinical decision support. Data were reviewed for types of variability within the social and technical subsystems and the external environment as described in the sociotechnical systems theory. Two researchers independently identified examples of variation and their sources, and then met with each other to discuss them until consensus was reached. Sources of variation were categorized as environmental (clinic staffing and clinic pace), social (perception of health information technology and real-time use with patients), or technical (computer access and information access). Examples of sources of variation within each of the categories are described and discussed in terms of impact on clinical workflow. As technologies are implemented, barriers to use become visible over time as users struggle to adapt workflow and work practices to accommodate new technologies. Each source of variability identified has implications for the effective design and implementation of useful health information technology. Accommodating moderate variability in workflow is anticipated to avoid brittle and inflexible workflow designs, while also avoiding unnecessary complexity for implementers and users.

Introduction

The Medicare and Medicaid Electronic Health Record (EHR) Incentive Programs, created to encourage the meaningful use of EHRs, have stimulated a flurry of research regarding successful implementation of health information technology (HIT). While the legislation is focused on the implementation and “meaningful use” of technology, the overarching goal of this financial incentive is to streamline workflow by increasing access to medical records, supporting clinician decision making, and increasing patients’ quality of care while decreasing costs and errors.

A key component of successful HIT usage is the effective integration of cognitive support into workflow.^1–6 This is generally accomplished via the use of clinical decision support (CDS) systems and tools. CDS tools are incorporated into many of the HIT products available today, including EHRs and computerized provider order entry. However, successful integration of CDS into clinical practices is much like solving a puzzle with many different pieces, where the degree of success has clear implications for quality of care. ⁷ Successful integration requires that the benefits of the CDS to the provider and the patient are immediately visible, so that the CDS is seen as a tool that facilitates core work activities. Therefore, successful CDS integration into workflow is important because it reduces the likelihood that it will be viewed as an extra task in an already busy clinic setting, and thus ignored. A recent Institute of Medicine committee on Computational Technology for Effective Healthcare report highlights both the complexity of integrating CDS into workflow and the importance of addressing the associated issues as part of a major paradigm shift in informatics. They strongly advocate for research and development as a necessary component of achieving the Institute of Medicine’s vision for healthcare quality. ⁸

The term “workflow” is widely used, but rarely defined. Unertl et al. ⁹ recently conducted a review of articles in scientific journals addressing the topic of workflow and found that workflow is studied from a broad range of perspectives ranging from static processes that can be captured on a flowchart to more complex conceptualizations that acknowledge the evolving nature of work. For the purposes of this analysis, we consider workflow as a dynamic construct that includes the three pervasive elements articulated by Unertl et al. ⁹ : temporality, aggregation of actors and actions, and context that constrains and enables actions. ⁹

We have applied sociotechnical systems theory in assessing the integration of CDS into workflow.^7,10–13 This approach has been widely applied in recent years to study the impact of computerized order entry systems.^{14 –17} Sociotechnical systems theory advocates for joint optimization of both social and technical subsystems taking into account constraints imposed by the larger external subsystem (Figure 1). The term “joint optimization” suggests that social and technical subsystems can be harmonized and balanced to optimize productivity, worker satisfaction, and safety.^18,19 Regretfully, many systems fall short of this ideal.^7,10 A recent commentary in JAMA highlights similar issues in healthcare, but from a systems integration perspective—reporting that “clinicians often feel burdened rather than supported by technology, as if vendors’ needs are setting the agenda” (p. 934). ²⁰ In their widely cited article, Pasmore et al. ¹⁰ called for a more balanced approach. They emphasized the importance of paying closer attention to the development of technology to anticipate its impact on behavior or practice. This speaks to the all-too-common gaps between developers of technology, behavioral scientists, and clinicians. Methods from the behavioral sciences such as ethnographic observation and interviews are powerful tools for anticipating and assessing the impact of technology on behavior or practice. However, bridging the gap between the findings from ethnographic studies and implications for technology design is not always straightforward. We suggest that sources of workflow variation is one type of finding from ethnographic research that has practical implications for the design of HIT. For example, designers may assume greater uniformity across healthcare settings than there actually is because the main tasks are very similar. However, it is the overall workflow that varies greatly, and this is what designers must learn to accommodate to create a successful HIT system.

Figure 1.

Sociotechnical systems theory.

The goal of the analysis presented in this article was to identify common types of variability found in primary care clinics that will aid technology designers in incorporating appropriate types of flexibility and functionality into HIT design for this setting. Although variability in workflow is generally positive as a source of adaption and innovation, it presents challenges for designers at all levels of a system. When variability is not anticipated, unintended consequences emerge, often resulting in reduced productivity, increased error, and other negative outcomes. In this article, we analyze ethnographic data collected in two studies that focused on the use of computerized clinical reminders and other CDS tools in primary care clinics, across a range of healthcare settings. Previous studies focused on identifying barriers and facilitators to CDS use. Here, the data are reviewed with the goal of identifying more general sources of workflow variation. Results include six key sources of variability and a definition of each. We also discuss implications of each of the designs, and provide illustrative examples. The implications offered are high level in order to increase generalizability, and to highlight the potential value of exploring these types of variability at all levels of design. More concrete recommendations for improving specific CDS tools have been published elsewhere.^21–23

Approach

Cognitive field research, including methods such as ethnographic observation and interviews, are used increasingly across disciplines to integrate information technology (IT) into complex work settings. ²⁴ This approach is especially useful with respect to HIT because of the ability to discover and understand the impact of new technologies on workflow early. More and more, human factors professionals, industrial engineers, and psychologists are adapting classic ethnographic methods to develop qualitative methods tailored to address such applied problems. Several recent studies have specifically identified integration of CDS into clinical workflow as a major fault of current HIT efforts and as a key barrier to HIT use.^25–27

For this project, researchers reviewed data they had collected in previous studies of CDS. By definition, ethnographic methods result in large sets of context-rich data. It is possible to conduct multiple rounds of thematic analyses, exploring the data from different perspectives, to address different research questions. Although the original studies used a mixed-methods approach, only the qualitative ethnographic data were analyzed for this article; other data collected were less relevant to the scope of this article.

Workflow

Before discussing variability in clinical workflow, it is helpful to first start with a discussion of workflow. Workflow is a broad term and can refer to multiple perspectives (i.e. the patient perspective, the provider perspective, etc.). ⁷ Furthermore, workflow can include different objects such as information, people, and equipment. For the purposes of design, it is generally helpful to consider workflow as an integration of these elements. By way of illustration, we offer a sample idealized workflow as a rhetorical device to aid the reader in envisioning the variations in workflow described in this article. This ideal workflow is based on discussions with primary care providers and IT specialists (see opportunistic interviews and key informant interviews described in the methods section of this article), as well as inferred intent from existing CDS and organizational structures (Figure 2), and does not represent a model of workflow observed at any individual site. This workflow focuses primarily on how information flows during a patient visit in a primary care clinic, but it also includes consideration of who (i.e. patient, intake nurse, primary care provider) is entering or retrieving information throughout the office visit, and by what means that happens.

Figure 2.

Idealized clinic workflow.

This idealized workflow comprises a scenario in which a patient moves through the clinic setting, and the EHR and CDS tools facilitate the sharing of key information at the optimal time across the healthcare team. The patient arrives at check-in and confirms demographic information, such as address and date of birth; describes chief complaint; and provides insurance information. This information is entered by the clerk, and a routing sheet is printed with key information about the patient for the provider. The patient returns to the waiting room until called by the intake nurse. The intake nurse brings the patient back to an intake or examination room, and measures vital signs. During intake, the nurse may also ask screening questions about tobacco use, alcohol use, functional status, and other prevention or health behaviors. The patient is then escorted to the exam room to await the visit with the provider. Either prior to or during the course of the visit, the provider may refer to the EHR to do the following:

• Review the patient’s recent history, previous evaluation and current condition, lab results, and consults;

• Document current symptoms, medications, and out-of-network care;

• Order prescriptions, diagnostic tests, or consults;

• Print reference material for the patient such as dietary guidelines, phone numbers for referrals, and directions to community resources.

In the context of this idealized workflow, information collected at intake is instantaneously uploaded into the patient record so that the provider has immediate access to vitals, screening results, prevention measures, functional status, and current problems. The information in the EHR is not only routinely updated prior to the exam but is also easy to scan while talking to the patient and to readily search for specific data such as a current problem list, findings from previous tests, current medications, and prior plans for diagnosis and treatment. The data in the EHR are complete and accurate.

Effective CDS tools such as clinical reminders direct the provider’s attention to the most cost-effective, evidence-based practices indicated for that patient, and are prioritized. During the exam, the provider and patient focus on reestablishing a personal connection, discussing ongoing health issues, new conditions, and preventive care. The provider enters notes as the exam unfolds without detracting from provider–patient communication. The provider is able to order diagnostic tests and consults during the exam, and is able to print new prescriptions, information resources, as well as phone numbers and directions to satellite facilities or community resources. Findings from procedures done in other facilities such as consults, imaging, and tests seamlessly appear in the patient’s record, and the provider is alerted to any follow-up action required.

While articulating a core, idealized workflow may be an effective technique from which to begin design, it is equally important to identify important variations from the idealized or prototypical workflow. Our observations indicate that workflow is highly variable across hospitals, and even across multiple clinics within a specific network. These variations influence the degree to which a healthcare team is able to leverage the strengths of any particular CDS. Although it is impossible to anticipate and design for all variations in workflow, consideration of the major types and extent of variation in a number of different settings can provide important insights to technology designers and work teams tasked with designing and integrating CDS technologies. Knowledge of variability in clinic workflow will aid technology designers in incorporating appropriate types of flexibility into HIT design.

This article identifies recurring sources of clinical workflow variations documented across a series of ethnographic studies in Veterans Affairs and academic medical centers.

Methods

The data used in this project were collected as part of two separate studies focused on understanding barriers and facilitators to the use of CDS in primary care.^{21–23,28–31} Observation and interview data were collected from 2004 to 2008. Both studies used a combination of key informant interviews, direct observations, and opportunistic interviews. The first study focused broadly on use of clinical reminders. Findings from the first study were published in an edited volume on Naturalistic Decision Making and Macrocognition, in the Journal of American Medical Informatics, and in the Journal of Biomedical Informatics.^21,23,28,29 The second study focused more specifically on CDS tools used to support colorectal cancer screening. Findings from the second study were published in BMC Medical Informatics and Decision Making, and in the American Medical Informatics Association Annual Symposium Proceedings.^22,30,31 Although the initial data collection took place in 2004, we find that the data remain relevant given that the initial sites were all Veterans Affairs Medical Centers (VAMCs). Few changes have been made to the computerized clinical reminders or the EHR (computerized patient record system) in the intervening years. The research was approved by the Institutional Review Board at each site.

Site selection

Study 1 was conducted in ambulatory care clinics of four VAMCs geographically distributed across the United States. Sites were selected based on geographic distribution nationally across four different Veterans Integrated Service Networks (VISNs) and willingness to participate in the research study.

Study 2 also included four sites: two VAMCs and two academic medical centers. The four facilities were selected due to being recognized leaders in the development of CDS. The two VAMC sites were selected based on having strong medical informatics programs, being geographically distributed nationally across two VISNs, and having developed and implemented unique CDS systems. All sites for this study were identified as benchmark institutions in a systematic review of HIT. ³² The intent was not to compare different systems, rather to explore common barriers and facilitators across different health systems.

At each of these eight sites, qualitative data were collected in primary care clinics. CDS technologies in use varied somewhat across sites. All six VAMCs used computerized clinical reminders integrated into the EHR used by intake nurses as well as primary care providers and registered nurses (RNs). One academic medical center used a CDS system that generated paper encounter reminders tailored to individual patients based on data in the EHR and information collected at intake, used primarily by the primary care provider. The second academic medical center used an electronic template health maintenance list that was an integral part of the EHR populated in part by data collected during intake.

Field study methods

The researchers conducted key informant interviews and direct observations with opportunistic interviews at each site. Key informant interviews^29,31 focused on mechanisms and best practices used to facilitate CDS implementation and integration into workflow. A set of core questions were asked for each interviewee. However, the semistructured nature allowed for flexibility and gave the interviewee an opportunity to elaborate on, or discuss important topics not anticipated. Key informants were administrators (e.g. the Chief Medical Informatics Officer) and clinical champions (e.g. clinic director). The key informant interviews were conducted either in-person during the site visit or afterward by phone and lasted approximately 60 min. A total of 15 key informant interviews were conducted across the eight sites. This includes one key informant interview at each site in Study 1, and two to three key informant interviews at each site in Study 2.

Direct observations^29,31 were conducted to explore the ways in which providers interacted and used CDS tools in situ. Data were collected focusing on the context and process surrounding CDS use. During observations, two to four observers experienced in ethnographic observation separately shadowed technical support staff (i.e. licensed practical nurses, licensed vocational nurses, medical assistants (MAs), health technicians, and RNs), nontechnical support staff (front desk clerks), and providers (i.e. physicians, physician assistants, and nurse practitioners) with their permission as they interacted with CDS tools during a work shift. We observed a total of 86 technical support staff, 11 nontechnical support staff, and 108 providers, spending 2 days at each site. Observations were recorded via handwritten notes during participant interaction with the CDS, capturing observable activities and verbalizations.

Observers also conducted opportunistic interviews^29,31 with providers on their use of CDS to better understand the observational data. These interviews were impromptu and time-permitting, so as not to disrupt the natural workflow of the providers. (Note: Opportunistic interviews are an integral part of ethnographic observation and cannot be separated in a meaningful way. Thus, the number of questions asked and to whom was not recorded and are not reported here.) Interviews covered why providers took certain actions, as well as opinions and feedback about the CDS and how it helps or impedes their workflow. This opportunistic feedback was recorded in the field notes. This feedback supplemented and aided understanding of corresponding observations. Each observer employed a maximum sampling strategy in which he or she stayed with one provider through an average of two patient encounters before observing another provider. The observers met at the end of each observation day to reflect and record key findings and observations. The handwritten observations were typed after each site visit, and a scheme was applied to track observer, site, clinic, and day.

Data analysis

Data from both studies were reviewed for examples of workflow variability and potential sources within each of the sociotechnical subsystems. Two researchers independently identified examples of variability and their sources, and then met to discuss them until consensus was reached. No a priori menu of types of variability was used. Rather, researchers identified emergent variables observed across study sites. Following a constructivist view,^33,34 no attempt was made to establish interrater reliability. The goal of this analysis was not to exhaustively identify every variation in clinical workflow. Such an effort would never be complete as humans continually adapt to the changing world, resulting in new sources of variability. Rather, the intent was to identify a small set of prototypical examples in each of the three sociotechnical categories. We identified patterns of variation that influenced integration of CDS at each site. Each example included here was observed to dramatically influence the use of CDS and/or require observable workarounds. From these patterns of variation, sources were identified that would aid technology designers in anticipating the types of variability that impact the integration of HIT. These sources of variability are likely to persist over time, even as specific workflow variations evolve and change.

Results: sources of workflow variability

Results of this analysis consist of key sources of variability and their definitions. Two categories of sources of workflow variability were identified within each sociotechnical subsystem and the larger environment (Table 1). While there are many sources of variability, in this article, we highlight those most relevant to design. With regard to environmental sources, clinic pace and staffing were highly visible examples. Although these might be considered part of the social system, in the sites we observed, decisions about clinic pace and staffing were made at a level higher than the work system studied and are thus considered part of the external environment. It is also important to point out that there are many factors found in the external environment that influence the use and adoption of HIT. We did not systematically identify all relevant external factors. Our analysis focused instead on identifying salient examples. In other words, we recorded those that stood out as variables that influenced use and adoption of HIT across the different observation sites. Within the social system, perceptions of CDS and technology used during exams were particularly salient examples. Within the technical subsystem, access to computers and access to historical information (paper medical records) and information from care received off-site were frequently noted sources of variation. Because the aim of the original studies was to explore the interaction of the primary care provider with the CDS, we did not focus on patient interactions. A brief definition of each is presented in Table 1.

Table 1.

Sources of variability and definitions.

Sources	Description
Environment
Clinic Staffing	Number of personnel; their educational background and experience level
Clinic Pace	Length of exam
Social
Perceptions of CDS	Attitudes and beliefs about CDS functionality, goals, reliability, and usability
Technology use during exams	Individual and social norms for computer use during exams
Technical
Computer access	Availability of relevant hardware and software through the clinic
Information access	Availability of information across HIT systems and capability to find information within an HIT

CDS: clinical decision support; HIT: health information technology.

Environmental sources

Staffing

Clinic staffing influenced highly visible aspects of workflow, including role allocation and the amounts and types of information shared from intake to exam. For example, intake may be conducted by RNs, licensed practical nurses, licensed vocational nurses, or MAs, all of whom have different types of training and experience. In clinics staffed largely with RNs, the intake exam was more likely to include screening and educational functions requiring more complex clinical judgment, in addition to measurement of vital signs. In this case, the need for intake-provider communication was critical as information collected during intake is more likely to be contextually complex and relevant to the provider exam. In clinics staffed largely with MAs, providers were more likely to duplicate screening conducted by the MA, relying primarily on the MA for readily quantifiable information such as vital signs (e.g. blood pressure, weight, and temperature).

Clinic pace

Clinic pace was another source of workflow variability. Across the eight clinics observed, the scheduled length of an exam was based on predefined schedule parameters at each clinic. Exam times varied from 15 to 30 min, with up to 45 min for residents who had to consult with an attending physician during this time. Although primary care providers could certainly choose to extend the exam time for a specific patient if needed, this would likely create a backlog throughout the day, leaving the provider with the task of making up the time by shedding tasks, skipping breaks, or shortening exam time for patients later in the day. Perhaps the most salient implication of variations in exam time and workload was the time available for the provider to balance preventive care versus acute care needs. As providers became more time pressured, they tended to focus on the most urgent, acute issues, leaving less time for discussion of preventive care and other potentially high-impact, early identification screening opportunities. An additional time challenge was with documentation. Documentation may become more sparse and less detailed when less time is allotted for each encounter. One provider reported jotting down a few notes during each exam, and then trying to find a few minutes after every two to three exams to add enough detail that the note would be meaningful. Even though the provider was fully aware that delaying documentation would likely reduce accuracy and completeness due to limited recall, the provider did not have time to document thoroughly during the exam.

Social sources

Perceptions of CDS

Provider’s perceptions of CDS had tremendous influence over how CDS was integrated into workflow. In the best cases, the CDS tool was seen as a useful, helpful tool with a small set of issues that could be overcome, worked around, or improved in future releases. In the worst cases, the CDS tool was perceived to have little benefit for patient care, devised primarily to allow administration to generate reports, track statistics, and monitor or punish provider behavior. In the words of one study participant, “Most of the reminders are just there to satisfy the statisticians ... [they] need to have a certain percent satisfied or turned off.” In some cases, negative perceptions were related to instability of the software and fear of losing patient data due to a technology malfunction, or not having patient data available when needed. Many of those with positive perceptions of CDS explored and learned the capabilities of the technology, and learned how to overcome its limitations. Those with negative perceptions of the CDS tool had little motivation to invest time in learning the technology. Thus, the perception of the technology had a direct impact on how effectively it was integrated.

Technology use during exams

Technology use during exams is another important source of workflow variability. The insertion of the live electronic record is a relatively recent change to the traditional social interaction between patient and provider. Strategies for, and attitudes about, incorporating computer use into this social interaction varied enormously. Some providers preferred to review the patient record before the exam and complete documentation after the patient has departed. Others preferred to treat each visit as a self-contained unit, using the computer to review the patient record; conduct the history and physical; and record the findings, assessments, and plans, all within the scheduled visit time. Some found the computer to be an obstacle to establishing rapport, and preferred to use the computer as little as possible during the exam. (Note: Although it may seem counterintuitive to include technology use during exams as an example of variation in the social system, we found it fit best here because of the focus on the role of technology in a social interaction.)

Technical sources

Computer access

Computer access was highly variable across clinical settings. We observed one setting in which computers were available in every exam room for use by providers, and laptops were transported on carts to each exam room for nurses to enter data gathered during intake. In other settings, there were no stationary or portable computers in the exam rooms. Rather, all documentation took place at workstations separate from the exam rooms (either shared or individual workstations). Other configurations observed included the use of handheld computing devices. Access to computers has an enormous impact on when and how information is entered and reviewed in the EHR, as well as the effectiveness of specific CDS tools. Limited access often resulted in redundant paper systems in which key information is written on post-it notes, postcards, or paper scraps—or relying on individuals to remember information—until it can be entered into the computer. Printouts of clinical reminders are used in some sites to accommodate limited computer access.

Access to clinical data

Access to clinical data is another source of variation. HIT offers the promise of increased access to clinical data. Ideally, providers would be able to access historical data for a specific patient; view X-ray, computed tomography (CT), and ultrasound images from other facilities; and refer to notes from consults—all at the click of a mouse. In reality, however, access to these data is quite varied. While the Veterans Health Administration and some academic research centers are approaching this ideal, many healthcare systems have only recently begun implementing EHRs. As recently as 2008, as few as 17 percent of US physicians had EHRs. ³⁵ For the healthcare systems that have adopted EHRs in the last few years, most patient histories are only available in a paper chart. The provider must therefore refer to both the electronic and paper records.

Data from procedures and tests administered off-site may or may not be accessible. One provider observed:

We do not scan things. If patients get things done outside, then it’s almost as like they never happened because I do not see them. Ideally I would have a box here [points to wall] and I could read things, results, and then someone would scan them in ..., but that can’t happen. We’re all too busy.

In some cases, paper charts were digitally scanned so that they could be attached to the electronic record. These scanned charts, however, were typically stored as images, rather than data files, and therefore were not searchable, limiting their usefulness as part of the electronic record.

In addition, access to the scanned documents was highly dependent on adequate labeling and correct filing within the EHR. One participant reported, “Sometimes [there] is a problem if there is no date of birth on the correspondence; sometimes you just have to use your best guess [about which patient the document pertains to].” Finally, in some cases, although a search capability may have been available, the time required to retrieve and identify the data of interest was prohibitive within the time constraints of an exam.

Discussion: implications for design

Each type of variability described above has implications for HIT design and implementation. In fact, most could be addressed in any combination of ways, including changes in the design of the organization (i.e., staffing, processes), the design of the HIT, and the environment within which the HIT is implemented. For example, staffing dictates who will use a particular CDS tool and for what purpose, and how they will coordinate. While many CDS tools are designed to support coordination from intake to provider exam, this aspect of CDS has been met with limited success. Inadequate anticipation of what types of information must be transferred, and in what form, often leads to redundant activities. A system designed to transfer quantitative information such as blood pressure and temperature readings taken by an MA at intake to the provider note in real-time may not as readily support an RN in highlighting a positive depression screen or other timely clinical judgments to the provider.

Clinic place is likely to have considerable influence on the use of CDS. Reduced exam times require providers to carefully prioritize activities (i.e. focus on acute issues over prevention, limit documentation). We suspect that time pressure also reduces the likelihood that a provider will take advantage of CDS such as clinical reminders, making it difficult to routinely incorporate the reminders into the exam. Thus, it is important to design CDS that balances these very important but competing demands on the provider’s time. The CDS tool must not hinder the provider’s ability to deal with acute issues in a limited time, and must provide easy access to relevant preventive care guidelines when time allows, while facilitating thorough and accurate documentation.

Our observations regarding perceptions of CDS converge with other research on this topic, suggesting that even though benefits of a technology have been demonstrated, there may still be resistance to use. Nielsen ³⁶ indicates that users, even when given evidence that their performance is better when using a given technology, will not use that technology if they do not prefer it over their current work process. Lack of comfort with the technology exacerbates real limitations of the tool, often resulting in highly frustrated users. For technology developers, making the benefit of the CDS to core tasks and values (i.e., patient care) highly salient is critical to user acceptance. CDS developers should explore high-priority concerns for end users, and make sure that they are addressed in the technology solution. In this case, understanding the importance of protecting patient data, and then making visible the strategies in place to ensure that data will not be lost, and that it will be available when needed, will go a long way toward reducing negative perceptions that hinder acceptance and integration of CDS.

Technology use during exams is likely to continue to evolve as HIT becomes more ubiquitous. As providers transition from paper to electronic records, there is little consensus about how to best leverage the strengths of HIT and CDS. Many reported that computer-based tasks competed for their time with the patient. The more time spent scanning the computer screen, navigating electronic records, and entering information, less was the time available for interacting with the patient. Some providers reported that with paper charts, they were better able to balance these competing needs. It was easier to page through paper notes and to write down key information to be documented more fully later. They found the computer screen to be a physical barrier, making it difficult to maintain eye contact with the patient while scanning or entering information. Furthermore, some decision support tools are designed to encourage full documentation; data search tasks; and the ordering of prescriptions, consults, and tests during the exam—increasing the number of tasks competing for the provider’s attention during the limited exam time. The degree to which these functions are incorporated into the exam via CDS tools is highly dependent upon at least three things. These include how well the CDS design complements and integrates with the provider’s work style; the provider’s facility with the EHR and typing; and, in some cases, the social norms within a clinic for incorporating computer use into the patient exam.^37–39 In contrast, we observed other providers who had effectively integrated the EHR into the interaction to demonstrate findings and reinforce patient understanding, and develop mutually agreeable follow-up visits.

Computer access will continue to be a source of variability given the rapid rate at which advances in hardware become available, and the increasing options available in desktop, tablet, and mobile platforms. CDS designers may need to consider that CDS may be used on a range of platforms, creating the need for different input and display designs tailored to different devices. Furthermore, it is not safe to assume that there will be universal access to computers throughout every clinic. Flexible designs should accommodate differences in computer access. For example, one design might allow for printed forms to be used in clinics without exam room computers. The printed form might contain patient-specific information, such as relevant clinical reminders, and allow for real-time documentation on the paper form. These forms would retain many of the decision support elements available electronically such as which questions should be asked and critical data to be gathered to inform decision making. Recent trends to eliminate paper altogether have met with limited success. Until computers are, in fact, ubiquitous, it is important to design systems that accommodate the use of paper along with electronic systems. ⁴⁰ Organizations planning to introduce new CDS should consider how and where the healthcare team will access the technology and strategies for incorporating additional data.

It will likely be several years before all clinical data are accessible in the EHR. In the meantime, therefore, CDS designers and those involved in its integration into clinical workflow must design technology and processes to accommodate access to clinical data via traditional means (i.e., mail, telephone, etc.), and transitional solutions (i.e., digitally scanned documents, email). In addition, CDS designs should include enhanced search functions to ensure that providers are able to quickly find the data that are currently available digitally.

Conclusion

We have identified sources of variability in workflow that are relevant to the design and implementation of CDS in primary care settings. As new HIT applications are integrated into the clinic setting, variations in clinical workflow and work practices become increasingly evident across sites and over time. Information technologies designed to support one specific, prescribed workflow may be quickly discovered to be inflexible, and often abandoned because workers find them unusable. In contrast, too much—or the wrong types of—flexibility can lead to unnecessary complexity. The challenge for designers, then, is to determine how much and what types of variability to accommodate, while still designing CDS to encourage and support adherence to evidence-based practice and effective workflows. It is worth noting that even when one becomes aware of the types of variability to be accommodated, making choices about when to attempt to “design for all” versus tailor an application to an individual work setting is not trivial. However, understanding the types of variability present and the challenges they represent is an important step toward making informed choices and predicting the impact of specific design choices.

This secondary analysis highlights important sources of variation in clinical workflow, suggesting that further, targeted research would provide additional insights. It is likely that an ethnographic study focusing primarily on variation in workflow would highlight additional sources of variability and richer examples. Our hope is that this article highlights the value of examining workflow variability and strategies for characterizing important aspects of workflow in complex healthcare environments. We would encourage designers of health informatics to explore variations in workflow early in the design process so that they can be considered as design concepts are generated, use cases developed, and product evaluation occurs.

This article described six sources of variability across ambulatory care clinics relevant to the use of EHRs and other CDS tools: clinic staffing, clinic pace, perceptions of CDS, technology use during exams, computer access, and information access. These variations in workflow present challenges to designers of CDS tools. In order to be successful, CDS tools must remain a support (rather than a hindrance) to cognition and effective practice in each of these settings. Accommodating moderate variability in workflow is expected to result in increased HIT adoption, improved efficiency, and user satisfaction.

Clinical relevance statement

The sources of workflow variation documented here should be considered in the design of systems. In addition, these sources of workflow variation have important implications for the integration of new technologies and procedures into the clinic setting. An awareness of these sources of variation will support designers and implementers in anticipating a range of use scenarios.

Human subjects protections

The study was performed in compliance with the World Medical Association Declaration of Helsinki on Ethical Principles and was reviewed by the Indiana University Institutional Review Board, the Indianapolis VA Medical Center Research Committee, and the Institutional Review Board of each data collection site.