Organizational learning in the implementation and adoption of national electronic health records: Case studies of two hospitals participating in the National Programme for Information Technology in England

Abstract

Objective:

To explore the role of organizational learning in enabling implementation and supporting adoption of electronic health record systems into two English hospitals.

Methods and setting:

In the course of conducting our prospective and sociotechnical evaluation of the implementation and adoption of electronic health record into 12 “early adopter” hospitals across England, we identified two hospitals implementing virtually identical versions of the same “off-the-shelf” software (Millennium) within a comparable timeframe. We undertook a longitudinal qualitative case study–based analysis of these two hospitals (referred to hereafter as Alpha and Omega) and their implementation experiences. Data included the following: 63 in-depth interviews with various groups of internal and external stakeholders; 41-h on-site observation; and content analysis of 218 documents of various types. Analysis was both inductive and deductive, the latter being informed by the “sociotechnical changing” theoretical perspective.

Results:

Although Alpha and Omega shared a number of contextual similarities, our evaluation revealed fundamental differences in visions of electronic health record and the implementation strategy between the hospitals, which resulted in distinct local consequences of electronic health record implementation and impacted adoption. Both hospitals did not, during our evaluation, see the hoped-for benefits to the organization as a result of the introduction of electronic health record, such as speeding-up tasks. Nonetheless, the Millennium software worked out to be easier to use at Omega. Interorganizational learning was at the heart of this difference.

Conclusion:

Despite the turbulent overall national “roll out” of electronic health record systems into the English hospitals, considerable opportunities for organizational learning were offered by sequential delivery of the electronic health record software into “early adopter” hospitals. We argue that understanding the process of organizational learning and its enabling factors has the potential to support efforts at implementing national electronic health record implementation endeavors.

Keywords

Adoption electronic health record implementation national organizational learning sociotechnical changing

Background

There is increasing international interest in the potential of electronic health record (EHR) systems for modernizing health care and improving its quality, safety, and efficiency.^1–3 Such endeavors are increasingly being undertaken on a system-wide or national scale, presenting new challenges both for the delivery of these initiatives and for their evaluation.^4–7 As used in this article, the term EHR refers to a digital longitudinal record of a citizen’s health and health-care interventions that is potentially accessible by health-care providers across a range of health-care settings.⁸

The National Programme for Information Technology (NPfIT) (2002–2011)^9–11 was the first sustained attempt to implement EHRs into National Health Service (NHS) hospitals throughout England.^12–14 We undertook a longitudinal, sociotechnical,¹⁵ and case study–based evaluation of the implementation and adoption of EHR systems in 12 early adopter NHS hospitals across England.^6,8,12 This work revealed that few of the hoped-for benefits were realized, at least in the short-term, but an important unanticipated benefit was the considerable interhospital organizational learning, this being exemplified by comparing the experiences of the two hospitals discussed in this article.

Methods

Ethical considerations

Our research was classified as a service evaluation (ref. 08/H0703/112). Informed consent was obtained from the participating hospital and individuals, and their anonymity was guaranteed.

Research settings

The two hospitals focused on in this article (and hereafter referred to as Alpha and Omega) were selected because they were the only hospitals implementing identical versions of the same software (Millennium) within the timeframe of our 30-month evaluation (September 2008 to February 2011). Our wider research had revealed organizational learning as an important benefit to participating hospitals, and these hospitals provided us with a welcome opportunity to consider this in more detail by studying the impact of vision, strategy, and culture with particular reference to the sharing of experiences and insights between these two hospitals, beyond the technical specification of particular software.

Sampling and data collection

We conceptualized each hospital as a case study to reflect the importance of local contingencies.¹⁶ This allowed the specific character of the implementation of the EHR software in each hospital to be revealed, while attempting to make general inferences transferable to another context and facilitate organizational learning. A designated lead researcher (A.T.) collected a broad array of longitudinal qualitative data: that is, purposefully (and, at times, opportunistically) semistructured interviews with a diverse range of internal and external stakeholders with broad range of perspectives, complemented by researcher’s field notes and observational and documentary data of various types with regard to planning, implementing, and using EHR systems at both hospitals. Triangulating between these data sources helped increase our understanding of the evolving process of implementation. We used generic interview guides that were then customized and reflexively utilized for specific participants. To explore organizational learning between the two case studies, we ensured that more or less similar groups of research participants (occupation, position, and relevance) were recruited at Alpha and Omega. Most interviews were audio-recorded and transcribed verbatim. In order to trace changes over time, we collected data in two phases, times 1 and 2 (T1, T2) with 9 months gap in between (see Table 1 for details).

Table 1.

Detailed dataset from Alpha & Omega.

Hospital	Total number of site interviews	Hours of on-site observations	Number of site documents
Alpha (May 2009 to December 2010)	27, including senior manager: 4; implementation team and IT: 5; health-care practitioners: 7; divisional and clinical managers: 4; administrative: 3; strategic health authorities (SHA): 2; local service provider (LSP): 1; Cerner: 1	19	94
Omega (December 2009 to December 2010)	36, including senior manager: 8; implementation team and IT: 10; health-care practitioners: 11; administrative staff: 2; SHA: 2; LSP: 2; Cerner: 1	22	124

IT: information technology.

Data analysis

Data analysis was an iterative process and followed a two-step approach: initially, at each individual case study level using a combination of deductive and inductive approaches,^17,18 and then a meta-synthesis that drew upon the analytical themes from the two case studies. We used an adapted sociotechnical framework for categorizing data and classifying the findings¹⁵ (see Table 2), and “sociotechnical changing” perspective^19,20 for analyzing and interpreting data, which we discuss below. Primary findings from each case study were discussed in regular analysis workshops with the wider evaluation team as well as through formative feedback sessions with hospitals’ representatives. This helped us validate individual case study findings and integrate them to draw organizational learning. Case studies were consequently analyzed in a cross comparison to identify common and contrasting themes and to clarify local consequences.

Table 2.

The sociotechnical model used in framing the study.¹⁵

	System functions	Human perspectives	The health-care system
Structure: What has been put in place?	Technology: What does the intervention look like?	Work organization: What new work practices emerge?	Role of medicines: What is the role of the intervention in the wider health-care system?
Process: What is done that is different?	Processing: What processes emerge and change?	Social interactions: How do communication patterns and workflow change?	Management of care delivery: What organizational changes emerge?
Outcome: What has been achieved?	Validity of processing: Is the service safe and reliable?	Quality of the service: Is the experience of the service satisfactory?	Potential for change: How might the intervention be used in the future?

Our theoretical approach to organizational learning

We acknowledge the lack of a very clear understanding of the terms “organizational” and “learning”²¹ and concerns about the fragmented state of both theoretical and empirical work on organizational learning.^22,23 A number of theoretical perspectives have been developed in an attempt to explain organizational learning and transformation through the adoption of technology—for example, planned change, technological imperative, and punctuated equilibrium.^24,25 These conventional approaches fail, however, to consider change as a way of organizational learning, underpinned by emerging social, technological, and economic conditions. They also overlook distinction between deliberate and emergent strategies: “emergent change,”²⁶ which is far beyond portraying change as only a background activity.^27,28

Recent literature highlights the role of leadership, envisioning the implementation of EHR as change management;²⁹ engaging with various groups of stakeholders,³⁰ taking wider social context into consideration;³¹ integration of EHR with workflows and care pathways;³² organizational culture and behavior;³³ the ever-evolving contextual flux,^26,34 and the interaction between system’s functioning, the organization’s needs and working patterns: the sociotechnical approach^20,35 in organizational transformation. From our perspective, stakeholders’ interpretations of EHR are critical to understanding their interactions with it. This, in turn, may significantly influence how actors in both organizations (Alpha and Omega) think about and act toward EHR systems in use.³⁶

We consider “cognition and micro-level processes are keys to understanding the organizational impact of new technologies.”³⁷ In this study, we adapted a practical definition of the construct of organizational learning in which the managers’ cognition, the incremental change of process of EHR implementation, the cooperation and exchange of experience between the settings of investigation, the cultural and historical relationship between the learning processes, and the socio-cognitive characteristics of the cooperative environment are all important aspects of organizational learning.^38–41 Our focus was therefore on the dynamics of interorganizational learning (hospitals learning from each other and external sources, learning from earlier experience, taking the result of our formative evaluation into consideration, and so on), the emerging nature of change (“changing”) and its characteristics, and its influence on the process of implementing EHR systems.

We adapted “sociotechnical changing” (present participle)¹⁹ perspective for data analysis. This implies that the EHR software, the care giving, the clinical practice, the organizational structures, and the carriers of institutional and professional norms were all in a state of flux, moving from somewhere now lost in the past to somewhere in the uncertain future: “becoming.”⁴² We thus concentrated on the activity “in between” the period of implementing EHR during which things were changing, rather than some predicted state of achieved change.²⁰ We sought to explain how stakeholders’ understandings and actions shaped the degree of implementation and adoption of EHR at Alpha and Omega and portray the mechanisms, which facilitated organizational learning between them.⁴³ We used the notion of “working-out” to signify a dynamic process of change over time that involved the ensemble of people, existing and emerging work practices and tools, individuals and organizational beliefs, assumptions, and expectations,^44,45 which can be understood as both cause and consequence of longer term processes of changing.⁴⁶ Through narrating the stories of EHR “in-the-making,”^47,48 we focused on exploring what stakeholders understood about EHR (perceptions, hopes, and fears), what they actually did in their routine practices to “make it work,” and how the consequent implementation at Omega benefitted from lessons learned at Alpha.

Results

The context

Both hospitals were London-based acute NHS hospitals that implemented the same version of Millennium software: LC1 (London Configuration).¹² Millennium is an established, large-scale “off-the-shelf” software, mainly developed in America by Cerner corporation (http://www.cerner.com). Alpha was a large-sized, urban, teaching hospital with over 5000 employees and serving a population of 700,000 people; it had substantial in-house information technology (IT) expertise spanning at least two decades. A few weeks after “go-live,” Alpha publicized a substantial loss of income directly associated with the implementation of Millennium. This led to a 90-day rescue plan, during which plans for further Millennium deployments were temporarily suspended.

Omega was a medium-sized, urban, district hospital with over 2500 staff and serving a population of around 320,000. It was the first post-Alpha deployment of Millennium in London, occurring 17 months after the deployment in Alpha; Omega received Millennium under the glare of substantial political and media scrutiny.

Both Alpha and Omega had similar governance structures in place to help support deployment of Millennium. They, however, had important differences in their visions of EHR and their respective implementation strategies, which contributed to their distinct approaches and experiences. Below, we describe these, with particular reference to the organizational learning that took place across and between the two hospitals. We provide some illustrative data below, with further supporting data provided in Appendix 1.

Contrasting visions of EHR

We found evidence of different interpretations of what the EHR was perceived to be, what it ended up being, and what it might become in the future. Alpha-based staff held a predominantly data-centric view, seeing EHR as an administrative tool and a digital container of information for joining-up health-care delivery and monitoring. In contrast, a more process-centric perception of EHR was dominant at Omega. Partly through various communications with Alpha (described later), EHR was perceived as a means to initiate business change and improve health-care service to help the hospital to become “more patient-focused” (Senior Manager, Omega). EHR was considered as an IT support to “improve what we do, both clinically and managerially” (Clinician, Omega) and to a change management process. Contrary to Alpha, Omega viewed the EHR as a multidimensional and sociotechnical journey: its aims were transforming clinical behaviors and improving work practices. This in turn had positive effects on the leadership, strategy, and consequences of implementation.

Distinct implementation strategies

The two hospitals pursued radically different strategic approaches. Implementing the EHR software was an end for Alpha, but was, in sharp contrast, for Omega more of a means of achieving other desired end points. With over 20 year’s history of computerization, Alpha considered the integrated EHR as another big IT project, which they believed they possessed enough in-house experience and expertise to implement. It was therefore not considered important to brief staff about the potential challenges of “going live” at Alpha.

One year into the implementation, many key people at Alpha still perceived the EHR as a complex replacement to the old Patient Administration System (PAS). While being skilled, the senior management and implementation teams had no specific prior experience of a hospital-wide EHR implementation. The composition of the implementation team was not grounded enough on operational users, which jeopardized the adoption of EHR at Alpha.

Alpha prepared the EHR business case with minimum insight about the scope of the task. From the beginning, the hospital heavily emphasized “benefits realization.” This raised users’ expectations and disappointed them when the (unrealistic) proposed benefits were either delayed or not achieved. Alpha underestimated the required cultural changes required to implement EHR, this being compounded by it overestimating its capability in relation to managing implementation. Worse still, NPfIT provided Alpha with no choice other than to use a generic training package that looked very different from the real Millennium environment. No tailored training materials or user guides were provided. Due to lack of mutual communication between the implementation team and other staff, Millennium was perceived as being imposed on users with no backfill for clinicians at the time of “go-live,” leading some staff to express the view that they “hated it” (Clinician, Alpha).

In contrast, Omega interpreted the EHR as “not the implementation of the technology but about the changes to the way in which staff undertake their daily clinical and support work” (IT Manager, Omega). The Board of Directors described EHR as a means toward change management, “a vehicle for improving the hospital performance” (Manager, Omega) and “an enabler to improve services” (Senior Manager, Omega). This was why Omega employed a Chief Executive Officer (CEO) with relevant prior experience to implement EHR:

I think that the advantage of the CEO who has already implemented Millennium twice before in other Trusts was phenomenal … She was able to put it very coherently into the strategic thinking of all the key people in the Trust. (Doctor, Omega)

Painful implementation of EHR at Alpha taught NPfIT to conduct “a softer introduction to Millennium (at Omega), because they didn’t have to take that first hit of and they were able to be on the coat tails of (Alpha) to a certain extent” (NPfIT 1). This was described “a real opportunity to jump forward, well, leap forward” (IT Manager, Omega). Instead of overestimating its internal resources, Omega outsourced several hardware and software prerequisites of Millennium. Contrary to Alpha, Omega deliberately decided not to consider benefits until they went back to “business as usual.” The Trust saw “no point in talking about benefits. You’ve just got to talk about doing it and being able to manage through it” (Manager, Omega).

Organizational learning between Alpha and Omega

The challenging implementation of EHR at Alpha resulted in “go-live” being repeatedly postponed at Omega. This taught NPfIT to conduct “a softer introduction to Millennium (at Omega)” (NPfIT 1), which brought Omega additional time and the opportunity to obtain additional information and generate insights to plan, test, reflect, and prepare in detail for a better-informed implementation. NPfIT facilitated various forums and meetings for representatives from early adopter hospitals to meet with software developers and NPfIT representatives, for example, the monthly London-wide user group meetings. Because of the detailed scrutiny on Omega’s “go-live” with Millennium, NPfIT organized more frequent project management meetings between Alpha and Omega, which resulted in an overall more appropriate implementation strategy and cross-site learning.

Furthermore, Omega delegated some clinical and administrative staff to visit Alpha and another hospital that implemented Millennium through non-NPfIT routes. This led users to “see the actual Millennium product in actual working live as it were” (Nurse, Omega), to gain more insights into it and to remember the scope of the task, which resulted in the implementation team thinking of procedures prior to deployment. Omega “embedded into the organization in advance of deployment” (NPfIT) a “Design Authority” consisting of different users including clinicians, to boost users’ engagement, and prepared user guides for the Millennium. Instead of relying only on classroom-style generic training, Omega conducted hundreds of workflow familiarization sessions for their staff.

A smoother implementation at Omega as a result of organizational learning

The differences between Alpha and Omega and the flow of insights and learning between the two hospitals, described above, led Omega to make Millennium work and experience a smoother implementation process, which resulted in more fruitful local consequences, as described below.

Most interviewees at Omega considered themselves “potential big winners from (EHR)” (Manager, Omega), partly because they learned lessons, experiences, and expertise from Alpha and applied them. For instance, reflecting on software difficulties at Alpha, which led to staff resistance, Omega tailored the Millennium software to avoid such challenges. Unlike Alpha, Omega did not “spend massive time trying to fix all sorts of problems arising during the deployment” (Nurse, Alpha). It “lost some money, but it’s trivial in comparison to (Alpha) and it’s thousands not millions” (Manager, Omega). Omega went back to “business as usual” 4 months after “go-live,” whereas this took 10 months at Alpha. Having learned about the difficulties at Alpha due to the bureaucratic deployment chain, Omega encouraged NPfIT representatives and Cerner to stay and work together at the heart of the hospital prior, during, and some months after go-live.

Discussion

Both Alpha and Omega did not, during our evaluation, see the hoped-for benefits to the organization as a result of the introduction of EHR, such as speeding-up tasks.⁴⁹ Nonetheless, the Millennium software worked out to be easier to use at Omega.⁵⁰ Organizational learning as defined earlier, we argue, was at the heart of such a difference.

Sequential implementation facilitated interorganizational learning at Omega, 17 months after Alpha went live.^51,52 The learning “on the go” between the two sites was supported by both formal processes, for example, delegates visit, workflow familiarization, lessons learned documents, local interim evaluations, training, forums, and meetings organized by NPfIT as well as often unintentional trial and error (learning-by-doing) attempts, that is, informal organizational relationships between staff, which were sometimes more beneficial than formal means of communication. Omega learned to get engaged with users from an early stage^53,54 and encouraged learning through user-friendly and innovative tailored training, which led to more insightful and positive attitudes toward EHR and a greater chance to make sense of it among staff. Such sense-making process contributed to developing particular assumptions, expectations, and knowledge of EHR at Omega, which then influenced users’ subsequent positive action toward it.³⁶ Crucial to this was a knowledgeable, experienced, and insightful leadership and managerial team who were signed-up to the vision of EHR,⁵⁵ had greater ownership of the implementation than Alpha,⁵⁶ and acted as “boundary spanners” that bridged the gap between the implementation team and end users.^57,58

Our findings indicate that users’ perceptions and values are part of their examination of the social aspects of EHR.^59,60 As an example, bilateral exchange of staff between the two hospitals also transferred cross-site knowledge and expertise. This cooperative environment led to development of common meaning and institutionalization (routinization of new work practices by embedding them into existing work practices).⁶¹

In addition, our evaluation team provided formative feedback to the sites, enabling additional learning processes,⁶² for different level of stakeholders involved.⁵¹ Beyond the simplistic dichotomy of judging the implementation of EHR as success or failure, our theoretical perspective, we argue, helped ensure that we did not reduce the EHR to delivery, implementation, and immediate use,⁴⁶ but understand it as both cause and consequence of longer term processes of changing and organizational learning, during which people and EHR came together to perform actions and tasks⁶³ as co-constitutive entities.^26,47

Policy implications

National-scale EHR systems are major new initiatives, which are likely to be difficult to manage, costly to maintain, and hard to sustain.⁶⁴ Their challenges are emerging, demanding patience, clear vision, and sustainable leadership. Their core vision may also get interpreted diversely and modified by different stakeholders. Despite inevitable and painful challenges that premier implementers may face, the cascade of organizational learning facilitated by gradual “roll out” may pave the way for followers through creation of a common language among stakeholders.

Direct and close channels of communication between the implementer hospitals and software suppliers are pivotal. It might seem trivial, but lack of shared understanding among all stakeholders of the purpose and content of the software to be chosen or produced could pose severe challenges to all parties involved. Contextual specifications are crucial in planning and selecting desired software. The decision to implement EHR should be a conscientious choice of individual organizations. Before an EHR application is designed or implemented, current work practices and workflows should be studied, the ways to improve them must be understood, and potential problems that they might bring about need to be identified.

In particular, EHR applications should satisfy both patients’ care and organizational management needs. Stakeholders’ mapping in design, implementation, and education processes and a clear reason and a long-term strategy that will maintain users’ enthusiasm and engagement are essential.⁶⁴ Training strategy needs to be ongoing, embedded in workflows, and as flexible as possible, that is, opportunistic, changing with the circumstance, and tailored to diverse users’ needs and their roles. Finally, our evaluation suggests that organizational learning within and between the implementing hospitals may be more effectively achieved through supporting and fostering the creation of informal channels and relationships than more formal means such as “lessons learned documents.” Health-care organizations that are planning to implement EHR need to consider approaches that maximize the opportunity for such sharing of experiences.

Rigor of this study

The strengths of this study were the contemporaneous real-time longitudinal data capture and the theoretically based approach to analysis and interpretation. Looking across our analytical and thematic findings enabled us to understand the process of “working-out” EHR and contributed to a cross-site organizational learning. We shared with hospitals our formative findings and early lessons learned at the time of implementation, perhaps when they needed these most. Given insufficient body of research and relatively small literature about organizational aspects of EHR implementation,⁶⁵ this study may contribute to broader understanding of the discipline.

This work should be also seen as hypothesis generating, and so our findings need to be interpreted with caution. Our evaluation only covered a relatively short period of EHR implementation in the beginning of what is likely to be a very long journey to full integration of the software. Both Alpha and Omega were “early adopters” with challenges, which were faced for the first time; these may eventually be resolved as subsequent implementers learn from these early experiences. This was inevitable given the limited number of implementations and crucial need for formative evaluation. The procedural and contextual factors explored above might have led to differentiations in terms of our participants’ perception of the process of implementation and adoption of EHR in Alpha and Omega. We thus cannot claim that if Alpha had not experienced a painful implementation, Omega could have enjoyed a much smoother experience.

Furthermore, we focused on the process of implementation not outcomes. We neither studied the impact of EHR nor did we compare the two hospitals systematically. We acknowledge that many dimensions to organizational learning may not be seen for several years, as there is a natural learning curve in any organizational change initiative.

Conclusion

Through highlighting the importance of organizational learning between two hospitals in the context of national implementation of EHR systems in England, this article adds to the hitherto limited literature on organizational considerations in relation to EHR implementation.^65,66

As NPfIT has been dismantled amid criticisms,⁶⁷ appreciating and enhancing a wealth of insightful lessons and interorganizational learning it created may improve adoption of EHR systems among health-care organizations. Albeit not explicitly, the shift of discourse and dialogue among policy makers and other stakeholders from seeing EHR as IT projects toward sociotechnical entities looks promising. Indeed, a real appreciation of the insights offered by this historic attempt to create national integrated EHR in England will, we envisage, only really become apparent in years to come.

Footnotes

Appendix

Appendix 1

	Visions of EHR	The implementation strategy	The organizational learning between Alpha and Omega	A smoother implementation at Omega as a result of learning from Alpha
Alpha	Data-centric view towards EHR: EHR is an opportunity to see real progress in medical computing in the UK. (Clinician, Alpha)	Lack of preparation: We had no time for end to end testing. We were not prepared enough. (Manager, Alpha)We did make sure the processes work and that the build is robust and that people understand their individual roles. (Doctor, Alpha) The not fit-for-purpose implementation team:You need people who are getting their hands dirty every day. We probably needed to devolve it down to a lower level and actually get operated input earlier on. (Manager, Alpha) I think the biggest problem is the expectation that it’s a little bit like going round installing three thousand copies of Microsoft Word and the following day everybody has got a word processor. It just is not that simple. (Manager, Alpha) Insufficient and inappropriate training:People were always crying out for a training domain to look and feel like the live system. (LSP 2) Lack of communication: Nobody said that they’ve got to do it now, they thought that somehow Cerner would do it but it doesn’t. (Manager, Alpha) As a result of all above: When the system didn’t fit the way they worked (at Alpha), then they had to start layering workarounds on top of workarounds and they just got into more and more trouble. (LSP 1)	Amending the implementation strategy at Omega as a result of learning from Alpha:Omega didn’t have to take that first hit of and they were able to be on the coat tails of (Alpha) to a certain extent. (NPfIT 1) Learning to get better-prepared at Omega:We did not wait till we’d had all the problems. (Manager, Omega) Better collaboration to implement EHR at Omega: What we’ve done differently between the two sites is where there is locally configurable changes that can be made and that’s been done working with the teams here (Omega). (LSP 1)	Millennium was worked-out at Omega: We made Millennium work here and we did it very well. (Doctor, Omega) A more tailored software at Omega: (Omega) applied their own build changes and their own boards, clinics, beds, bays, so it had that look and feel of what their life system would be. (LSP 1) As a result:The organizational and operational buying was better here (Omega) than I had seen in other Trusts. (LSP 1) The implementation of Millennium at Omega was “a major improvement from where we’ve been before and it’s meant that we can be much more open and clear with each other. (LSP 2)
Omega	Process-centric view toward EHR: I don’t think we are deploying Millennium here, I think we are reviewing and improving our services … The core is a change management program. (IT Manager, Omega) EHR as a change management process: EHR is 20% system and 80% user change issues” (IT Manager, Omega) for “staff to update the record in real time, so EHR became accepted as a normal part of their work. (Manager, Omega)	Not prioritizing benefits realization: There is no point in talking about benefits. You’ve just got to talk about doing it and being able to manage through it. (Manager, Omega)

Acknowledgements

We are very grateful to the Alpha and Omega hospitals for making this work possible and to all individuals who kindly gave their time. We acknowledge the intellectual support from our entire research team (National NHS CRS Evaluation team: CFHEP 005) to conduct this work. We are grateful to both our anonymous reviewers. We also thank our Independent Project Steering Committee, which was chaired by Professor David Bates. A.T. conceived the article, conducted data collection and analysis, and drafted the first version of the manuscript with A.S. and N.B., who both extensively contributed to several revisions and intellectual development of the article. All authors read and approved the final manuscript.

Declaration of conflicting interests

The views expressed in this publication are those of the authors and not necessarily those of the NHS, the National Institute for Health Research, or the Department of Health. The authors declare that they have no competing interests.

Funding

This report is independent research commissioned by the National Institute for Health Research. AS is supported by a Harkness Fellowship in Health Care Policy and Practice from The Commonwealth Fund.

References

Black

Car

Pagliari

. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011; 8(1): e1000387.

Catwell

Sheikh

. Evaluating eHealth interventions: the need for continuous systematic evaluation. PLoS Med 2009; 6(8): e1000126.

Institute of Medicine. Health and human sciences in the 21st century: charting a new course for a healthier America. New York: National Academies Press, 2009.

Institute of Medicine. Health IT and patient safety: building safer systems for better care, http://www.iom.edu/hitsafety (2011, accessed 10 February 2013).

Shekelle

Goldzweig

. Costs and benefits of health information technology: an updated systematic review. London: Health Foundation for Southern California Evidence-Based Practice Center, RAND Corporation, 2009.

Takian

Petrakaki

Cornford

. Building a house on shifting sand: methodological considerations when evaluating the implementation and adoption of national electronic health record systems. BMC Health Serv Res 2012; 12: 105.

Scott

Rundall

Vogt

. Kaiser Permanente’s experience of implementing an electronic medical record: a qualitative study. BMJ 2005; 331: 1313–1316.

Robertson

Cresswell

Takian

. Implementation and adoption of nationwide electronic health records in secondary care in England: qualitative analysis of interim results from a prospective national evaluation. BMJ 2010; 341: c4564.

Department of Health. Delivering 21st century IT support for the NHS: national strategic programme. London: Department of Health, 2002.

10.

Brennan

. The biggest computer programme in the world ever! How’s it going? J Inform Technol 2007; 22: 201–211.

11.

Cabinet Office. Major projects authority programme assessment review of the National Programme for IT, http://www.cabinetoffice.gov.uk/sites/default/files/resources/mpa-review-nhs-it.pdf (2011, accessed 10 February 2013).

12.

Sheikh

Cornford

Barber

. Implementation and adoption of nationwide electronic health records in secondary care in England: final qualitative results from prospective national evaluation in “early adopter” hospitals. BMJ 2011; 343: d6054.

13.

Warner

. A suitable case for treatment: the NHS reform. Surrey: Grosvenor House, 2011.

14.

Brennan

. THE NHS IT PROJECT: the biggest computer programme in the world ever! Oxford: Radcliffe, 2005.

15.

Cornford

Doukidis

Forster

. Experience with a structure, process and outcome framework for evaluating an information system. Omega 1994; 22: 491–504.

16.

Stake

. The art of case study research. London: SAGE, 1995.

17.

Corbin

Strauss

. Basics of qualitative research: techniques and procedures for developing grounded theory. Thousand Oaks, CA: SAGE, 2008.

18.

Mays

Pope

. Quality in qualitative health research: qualitative research in health care. London: BMJ Publication Group, 1999.

19.

Petrakaki

Cornford

Klecun

. Sociotechnical changing in healthcare. Stud Health Technol Inform 2010; 157: 25–30.

20.

Takian

Sheikh

Barber

. We are bitter, but we are better off: case study of the implementation of an electronic health record system into a mental health hospital in England. BMC Health Serv Res 2012; 12: 484.

21.

Schein

. Organizational learning: what is new? Working paper no. 3912-96, July 1996. Cambridge, MA: Massachusetts Institute of Technology (MIT) Sloan School of Management, http://hdl.handle.net/1721.1/2628 (accessed 18 February 2013).

22.

Hendry

. Understanding and creating whole organizational change through learning theory. Hum Relat 1996; 49(5): 621–641.

23.

Tsang

EWK

. Organizational learning and learning organization: a dichotomy between descriptive and prescriptive research. Hum Relat 1997; 50(1): 73–90.

24.

Pettigrew

. The awakening giant. Oxford: Blackwell Publishers, 1985.

25.

Wilson

. A strategy of change: concepts and controversies in the management of change. London: Routledge, 1992.

26.

Aarts

Doorewaard

Berg

. Understanding implementation: the case of a computerized physician order entry system in a large Dutch university medical center. J Am Med Inform Assoc 2004; 11(3): 207–216.

27.

Orlikowski

. Improvising organizational transformation over time: a situated change perspective. Inform Syst Res 1996; 7(1): 63–92.

28.

Mintzberg

. Crafting strategy. Harvard Business Review, July–August 1987, pp. 66–75.

29.

Lorenzi

Riley

. Managing change: an overview. J Am Med Inform Assoc 2000; 7: 116–124.

30.

Ash

Gorman

Lavelle

. Perceptions of physician order entry: results of a cross-site qualitative study. Methods Inf Med 2003; 42(4): 313–323.

31.

Kling

Scacchi

. The web of computing: computing technologies as social organization. In: Yovits

(ed.) Advances in computers. New York: Academic Press, 1982. 21: 2–90

32.

Berg

. The search for synergy: interrelating medical work and patient care information systems. Methods Inf Med 2003; 42(4): 337–344.

33.

Massaro

. Introducing physician order entry at a major academic medical center: I. Impact on organizational culture and behavior. Acad Med 1993; 68(1): 20–25.

34.

Orlikowski

. Using technology and constituting structures: a practice lens for studying technology in organizations. Organ Sci 2000; 11(4): 404–428.

35.

Berg

Aarts

Van Der Lei

. ICT in health care: sociotechnical approaches. Methods Inf Med 2003; 42(4): 297–301.

36.

Orlikowski

Gash

. Technological frames: making sense of information technology in organizations. Acm T Inform Syst 1994; 2(2): 174–207.

37.

Weick

. Technology as Equivoque: sensemaking in new technologies. In: Goodman

Sproull

and Associates (eds) Technology and organizations. San Francisco, CA: Jossey-Bass, 1990, pp. 1–44.

38.

Brown

Duguid

. Organizational learning and communities of practice: toward an unified view of working, learning and innovation. Organ Sci 1991; 2(1): 40–57.

39.

Argyris

. Reasoning, learning and action: individual and organizational. San Francisco, CA: Jossey-Bass, 1982.

40.

Bartunek

. The dynamics of personal and organizational reframing. In: Quinn

Cameron

(eds) Paradox and transformation: toward a theory of change in organization and management. Cambridge, MA: Ballinger, 1988: 137–62.

41.

Geresick

CJG

. Revolutionary change theories: a multilevel exploration of the punctuated equilibrium paradigm. Acad Manage Rev 1991; 16(1): 10–36.

42.

Clegg

Kornberger

Rhodes

. Learning/becoming/organizing. Organization 2005; 12(2): 147–167.

43.

Cho

Mathiassen

Nilsson

. Contextual dynamics during health information systems implementation: an event-based actor-network approach. Eur J Inform Syst 2008; 17: 614–630.

44.

Davidson

Chiasson

. Contextual influences on technology use mediation: a comparative analysis of electronic medical records systems. Eur J Inform Syst 2005; 14: 6–18.

45.

Klecun

Cornford

. A critical approach to evaluation. Eur J Inform Syst 2005; 14: 229–243.

46.

Lin

Cornford

. Sociotechnical perspectives on emergence phenomena. In: Coakes

Willis

Lloyd-Jones

(eds) The new sociotech: Graffiti on the long wall. Godalming: Springer, 2000, pp. 51–60.

47.

Berg

. Patient care information systems and health care work: a sociotechnical approach. Int J Med Inform 1999; 55(2): 87–101.

48.

Latour

. Science in action: how to follow scientists and engineers through society (New edition). Cambridge, MA: Harvard University Press, 1988.

49.

Bates

Kuperman

Wang

. Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality. J Am Med Inform Assoc 2011; 10(6): 523–530.

50.

Yarbrough

Smith

. Technology acceptance among physicians: a new take on TAM. Med Care Res Rev 2007; 64(6): 650–672.

51.

Crossan

Lane

White

. An organizational learning framework: from intuition to institution. Acad Manage Rev 1999; 24: 522–537.

52.

Keshavjee

Bosomworth

Copen

. Best practices in EMR implementation: a systematic review. AMIA Annu Symp Proc 2006: 982.

53.

Puffer

Ferguson

Wright

. Partnering with clinical providers to enhance the efficiency of an EMR. J Healthc Inf Manag 2007; 21(1): 24–32.

54.

May

. A rational model for assessing and evaluating complex interventions in health care. BMC Health Serv Res 2006; 6: 86.

55.

Van Ginneken

. The computerized patient record: balancing effort and benefit. Int J Med Inform 2002; 65(2): 97–119.

56.

Takian

. Envisioning electronic health record systems as change management: the experience of an English hospital joining the National Programme for Information Technology. Stud Health Technol Inform 2012; 180: 901–905.

57.

Yusof

Kuljis

Papazafeiropoulou

. An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008; 77(6): 386–398.

58.

Greenhalgh

Robert

Macfarlane

. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004; 82(4): 581.

59.

Kling

Iacono

. The institutional character of computerized information systems. Office Tech People 1989; 5(1): 7–28.

60.

Markus

. Systems in organizations: bugs and features. Boston, MA: Pitman, 1984.

61.

Argyris

Schön

. Organisational learning: a theory of action perspective. Reading, MA: Addison-Wesley, 1978.

62.

Daft

Weick

. Toward a model of organizations as interpretation systems. Acad Manage Rev 1984; 9(2): 284–295.

63.

Law

. Organizing modernity: social order and social theory. Wiley-Blackwell, 1993. Oxford, UK & Cambridge, MA, USA.

64.

Takian

Cornford

. NHS information: revolution or evolution? Health Policy Technol 2012; 1: 193–198.

65.

Braude

. People and organizational issues in health informatics. J Am Med Inform Assoc 1997; 4(2): 150–151.

66.

England

Stewart

Walker

. Information technology adoption in healthcare: when organisations and technology collide. Aust Health Rev 2000; 23(3): 176–185.

67.

Robertson

Bates

Sheikh

. The rise and fall of England’s National Programme for IT. J R Soc Med 2011; 104(11): 434–435.