Barriers to the secondary use of data in critical care

Abstract

Purpose

We explore the challenges of the secondary use of data in clinical information systems which critical care units in the National Health Service (England) are facing.

Methods

We conducted an online survey of critical care units in England regarding their practices in collecting and using clinical information systems and data.

Results

Critical care units use clinical information systems typically independently of hospital information systems and focus mainly on using data for auditing, management reporting and research. Respondents reported that extracting data from their clinical information system was difficult and that they would use stored data more if it were easier to access. Data extraction takes time and who extracts data, the training they receive and the tools they use affect the extraction and use of data.

Conclusion

A number of key challenges affect the secondary use of data in critical care: a lack of integration of information systems within critical care and across departments; barriers to accessing data; mismatched data tools and user requests. Data are predominantly used for reporting and research with less emphasis on using data to inform clinical practice.

Background

The Institute of Medicine^1,2 identifies digital data held in clinical information systems (CIS) as a key resource in the improvement of the quality of health care delivery. While the positive effects of CIS on healthcare quality has been supported by a number of systematic reviews,^3,4 this has mainly been due to improvements in the quality of the care records, ⁵ rather than the use of the digitalized data outside the immediate episode of care. The potential of the secondary use of CIS data to identify opportunities for quality improvement is recognized to be important, ⁶ and shown in retrospective analytical studies.^7,8 Such reuse of CIS data is actively promoted in national health care strategies^9,10 although the cost and feasibility of accomplishing this has not been well researched.

We aimed to provide foundational evidence for how CIS’s are currently being used in critical care units in England. We explored aspects of the work required to collect and use data, to highlight some of the barriers to the secondary use of data and chose critical care units as they generate large volumes of data recorded both through automated monitoring and ad hoc manual data entry. We conducted a survey of English critical care units to provide a descriptive overview of the context and current issues related to the clinical use of digital data.

Methods

We conducted a survey of all Critical Care Units (ICU) in England registered with the Intensive Care National Audit and Research Centre (ICNARC; https://www.icnarc.org) program. A link to complete the online survey was sent by ICNARC to nominated contacts in each unit. The first request to complete the survey was followed by a second invitation sent to non-respondents after a few weeks.

Survey respondents were asked to complete an online survey form that included a list of questions regarding the use of data in critical care (see Appendix 1). To gain an understanding of the context in which data is generated within critical care units we asked respondents to inform us about the CIS implemented in their units. Further, in order to ascertain what was being done with data and what some of the challenges were, we asked respondents a range of questions about data extraction practices in their units.

The survey was created and run using online survey software, Qualtrics (Utah, USA). The survey was designed and tested within the research team. Following this, we presented the survey to a project steering group of eight professionals from the medical, computing and information systems and informatics sectors, for external comment and made suitable amendments as suggested. Three typical question types were employed in the survey. For specific issues, such as which CIS provider a unit used, respondents could only select a single option. For issues where we sought to understand the range of challenges units were facing, such as how data are extracted, respondents could select multiple options. We also used a five-point scale (from strongly disagree to strongly agree) in order to gauge how respondents rated particular statements, such as whether it was easy to extract data from their CIS. The survey questions are shown in Appendix 2.

Once the survey was complete the data were exported and reviewed. Responses from users who started the survey but either did not answer questions or stopped their response after too few questions were excluded. The data were analyzed in a statistical software package, IBM SPSS Statistics. Although we performed statistical analyses on the data, the focus for this study was on providing a description of the use of data in critical care.

Results

Of the 283 individuals contacted, 148 (52.3%) responses to the survey were received. Of the 148 responses, 57 responses were not sufficiently completed to be included in the results of the survey. Of the remaining 91 (32.2%) responses, 25 individuals indicated their units did not use a CIS. The following results are based on the remaining 66 (23.3%) survey responses.

The CIS installed base varied across CIS vendors, with 16 different suppliers mentioned by respondents. Aside from prominent CIS vendors, such as iMDSoft (Dusseldorf, Germany) and Philips Healthcare (Amsterdam, Netherlands), which accounted for 46.1% of CIS implementations, there were a range of smaller suppliers including critical care units which had built their own solutions. Critical care CIS implementations were generally independent of wider hospital information systems. 68.2% of critical care CIS installations were not integrated with other hospital information systems at all and only 3% were reported as being fully integrated.

Respondents confirmed that the main purposes reported for using the collected data were auditing, management reporting and research; 93.8% of critical care units reported providing data to ICNARC while almost half provide data to other national audit bodies; 87.7% of units used the data for internal audits. Aside from auditing, 76.9% used the data for management reporting and 66.2% used data for research purposes.

The majority of Respondents (79.1%) agreed that they would be able to use data stored in their CIS more if they were easier to extract; 56.5% of respondents stated that it was not easy to extract data from their CIS, compared with 24.2% of respondents who stated that it was easy to extract data.

Data were extracted by individuals with various roles across the critical care department. The most prominent of these were consultants, data clerks and nurses (both clinical and research). IT department staff provided some support, although to a lesser degree. Even when data could be extracted though, it was considered challenging to interpret the resulting datasets. More respondents (41.9%) thought that the data extracted could not be readily understood by clinicians compared with those who thought that extracted data could be readily understood by clinicians (33.9%).

Given the challenges individuals may face in working with data and the CIS we asked respondents about the training users received for such tasks. While many reported receiving training from the CIS vendor directly (44.6%) or from another supplier (13.8%), many relied on their vocational training they previously undertook (30.8%), self-training (40%) or in-house training (12.3%). Of note was the fact that about 23% of respondents reported having received no training at all.

In all, 80.9% of units reported spending more than 1 day per month extracting data, with a reported 20.6% of respondents saying their units spent more than 1 week per month on such tasks. Despite this, 36.8% of respondents reported that no funding was allocated to data extraction specifically. For those that did receive funding, 59.7% reported spending between £0 and £50,000 per annum. Typically such work was not separately funded with 76.2% of respondents reporting that data extraction tasks were included as part of normal duties.

We asked units to tell us what methods they used to extract data. In all, 62.5% of units relied on tools provided by the CIS vendor. Some units also sourced locally written software (42.2%) or used generic tools (34.4%). Significantly, more than half of respondents (54.7%) reported that data were manually extracted from the CIS, that is, individuals manually transferred data by viewing a CIS data screen and transcribing values into another tool.

Critical care units used a combination of methods to support data extraction for analysis. While clinical staff in some sites were reportedly able to extract data to support data analysis work (36.1%), much of the data extraction to support data analysis was done by specific individuals tasked with such work (up to 70.5% of respondents reported having to request support from responsible staff). These individuals not only supplied raw data but often undertook data analysis as well. Nearly two-thirds of respondents (64.5%) observed that clinical staff did not understand how to make data requests in a way that suited how data were stored in the CIS. Only 11.3% of sites state that clinical staff were able to do this.

Even if data could be extracted, about half of respondents (51.6%) reported that the current tools to analyze extracted data were not sufficient to support clinical decision making. This compares with 16.2% of respondents who suggested that current tools were sufficient to support clinical decision making and 32.2% who regarded current tools as neither sufficient nor not sufficient.

Discussion

While the overall response rate for this survey is low, the study does highlight the issues that a significant number of critical care units in NHS England are facing. We recognize that interpreting these results provide limited opportunity for generalization. Despite this, even if non-respondents face no issues in the secondary use of data and their CIS implementations were considered unproblematic, which is an unlikely scenario, the results of the survey still bear relevance for understanding current issues of data and CIS use in critical care. We should also point out that of the responses received 16.9% of respondents suggested they did not use a CIS at all, highlighting a challenge for these units of managing data and their reuse without the use of technology.

The results of this survey suggest that there are significant obstacles to the use and reuse of data within critical care. The key obstacles include: first, a lack of integration of information systems within critical care and across departments; second, barriers to accessing data constrain their reuse; third, data tools and mismatched user requests constrain the efficiency of data work; and fourth, data are predominantly used for reporting and research with less emphasis on using data to inform clinical practice.

Given the variety of CIS’s being used in critical care and that most systems are not integrated with other hospital information systems, integration and standardization remain a challenge. Critical care units across England rely on a CIS to facilitate the delivery of health care but this is often done in isolation, both within and beyond the hospital. Since critical care CIS’s are often not integrated or, if there is integration it is only partial, data held in a CIS are rarely accessible elsewhere in the hospital. Additionally, for various reasons different units choose different systems which often have limited ways of sharing data. Standards for sharing a wide range of data are limited, constraining opportunities to find new ways of using or repurposing data. A further challenge is that while critical care units may face similar issues in relation to data reuse, any solution is unlikely to work across all units.

Data access is a significant issue which critical care departments face. Our study shows that there is significant latent demand for data (80% agreed they would use data more if they had it) but there are significant hurdles in extracting the data and subsequently analyzing it. While not often highlighted, in either practice or research, the very process of getting the data out of an information system is challenging. It takes a significant amount of time, costs money, and requires skilled users to accomplish. Furthermore, a wide range of different staff are involved in data extraction but training is limited and users are skilled to varying degrees. Additionally, while CIS tools exist these are not necessarily easy to use or fit for purpose. Most often the responsibility for extracting data and/or conducting data analysis falls to particular individuals. Given that much of this work appears to be done as part of normal duties and there is limited funding to support such work separately, it begs the question as to how effective this approach is likely to be in delivering the sorts of evidence that clinicians require.

A compounding challenge is that data workers are often constrained by the CIS they are using. They are reliant on vendor-supplied tools that may not suit their needs, have to develop tools locally at a cost or adapt generic tools in some way. How much of these efforts are being duplicated across critical care units raises a question regarding the efficiency of such processes. That data workers often turn to manually working with data is an indication of the inefficiencies in data working processes. That users have low awareness of data availability and structure means they are often unable to articulate requests for data and/or analysis appropriate to the data that can be obtained.

Finally, when data are reused the key focus is on reporting, whether for auditing or management purposes and research. Furthermore, even when data are available to clinicians current tools to analyze the data do not facilitate clinical decision-making. This highlights an important area for data reuse to affect the quality of health care delivery. From the results of our survey, we contend that one of the significant consequences of all the aforementioned challenges is that data are currently underutilized and may not be easily mobilized to support care delivery unless the underlying issues are addressed. For example, by being able to access, review and analyse all of a unit’s historical data, clinicians may be able to use the data to aid their decision-making for the treatment of existing patients. If this access was relatively straightforward, such a process could be real-time which would overcome the constraints faced when using audit or research data that typically takes time to process. Developing such systems and processes might be considered considerably challenging but doing so would yield the opportunities for improving the quality of care that the reuse of data offers.

Given the range of challenges highlighted in this study we argue that a multipronged approach is required to address these. For example, while national strategies and industry programs may be necessary to agree to set standards and integration requirements, it is necessary for individual hospitals to drive such actions from the user perspective. In the course of our wider research, for example, we still encounter new CISs implementations where integration and data standards are not key focuses. A pragmatic focus on ensuring that systems can at least provide a basic set of useful data to be shared amongst relevant users would be a productive start.

Our study points to the need for the recruitment of staff with the requisite skills to work with data and CISs. At present, it is unlikely that clinicians, in general, have all the skills to deal with many of the practical challenges in working with data. Improved training may help motivated individuals who have the time to focus on such tasks but it may be useful to consider specific data work roles. The uneven spend on data work and training across our survey respondents suggest that this may be difficult to achieve everywhere. One pragmatic opportunity may be to make shared data workers available across hospitals to overcome the uneven availability of such skills.

It would also be useful to consider some of the technical issues, particularly the suggestion that current tools are not as effective as needed. While replacing an entire CIS is impractical and changing a CIS dramatically is unlikely, it may be possible to develop tools that address particular time-consuming, costly issues users have to deal with. We have in our wider research observed how users attempt to create local solutions that save time and money. A further pragmatic opportunity exists to create a centralized hub for users to share ideas and solutions addressing issues that could lead to more effective use of data. It is also necessary to note that critical care units are supported by a range of CIS providers. These solutions are fundamentally different and even implementations of the same CIS could have significant differences in, for example, how data are stored. Using more direct methods for querying databases directly (e.g. using SQL) are unlikely to be straightforward and would need the support of CIS providers along with significant additional training for users.

Finding new ways to enable users to engage with data could therefore be vital. For example, end-user development (EUD) techniques offer the potential for system users to become more active in modifying and customizing software tools for their own needs. ¹¹ Use of EUD methods in the critical care context has the potential to increase access to data, and potentially overcome some of the issues we have identified. ¹² One example of a software approach that applies these methods is the use of program synthesis methods so that practitioners can create new software functionality by demonstrating the data formats that they need to work with, as in the ‘Data Noodles’ prototype described by Gorinova et al. ¹³

Conclusion

While many of these challenges and considerations may not be new to the health care sector’s discussion on the relevance and use of data, the study we presented seeks to highlight the real concerns users are facing in their daily work. We do not suggest that dealing with these issues will be straightforward but addressing the challenges in practical ways, along with national strategies and policy, could help to ease the burden and facilitate quality health care delivery.