Sage Journals: Discover world-class research

Abstract

In the last decade, health providers have implemented information systems to improve accuracy in medical diagnosis and decision-making. This article evaluates the impact of an electronic health record on emergency department physicians’ diagnosis and admission decisions. A decision analytic approach using a decision tree was constructed to model the admission decision process to assess the added value of medical information retrieved from the electronic health record. Using a Bayesian statistical model, this method was evaluated on two coronary artery disease scenarios. The results show that the cases of coronary artery disease were better diagnosed when the electronic health record was consulted and led to more informed admission decisions. Furthermore, the value of medical information required for a specific admission decision in emergency departments could be quantified. The findings support the notion that physicians and patient healthcare can benefit from implementing electronic health record systems in emergency departments.

Keywords

Bayesian statistics decision support systems decision trees electronic health records emergency department medical decision-making

Introduction

In the last decade, many health providers have turned to medical technologies to improve medical diagnoses, managerial capabilities, and decision-making processes.^1–3 The increase in national health expenditures and the drive to improve the quality of healthcare have prompted the widespread adoption of health information technologies (HIT).⁴ Early studies suggested that information technologies (ITs) could reduce risks and uncertainties that stem from a lack of information.^5,6 In their everyday work, medical staff are under time constraints and cannot always wait for diagnostic test results.⁷ Electronic health record (EHR) information systems (ISs) are designed to facilitate the retrieval of patients’ digital records from many decentralized health providers (known as interoperability). Accessing an EHR has been shown to improve diagnostic accuracy.⁸ The implementation of these systems in the healthcare sector has been the subject of many studies. The findings show that design and ease of use are critical for the adoption of EHR since the first experience and the system interface with its users are highly influential.⁹ Huerta et al.¹⁰ found that hospitals that adopted EHR system are more productive. Others have found that EHR use increases quality of care.^11,12 Goh et al.¹³ suggested an EHR design that is compatible with physicians’ daily routines.

However, the impact of EHR on high-stress emergency departments (EDs), which often deal with enormous numbers of patients under heavy time constraints, is less straightforward.^14,15 Projections suggest there could be US$1.9 million drop in ED costs in a single region in the United States if an interoperable system was put in place.¹⁶ A lack of theory and experience with EHR systems has led to ineffective governmental spending on redundant testing or other assessments in the ED.¹⁷ EHR systems require a large investment and thus constitute an economic challenge. However, Boonstra and Broekhuis¹⁸ listed eight other obstacles (besides the financial investment).

EHR systems coordinate the storage and retrieval of patient records from multiple health sources such as laboratories, other hospitals, and specialized clinics, thus providing vital historical medical information that is required for critical decision-making. Information retrieved from EHR is not random; there are specific patient factors that increase the odds of information access.² There is a strong connection between viewing information stored in the EHR and the clinical decision process.¹⁹ EHR can provide professionals or non-professional caregivers a complete, accurate, and searchable health information that is available at the point of care, thus allowing for a more informed decision-making process and enhancing the quality and reliability of healthcare delivery.²⁰ Medical professionals value quick access (one click) to results that are automatically found based on the diagnosis in the EHR.²¹ Primary care physicians with substantial experience in EHR use have attested to the system’s advantages such as avoiding medical errors, delivering preventive care, and improving quality of care.²²

Contribution of EHR systems to medical decisions

Despite the increasing availability of medical technologies to clinicians, there has been less research documenting the effectiveness of their use.¹⁴

Medical decision-making usually takes place in the context of a healthcare institution or organization and should ideally be structured. EHR can possibly simplify the process by providing access to medical history, tests, imaging, and so on, in a computerized format—all in one place. Providing information and therapeutic options can often reduce the complexity of the decisions medical staff are required to make.²³ Several theoretical frameworks for assessing the potential value of electronic medical information have been put forward in recent years.^24,25 Claxton et al.²⁶ and Walker et al.¹⁹ showed that computer-assisted interoperability between healthcare providers could lead to a reduction in redundant tests, improve medical performance, and also provide a positive financial return on investment. Presumably, HIT can elevate the level of care and save costs by reducing unnecessary tests and avoiding mistakes.²⁷ In addition, the system provides better communication between different physicians and can help avoid certain mistakes.²⁸ It also gives the physician access to several types of information that are usually unavailable through conversation with the patient (e.g. past laboratory and imaging results). The EHR can help lessen physicians’ cognitive overload by providing the information and environment (less time spent searching for paper tests, histories, etc.) needed for the decision-making process.²⁹ Ben-Assuli and colleagues.^30,31 evaluated the contribution of IT to medical decision-making processes and concluded that physicians who use information components from EHR systems can improve decision-making and thus enhance medical efficiency. EHR influences and improves physicians’ decision-making.^32,33 Patients in the emergency room are less likely to be hospitalized if physicians use EHR during their examination and treatment.³⁴ James and Savitz³⁵ found a reduction in admission rates in hospitals that implemented EHR.

However, studies have also failed to report a positive impact of ISs in terms of medical outcomes and decision-making processes.^23,36,37 Many factors influence medical decision-making, including stress and time pressure,³⁸ experience,³⁹ risk and uncertainty,⁴⁰ and amount of information. Physicians in the ED were found to retrieve only a limited amount of relevant information, even when there were no time constraints,⁴¹ and to use EHR systems for far fewer tasks than the systems supported.⁴¹ One explanation may be ineffective staff education and improper integration with the existing workflow.^42,43 Alternatively, doctors may perceive this technology as a threat to their professional autonomy.⁴⁴ Lack of use may also be related to failed implementation due to organizational challenges such as poor fit between the ISs and the organization’s structure and policy.⁴⁵ Consequently, there is a gap between the literature and studies describing the impact of EHR on workflow, care, and costs and their influence on the way these systems are designed or implemented. This was best described by Brynjolfsson and Hitt³⁶ as the productivity paradox. This paradox suggests that the influence and improvement in medical decision-making and medical care of EHR systems are not easily identifiable. In a more recent work, Tambe et al.⁴⁶ found that IT can influence organizational change and productivity. Nevertheless, obstacles and difficulties in implementing EHRs are still limiting its potential success. Our intent is to overcome this gap by demonstrating a beneficial and advantageous aspect of the system to the healthcare sector.

Decision analytic approach to assessing the value of information

The subjective approach to assessing the value of information reflects users’ perspectives on the advantages and disadvantages of ISs. The realistic approach to assessing the value of information measures the difference in performance before and after obtaining information.

The decision analytic approach to assessing the value of information predicts a system’s value based on a theoretical model derived from decision analytic models and from the predicted behavior of the decision-maker. For instance, under the assumption that the decision-maker behaves rationally, the expected utility (EU) or expected profit can be calculated. Each approach has certain advantages as well as shortcomings, and the optimal approach to a given situation depends on the information to be assessed and the variables of the situation.

A decision analytic evaluation of an IS such as an EHR system is usually undertaken when the information quality cannot be entirely objectively measured, for instance, when the reliability of the information is difficult to evaluate⁴⁷ as in many clinical cases. The more unstructured the decision, the more difficult it is to measure the value of information. Unstructured decisions usually involve numerous unknowns and complexities and may involve extensive intuition as well; altogether, this can make it rather difficult to assess the quality of the decision process.⁴⁸ In healthcare, for instance, a measure of information can cover real-world issues such as medical risks, people skills, and organization politics.^49,50 In the case of physicians, decision-making is fairly amorphous and usually extremely complex; there is no way to work out “the best answers.” Therefore, in this case, assessment of information is considered the most reliable indicator, and hence was used in this study.

Another advantage of using the decision analytic approach to assess the value of information is the feasibility of obtaining results before the EHR is up and running. It is very useful to carry out measures on planned systems that are not yet part of the users’ or decision-makers’ environment.⁵⁰ In this study, the assessment of the value of information was carried out by preparing a theoretical analytical model and running a clinical decision-making simulation. The use of simulations as a research method in empirical studies on EHR nevertheless remains the exception to the rule. Simulating an EHR can provide a realistic environment for the system’s use and human–machine interaction studies.⁵¹ A previous study investigating genetic testing using an EHR simulation reported a positive impact.⁵² A simulation also improves quality of care because there are no medical outcomes and patients’ privacy issues are not violated.⁵³ Russ et al.⁵⁴ showed in an EHR simulation study that the system reduces human error by adding medication alerts. We used decision trees as presented by Pauker and Kassirer^55,56 and implemented in numerous studies such as those by Dotan et al.⁵⁷ and Golan et al.⁵⁸

Below we present the decision tree and explain the nomenclature and calculations for each step. We then describe the approach selected for the assessment of the payoff utilities and the EU of the physicians. Finally, we detail the results, followed by the discussion, limitations, and suggestions for further research.

Objectives

This article presents a decision analytic approach to assessing the value of information, in terms of its contribution to efficient decision-making among ED physicians dealing with patients with coronary artery disease (CAD). Efficient decision-making within the framework of the ED means an increase in justified admissions and a reduction in potentially redundant admissions. The purpose of this study was to contribute to further elucidate the role of EHR in medical decision-making, by analyzing what constitutes efficient use of the system. It was conducted by applying a decision-tree model that depicts the typical steps in the medical decision-making of ED physicians, as elaborated in the “Materials and method” section.

Materials and method

Role of Bayesian statistics in the proposed analysis

Bayesian inference uses the evidence previously gathered from a certain field and deduces the probabilities accordingly. Bayesian inference is based on calculating the posterior possibilities of hypotheses.⁵⁹ The Bayesian statistic generates a stochastic distribution of the parameters predicting an event.⁶⁰ The end result of a Bayesian analysis is a joint distribution that fits both the observed data (evidence from studies) and probabilities or projections. The Bayesian model is frequently used in decision-making studies since researchers can develop a predictive decision model. Cho et al.⁶¹ used a Bayesian inference method to predict a specific risk factor for length of stay in the intensive care unit (ICU). They found that using a clinical support system reduced significantly the length of stay and hospital admission–related stress conditions. Others have used Bayesian models to predict the prevalence of diseases such as pancreatic cancer by combining information from EHR and PubMed to calculate the weighted risk factors.⁶² Kawaler et al.⁶³ were able to predict risk factors for developing blood clots after hospitalization based on information retrieved from the EHR. Data retrieved from these systems may also contribute to studies determining factors and projections of medical processes and therapeutic alternatives. Nevertheless, the probabilities assigned to clinical decision-making can be subjective rather than mathematical. The tendency of clinicians to over-estimate diagnosis probabilities may cloud their decision-making process.⁶⁴ Specifically, physicians have problems understanding receiver operating characteristics and likelihood ratios, which are critical in deciding on diagnosis informed by prior evidence-based probabilities.⁶⁵ Florkowski⁶⁶ argued that determining the pretest probability is a prerequisite to undertaking any diagnosis test, and presupposes that diagnosis uncertainty exists. Furthermore, the author noted that the quality of the studies that are determining the prior probabilities must be based on rigorous to avoid bias. To overcome this hurdle, Bayesian statistics employ results from previous studies to calculate the posterior probabilities.

In this study, we used Bayesian inference in the decision-tree model. The EU was calculated based on knowledge and studies on the clinical situations^{49,50,67–69} physicians face in previous studies. Using Bayes’ rule as a guideline, posterior probabilities were calculated, and based on these, our decision analytic model was formulated.

Decision tree with the EU approach

A physicians’ EU approach was chosen to analyze the relationships between correct medical decisions and various payoffs.

Below is an illustration of a classical admission decision node (Figure 1).

Figure 1.

Admission decisions and general utilities.

We employed several definitions as follows:

$P$ —the probability of having a CAD differential diagnosis (DD);

$U_{11}$ —the utility when the DD is CAD (D+) when an admit decision was made;

$U_{12}$ —the utility when the DD is not CAD (D−) when an admit decision was made;

$U_{21}$ —the utility when the DD is CAD (D+) when a discharge decision was made;

$U_{22}$ —the utility when the DD is not CAD (D−) when a discharge decision was made;

$D$ — $D +$ : DD of CAD (admission required) and $D -$ : non-CAD-related DD admission). We used one threshold probability (having CAD), which represents an indifference point between admission and discharge decisions. Guyatt et al.⁶⁷ emphasized the importance of the threshold-value approach in medical resource allocation issues;

$E_{A} (U)$ , $E_{D} (U)$ —the EU of admit and discharge decisions, respectively.

According to Figure 1, the EU of admission and discharge would be respectively

E_{A} (U) = P \cdot U_{11} + (1 - P) \cdot U_{12}

(1)

E_{D} (U) = P \cdot U_{21} + (1 - P) \cdot U_{22}

(2)

Equation 1 presents the physicians’ EU in an admission decision which is calculated by multiplying the probability of a CAD DD (P) with the utility of CAD DD when the patient is admitted to the hospital (U₁₁) and adding the complementary cases of the probability of no utility with a CAD DD when the patient is discharged by calculating the multiplication of one minus all the probabilities (1 − P) of the utility when the diagnosis is not CAD and the patient is admitted (U₁₂). Equation 2 presents calculations of the physicians’ EU when they decide to discharge the patient after diagnosing CAD. The probability of a patient with CAD DD (P) is multiplied by the utility of a CAD DD when the patient is discharged (U₂₁). We also add the complementary cases of the probability (1 − P) for a patient without CAD DD and discharged from the hospital (U₂₂).

The threshold probability of having the CAD DD ( $P$ ) can be calculated by comparing the EU of admission and discharge ( $E_{A} (U) = E_{D} (U)$ )

We assumed that the payoff utility for the discharge of a patient having a CAD DD is 0 (the worst option), and the payoff utility for the discharge of a patient not having a CAD DD is 1 (the perfect option). Consequently, the payoff utility of an admission decision ranges from 0 to 1: $1 > U_{12} > U_{11} > 0$ . Hence

{\begin{cases} E_{A} (U) = E_{D} (U) \\ s . t : 1 > U_{12} > U_{11} > 0 \end{cases}

(3)

Equation 3 depicts the constraint $U_{12} > U_{11}$ since $U_{11}$ represents sicker patients (the utility in the case where the DD is CAD (D+) when an admit decision was made) than patients represented by $U_{12}$ (the utility in the case where the DD is not CAD (D−) when an admit decision was made), since the utilities represent the medical state of the patients. The calculated threshold P is as follows

E_{A} (U) = P \cdot U_{11} + (1 - P) \cdot U_{12}

(4)

E_{D} (U) = P \cdot 0 + (1 - P) \cdot 1 = 1 - P

(5)

E_{A} (U) = E_{D} (U)

\begin{array}{l} P \cdot U_{11} + (1 - P) \cdot U_{12} = 1 - P \\ ⇓ \end{array}

(6)

P = \frac{1 - U_{12}}{1 + U_{11} - U_{12}}

(7)

We selected the threshold $P$ value based on previous studies dealing with comparable CAD cases,^68–71 and we calculated the appropriate values for $U_{11}$ and $U_{12}$ (equation 4). The final values with their range for the sensitivity analysis appear in Table 1. Sensitivity and specificity were based on calculations of positive and negative predictive values from previous clinical case studies.^68,69,72,73 These values were validated by consulting with a panel of senior physicians working in conjunction with MSR, an international Medical Simulation Center (http://www.msr.org.il/e). The panel also included internal physicians (some specialized in cardiology) and emergency physicians.

Table 1.

The values of the variables.

Variable	Definition	Value [Range of sensitivity analysis]
$P (T_{1}^{+} \| D^{+}) = s e n T_{1}$	The sensitivity of the T₁ examination (ECG)	73.68% [65%–81%]
$P (T_{1}^{-} \| D^{-}) = s p e T_{1}$	The specificity of the T₁ examination (ECG)	90.15% [80%–95%]
$P (T_{2}^{+} \| D^{+}) = s e n T_{2}$	The sensitivity of the T₂ examination (CE)	72.63% [65%–79%]
$P (T_{2}^{-} \| D^{-}) = s p e T_{2}$	The specificity of the T₂ examination (CE)	91.45% [85%–95%]
	The probability of having a CAD diagnosis without information via the EHR	Case 1: 0.55 [0.45–0.65]
$P (D^{+})$		Case 2: 0.22 [0.16–0.28]
$P (D^{+})$ ^a	The probability of having a CAD diagnosis with information via the EHR	Case 1: 0.05 [0.02–0.08]
		Case 2: 0.2 [0.15–0.25]
U₁₁	The utility in the case where the DD is CAD (D+) when a decision to admit was made	0.8 [0.72–0.88]
U₁₂	The utility in the case where the DD is not CAD (D−) when a decision to admit was made	0.97 [0.89–1]
U₂₁	The utility in the case where the DD is CAD (D+) when a discharge decision was made	0^b
U₂₂	The utility in the case where the DD is not CAD (D−) when a discharge decision was made	1^b

ECG: electrocardiogram; CE: cardiac enzymes; EHR: electronic health record; CAD: coronary artery disease; DD: differential diagnosis.

The sensitivity analysis did not involve changes in U₂₁ and U₂₂, which are regarded as an anchor.

The same case with information via the EHR.

Description of the decision tree in the model

We depict the whole decision tree (Figure 2) step by step starting from the root node with access to the EHR. This is followed by the first chance node with the test results and then the second decision node containing the admission decision, and the tree is ended with the final chance node representing the impact of each admission decision on the medical outcomes, in utility values.

Figure 2.

Stage 4—the full decision tree.

Stage 1: the root node in the decision tree—access to the EHR

First Scenario—Full mandatory access to a simulated EHR.

Second Scenario—No access to a simulated EHR.

It is important to note that the same scenario was tested in both cases (with and without access to the EHR). The only difference between the scenarios was that when access to the EHR was available, there was access to additional information, including previous encounters, diagnoses, lists of medications, previous lab tests, previous clinical advanced tests, percutaneous coronary intervention results, and so on.

Stage 2: the first chance node—examination results

Clinical information on patients can affect the posterior probability that the subject has CAD and thus impact the decision to admit the patient. Electrocardiogram (ECG) and cardiac enzymes (CE) (troponin I, troponin T, CK-MB, and myoglobin) are ordered extensively in cases of chest pain. Previous research has indicated that ECG and CE together are the current diagnostic gold standard of CAD diagnosis and complement the clinical assessment.^68,71 Theoretically and practically, each examination can lead to results which can be interpreted in two ways: they can either confirm the CAD DD and indicate a justified need for admission ( $T_{n}^{+}$ ) or disconfirm this DD and lead to an unnecessary admission ( $T_{n}^{-}$ ). We begin by examining the impact of ECG (T₁) and CE (T₂) results on a CAD DD. We define the following related probabilities:

${P (D}^{+})$ —The prior probability of CAD DD is based on the patient’s basic medical history. The prior probabilities in each case were taken from the medical literature and confirmed by six senior physicians and the MSR.

$s e n T_{1} = P (T_{1}^{+} | D^{+})$ —Sensitivity of medical test T₁ is defined as the proportion of cases in which the medical test results correctly identify the CAD DD.

$s p e T_{1} = P (T_{1}^{-} | D^{-})$ —Specificity was defined as the proportion of cases in which the test results correctly reject a CAD DD.

Data on specificity and sensitivity of ECG and CE tests were retrieved from previous articles^68,69,71,72 and constituted a weighted average including the relative number of patients participating in each study. The final values with their sensitivity analysis ranges appear in Table 1.

$P (T_{1}^{+})$ —The probability of achieving positive results for identifying the CAD DD by the ECG (the first medical test). This was calculated by the complete probability equation

P (T_{1}^{+}) = s e n T_{1} \cdot P (D^{+}) + P (T_{1}^{+} | D^{-}) \cdot P (D^{-}) = s e n T_{1} \cdot P (D^{+}) + [1 - s p e T_{1}] \cdot [1 - P (D^{+})]

This equation can be expanded to capture the effects of results of multiple medical tests on a correct CAD DD as follows. For illustrative purposes, we show only the calculated probability in the case where both ECG (T₁) and CE (T₂) tests indicated a DD of CAD ( $T_{1}^{+} T_{2}^{+}$ )

P (T_{1}^{+} T_{2}^{+}) = s e n T_{1} \cdot s e n T_{2} \cdot P (D^{+}) + [1 - s p e T_{1}] \cdot [1 - s p e T_{2}] \cdot [1 - P (D^{+})]

The remainder of the probabilities ( $P (T_{1}^{+} T_{2}^{-})$ , $P (T_{1}^{-} T_{2}^{+})$ , $P (T_{1}^{-} T_{2}^{-})$ ) were calculated in a similar way.

Stage 3: the second decision node—the admission decision

At this node, the physician must choose whether to admit or discharge the patient.

Stage 4: the final chance node—the impact of each admission decision

In general, when there is one medical test, there are two options for each endpoint: $D +$ (positive CAD diagnosis) and $D -$ (negative CAD diagnosis). In Figure 2, CAD refers to the D+ and No CAD refers to D−.

This stage contains posterior probability calculations based on the results of multiple medical tests. The posterior probabilities were calculated with respect to their support or rejection of a CAD DD based on the results of both examinations ( $T_{1}, T_{2}$ ), using Bayes’ rule for conditional probabilities. For illustrative purposes, we show only the posterior probability in the case where the DD is CAD D+ when the results obtained from both the ECG and CE tests indicated a DD of CAD ( $T_{1}^{+} T_{2}^{+}$ )³¹:

1. ${P (D}^{+} | T_{1}^{+} T_{2}^{+}) = \frac{s e n T_{1} \cdot s e n T_{2} \cdot {P (D}^{+})}{s e n T_{1} \cdot s e n T_{2} \cdot {P (D}^{+}) + [1 - s p e T_{1}] \cdot [1 - s p e T_{2}] \cdot [1 - {P (D}^{+})]}$

The remaining posterior probability equations

2. ${P (D}^{+} | T_{1}^{+} T_{2}^{-})$ , ${P (D}^{+} | T_{1}^{-} T_{2}^{+})$ , ${P (D}^{+} | T_{1}^{-} T_{2}^{-})$ were calculated in a similar way.³¹

The summary of all values of the variables in both cases appears in Table 1.

Application of the model to medical cases

To test the decision-tree model, we simulated two medical cases where we contrasted the decision-making of physicians who had access to the EHR data with that of physicians who lacked such access. The physicians were aware of the two conditions and based their medical decisions solely on what was presented to them in the system. The cases were chosen from the most common clinical scenarios at the National Center for Health Statistics (NCHS), the principal health statistics agency in the United States. These cases also appear in the textbooks of the Educational Commission for Foreign Medical Graduates (ECFMG), which assesses the readiness of international medical graduates to enter residency or fellowship programs in the United States. According to the NCHS, the most frequent reason given by adult patients for visiting the ED is chest pain. Consequently, chest pain was chosen here, with the potential diagnosis of CAD.

The simulated cases were developed by a panel of six senior physicians in conjunction with the MSR. The technical data were added to the ECFMG instructions from several previous works on chest pain and CAD diagnosis.^70,74–76 An example of the simulated medical case can be found in online Appendix 1 (Section 1.1: Case 1).

The EHR system contains medical history such as previous tests and visits to the ED. It accurately represents the state of EHR in Israel. The system is interoperable, and medical staff have access to information from other healthcare providers.

Case 1 was designed so that the admission decision could be made solely from the patient’s medical information (medical complaint, physical examination, and results of previous tests) and potential additional information from the EHR system. The case was designed to ensure that a CAD DD would be refuted by the results of the two medical tests used (ECG and CE) without access to the simulated EHR. Consequently, the physician should discharge the patient (Bayes’ posterior probability that the patient has CAD was 0.5%). However, with access to the EHR system, the decision to admit was supported by the patient’s medical history in the EHR (inferior-wall myocardial infarction (MI) and the results of a percutaneous coronary intervention for stenosis of 85 percent in left anterior descending with stent implantation and stenosis of 60 percent in right coronary artery). Therefore, with access to the EHR, the physician should admit the patient (Bayes’ posterior probability that the patient has CAD was 9.6%).

In Case 2, without access to the simulated EHR, the decision to admit the patient was consistent with accepted medical judgment (Bayes’ posterior probability of CAD was 38.7%). However, with access to the EHR, the conventional medical decision should be to discharge this patient, after concluding that the main DD was not related to CAD (Bayes’ posterior probability of CAD was 2.1%). Without access to the EHR, medical tests (ECG and CE) both supported and disconfirmed a diagnosis of CAD DD. With access to the EHR, the results from both tests disconfirmed the CAD DD (since the current results and the current medical status of the patient could be compared to previous results by access to the EHR).

Results

The findings are organized in terms of the two cases. We first show the variable values, including the range of sensitivity analyses and then obtain and compare the posterior probabilities ( ${P (D}^{+} | T_{1}^{-} T_{2}^{-})$ and ${P (D}^{+} | T_{1}^{+} T_{2}^{-})$ ) for the two scenarios with and without access to the EHR. The summary of all values of the variables in both cases appears in Table 1.

Case 1

According to Figure 3, without access to the EHR, the medical decision of the physician should be to discharge the patient. By contrast, with access to the EHR, the medical decision of the physician should be to admit this patient.

Figure 3.

Case1: Results with and without the EHR when the signs of tests T₁ and T₂ are both negative.

The comparison between the posterior probabilities of ${P (D}^{+} | T_{1}^{-} T_{2}^{-})$ and the prior probabilities ${P (D}^{+})$ in both access patterns to the EHR is shown in Table 2.

Table 2.

Comparing probability results (Case 1).

Medical decision	Decrease in percentage (absolute values)	${P (D}^{+} \| T_{1}^{-} T_{2}^{-})$ posterior	${P (D}^{+})$ a priori	Probability
Discharge	4.5%	0.5%	5%	EHR was not accessed
Admit	45.4%	9.6%	55%	EHR was accessed

EHR: electronic health record.

It is worth noting that the posterior probabilities were lower in both scenarios (with and without access to the EHR), exposing the negative results that were obtained from both the ECG and the CE examinations (disconfirming the DD of CAD). We expanded our measurement with a sensitivity analysis (in online Appendix 2, Section 2.2), and we further validated our findings using a Monte Carlo simulation on 100,000 trials.

Case 2

According to Figure 4, without access to the EHR, the physician’s medical decision should be to admit this patient. By contrast, with access to the EHR, the decision should be to discharge the patient.

Figure 4.

Case 2: Results with and without the EHR when the T₁ sign in the case without the EHR is positive and with the EHR is negative and the T₂ sign for both situations is negative.

Similarly, the contribution of the medical tests (ECG and CE) in the situation without access to the EHR was a positive result from the ECG and a negative result from the CE examinations, confirming and disconfirming, respectively, the CAD diagnosis. This was unlike the situation with access to the EHR, in which both tests disconfirmed the CAD diagnosis. Consequently, there are two important comparisons between probabilities that reflect the contribution of the medical tests:

Without access to the EHR—the comparison between the posterior probabilities of ${P (D}^{+} | T_{1}^{+} T_{2}^{-})$ and the prior probability ${P (D}^{+})$ .

With access to the EHR—the comparison between the posterior probabilities of ${P (D}^{+} | T_{1}^{-} T_{2}^{-})$ and the prior probability ${P (D}^{+})$ .

From Table 3, it can be seen that the posterior probability increased in the situation without access to the EHR, thus showing the effect of the positive result that was obtained from the ECG examination (confirming the DD of CAD).

Table 3.

Comparing probability results (Case 2).

Medical decision	Decrease (absolute values)	${P (D}^{+} \| T_{1}^{-} T_{2}^{-})$ posterior	Increase (absolute values)	${P (D}^{+} \| T_{1}^{+} T_{2}^{-})$ posterior	${P (D}^{+})$ a priori	Probability
Admit	–	–	16.7%	38.7%	22%	EHR was not accessed
Discharge	19.9%	2.1%	–	–	20%	EHR was accessed

EHR: electronic health record.

By contrast, the posterior probability decreased in the situation with access to the EHR, thus showing the effect of the negative results that were obtained from both the ECG and the CE examinations (disconfirming the DD of CAD). We expanded our measurement with a sensitivity analysis (in online Appendix 2, Section 2.3) that further validated the findings.

Discussion and contribution

In recent years, the adoption of EHR systems in the healthcare sector has increased substantially as more organizations have come to recognize the importance and potential applications of such systems.⁴ Physicians need medical information to assist them in providing medical services. One of the major issues in this context is how information regarding patients supplied by EHR under the severe time constraints and overcrowding of an ED can improve decision-making and its outcomes. This study attempted to answer this question by providing further insights into the field of medical informatics. We found that using EHR contributes to admission decisions. In fact, accessing EHR helps determine the correct main DD and, in particular, to identify or refute a CAD diagnosis (in our specific medical cases of chest pain). We sought to investigate this issue by using simulated scenarios. Despite the disparities between simulated settings and the real-world, we felt that a rather controlled and limited exploration was needed to better understand the implications of using EHR in the complex reality of the ED.

Our main goal in this study was to contribute to the body of scientific knowledge and to researchers by providing additional insights into the fields of medical informatics, decision theory, and management ISs. The main conclusions of this study serve to better highlight the importance of medical history presented to physicians at the point of care. First, physicians take advantage of medical history because they realize its importance. This study reflected the potential of an EHR to contribute to physicians’ performance. Second, we contribute by quantifying the value of medical information required for a specific admission decision in EDs. We constructed a decision tree that contains typical steps in medical care and ends with leaves containing utility values (unlike a standard medical decision tree that compares several therapy alternatives).⁷⁷ Third, we contribute by designing testable medical scenarios. The medical scenarios (taken from the standard medical textbook literature) were designed by experts (a panel of senior physicians from several fields) with the cooperation of the MSR (which specializes in medical scenarios). Fourth, we found a positive relationship between viewing the medical history and a more accurate DD, specifically in chest pain scenarios.

A secondary goal of this study was to contribute to greater efficiency in the implementation and use of EHR,⁷⁸ in order to improve the healthcare services provided to patients. Efficient implementation of such systems means mapping potential obstacles (e.g. when physicians refrain from using the system, information overload) and determining the optimal ways of using the system in terms of scope and training of medical staff. Over the past few years, there has been an increase in national expenditures on health in general and on healthcare technology in particular. Our results and conclusions may aid policy makers in deciding whether national EHRs, which condense historical information from all health service providers, can improve healthcare services to citizens.

Limitations and avenues for future research

Our study only implemented two levels of information: fully provided mandatory access to a simulated EHR or unavailability of such access. In future research, it would be beneficial to examine additional levels of information. Furthermore, it is important to keep in mind that our results were obtained via a simulated procedure of two simulated cases and not in a real-world setting. The simulation was conducted on a computer. Physicians participating in this simulation were presented with a situation (symptoms, medical history, testing, etc.) and were asked to choose their course of action and therapeutic process. This may limit the ability to generalize our findings. In reality, doctors may not always be able to retrieve information from the system in an efficient and timely manner, due to workload and/or lack of training with the system. Therefore, these results still need to be considered from a limited perspective, and a more ecologically realistic investigation is needed. Furthermore, our study did not examine task complexity effects (the complexity of the medical cases). Our methodology included a sensitivity analysis to further confirm our results. However, it has been claimed that sensitivity and specificity may vary with the cut-off chosen for a clinical test. Currently, these statistical techniques for interpreting prior probability evidence diagnosis are not well understood or widely used in medical practice.⁶⁶ The complexity of medical tasks has been studied and is known to impact decision-making in clinical settings.⁷⁹ Decision-making in complex and cognitive-demanding tasks has been shown to benefit from technological aids.⁸⁰ This effect should be considered in future research of EHR, along with the need to explore other scenarios as well, to ensure that the results can be generalized to other medical cases, since examining only two medical cases as done here is not sufficient to make generalizations.

Clearly, the assessment of the value of information via the decision analytic approach does not replace an actual performance examination on the part of decision-makers. Assessing the value of information via a realistic approach is nevertheless possible by implementing several methods such as analyzing real EHR files.^2,80

Footnotes

Acknowledgements

This article is based on research that was not funded entirely or partially by an outside source.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Rosenbloom

Denny

. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc 2011; 18(2): 181–186.

Vest

Zhao

Jasperson

. Factors motivating and affecting health information exchange usage. J Am Med Inform Assoc 2011; 18: 143–149.

Wright

Soran

Jenter

. Physician attitudes toward health information exchange: results of a statewide survey. J Am Med Inform Assoc 2010; 17(1): 66–70.

Goldschmidt

. HIT and MIS: implications of health information technology and medical information systems. Comm ACM 2005; 48(10): 68–74.

Ahituv

Neumann

Riley

. Principles of information systems for management. 4th ed. Dubuque, IA: Business and Educational Technologies, 1994.

Hripcsak

Sengupta

Wilcox

. Emergency department access to a longitudinal medical record. J Am Med Inform Assoc 2007; 14(2): 235–238.

Walter

Kostopoulos

Haass

. Point-of-care laboratory halves door-to-therapy decision time in acute stroke. Ann Neurol 2011; 69(3): 581–586.

Faisal

Parveen

Badsha

. Computer assisted diagnostic system in tumor radiography. J Med Syst 2013; 37(3): 1–10.

Grudin

. Human-computer interaction. Annu Rev Inform Sci 2011; 45(1): 367–430.

10.

Huerta

Thompson

Ford

. Electronic health record implementation and hospitals’ total factor productivity. Decis Support Syst 2012; 55(2): 450–458.

11.

Cebul

Love

Jain

. Electronic health records and quality of diabetes care. N Engl J Med 2011; 365(9): 825–833.

12.

Linder

Schnipper

Middleton

. Method of electronic health record documentation and quality of primary care. J Am Med Inform Assoc 2012; 19(6): 1019–1024.

13.

Goh

Gao

Agarwal

. Evolving work routines: adaptive routinization of information technology in healthcare. Inform Syst Res 2011; 22(3): 565–585.

14.

Cooke

Higgins

Kidd

. Use of emergency observation and assessment wards: a systematic literature review. Br Med J 2006; 20: 138–142.

15.

Bair

Song

Chen

. The impact of inpatient boarding on ED efficiency: a discrete-event simulation study. J Med Syst 2010; 34: 919–929.

16.

Frisse

Johnson

Nian

. The financial impact of health information exchange on emergency department care. J Am Med Inform Assoc 2012; 19(3): 328–333.

17.

Connelly

Park

. Focus on health information technology, electronic health records and their financial impact: the impact of electronic health records on care of heart failure patients in the emergency room. J Am Med Inform Assoc 2012; 19(3): 334–340.

18.

Boonstra

Broekhuis

. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res 2010; 10(231): 1–17.

19.

Walker

Pan

Johnston

. The value of health care information exchange and interoperability. Health Aff 2005; 24: 5–10.

20.

Ben-Assuli

Shabtai

Leshno

. The impact of EHR and HIE on reducing avoidable admissions: controlling main differential diagnoses. BMC Med Informat Decis Making 2013; 13(1): 49.

21.

Fowler

Yaeger

. Electronic health record: integrating evidence-based information at the point of clinical decision making. J Med Libr Assoc 2014; 102(1): 52–55.

22.

Makam

Lanham

Batchelor

. The good, the bad and the early adopters: providers’ attitudes about a common, commercial EHR. J Eval Clin Pract 2014; 20(1): 36–42.

23.

Redelmeier

Shafir

. Medical decision making in situations that offer multiple alternatives. JAMA 1995; 273(4): 302–305.

24.

Basu

Meltzer

. Value of information on preference heterogeneity and individualized care. Med Decis Making 2007; 27(2): 112–127.

25.

Häyarinen

Saranto

Nykänen

. Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform 2008; 77: 291–304.

26.

Claxton

Sculpher

Drummond

. A rational framework for decision making by the National Institute for Clinical Excellence (NICE). Lancet 2002; 360(9334): 711–715.

27.

Yasnoff

Sweeney

Shortliffe

. Putting health IT on the path to success. JAMA 2013; 309(10): 989–990.

28.

Bates

Gawande

. Improving safety with information technology. N Engl J Med 2003; 348(25): 2526–2534.

29.

Stead

Searle

Fessler

. Biomedical informatics: changing what physicians need to know and how they learn. Acad Med 2011; 86(4): 429–434.

30.

Ben-Assuli

Leshno

Shabtai

. Using electronic medical record systems for admission decisions in emergency departments: examining the crowdedness effect. J Med Syst 2012; 36(6): 3795–3803.

31.

Ben-Assuli

Leshno

. Using electronic medical records in admission decisions: a cost effectiveness analysis. Decision Sci 2013; 44(3): 463–481.

32.

Rahimi

Liaw

Ray

. Ontological specification of quality of chronic disease data in EHRs to support decision analytics: a realist review. Decis Anal 2014; 1(1): 1–31.

33.

Shortliffe

Barnett

. Biomedical data: their acquisition, storage, and use. In: Shortliffe

Cimino

(eds) Biomedical informatics. London: Springer, 2014, pp. 39–66.

34.

Fontaine

Ross

Zink

. Systematic review of health information exchange in primary care practices. J Am Board Fam Med 2010; 23(5): 655–670.

35.

James

Savitz

. How Intermountain trimmed health care costs through robust quality improvement efforts. Health Aff 2011; 30(6): 1185–1191.

36.

Brynjolfsson

Hitt

. Paradox lost? Firm-level evidence on the returns to information systems spending. Manag Sci 1996; 42(4): 541–558.

37.

Chaudhry

Wang

. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 2006; 144(10): 742–752.

38.

Mohan

Angus

Ricketts

. Assessing the validity of using serious game technology to analyze physician decision making. PLoS One 2014; 9(8): e105445.

39.

Wainwright

Shepard

Harman

. Factors that influence the clinical decision making of novice and experienced physical therapists. Phys Ther 2011; 91(1): 87–101.

40.

Xue

Levin

. The impact of prior risk experiences on subsequent risky decision-making: the role of the insula. Neuroimage 2010; 50(2): 709–716.

41.

Hersh

Hickam

. How well do physicians use electronic information retrieval systems? JAMA 1998; 280(15): 1347–1352.

42.

Laerum

Karlsen

Faxvaag

. Effects of scanning and eliminating paper-based medical records on hospital physicians’ clinical work practice. J Am Med Inform Assoc 2003; 10(6): 588–595.

43.

Graham

Kushniruk

Bullard

. How usability of a web-based clinical decision support system has the potential to contribute to adverse medical events. In: AMIA annual symposium proceedings, Washington, DC, 6 November 2008, pp. 257–261.

44.

Walter

Lopez

. Physician acceptance of information technologies: role of perceived threat to professional autonomy. Decis Support Syst 2008; 46(1): 206–215.

45.

Lluch

. Healthcare professionals’ organisational barriers to health information technologies—a literature review. Int J Med Inform 2011; 80(12): 849–862.

46.

Tambe

Hitt

Brynjolfsson

. The extroverted firm: how external information practices affect innovation and productivity. Manag Sci 2012; 58(5): 843–859.

47.

Watts

Shankaranarayanan

Even

. Data quality assessment in context: a cognitive perspective. Decis Support Syst 2009; 48: 202–211.

48.

Gorry

Morton

. A framework for management information systems. Sloan Manag Rev 1971; 13: 55–70.

49.

Anderson

. Managerial considerations in participative design of MIS/DSS. Inform Manag 1985; 9: 201–207.

50.

Işık

Saraçoğlu

Harmanci

. Design of a cervical collar device to facilitate and accelerate implementation of first aid. J Med Syst 2010; 34: 573–578.

51.

Landman

Redden

Neri

. Using a medical simulation center as an electronic health record usability laboratory. J Am Med Inform Assoc 2014; 21(3): 558–563.

52.

Deshmukh

Hoffman

Arnoldi

. Efficiency of CYP2C9 genetic test representation for automated pharmacogenetic decision support. Methods Inf Med 2009; 48(3): 282–290.

53.

Stephenson

Gorsuch

Hersh

. Participation in EHR based simulation improves recognition of patient safety issues. BMC Med Educ 2014; 14(1): 224.

54.

Russ

Zillich

Melton

. Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation. J Am Med Inform Assoc 2014; 21(e2): e287–e296.

55.

Pauker

Kassirer

. The threshold approach to clinical decision making. N Engl J Med 1980; 302(20): 1109–1117.

56.

Pauker

Kassirer

. Medical progress: decision analysis. N Engl J Med 1987; 316(5): 250–258.

57.

Dotan

Pinchuk

Lichtenberg

. Decision analysis supports the paradigm that indiscriminate supplementation of vitamin E does more harm than good. Arterioscler Thromb Vasc Biol 2009; 29(9): 1304–1309.

58.

Golan

Wolf

Pauker

. Empirical anti-Candida therapy among selected patients in the intensive care unit: a cost-effectiveness analysis. Ann Intern Med 2005; 143(12): 857–869.

59.

Gelman

Shalizi

. Philosophy and the practice of Bayesian statistics. Br J Math Stat Psychol 2013; 66(1): 8–38.

60.

Guttorp

. Stochastic modeling of scientific data. London: Chapman & Hall, 1995.

61.

Cho

Park

Kim

. Using EHR data to predict hospital-acquired pressure ulcers: a prospective study of a Bayesian Network model. Int J Med Inform 2013; 82(11): 1059–1067.

62.

Zhao

Weng

. Combining PubMed knowledge and EHR data to develop a weighted Bayesian network for pancreatic cancer prediction. J Biomed Inform 2011; 44(5): 859–868.

63.

Kawaler

Cobian

Peissig

. Learning to predict post-hospitalization VTE risk from EHR data. In: AMIA annual symposium proceedings, Chicago, IL, 3 November 2012, pp. 436–445.

64.

Cahan

Gilon

Manor

. Probabilistic reasoning and clinical decision-making: do doctors overestimate diagnostic probabilities? QJM 2003; 96(10): 763–769.

65.

Ghosh

Erwin

. Do medical students and physicians understand probability. QJM 2004; 97: 53–56.

66.

Florkowski

. Sensitivity, specificity, receiver-operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests. Clin Biochem Rev 2008; 29(Suppl. 1): S83–S87.

67.

Guyatt

Baumann

Pauker

. Addressing resource allocation issues in recommendations from clinical practice guideline panels: suggestions from an American College of Chest Physicians task force. Chest 2006; 129(1): 182–187.

68.

Reichlin

Hochholzer

Bassetti

. Early diagnosis of myocardial infarction with sensitive cardiac troponin assays. N Engl J Med 2009; 361(9): 858–867.

69.

Keller

Zeller

Peetz

. Sensitive troponin I assay in early diagnosis of acute myocardial infarction. N Engl J Med 2009; 361(9): 868–877.

70.

Pope

Aufderheide

Ruthazer

. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med 2000; 342: 1163–1170.

71.

Andersen

Nielsen

Christensen

. A study of police operated dispatch to acute coronary syndrome cases arising from 112 emergency calls in Aarhus county, Denmark. Emerg Med J 2006; 23(9): 705–706.

72.

AHB

Smith

Christenson

. Evaluation of a point-of-care assay for cardiac markers for patients suspected of acute myocardial infarction. Clin Chim Acta 2004; 346(2): 211–219.

73.

Sesso

Chen

L’Italien

. Blood pressure lowering and life expectancy based on a Markov model of cardiovascular events. Hypertension 2003; 42(5): 885–890.

74.

Lee

Goldman

. Evaluation of the patient with acute chest pain. N Engl J Med 2000; 342: 1187–1195.

75.

Panju

Hemmelgarn

Guyatt

. Is this patient having a myocardial infarction? JAMA 1998; 280: 1256–1263.

76.

Yen

Bakken

. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012; 19: 413–422.

77.

Hlatky

. What we can learn from a decision model: comment on “Cost effectiveness of adding magnetic resonance imaging to rheumatoid arthritis management.” Arch Intern Med 2011; 171: 667–668.

78.

Ovretveit

Scott

Rundall

. Improving quality through effective implementation of information technology in healthcare. Int J Qual Health Care 2007; 19(5): 259–266.

79.

Laxmisan

Hakimzada

Sayan

. The multitasking clinician: decision-making and cognitive demand during and after team handoffs in emergency care. Int J Med Inform 2007; 76: 801–811.

80.

France

Levin

Hemphill

. Emergency physicians’ behaviors and workload in the presence of an electronic whiteboard. Int J Med Inform 2005; 74(10): 827–837.

Assessing electronic health record systems in emergency departments: Using a decision analytic Bayesian model

Abstract

Keywords

Introduction

Contribution of EHR systems to medical decisions

Decision analytic approach to assessing the value of information

Objectives

Materials and method

Role of Bayesian statistics in the proposed analysis

Decision tree with the EU approach

Description of the decision tree in the model

Stage 1: the root node in the decision tree—access to the EHR

Stage 2: the first chance node—examination results

Stage 3: the second decision node—the admission decision

Stage 4: the final chance node—the impact of each admission decision

Application of the model to medical cases

Results

Case 1

Case 2

Discussion and contribution

Limitations and avenues for future research

Footnotes

Acknowledgements

Declaration of conflicting interests

Funding

References