Abstract
The purpose of this study was to evaluate the accuracy of using a combination of International Classification of Diseases (ICD) diagnostic codes and imaging procedure codes for identifying deep vein thrombosis (DVT) and pulmonary embolism (PE) within administrative databases. Information from the Alberta Health (AH) inpatients and ambulatory care administrative databases in Alberta, Canada was obtained for subjects with a documented imaging study result performed at a large teaching hospital in Alberta to exclude venous thromboembolism (VTE) between 2000 and 2010. In 1361 randomly-selected patients, the proportion of patients correctly classified by AH administrative data, using both ICD diagnostic codes and procedure codes, was determined for DVT and PE using diagnoses documented in patient charts as the gold standard. Of the 1361 patients, 712 had suspected PE and 649 had suspected DVT. The sensitivities for identifying patients with PE or DVT using administrative data were 74.83% (95% confidence interval [CI]: 67.01–81.62) and 75.24% (95% CI: 65.86–83.14), respectively. The specificities for PE or DVT were 91.86% (95% CI: 89.29–93.98) and 95.77% (95% CI: 93.72–97.30), respectively. In conclusion, when coupled with relevant imaging codes, VTE diagnostic codes obtained from administrative data provide a relatively sensitive and very specific method to ascertain acute VTE.
Keywords
Introduction
Venous thromboembolism (VTE), which includes pulmonary embolism (PE) and deep vein thrombosis (DVT), is the third most common acute cardiovascular condition after myocardial infarction and stroke.1,2 Population-based studies report an annual incidence of clinically-recognized acute VTE, including both DVT and PE, between 1.0 and 1.5 per 1000 adults.3–5 VTE recurs in about 20% of patients after five years, but the rate varies depending on the presence of risk factors. 6
To better assess the population burden of acute VTE, administrative healthcare databases such as hospital discharge abstract databases, pharmacy records, and physician or medical services claims databases are increasingly being used in epidemiologic research7–9 mainly because they allow for rapid identification of large patient cohorts and provide comprehensive data at reasonable cost. Based on Alberta Health (AH) administrative databases, we established a provincial venous thromboembolism database (AB-VTE; inpatient data, outpatient data from ambulatory data, claims data from Alberta Health Services [AHS], registry data, vital statistics, and Alberta Blue Cross data) from 2000 to 2010. These databases include demographic information, procedure codes, and International Classification of Diseases (ICD) diagnosis codes, which can be used to identify patients diagnosed with venous thromboembolism as well as other relevant comorbidities and treatments. However, assignments of ICD codes are subject to errors resulting from limitations in available clinical data, diagnostic errors, and coding errors made by human operators. Before analysis can take place, it is imperative to assess the validity of this cohort by examining the accuracy of using administrative data to ascertain acute VTE.
Using a random sample selected from all patients who received imaging studies to exclude acute VTE at one centre, we sought to evaluate the accuracy of using a combination of ICD diagnostic codes and imaging codes for identifying acute DVT and PE diagnoses within our administrative database by comparison to the gold standard of physician diagnosis obtained by chart review.
Methods
We used an algorithm comprising one imaging code (VTE-related imaging, defined below) and one VTE-related diagnostic code to identify incident VTE cases.
Study population
The University of Alberta Health Research Ethics Board (Pro00048871) approved this study. Using the University of Alberta Hospital (UAH) radiology database, a sample of probable VTE patients, ⩾18 years old, who underwent imaging to diagnose VTE, were chosen randomly from the period 1 January 2000 to 31 December 2010. Regardless of the results of the radiological investigation, all patient charts were reviewed to confirm the presence or absence of VTE. For cases with suspected pulmonary embolism, the use of computerized tomographic angiography (CTA) and/or ventilation perfusion (V/Q) scan were the two modalities of choice. For suspected DVT cases, we relied solely on the lower limb compression duplex ultrasound (CUS).
Chart review (gold standard)
For all identified patients, a trained abstractor with a medical background (who was blinded to patient diagnostic codes during abstraction) used a standardized data collection form to abstract information from the entire medical record. Data elements included documented symptomatic VTE, diagnostic modality, and timing of the objective confirmation of the event. Patients were considered to have PE if they were symptomatic at presentation and had a positive imaging test (positive CTA scan or high probability V/Q scan). In patients with suspected DVT, an event was considered positive if the patient was symptomatic at presentation and CUS showed a proximal DVT.
Health administrative data sources
Once patients were classified as having or not having VTE as per our gold standard reference, corresponding discharge abstract database (DAD) or ambulatory care administrative records were obtained for each patient. Discharge abstract database records all admissions to acute care facilities with most responsible diagnosis, up to 25 other diagnoses or comorbidities, and procedures in International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) (1 January 2000 to 31 March 2002) or International Classification of Diseases, Tenth Revision (ICD-10-CA) (1 April 2002 to 31 December 2010); while the ambulatory care database (using ICD-9-CM from 1 January 2000 to 31 March 2002 and ICD-10-CA from 1 April 2002 to 31 December 2010) tracks all visits to hospital-based physicians’ offices (i.e. specialist offices) and the emergency department (ED) and allows for coding of up to six conditions for the fiscal years 2000 to 2002 and 10 conditions from 2003 to 2005.
Sample size and statistical analysis
We aimed for a sample size that would allow us to detect a sensitivity and specificity of approximately 0.80 with 95% confidence intervals (CI) within 0.1 of the estimate.
To estimate the accuracy of our algorithm, we compared the current estimates of PE and/or DVT prevalence among patients tested with imaging for PE which ranged from 19% to 79% and for DVT was about 23%.10,11 A calculated sample of 627 patients screened for PE by CTA or V/Q scans and 615 patients screened by CUS for lower limb DVT was required to detect a sensitivity and specificity of 80% with a 95% CI width of 10%. 12 The total number of required cases was 1366 after accounting for 10% missing charts. DVT and PE occurrence was determined if there were a priori-defined ICD codes for DVT in any diagnostic field (ICD-9-CM: 451.1, 451.2, 451.8, 451.9, 453.2, 453.8, and 453.9; ICD-10-CA: I80.2, I80.3, I80.1, I82.8, I80.9, I82.9, I80.8, O22.3, O22.9, O87.1) and PE (ICD-9-CM: 415.0 and 415.1; ICD-10-CA: I26.9, I26.0). For VTE events, we used the combination of PE and DVT diagnostic codes. For patients admitted to hospital (inpatient administrative records), we relied on the presence of the ICD diagnosis and imaging codes that occurred in the same medical act. While in patients diagnosed in ED and outpatient clinics (ambulatory care records), we used a seven-day window. A post hoc sensitivity analysis was done to examine the accuracy of ICD codes for VTE recorded as principle discharge diagnosis fields (first two fields). The gold standard diagnosis was determined from review of the patient chart. Descriptive statistics were used to characterize the study population. We also computed sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, positive predictive value (PPV), and negative predictive value (NPV) of the algorithm with corresponding 95% CIs. All analyses were carried out with SPSS® 21.0 (SPSS Inc, Chicago, IL, USA).
Results
We randomly sampled 1361 patients with probable VTE, 712 patients with suspected PE and 649 patients with suspected DVT. The median age for all patients was 67 years (IQR 54–76), and 734 (53.9%) were female. Based on chart review, 147 patients (20.65%) were diagnosed as having PE and 105 (16.18%) were diagnosed as having DVT (including 227 patients [17%] with both DVT and PE) (Table 1). All cases of PE identified by chart review were confirmed with either V/Q or CT scan. All cases of DVT identified by chart review were confirmed by compression ultrasonography.
Characteristics of patients with clinical suspicion for acute PE (N=712) and/or DVT (N=649) (Total N=1361).
PE, pulmonary embolism; DVT, deep vein thrombosis; IQR, interquartile range.
Applying the predefined ICD diagnostic codes to the 1361 patients, we identified 258 (18.96%) patients with VTE, 156 (21.91%) patients with PE and 102 (15.72%) patients with DVT. For identifying patients with VTE by discharge diagnoses (in all diagnosis positions) in the administrative data, the diagnostic statistics were: sensitivity 75% (95% CI: 69.18–80.22), specificity 93.78% (95% CI: 92.19–95.13), positive likelihood ratio 12.05 (95% CI: 9.49–15.31), negative likelihood ratio 0.27 (95% CI: 0.22–0.33), PPV 73.26% (95% CI: 67.41–78.56%) and NPV 94.29% (95% CI: 92.75–95.58). For identifying patients with PE, the diagnostic statistics were: sensitivity 74.83% (95% CI: 67.01–81.62), specificity 91.86% (95% CI: 89.29–93.98), positive likelihood ratio 9.19 (95% CI: 6.86–12.31), negative likelihood ratio 0.27 (95% CI: 0.21–0.36), PPV 70.51% (95% CI: 62.69–77.53%) and NPV 93.35% (95% CI: 90.94–95.27). Lastly, for identifying patients with DVT, the diagnostic statistics were: sensitivity 75.24% (95% CI: 65.86–83.14), specificity 95.77% (95% CI: 93.72–97.30), positive likelihood ratio 17.8 (95% CI: 11.75–26.94), negative likelihood ratio 0.26 (95% CI: 0.19–0.36), PPV 77.45% (95% CI: 68.11–85.31%) and NPV 95.25% (95% CI: 93.11–96.87).
A post hoc sensitivity analysis using VTE, PE or DVT in only the primary discharge diagnosis field was done in an attempt to increase PPV in our data (Table 3). We found that the selective use of primary diagnosis fields resulted in an increase in the PPV to 83.07% (95% CI: 76.95–88.12) for VTE, 83.76% (95% CI: 78.81%, 89.93%) for PE and to 81.94% (95% CI: 71.10%, 90.01%) for DVT. However, the sensitivity of the test dropped significantly to 62.30% (95% CI: 56.00–68.31) for VTE.
Tables 2 and 3 summarize the performance of the algorithm in any diagnosis position, and in the principal diagnosis position of VTE, respectively.
Accuracy of ICD-9-CM and ICD-10 hospital discharge diagnosis codes (in any discharge diagnosis positions) coupled with radiology diagnostic codes for identifying patients with acute symptomatic PE and DVT.
PE, pulmonary embolism; DVT, deep vein thrombosis; VTE, venous thromboembolism; CI, confidence interval.
ICD-9-CM and ICD-10 hospital discharge diagnosis codes, in any diagnosis position, were combined with radiology diagnostic codes to identify acute events.
Accuracy of ICD-9-CM and ICD-10 hospital discharge diagnosis codes in principle diagnosis positions coupled with radiology diagnostic codes for identifying patients with acute symptomatic PE and DVT.
PE, pulmonary embolism; DVT, deep vein thrombosis; VTE, venous thromboembolism; CI, confidence interval.
ICD-9-CM and ICD-10 hospital discharge diagnosis codes, in principle diagnosis positions, were combined with radiology diagnostic codes to identify acute events.
Discussion
Our study explored the validity of using ICD discharge diagnoses when coupled with an imaging procedure code from health administrative data to identify patients presenting with acute venous thromboembolism. We found that about 19% of all patients who were radiologically investigated for VTE at a tertiary care facility were found to have positive imaging studies. The identification of ICD codes, in any diagnosis position, with one procedure code was moderately sensitive but remarkably specific with good predictive abilities for VTE, PE and DVT. Additionally, using ICD discharge diagnoses with an imaging procedure code to ascertain acute VTE gave reliable estimates of numbers and proportions as well as a reasonable degree of validity, despite imperfect diagnostic accuracy. Furthermore, the use of principle diagnosis fields only, resulted in moderate increase in PPVs at the expense of the test sensitivities. The use of ICD codes (in any position) with the imaging procedure codes resulted in a good balance between sensitivity, specificity and predictive values. The approach of using imaging procedure codes combined with ICD discharge codes is an improvement over the use of ICD discharge codes alone, which have been shown in a recent systematic review to yield a very low PPV of 30–40%. 13 Our study expands on prior published literature, and suggests that ICD codes in Canadian administrative data are moderately sensitive but have a good specificity for identifying acute DVT/PE events.14–16 Additional criteria could be added to enhance positive predictive value, such as prescription of an anticoagulant, but this would come at the cost of further reduction in sensitivity and negative predictive value. 17
Our diagnostic statistic estimates should be considered in the context of the data used. The PPV and sensitivity estimates depend on the accuracy and completeness of the discharge abstract databases, which vary substantially from system to system. The Alberta Health Discharge Abstract Database allows 25 diagnosis or comorbidity codes, and procedure codes (using ICD-9-CM for 1994–2002 and ICD-10-CA for 2002–2010); and the Ambulatory Care Database (using ICD-9-CM for 1994–2002 and ICD-10-CA for 2002–2010) tracks all visits to hospital-based physician offices (i.e. specialist offices) and the ED and allows for coding of up to 6 conditions for the years 2000 to 2002 and 10 conditions from 2003 to 2010. It is likely that some diagnostic codes were recorded on the basis of patient complaints and physician examinations. All cases of VTE identified by chart review were symptomatic and were treated with anticoagulant treatment or inferior vena cava filter, making failure to code the most likely cause for the absence of ICD diagnosis codes for these cases in our study. The utilization of our algorithm (i.e. combining imaging codes with diagnostic codes) to identify actual cases maximizes predictive ability and specificity.
It is likely that when ICD codes are used alone for case identification, there is under-coding of both PE and DVT. Previous literature shows that in addition to missing data issues, coders generally abstract diagnoses from physician notes rather than from diagnostic imaging test reports.18,19 Therefore, a clot found on lower limb compression ultrasonography would not trigger coding of DVT unless this diagnosis was explicitly documented by the physician in the chart. This may also explain the lower sensitivity of ICD codes for PE in patients with acute respiratory failure or exacerbation of chronic obstructive pulmonary disease because the clinical presentations the two conditions frequently overlap. 18
Only a few studies have assessed the validity of hospital discharge diagnosis codes for identifying VTE, and none have utilized procedure codes with the diagnosis codes to identify events. Differences in sensitivity across studies are likely due to heterogeneity in the study population, recruitment period, and the choice of gold standard reference. The selection of a surgical cohort and the nature of VTE may explain the lower sensitivity reported by Zahn et al. 20 Nevertheless, our sensitivity estimates are consistent with those reported by Heckbert et al 21 and even compare favourably with those reported by Tagalakis et al 16 during a 60-day window. This also extends the findings of previous studies that showed that coding sensitivity had been preserved in the transition from ICD-9-CM to ICD-10 for other conditions. 22
Several limitations of our study deserve mention. First, estimating the accuracy of ICD discharge diagnosis codes for PE and DVT alone was not within the scope of the present study. Second, our study was not powered to determine statistically-significant differences in the sensitivity of ICD discharge diagnosis codes alone, and the results of our analyses should be considered to be only supportive of the use of imaging procedure codes along with specific VTE ICD diagnosis codes. Third, we were not able to review medical charts or ask clinicians and coders for the reasons for omitting appropriate discharge diagnosis codes in patients with objectively-confirmed DVT or PE because of the retrospective design of our study. Fourth, our study sample was drawn from one major hospital in Canada and we cannot exclude that the results would be different in other countries or settings. Though our study has limitations, our data support that, when coupled with a relevant diagnostic imaging code, VTE diagnostic codes from administrative data provide a relatively sensitive and remarkably specific method of ascertaining acute VTE.
In conclusion, we found that the combination of an imaging procedure code with ICD diagnostic codes drawn from administrative data is a valid approach for ascertaining acute VTE. Our data support the use of administrative data to perform epidemiological research about acute VTE.
Footnotes
Acknowledgements
MS McMurtry is supported by the Heart and Stroke Foundation of Canada. GS Alotaibi received a scholarship from King Saud University, Riyadh, Saudi Arabia.
Declaration of conflicting interest
The authors declare that there is no conflict of interest.
Funding
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.