Abstract
Objectives:
To evaluate the impact of International Classification of Disease, 10th revision, Clinical Modification (ICD-10-CM) implementation on pneumonia hospitalizations rates, which had declined following pneumococcal conjugate vaccine introduction for infants in 2000.
Methods:
We randomly selected records from a single hospital 1 year before (n = 500) and after (n = 500) October 2015 implementation of ICD-10-CM coding. We used a validated ICD-9-CM algorithm and translation of that algorithm to ICD-10-CM to identify pneumonia hospitalizations pre- and post-implementation, respectively. We recoded ICD-10-CM records to ICD-9-CM and vice versa. We calculated sensitivity and positive predictive value (PPV) of the ICD-10-CM algorithm using ICD-9-CM coding as the reference. We used sensitivity and PPV values to calculate an adjustment factor to apply to ICD-10 era rates to enable comparison with ICD-9-CM rates. We reviewed primary diagnoses of charts not meeting the pneumonia definition when recoded.
Results:
Sensitivity and PPV of the ICD-10-CM algorithm were 94% and 92%, respectively, for young children and 74% and 79% for older adults. The estimated adjustment factor for ICD-10-CM period rates was −2.09% (95% credible region [CR], −7.71% to +3.0%) for children and +6.76% (95% CR, −3.06% to +16.7%) for older adults. We identified a change in coding adult charts that met the ICD-9-CM pneumonia definition that led to recoding in ICD-10-CM as chronic obstructive pulmonary disease (COPD) exacerbation.
Conclusions:
The ICD-10-CM algorithm derived from a validated ICD-9-CM algorithm should not introduce substantial bias for evaluating pneumonia trends in children. However, changes in coding of pneumonia associated with COPD in adults warrant further study.
Three-Question Summary Box
What Is the Current Understanding of This Subject?
Pneumonia hospitalizations in the United States declined after adoption in 2000 of universal vaccination of infants with conjugate pneumococcal vaccine. Trends in pneumonia discharges since switching from ICD-9-CM to ICD-10-CM for hospital discharge coding in October 2015 have not been evaluated.
What Does This Report Add to the Literature?
This report suggests that translating a validated ICD-9-CM algorithm for pneumonia to ICD-10-CM will not substantially impact the ascertainment of events in the ICD-10-CM era for children.
What Are the implications for Public Health Practice?
The impact of changes in ICD-10-CM coding of pneumonia among adults with chronic obstructive pulmonary disease warrants further monitoring.
Introduction
Pneumococcal conjugate vaccines, introduced into the US childhood immunization schedule in 2000, have had a major impact on pneumonia incidence for both vaccinated (direct effect) and unvaccinated persons (indirect protection). Grijalva et al reported a major decline in all-cause pneumonia hospitalization rates in children aged <2 years following the 7 valent pneumococcal vaccine (PCV7) introduction, using a time series analysis, an all-cause pneumonia definition and hospital discharge data from the National Inpatient Sample. 1 Griffin et al using the same database and a similar all-cause pneumonia definition documented that the decline in pneumonia hospitalizations in children aged <2 years was sustained through 2010, and also reported significant declines in most other age groups. 2 This all-cause pneumonia algorithm was subsequently shown to have a positive predictive value (PPV) of 95.5% for identifying pneumonia that led to hospitalization in adults. 3
In 2010, PCV13, a conjugate vaccine with increased pneumococcal serotype coverage, replaced PCV7 for universal childhood immunization in the United States. In addition, the Food and Drug Administration approved PCV13 for adults in 2010 and the Advisory Committee on Immunization Practices recommended it for all adults aged 65 years and older in 2014, although that recommendation was changed again in 2019 to one of shared decision-making. Continued monitoring of pneumonia hospitalization trends in both children and adults in the United States can help assess the impact of these recommendations.
Prior to the implementation of the International Classification of Disease, 10th revision, Clinical Modification (ICD-10-CM) in 2015, algorithms used to identify hospital admissions for pneumonia from hospital administrative data were based on the International Classification of Diseases, 9th revision, Clinical Modification (ICD-9-CM). 4,5 The transition between coding systems creates challenges in assessing pneumonia trends. Given that pneumococcal conjugate vaccines confer indirect as well as direct effects, measuring population trends may give a more accurate estimate of vaccine impact than traditional vaccine effectiveness analyses. Traditional analyses based on individual-level vaccination status only measure direct effects since they exclude benefits to unvaccinated individuals. In addition, those studies likely underestimate benefits to vaccinated persons by using as comparators unvaccinated individuals whose disease risk decreased due to the vaccination program.
An understanding of the comparability of algorithms using ICD-9-CM and ICD-10-CM is important when analyzing discharge diagnosis data to assess impact of vaccine on pneumonia and other conditions. 6 -9 Other investigators have found lack of a direct correspondence between ICD-9-CM and ICD-10-CM coding systems for several conditions; therefore, additional examination is needed to enable evaluations that encompass periods including both coding eras. 8
Methods
Our aim was to evaluate the impact of ICD-10-CM coding implementation on pneumonia trends. We aimed to identify whether there was a systematic difference in pneumonia coding between the ICD-9-CM and ICD-10-CM systems, and if so to calculate an adjustment factor that could be used to account for such a difference. To identify systematic differences in coding, hospitalizations meeting pneumonia criteria in the year prior to ICD-10 implementation (ie, using ICD-9-CM) were recoded using ICD-10-CM, and hospitalization meeting pneumonia criteria in the year following ICD-10-CM implementation were recoded using ICD-9-CM. This analysis was considered public health activity by the Centers for Disease Control and Prevention and Vanderbilt University Medical Center Human Subject Research Protection Offices and institutional review board review was not required by either institution.
Study Population and Data Sources
The Hospital Discharge Data System (HDDS) of the Tennessee Department of Health receives information from UB-92 (HCFA-1450) forms on all inpatient discharges from Tennessee hospitals. Each form contains information on patient diagnoses, procedures performed on the patients, charges for services provided, and selected patient demographics. Hospitalization data from one large academic hospital were used to recode pneumonia hospitalizations identified from 2 periods: 1 year before (October 1, 2014, through September 30, 2015) and 1 year after (October 1, 2015, through September 30, 2016) implementation of ICD-10-CM.
Pneumonia definition
We based our algorithm to identify hospitalizations for all-cause pneumonia in the HDDS data on that used by Griffin et al to analyze national pneumonia trends. The algorithm required a first-listed discharge diagnosis of pneumonia or a first-listed discharge diagnosis of meningitis, septicemia, empyema, or acute respiratory failure in addition to a diagnosis of pneumonia in another diagnostic field. 2
The ICD-9-CM pneumonia algorithm was translated to an ICD-10-CM algorithm using General Equivalence Mappings (GEMS). The National Center for Health Statistics, Centers for Medicare & Medicaid Services, American Health Information Management Association, American Hospital Association, and 3M Health Information Systems developed GEMS as a publicly available reference map, to aid in navigating the complex meanings between code sets. 10,11 Codes used for this study are listed in Table 1. 10
Codes Used Pneumonia Algorithm.a
Abbreviations: ICD-9-CM, International Classification of Disease, 9th revision, Clinical Modification; ICD-10-CM, International Classification of Disease, 10th revision, Clinical Modification.
a ICD-9-CM codes were mapped to ICD-10-CM codes using general equivalency mapping.
Data Collection
We used the ICD-9-CM published and ICD-10-CM translated pneumonia algorithms to identify hospital medical records with a discharge diagnosis of pneumonia during the 1-year period before and 1-year period after ICD-10-CM implementation, respectively. For each of these 2 periods, we randomly selected 500 records (150 from children aged ≤5 years and 350 from adults aged ≥65 years). Pairs of certified billing coders from 2 different medical systems recoded each selected record using ICD-10-CM codes for records with discharge dates October 1, 2014, through September 30, 2015, and ICD-9-CM codes for records with discharge dates October 1, 2015, through September 30, 2016. Coders were masked to the original coding and to that of their co-coder. We employed 4 different coders for adult charts and 3 coders for children’s charts, and pairs changed over the course of the study, preventing calculation of intercoder reliability.
Data Analysis
We assessed the accuracy of the translated ICD-10-CM algorithm using the ICD-9-CM algorithm as the reference. We determined the sensitivity of the ICD-10-CM coding algorithm by calculating the proportion of hospitalizations that met the pneumonia definition using ICD-9-CM coding that also met the definition when recoded with ICD-10-CM. We determined the PPV of the ICD-10-CM coding algorithm by calculating the proportion of hospitalizations that met the pneumonia definition when charts that met the algorithm by ICD-10-CM coding were recoded using ICD-9-CM. For both the sensitivity and PPV calculations the denominators were 300 charts for children and 700 for adults.
We used the estimated sensitivity and PPV of the ICD-10-CM coding algorithm to estimate an adjustment factor that accounts for the transition from ICD-9-CM to ICD-10-CM and could be used to obtain an adjusted incidence for the post-ICD-10 period, (Supplemental material). The formula calculated the adjusted incidence post-ICD-10-CM implementation as the observed incidence post-ICD-10-CM implementation multiplied by the quotient of PPV and sensitivity.
We considered the means of sensitivity and PPV to be distributed uniformly within the observed range of sensitivity and PPV across the coders since only 3 or 4 coders did the recoding for each group of charts (children pre-ICD-10-CM implementation, children post-ICD-10-CM implementation, adults pre-ICD-10-CM implementation, adults post-ICD-10-CM implementation). To further account for the sampling variability from finite sample sizes, the actual sensitivity and PPV values were considered to follow a binomial distribution. Based on these modeling assumptions, a Monte-Carlo sampling approach was applied to obtain the sampling distributions of the adjustment factors. For children and adults, separately, we uniformly sampled 10,000 times from the range of observed values of the sensitivity and PPV to get the means and variability of the respective values. For each of the 10,000 means and the actual sample sizes, we generated simulated sensitivity and PPV values from the corresponding binomial distributions. We then applied the adjustment formula to obtain a sampling distribution of the adjustment factor. The means of the sampled sensitivities and PPVs and the 95% credible regions (CRs) of the sampling distributions were calculated. To express the predictive distribution of where these parameters fell, we used a Bayesian approach and thus the term CR rather than confidence interval was used. The percent needed to be adjusted to account for the transition to ICD-10-CM was calculated as (1 − adjustment factor) × 100. Considering that the underlying population denominator remained generally constant during the year following ICD-10-CM implementation, this percent adjustment could be applied to hospitalization rates after October 1, 2015, to account for the switch to the new coding system. Our 1000-chart sample size was initially selected to detect a difference (adjustment factor) of 5% or greater between the 2 time periods. However, our initial analysis revealed differences in results between children (300 charts) and adults (700 charts), so all analyses are reported separately for the 2 age groups.
Results
We recoded 500 charts (150 children, 350 adults) that met the pneumonia definition using an ICD-9-CM coding algorithm and 500 charts (150 children, 350 adults) which met the pneumonia definition after translating this algorithm to ICD-10-CM. The crude sensitivity and PPV of the ICD-10-CM algorithm for detecting pneumonia, using ICD-9-CM charts as the gold standard were 94% (282/300) and 92% (276/300), respectively, for children (Table 2). These metrics were lower for adults, with a sensitivity of 74% (516/700) and a PPV of 79% (553/700). For children, the observed range for sensitivity and PPV across coders was 94% to 94% and 88% to 96%, respectively. For adults, the range was 72% to 78% for sensitivity and 74% to 86% for PPV. The calculated adjustment factor was −2.09% (95% CR: −7.71% to +3.0%) for children and +6.76% (95% CR: −3.06% to +16.7%) for adults (Supplemental figure).
Estimated Sensitivity, Positive Predictive Value, and Range of These Values for ICD-10-CM Coding Using ICD-9-CM Coding as the Reference.
Abbreviations: ICD-9-CM, International Classification of Disease, 9th revision, Clinical Modification; ICD-10-CM, International Classification of Disease, 10th revision, Clinical Modification.
For children, there were only 3 (2%) and 2 (1%) of 150 charts in which neither of 2 coders indicated that charts met the pneumonia definition, when using the alternate coding scheme. Table 3 presents the specific alternative diagnoses. The small number of charts in both periods for which there was agreement between the 2 coders about alternative diagnoses to pneumonia suggests no large systematic difference in coding between the 2 eras for children. For adults, however, there were 41 (12%) and 32 (9%) of 350 charts for which both coders’ results did not meet the pneumonia definition using the alternate coding scheme. Table 3 presents the specific alternative diagnoses. The percent was higher for the translation from ICD-9-CM to ICD-10-CM than vice versa, consistent with the relatively low sensitivity of 74%. Of the 41 charts that did not meet the pneumonia definition by either of 2 coders when recoded using ICD-10-CM, 29 (71%) had a primary diagnosis of chronic obstructive pulmonary disease (COPD) exacerbation; all of these had pneumonia as a secondary diagnosis but did not meet the algorithm-defined pneumonia definition.
Charts That Failed to Meet Pneumonia Definition by Both of 2 Coders When Recoded by Different Coding Scheme.
Discussion
Our results suggest that an ICD-10-CM algorithm derived from a validated ICD-9-CM algorithm will not introduce substantial bias if used for evaluating trends in pneumonia for young children. This is important information since it suggests that observed trends in pneumonia based on analysis of hospital discharge data are unlikely to be attributable to adoption of ICD-10-CM.
For older adults, our results suggest that the algorithm we examined may not systematically impact the identification of pneumonia hospitalizations. However, the precision of the estimation is limited, and we cannot rule out that the algorithm may undercount events that would have been coded as pneumonia in the ICD-9-CM era. Although large underestimations seem unlikely, more modest impact (eg, <16.7%) would be difficult to rule out. This is an important consideration because the estimated efficacy of PCV13 against all-cause pneumonia among adults aged 65 years and older in a large clinical trial was 5.1% (95% CI, −5.1 to 14.2). 12 Our results suggest that unadjusted estimations of rates using the translated ICD-10-CM algorithm may magnify the relatively small real expected effects. In addition, the childhood vaccination program greatly reduces colonization with vaccine-type serotypes, leading to indirect protection of unvaccinated groups against pneumococcal infections, including pneumonia. The additional benefit of vaccinating adults directly would diminish as circulation of vaccine serotypes is markedly reduced in populations in which there is universal vaccination of children. 13 Detecting small changes in ecologic studies using administrative data is challenging. Our analysis suggests that changes in coding make the task more difficult.
The low sensitivity of the ICD-10-CM algorithm in our adult data appeared to be due in part to a systematic change in the way pneumonia associated with a COPD exacerbation was coded. This change appeared to occur with the transition to ICD-10-CM in 2015. The American Heart Association’s Coding Clinic for ICD-10-CM/PCS (2016 third quarter edition) clarified the use of the ICD-10-CM pneumonia and influenza codes when a patient has COPD. The COPD codes J44.0 and J44.1 require that the lower respiratory infection be sequenced after it, meaning the lower respiratory infection cannot be assigned as the primary diagnosis in patients with COPD, whether an acute exacerbation is present or not, as the code book states “use additional code to identify the infection.” This change would also affect other algorithms that used pneumonia as the primary discharge diagnosis since a substantial number of these events are now being coded as COPD exacerbation, with pneumonia listed as a secondary diagnosis. It is unclear whether other algorithms, such those based on listing of pneumonia in any diagnostic position, would give a more accurate representation of trends in pneumonia.
In an analysis of inpatient and outpatient Veteran’s Administration medical record data from 1 year before and 1 year after the ICD-10-CM transition, Yoon and Chow compared prevalence of 34 common conditions based on ICD-9-CM and ICD-10-CM codes. For several conditions, including pneumonia, their analysis suggested that there was a 27% higher odds of having pneumonia as a coded diagnosis during the ICD-10 than the ICD-9 year. 14 The inclusion of both outpatient and inpatient events in this analysis precludes a direct comparison with our results.
There has been increasing recognition of sepsis as an important contributor to morbidity and mortality and a frequent accompaniment to other conditions, including pneumonia. Lindenauer et al analyzed trends in hospital admissions and outcomes for patients with pneumonia, sepsis, and respiratory failure. They compared results using 2 alternative approaches for defining pneumonia: one that depends on pneumonia listed as the primary diagnosis and another that also includes patients with the primary diagnosis of sepsis when combined with a secondary diagnosis of pneumonia. 15,16 Overall, when requiring pneumonia to be listed as the primary diagnosis, there was a 27.4% reduction in the annual pneumonia hospitalization rate, and a 28.2% decline in in-hospital pneumonia mortality from 2003 to 2009. However, when including those with a primary diagnosis of either sepsis or respiratory failure combined with pneumonia listed as a secondary diagnosis, the decline in hospitalization rate was attenuated to 12.5%, and inpatient mortality changed little. This suggests a change in diagnostic coding, such that sepsis became a preferred primary diagnosis. These findings suggest that attempts to measure outcomes in patients with pneumonia by studying only those who receive a primary diagnosis of pneumonia will be biased toward inclusion of those with less severe disease. 17 This could be especially problematic for longitudinal studies that are subject to the effects of such temporal trends in coding practice.
The algorithm used for the current study was initially developed to assess national pneumonia trends. Pneumonia was defined as pneumonia as the primary discharge diagnosis or a first listed diagnosis of sepsis, meningitis, acute respiratory failure, or empyema in addition to a diagnosis of pneumonia in another diagnostic field. This algorithm was shown to have a PPV of 95.5% for identifying community-acquired pneumonia (pneumonia) in adults. 3 However, the algorithm was developed using only ICD-9-CM era data.
Some studies that converted ICD-9-CM algorithms to ICD-10-CM have assumed that trends in outcomes evaluated are stable and attributed changes in trends over the transition to the new coding. 9 However, pneumonia rates also change year to year as a result of changes in circulating respiratory viruses, including influenza and respiratory syncytial virus. In addition, introduction of pneumococcal conjugate vaccines has affected temporal trends in pneumonia. Therefore, we elected to re-code charts to specifically evaluate whether the change in coding was likely to affect trends. In this study, we only evaluated one algorithm; evaluation of other algorithms would require selection and evaluation of different charts. In addition, our study was performed at one large hospital and employed a total of 4 trained coders. Although coding manuals attempt to standardize the way primary discharge diagnoses are determined, there are likely both personal and institutional influences on coding such that our results may not generalize directly to other settings. Given that consistent pairs of coders were not used, we were unable to assess intercoder reliability. Finally, since sensitivity and PPV varied substantially between children and adults, we performed all analyses separately for these 2 groups, limiting the power to detect changes due to coding in each of these strata. Although a larger sample size in each of these strata would provide more precision, the credible intervals indicate the range of likely values.
We identified changes in coding of COPD as one reason for differences in pneumonia classification among adults between the 2 coding eras. In addition, charts for which the primary reason for admission was pneumonia in one era were classified as a wide variety of other conditions in the other era (Table 3). The attribution of admission to other chronic conditions was much less common in children. Williams and colleagues found that of the 12 ICD-9 algorithms they evaluated in children, those with the best performance characteristics excluded children with complex chronic conditions. 5 Although such children constitute the minority of those with pneumonia, most older adults with pneumonia have comorbidities, which may make standardization of coding for adults with pneumonia more challenging.
Conclusion
In summary, we found that the coding algorithm we assessed translated well to ICD-10-CM, but it is difficult to rule out a modest undercount of all-cause pneumonia in the ICD-10-CM compared to the ICD-9-CM era for adults. In addition, we identified a change in coding for pneumonia associated with COPD, such that COPD is preferentially listed as the primary discharge diagnosis in the ICD-10-CM era. This information will be important for others trying to assess trends in pneumonia.
Supplemental Material
Supplemental Material, Pneumonia_Hospitalization_Coding_Changes_associated_with_Transition_from_the_9th_to_10th_Revisionmg_-_Supplement_for_submission_anonymized - Pneumonia Hospitalization Coding Changes Associated With Transition From the 9th to 10th Revision of International Classification of Diseases
Supplemental Material, Pneumonia_Hospitalization_Coding_Changes_associated_with_Transition_from_the_9th_to_10th_Revisionmg_-_Supplement_for_submission_anonymized for Pneumonia Hospitalization Coding Changes Associated With Transition From the 9th to 10th Revision of International Classification of Diseases by Ryan B. Smithee, Tiffanie M. Markus, Elizabeth Soda, Carlos G. Grijalva, Wei Xing, Nong Shang, Marie R. Griffin and Fernanda C. Lessa in Health Services Research and Managerial Epidemiology
Footnotes
Authors’ Note
The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the US Centers for Disease Control and Prevention.
Acknowledgments
The authors thank their medical coding colleagues Teresa Surratt, Penny Brasfield, Jennifer Yates, and Elizabeth Porter who performed the medical record recoding to make this project possible.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: MRG has received research support from The Campbell Alliance, The Centers for Disease Control and Prevention, and the Food and Drug Administration; CGG has received consulting fees from Pfizer and Merck and received research support from Sanofi-Pasteur, Campbell Alliance, the Centers for Disease Control and Prevention, National Institutes of Health, The Food and Drug Administration, and the Agency for Health Care Research and Quality.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Centers for Disease Control and Prevention (CDC; grant number CK17-1701), and the National Institutes of Health Grant (UL1 TR000445).
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
