Sensitivity and Positive Predictive Value of Death Certificate Data Among Deaths Caused by Legionnaires’ Disease in New York City,2008-2013

Abstract

Objectives:

Death certificates are an important source of information for understanding life expectancy and mortality trends; however, misclassification and incompleteness are common. Although deaths caused by Legionnaires’ disease might be identified through routine surveillance, it is unclear whether Legionnaires’ disease is accurately recorded on death certificates. We evaluated the sensitivity and positive predictive value of death certificates for identifying deaths from confirmed or suspected Legionnaires’ disease among adults in New York City.

Methods:

We deterministically matched death certificate data from January 1, 2008, through December 31, 2013, on New York City residents aged ≥18 years to surveillance data on confirmed and suspected cases of Legionnaires’ disease from January 1, 2008, through October 31, 2013. We estimated sensitivity and positive predictive value by using surveillance data as the reference standard.

Results:

Of 294 755 deaths, 27 (<0.01%) had an underlying cause of death of Legionnaires’ disease and 33 (0.01%) had any mention of Legionnaires’ disease on the death certificate. Of 1211 confirmed or suspected cases of Legionnaires’ disease, 267 (22.0%) matched to a record in the death certificate data set. The sensitivity of death certificates that listed Legionnaires’ disease as the underlying cause of death was 17.3% and of death certificates with any mention of Legionnaires’ disease was 20.9%. The positive predictive value of death certificates that listed Legionnaires’ disease as the underlying cause of death was 70.4% and of death certificates with any mention of Legionnaires’ disease was 69.7%.

Conclusions:

Death certificates had limited ability to identify confirmed or suspected deaths with Legionnaires’ disease. Provider trainings on the diagnosis of Legionnaires’ disease, particularly hospital settings, and proper completion of death certificates might improve the sensitivity of death certificates for people who die of Legionnaires’ disease.

Keywords

Legionnaires’ disease legionella pneumonia and influenza respiratory infections mortality data

In 2015, New York City had the largest Legionnaires’ disease (LD) outbreak in its history, with 138 cases and 16 (12%) deaths.¹ In New York City, LD incidence, driven predominantly by transmission in the community (88%), has increased since 2002, consistent with national trends.^2,3 Case-fatality rates among community-acquired LD cases typically ranged from 7% to 10%, based on studies using data from 1993 to 2011.^2,4,5

Death certificate data indicate that pneumonia and influenza are the third-leading underlying cause of death (COD) in New York City.⁶ Death certificates are an important source of information for understanding life expectancy and mortality trends; however, misclassification and incompleteness are common.^7,8 Physicians use their best judgment and the medical information available at the time of death to report the causal chain of events leading to death and any other conditions that contributed substantially to death. The underlying COD (or the cause that precipitated the chain of events leading to death) is arguably the most important because it is used to categorize leading CODs in a jurisdiction.⁹ However, in some cases, the causal chain of events may be unclear or the medical information available may be limited. Results of LD tests might not be available until after a person has died; as such, LD deaths might be coded as pneumonia and influenza.² Because of reporting requirements for LD, it is also possible that a death is counted as an LD death during analysis of LD surveillance data but is not counted during analysis of death certificate data. In addition, although a physician might list LD on a death certificate, the decedent might not meet criteria to be a confirmed LD case as defined by the public health surveillance system. To our knowledge, no published studies have assessed the accuracy of death certificates for identifying LD deaths. The objective of our study was to evaluate the sensitivity and positive predictive value (PPV) of death certificates for identifying confirmed or suspected LD deaths.

Methods

Definitions

The terms “confirmed LD case” and “confirmed LD diagnosis” refer to a case or a diagnosis in the LD surveillance system, respectively, that meets the definition provided by the Council of State and Territorial Epidemiologists.¹⁰ A “suspected LD case” refers to a case that is clinically compatible with LD and meets at least 1 presumptive laboratory criterion for Legionella.¹⁰ “Confirmed or suspected LD death” is defined as a death that occurred ≤30 days after the diagnosis date of a confirmed or suspected LD case. “Underlying COD” refers to the condition that initiated the chain of events leading directly to death and is determined from the conditions entered by the physician on the death certificate. “Any mention of COD” refers to conditions listed on the death certificate that either were part of the chain of events leading directly to death or were “contributing CODs” (ie, CODs that contributed to death but were not part of the chain of events leading directly to death). Underlying CODs are nested within a list of any mention CODs.

Data Collection

In New York City, physicians and laboratories are required to report LD to the New York City Department of Health and Mental Hygiene (DOHMH) LD surveillance system within 24 hours of diagnosis. Most laboratory reports are for positive urine antigen tests, which detect Legionella pneumophila serogroup 1. DOHMH investigates all reported cases to determine whether they meet LD case definitions. Death certificates must be filed within 72 hours of death or body discovery.

Study Population

Using methods described previously, we deterministically matched death certificate data from January 1, 2008, through December 31, 2013, on New York City residents aged ≥18 years to surveillance data on confirmed and suspected LD cases from January 1, 2008, through October 31, 2013; no human review was performed.¹¹ We assumed people who matched were deceased and had confirmed or suspected LD. We excluded people who died from external causes (eg, accident, homicide) or suicide from this retrospective cohort study.

COD Determination

In New York City, CODs on death certificates are submitted in text; an automated algorithm assigns International Classification of Diseases, 10th Revision (ICD-10)¹² diagnostic codes and identifies underlying and contributing CODs. An ICD-10 code of A48.1 indicates that LD was listed on the death certificate. We also examined an expanded set of underlying CODs and any mention of CODs, including LD, pneumonia, or influenza (LDPI), by using ICD-10 codes A48.1 and J09-J18. We categorized all other non-LDPI ICD-10 codes into general categories (Table 1).

Table 1.

Non-Legionnaires’ disease, pneumonia, or influenza cause-of-death categories^a

Cause-of-Death Category	International Classification of Diseases, 10th Revision Code
Major cardiovascular disease	I00-I78
Malignant neoplasms	C00-C97
Diabetes mellitus	E10-E14
HIV/AIDS	B20-B24
Respiratory diseases, excluding pneumonia and influenza	J00-J99, excluding J09-J18
Other bacterial diseases, excluding Legionnaires’ disease	A30-A49, excluding A48.1
Chronic liver disease and cirrhosis	K70, K73-K74
Viral infections or other viral diseases	A80-B19, B25-B34
Other infectious diseases	B35-B99

^a Data source: International Classification of Diseases, 10th Revision.¹²

Statistical Methods

We calculated the case fatality rate as the proportion of cases that matched to a death record within 30 days of diagnosis. We estimated sensitivity and PPV with corresponding 95% confidence intervals (CIs) by using surveillance data as the reference standard. We used the Pearson χ² test or the Fisher exact test, when appropriate, to evaluate the extent of discordance between LD deaths identified through surveillance and through listed COD on the death certificate, stratified by sex, race/ethnicity, age at death, census tract–level poverty (based on residence at time of LD diagnosis), and type of place (hospital or other) at death.¹³ Census tracts were grouped into 4 poverty levels based on the percentage of residents living below the federal poverty level: <10% (low), 10%-19% (medium), 20%-29% (high), and ≥30% (very high). We conducted a post hoc sensitivity analysis to evaluate the robustness of our definition for LD deaths, using ≤15 and ≤60 days after the diagnosis date of a confirmed or suspected LD case.

We conducted all analyses by using SAS version 9.2.¹⁴ We considered P < .05 to be significant. This investigation was deemed to be nonhuman subjects research by DOHMH and to be public health nonresearch by the Centers for Disease Control and Prevention.

Results

Of 294 755 adult deaths, excluding those from external causes, from January 1, 2008, through December 31, 2013, a total of 27 (<0.01%) had an underlying COD of LD and 33 (0.01%) had any mention of LD on the death certificate (Table 2). Of the 33 decedents with any mention of LD on their death certificate, 23 (69.7%) had a confirmed or suspected LD diagnosis according to surveillance records.

Table 2.

Confirmed or suspected number of Legionnaires’ disease deaths by whether Legionnaires’ disease or Legionnaires’ disease, pneumonia, or influenza was listed as an underlying cause of death or mentioned at all on the death certificate, New York City, 2008-2013

Information on Death Certificate	No. of Confirmed or Suspected Legionnaires’ Disease Deaths^a	No. of Not Confirmed or Suspected Legionnaires’ Disease Deaths^b
Underlying cause of death listed on death certificate
Legionnaires’ disease listed	19	8
Legionnaires’ disease not listed	91	294 637
Legionnaires’ disease, pneumonia, or influenza listed	40	14 043
Legionnaires’ disease, pneumonia, or influenza not listed	70	280 602
Any mention of cause of death on death certificate
Legionnaires’ disease listed	23	10
Legionnaires’ disease not listed	87	294 635
Legionnaires’ disease, pneumonia, or influenza listed	58	26 422
Legionnaires’ disease, pneumonia, or influenza not listed	52	268 223

^a Deaths occurring ≤30 days after the diagnosis date of a confirmed or suspected Legionnaires’ disease case.

^b Deaths occurring >30 days after the diagnosis date of a confirmed or suspected Legionnaires’ disease case and deaths that did not match to a confirmed or suspected Legionnaires’ disease case from January 1, 2008, through October 31, 2013.

Of 1211 confirmed or suspected LD cases during January 1, 2008, through October 31, 2013, a total of 267 (22.0%) matched to a record in the death certificate data set. Of these 267 people, 110 (41.2%) died ≤30 days after LD diagnosis. The 30-day case fatality rate was 9.1% (110/1211). Of these 110 decedents, 59 (53.6%) were male, and the median age at death was 73 (interquartile range [IQR], 60-83) (Table 3). Fifty-six (50.9%) decedents were non-Hispanic white, 31 (28.2%) were non-Hispanic black, and 17 (15.5%) were Hispanic. Ninety-eight decedents (89.1%) died in a hospital.

Table 3.

Characteristics of decedents with confirmed or suspected Legionnaires’ disease in New York City, 2008-2013

Characteristics	Total (n = 267)	Deceased ≤30 Days After Diagnosis (n = 110)	Deceased >30 Days After Diagnosis (n = 157)
Sex, no. (%)
Male	151 (56.6)	59 (53.6)	92 (58.6)
Female	116 (43.4)	51 (46.4)	65 (41.4)
Race/ethnicity, no. (%)
Non-Hispanic white	125 (46.8)	56 (50.9)	69 (43.9)
Non-Hispanic black	80 (30.0)	31 (28.2)	49 (31.2)
Hispanic	49 (18.4)	17 (15.5)	32 (20.4)
Non-Hispanic Asian	13 (4.9)	6 (5.5)	7 (4.5)
Age at death, median (IQR), y	73 (60, 83)	73 (60, 83)	73 (60, 84)
Census tract–based poverty level,^a no. (%)^b
Low	74 (27.7)	34 (31.2)	40 (26.0)
Medium	68 (25.5)	31 (28.4)	37 (24.0)
High	54 (20.2)	19 (17.4)	35 (22.7)
Very high	67 (25.1)	25 (22.9)	42 (27.3)
Place of death, no. (%)
Hospital	190 (71.2)	98 (89.1)	92 (58.6)
Other	77 (28.8)	12 (10.9)	65 (41.4)

Abbreviation: IQR, interquartile range.

^a Census tracts were grouped into 4 poverty levels based on the percentage of residents living below the federal poverty level: <10% (low), 10%-19% (medium), 20%-29% (high), and ≥30% (very high).

^b Denominator is the number of decedents whose census tract at the time of diagnosis was available. A total of 263 decedents, 109 who died ≤30 days after diagnosis and 154 who died >30 days after diagnosis, had census tract data.

The sensitivity of death certificates that listed LD as the underlying COD was 17.3% (19/[19 + 91]; 95% CI, 10.7%-25.7%) and of death certificates with any mention of LD was 20.9% (23/[23 + 87]; 95% CI, 13.7%-29.7%). The PPV of death certificates that listed LD as the underlying COD was 70.4% (19/[19 + 8]; 95% CI, 49.8%-86.2%) and of death certificates with any mention of LD was 69.7% (23/[23 + 10]; 95% CI, 51.3%-84.4%). The sensitivity of death certificates that listed LDPI as the underlying COD was 36.4% (40/[40 + 70]; 95% CI, 27.4%-46.1%) and of death certificates with any mention of LDPI was 52.7% (58/[58 + 52]; 95% CI, 43.0%-62.3%). The PPV of death certificates that listed LDPI as the underlying COD was 0.3% (40/[40 + 14 043]; 95% CI, 0.2%-0.4%) and of death certificates with any mention of LDPI was 0.2% (58/[58 + 26 422]; 95% CI, 0.2%-0.3%).

Of death certificates that listed LD as the underlying COD, the use of a 15-day cut-off to identify confirmed or suspected LD deaths resulted in moderately improved sensitivity (19.1%) but decreased PPV (63.0%), whereas a 60-day cut-off resulted in decreased sensitivity (14.2%) and no change in PPV (70.4%).

Despite high levels of discordance between confirmed or suspected LD deaths identified through death certificates and LD surveillance, discordance did not differ significantly by sex, age, race/ethnicity, census tract–based poverty level, or place of death (Table 4).

Table 4.

Discordance of underlying and any mention cause-of-death data among confirmed or suspected Legionnaires’ disease deaths, by characteristics, New York City, 2008-2013

Characteristics	No. of Deaths ≤30 Days After Diagnosis of Legionnaires’ Disease (n = 110)	Underlying Cause of Death		Any Mention of Cause of Death
Characteristics		Discordant,^a No. (%)	P Value	Discordant,^a No. (%)	P Value
Sex^b
Male	59	39 (76.5)	.11	50 (84.8)	.12
Female	51	52 (88.1)	.11	37 (72.6)	.12
Race/ethnicity^c
Non-Hispanic white	56	49 (87.5)	.13	47 (83.9)	.18
Non-Hispanic black	31	25 (80.7)		23 (74.2)
Hispanic	17	14 (82.3)		14 (82.4)
Non-Hispanic Asian	6	3 (60.0)		3 (60.0)
Age at death,^c y
≤57	19	17 (89.5)	.62	16 (84.2)	.74
58-69	32	26 (81.2)		25 (78.1)
70-82	30	26 (86.7)		25 (83.3)
≥83	29	22 (75.9)		21 (72.4)
Census tract–based poverty level^c-e
Low	34	32 (94.1)	.13	30 (88.2)	.49
Moderate	31	26 (83.9)		24 (77.4)
High	19	14 (73.7)		14 (73.7)
Very high	25	19 (76.0)		19 (76.0)
Type of place at death^c
Hospital	98	79 (80.6)	.12	76 (77.6)	.45
Other	12	12 (100)	.12	11 (91.7)	.45

^a Discordant underlying and any mention cause of death was defined as not having Legionnaires’ disease listed on the death certificate.

^bP value determined by using Fisher exact test; P < .05 was considered significant.

^cP value determined by using Pearson χ² test; P < .05 was considered significant.

^d Census tracts were grouped into 4 poverty levels based on the percentage of residents living below the federal poverty level: <10% (low), 10%-19% (medium), 20%-29% (high), and ≥30% (very high).

^e Denominator is the number of people whose census tract at the time of diagnosis was available. A total of 109 decedents who died ≤30 days after diagnosis had census tract data.

Of 91 confirmed or suspected LD deaths without LD as the underlying COD on the death certificate, malignant neoplasms (n = 26, 28.6%), pneumonia or influenza (n = 21, 23.1%), and major cardiovascular disease (n = 18, 19.8%) were most commonly listed. Of 19 confirmed or suspected LD deaths with a listed underlying COD of LD on the death certificate, other contributing CODs included major cardiovascular disease (n = 10), respiratory diseases excluding pneumonia and influenza (n = 8), other bacterial diseases excluding LD (n = 5), malignant neoplasms (n = 3), and pneumonia or influenza (n = 2) (not mutually exclusive).

Discussion

We found that death certificates in New York City had limited ability to identify confirmed or suspected LD deaths and did not accurately reflect the burden of LD deaths identified through LD surveillance. For example, only about 1 in 5 confirmed or suspected LD deaths had a corresponding death certificate that listed an underlying COD of LD or any mention of LD. Not surprisingly, adding codes for pneumonia from other causes and influenza resulted in higher sensitivity for identifying confirmed or suspected LD deaths than examining the death certificate for LD only, but we still found that death certificates performed poorly.

Inaccuracies in COD reporting on death certificates have been documented, and failure to properly code deaths by etiology can lead health officials to underestimate the burden of disease and underinvest in prevention and control efforts.^15,16 In 2009, at large hospitals, DOHMH began conducting training sessions focused on reporting procedures and classification guidelines, such as avoiding the use of nonspecific categories such as “cardiorespiratory failure.” Training on LD is particularly important, given its rising incidence and public health importance, and particularly effective, given that most confirmed or suspected LD deaths occur in hospitals. From a public health perspective and as per national clinical recommendations, people with severe community-acquired pneumonia should be tested for Legionella pneumophila to determine etiology.¹⁷

We measured a 30-day case-fatality rate of 9.1% among people with a confirmed or suspected diagnosis of LD, consistent with previous studies.^2,4,5 Underlying conditions (eg, HIV, cancer) have been associated with increased susceptibility to LD and poor outcomes and were common among people with a confirmed or suspected LD death.⁴ Given the nonspecific clinical manifestations of LD, other underlying conditions could possibly overshadow the clinical presentation of LD and its contribution and attribution to death.

Limitations

This study had several limitations. First, because we focused on New York City, our results may not be generalizable to other jurisdictions. Second, although decedents had a diagnosis of confirmed or suspected LD, the study population represented only a subset of LD cases, and the sociodemographic profile of our study population differed from that of all LD cases in New York City.² Third, we did not conduct a human review on matches; as such, our outcome might have been subject to under- or overmatching. However, because we used a conservative match algorithm, we believe the risk of overmatching was low. Fourth, our data did not capture data on people diagnosed in New York City who died outside of New York City, and we assumed that people who did not match to a death certificate were still living, which may have led to underestimation. However, the case fatality rate was consistent with other estimates.^2,4,5 Fifth, underlying COD was coded by an automated algorithm, and the sensitivity of underlying COD was subject to hierarchical selection rules. Therefore, any inaccuracies in the order by which diagnoses were reported might subsequently result in misclassification of the underlying COD. Sixth, this study lacked autopsy data, which are the gold standard for determining COD. Despite not having autopsy data, we believe using 30-day case fatality to define confirmed and suspected LD deaths was reasonable based on the acute nature of LD and the PPVs observed. In addition, our sensitivity analysis demonstrated the robustness of this cut-off. Finally, we did not conduct a human review of pneumonia decedents in New York City; therefore, it is possible that some LD cases were missed entirely.

Conclusion

Our study identified major gaps in COD reporting on death certificates among people who died shortly after a diagnosis of confirmed or suspected LD. Underreporting of LD on death certificates may result in missed opportunities for public health funding and intervention. Provider trainings on the diagnosis of LD and proper completion of death certificates are needed to improve the accuracy of reporting for people who die of LD.

Footnotes

Disclaimer

The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention or the New York City Department of Health and Mental Hygiene.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Weiss

Boyd

Rakeman

. A large community outbreak of Legionnaires’ disease associated with a cooling tower in New York City, 2015. Public Health Rep. 2017;132(2):241–250.

Farnham

Alleyne

Cimini

Balter

. Legionnaires’ disease incidence and risk factors, New York, New York, USA, 2002–2011. Emerg Infect Dis. 2014;20(11):1795–1802.

Garrison

Kunz

Cooley

. Vital signs: deficiencies in environmental control identified in outbreaks of Legionnaires’ disease—North America, 2000-2014. MMWR Morb Mortal Wkly Rep. 2016;65(22):576–584.

Phin

Parry-Ford

Harrison

. Epidemiology and clinical management of Legionnaires’ disease. Lancet Infect Dis. 2014;14(10):1011–1021.

Dominguez

Alvarez

Sabria

. Factors influencing the case-fatality rate of Legionnaires’ disease. Int J Tuberc Lung Dis. 2009;13(3):407–412.

Huynh

Lee

. Summary of Vital Statistics, 2014. New York: New York City Department of Health and Mental Hygiene, Bureau of Vital Statistics; 2016.

Smith Sehdev

Hutchins

. Problems with proper completion and accuracy of the cause-of-death statement. Arch Intern Med. 2016;161(2):277–284.

Messite

Stellman

. Accuracy of death certificate completion: the need for formalized physician training. JAMA. 1996;275(10):794–796.

Brooks

Reed

. Principles and pitfalls: a guide to death certification. Clin Med Res. 2015;13(2):74–82.

10.

Council of State and Territorial Epidemiologists. Strengthening surveillance for travel-associated legionellosis and revised case definitions for legionellosis. Position statement 05-ID-01. 2005. http://c.ymcdn.com/sites/www.cste.org/resource/resmgr/PS/05-ID-01FINAL.pdf. Accessed May 30, 2018.

11.

Drobnik

Pinchoff

Bushnell

. Matching HIV, tuberculosis, viral hepatitis, and sexually transmitted diseases surveillance data, 2000-2010: identification of infectious disease syndemics in New York City. J Public Health Manag Pract. 2014;20(5):506–512.

12.

World Health Organization. International Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10). Geneva: World Health Organization; 2015.

13.

Toprani

Hadler

. Selecting and Applying a Standard Area-Based Socioeconomic Status Measure for Public Health Data: Analysis for New York City. New York: New York City Department of Health and Mental Hygiene; 2013.

14.

SAS Institute, Inc. SAS Version 9.2 Procedures Guide. Cary, NC: SAS Institute, Inc; 2011.

15.

Johns

Madsen

Maduro

Zimmerman

Konty

Begier

. A case study of the impact of inaccurate cause-of-death reporting on health disparity tracking: New York City premature cardiovascular mortality. Am J Public Health. 2013;103(4):733–739.

16.

Wexelman

Eden

Rose

. Survey of New York City resident physicians on cause-of-death reporting, 2010. Prev Chronic Dis. 2013;10:E76.

17.

Mandell

Wunderink

Anzueto

. Infectious Diseases Society of America/American Thoracic Society consensus guidelines on the management of community-acquired pneumonia in adults. Clin Infect Dis. 2007;44(suppl 2):S27–S72.