Sage Journals: Discover world-class research

Abstract

Objective

Sepsis is the leading cause of in-hospital mortality in the United States (US). Quality improvement initiatives for improving sepsis care depend on accurate estimates of sepsis mortality. While hospital 30-day risk-standardized mortality rates have been published for patients hospitalized with acute myocardial infarction, heart failure, and pneumonia, risk-standardized mortality rates for sepsis have not been well characterized. We aimed to construct a sepsis risk-standardized mortality rate map for the United States, to illustrate disparities in sepsis care across the country.

Methods

This cross-sectional study included adults from the US Nationwide Inpatient Sample who were hospitalized with sepsis between 1 January 2010 and 30 December 2011. Hospital-level risk-standardized mortality rates were calculated using hierarchical logistic modelling, and were risk-adjusted with predicted mortality derived from (1) the Sepsis Risk Prediction Score, a logistic regression model, and (2) gradient-boosted decision trees, a supervised machine learning (ML) algorithm.

Results

Among 1,739,033 adults hospitalized with sepsis, 50% were female, and the median age was 71 years (interquartile range: 58–81). The national median risk-standardized mortality rate for sepsis was 18.4% (interquartile range: 17.0, 21.0) by the boosted tree model, which had better discrimination than the Sepsis Risk Prediction Score model (C-statistic 0.87 and 0.78, respectively). The highest risk-standardized mortality rates were found in Wyoming, North Dakota, and Mississippi, while the lowest were found in Arizona, Colorado, and Michigan.

Conclusions

Wide variation exists in sepsis risk-standardized mortality rates across states, representing opportunities for improvement in sepsis care. This represents the first map of state-level variation of risk-standardized mortality rates in sepsis.

Keywords

sepsis risk standardized mortality rate machine learning mortality healthcare quality

Background

Over the past two decades, the Surviving Sepsis Campaign¹ and a multitude of quality improvement initiatives have aimed to standardize sepsis management and improve bundle compliance.^2,3 Central to the outcome evaluation of these initiatives are accurate estimates of sepsis mortality, as mortality reflects quality of care, and is the ultimate meaningful outcome to patients. Risk-standardized mortality rates (RSMRs) can reduce the confounding effect of case mix (e.g. age, comorbidity burden) and assess regional variation in sepsis mortality more precisely.

State-based differences in RSMR for acute myocardial infarction (AMI), heart failure (HF), and pneumonia (PNA) have been visualized based on public reporting from the Centers for Medicare and Medicaid Services (CMS),^4–7 but have not been performed for sepsis (eTable 1). Using similar methods, several studies demonstrated the inequality in sepsis care by reporting the mortality variation between hospitals. Hatfield et al. reported a difference of 11.5 percentage points in median sepsis mortality rate between the highest and lowest decile of hospitals.⁸ Wang et al. calculated the observed-to-expected mortality ratio in severe sepsis using administrative data from the University Health System Consortium.⁹ Darby et al. revealed that a hospital-specific RSMR for sepsis ranged from 12.2% to 24.5% using hospital discharge data only from the state of Pennsylvania.¹⁰ Walkey et al. also found a wide difference in RSMR in septic shock by both implicit diagnosis and explicit ICD codes.¹¹ However, these studies used mortality rates that were not risk-adjusted, employed traditional statistical regression techniques, and were restricted to populations at specific medical centers or age groups.

Thus, our proposed method differs from previous sepsis mortality measurement in two fundamental aspects. First, our RSMR estimates were based on a nationwide database with the integration of a supervised machine learning (ML)-based model allowing for relationships not based on conventional regression. Second, unlike previous studies which performed comparisons at a hospital level, we constructed a national sepsis mortality map of the United States to illustrate the geographic variation in sepsis quality of care.

Methods

Data sources

We extracted data from the 2010 and 2011 years of the US Nationwide Inpatient Sample (NIS), the largest all-payer, public claims database of inpatient encounters. The NIS contains a 20% stratified sample of all short-term, non-federal, non-rehabilitation hospitals, representing 5–8 million discharges per year. We applied the trend weights developed by the Healthcare Cost and Utilization Project to weight patient-level discharge data, to derive estimates for >35 million hospitalizations nationally. We could not use more recent data because the NIS did not provide state-specific identifiers after 2011.

Ethics approval

This study was approved by the Institutional Review Board at National Taiwan University Hospital, which waived the requirement for informed consent from patients because of the anonymous nature of this public database.

Case selection and definitions

In accordance with the Sepsis-3 definition, sepsis was defined as life-threatening organ dysfunction caused by a dysregulated host response to infection.¹² Sepsis hospitalizations were identified by Martin and Moss’ validated International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes or procedure codes from the electronic health record.¹³ We selected patients with any diagnosis of sepsis (995.92 severe sepsis, 785.52 septic shock) or systemic infection (038.* septicemia, 020.0 sepsis, 790.7 bacteremia, 117.9 disseminated fungal infection, 112.5 disseminated candidiasis, or 112.81 disseminated fungal endocarditis) and a diagnosis of acute organ dysfunction in any of seven organ systems: cardiovascular, respiratory, central nervous, hematologic, hepatic, renal and metabolic systems. We excluded patients not meeting Martin's sepsis criteria, missing mortality information, below 18 years of age, or who were transferred to a short-term hospital against medical advice. We collected demographic information, pre-existing comorbidities, and in-hospital mortality from claims records of the index hospitalization. Comorbidities, based on Gagne's validated ICD-9-CM codes,¹⁴ were not included in the risk adjustment unless they were also present in the 12 months before admission. Since the NIS lacks a code for ICU admission, mechanical ventilation >1 day was used as a proxy for ICU stay.¹⁵

Outcome

Thirty-day in-hospital crude mortality rate was calculated based on length of stay and in-hospital mortality data for each patient. The primary outcome was 30-day in-hospital risk-standardized all-cause mortality rate, as described below.

Descriptive statistics

Continuous variables were summarized as mean with standard deviation or median with interquartile range (IQR). Categorical variables were summarized as number and proportion. All analyses were performed in R (version 3.5, Vienna, Austria).

Risk-standardized mortality rates

Hierarchical logistic regression models were used to estimate hospital-level 30-day all-cause RSMRs for sepsis. This approach takes into account the two-level (patient and hospital) hierarchical structure of the data to account for patient clustering within hospitals. Each model includes age, sex, hospital-specific random effects (representing baseline mortality risk at that hospital), predicted mortality and an intercept. Higher RSMRs represent worse outcomes. RSMRs in our study are expressed as percentages except where otherwise noted as fractions of one.

Feature analysis and selection

For the reference model, we fit a logistic regression model using the widely-used Sepsis Risk Prediction Score model (SRS), which assigns patients a different number of points for each variable listed in Table 1.¹⁵ In the feature analysis step of the construction of the ML model, we used the same variables in Table 1 and performed multiple logistic regression to obtain adjusted odds ratios (aOR) in our sample. For each variable, all remaining covariates were mutually adjusted.

Table 1.

Baseline characteristics of inpatients with sepsis in 2010–2011, with adjusted odd ratios for all-cause mortality by demographic and clinical characteristics.

Variables	N (%)	OR (95% CI)
Demographics
Age < 40 (reference)	96,717 (5.56%)	Reference
Age 40–49	122,646 (7.05%)	1.48 (1.39–1.57)
Age 50–59	254,725 (14.65%)	1.81 (1.72–1.92)
Age 60–69	343,597 (19.76%)	2.17 (2.05–2.29)
Age 70–79	391,463 (22.51%)	2.86 (2.71–3.02)
Age 80–89	402,816 (23.16%)	4.17 (3.95–4.40)
Age 90 +	127,069 (7.31%)	6.12 (5.76–6.49)
Female sex (vs male sex)	876,769 (50.42%)	1.07 (1.05–1.09)
White/Other (reference)	1337,749 (76.92%)
African-American	262,360 (15.09%)	1.05 (1.02–1.08)
Hispanic	138,924 (7.99%)	0.99 (0.96–1.03)
Measures of acute illness severity (presence vs absence)
Early mechanical ventilation	304,659 (17.52%)	4.63 (4.53–4.75)
Late mechanical ventilation	88,067 (5.06%)	5.72 (5.43–6.02)
Shock	778,820 (44.78%)	2.06 (2.02–2.10)
Hemodialysis	202,522 (11.65%)	1.43 (1.39–1.48)
ICU care (at least one day)	167,975 (9.66%)	1.16 (1.11–1.20)
Comorbidities (presence vs absence)
Anemia	610,061 (35.08%)	0.64 (0.63–0.65)
Depression	170,290 (9.79%)	0.80 (0.78–0.83)
Diabetes	597,404 (34.35%)	0.86 (0.84–0.87)
Drug and substance abuse	41,357 (2.38%)	0.77 (0.72–0.82)
Chronic lung disease	433,508 (24.93%)	0.93 (0.91–0.95)
Congestive heart failure	443,337 (25.49%)	1.07 (1.04–1.09)
Hypertension	992,158 (57.05%)	0.78 (0.77–0.80)
Hypothyroid disease	225,333 (12.96%)	0.90 (0.88–0.93)
Liver disease	100,119 (5.76%)	2.07 (2.00–2.15)
Lymphoma	36,659 (2.11%)	1.62 (1.53–1.71)
Metastatic carcinomas	88,114 (5.07%)	2.78 (2.68–2.88)
Neurological conditions	271,067 (15.59%)	0.94 (0.92–0.97)
Obesity	185,958 (10.69%)	0.72 (0.70–0.75)
Paraplegia	121,273 (6.97%)	0.81 (0.78–0.84)
Perivascular conditions	172,268 (9.91%)	1.29 (1.25–1.33)
Psychiatric diseases	91,413 (5.26%)	0.71 (0.68–0.74)
Pulmonary-circulatory disorders	98,785 (5.68%)	1.14 (1.10–1.19)
Renal failure, chronic	585,859 (33.69%)	0.98 (0.96–1.00)
Malignant solid tumors	68,063 (3.91%)	1.59 (1.53–1.66)
Weight loss	352,449 (20.27%)	0.91 (0.89–0.93)

Early mechanical ventilation = 0–2 days from admission.

Late mechanical ventilation > 2 days from admission.

Selection, training, and validation of the ML model

Gradient boosted decision tree (GBDT; hereafter referred to as the “boosted tree”) is an ensemble supervised learning method that builds an additive model of decision trees in a “greedy” stage-wise process estimated by gradient descent.¹⁶ “Boosting” refers to the combination of weak prediction trees to produce a stronger prediction. We constructed a GBDT model with the xgboost package in R to predict state-level RSMR using the variables identified above to associate with mortality.¹⁷ We used a maximum depth of 6 and a gamma of 0. We used 580 variables as estimators during training, and 100 variables during testing. The 2010 data was used for training with 5-fold primary validation. The 2011 data was used for secondary validation (eFigure 4). To determine the degree of each variable's influence in the boosted tree model, we aggregated the reduction in error resulting from use of that variable as the splitting variable over all trees.

Evaluation of the two models

The goodness of fit of the two models was examined using Harrell's C-statistic, the R-square, and the area under the receiver operator characteristic curve (AUROC).

Construction of national RSMR maps

Hospital-level RSMRs were analyzed for 45 states. Data for five states (Idaho, Alabama, Hawaii, New Hampshire, and Delaware) was unavailable or incomplete in 2010–2011. States were classified by their Bureau of Economic Analysis regions (eTable 4) as the US Census administrative divisions were not adequately descriptive in name and did not adequately describe the “sepsis belt.” Maps were plotted using our in-house dashboard.¹⁸ For each state, inter-model RSMR differences were calculated by subtracting RSMRs of the SRS model from RSMRs of the boosted tree model (eFigure 3).

Hospital-level comparisons

The range of crude mortality rate and RSMR were visualized with box plots. Hospitals were divided into deciles by RSMR to examine the range in RSMR. Hospitals were divided into quintiles by annual number of sepsis admissions to examine the range in hospital volume, visualized by histograms. Differences in RMSR and sepsis admission counts across quintiles of sepsis admissions were tested using the Kruskal–Wallis test.

Results

Study population and encounter characteristics

Among 1,739,033 inpatients with sepsis identified in the NIS (eFigure 1), 15% were African-American, 8% were Hispanic, and 77% were White or Other; 50% were female (Table 1). Mortality increased with age, with 80–89-year-olds carrying an aOR for mortality of 4.17 (confidence interval (CI): 3.95–4.40) compared to <40-year-olds (Table 1). Mortality was slightly higher in women than men (aOR 1.07; CI: 1.05–1.09), and in African-American patients than White patients (aOR 1.05; CI: 1.02–1.08). Mortality was higher in patients requiring early mechanical ventilation (aOR 4.63; CI: 4.53–4.75) and late mechanical ventilation (aOR 5.72; CI: 5.43–6.02) than patients not requiring ventilation. Mortality was also higher in patients who had shock (aOR 2.06; CI: 2.02–2.10) or who required hemodialysis (aOR 1.43; CI: 1.39–1.48).

Two models of RSMR

Using the variables above, we constructed a reference model for predicted mortality rate using SRS logistic regression, and an alternative model for predicted mortality rate using boosted trees, for estimating 30-day RSMR of sepsis. In this boosted tree model, the variable with the highest influence on prediction was the need for intubation and ventilation, with a weight of 0.24 (eFigure 4), followed by age, respiratory failure, cardiac arrest and ventricular fibrillation, and shock (eTable 3).

After risk adjustment, the range of the hospital-level mortality narrowed from 0–100% to 4.7–58.2% (eFigure 6). The median RSMR for the top and bottom decile of hospitals by RSMR was 13.5% and 25.8%, respectively (eFigure 7). In the SRS model, median RSMR was 18.5% (IQR: 17.1–20.6%, representing a 3.4% difference between the 25th and 75th percentiles; Table 2). In the boosted tree model, median RSMR was 18.4% (IQR 17.0–21.0%, representing a 4.0% difference). RMSRs higher than 18.5% and 18.4% in the respective models indicate that the mortality from sepsis is higher than expected after risk adjustment.

Table 2.

Assessment of model fit for the sepsis risk prediction score (SRS) model and the boosted tree model in predicting median 30-day risk-standardized mortality rate (RSMR) of sepsis among inpatients hospitalized with sepsis in 2011.

	SRS	Boosted tree
Median 30-day RSMR (25th–75th %ile), %	18.46 (17.13–20.57)	18.39 (16.99–21.01)
Mean 30-day RSMR (SD), %	18.84 (3.20)	19.04 (3.86)
C-statistic	0.78	0.87
R-square	0.14	0.25
AUPRC	0.47	0.62
PPV^a	0.60	0.69

Abbreviations: AUPRC: area under the precision-recall curve; PPV: positive predictive value; %ile: percentile; SD: standard deviation.

Under a predefined cutoff of 0.5.

Goodness of fit of RSMR models

The boosted tree model demonstrated better calibration than the SRS model (eFigure 8). The boosted tree model also had better discrimination (AUROC: 0.87 vs 0.78; AUPRC: 0.62 vs 0.47; Table 2 and eFigure 2) and higher positive predictive value (0.69 vs 0.60) than the SRS model. Having established its improved calibration and discrimination compared to the reference SRS model, RSMR estimates in the Discussion will refer to those done by the boosted tree method, unless otherwise specified.

Table 3.

State-level median 30-day risk-standardized mortality rate (RSMR) of sepsis predicted by the Sepsis Risk Prediction Score (SRS) model (left) and the boosted tree model (right) among inpatients hospitalized with sepsis in 2011, ranked in order of decreasing RSMR.

	Sepsis Risk Prediction Score model			Boosted tree model
Rank	State	Median 30-day RSMR (25th–75th %ile)	Region	State	Median 30-day RSMR (25th-75th %ile)	Region
1	WV	23.35 (21.22–26.19)	Southeast	WY	23.19 (20.33–24.87)	Mountain
2	WY	21.97 (20.33–23.00)	Mountain	ND	20.45 (18.48–22.42)	Plains
3	MA	20.94 (18.47–24.27)	New England	MS	20.43 (18.32–22.09)	Southeast
4	OH	20.69 (17.76–22.44)	Great Lakes	RI	20.13 (18.25–22.33)	New England
5	ND	20.44 (19.80–21.07)	Plains	ME	20.11 (15.36–21.55)	New England
6	VA	20.09 (18.01–22.68)	Southeast	MA	20.08 (17.95–21.47)	New England
7	CT	19.61 (15.78–21.51)	New England	MT	20.08 (17.94–21.23)	Mountain
8	AR	19.52 (18.37–21.49)	Southeast	WV	20.00 (18.71–22.76)	Southeast
9	NY	19.49 (17.56–21.09)	Mideast	AR	19.96 (18.28–23.41)	Southeast
10	MS	19.47 (18.02–21.59)	Southeast	KY	19.94 (17.04–23.18)	Southeast
11	AK	19.37 (18.36–20.38)	Far West	MD	19.89 (18.68–21.48)	Mideast
12	GA	19.28 (17.68–20.92)	Southeast	AK	19.80 (19.17–20.44)	Far West
13	RI	19.25 (17.29–21.80)	New England	NJ	19.79 (18.16–22.92)	Mideast
14	MT	19.24 (18.63–20.07)	Mountain	NV	19.71 (18.00–21.10)	Far West
15	UT	19.17 (17.51–22.55)	Mountain	GA	19.69 (17.28–21.20)	Southeast
16	MD	19.12 (16.35–20.18)	Mideast	VT	19.57 (18.93–19.92)	New England
17	NV	19.09 (17.39–20.08)	Far West	NY	19.50 (17.27–21.67)	Mideast
18	ME	19.07 (16.37–20.20)	New England	IN	19.44 (16.70–22.41)	Great Lakes
19	TN	19.04 (17.80–21.88)	Southeast	OH	18.93 (16.68–21.05)	Great Lakes
20	MN	18.95 (18.02–20.46)	Plains	KS	18.91 (17.61–22.39)	Plains
21	OK	18.94 (17.98–22.55)	Southwest	MN	18.83 (18.04–21.52)	Plains
22	KY	18.87 (17.11–21.24)	Southeast	OK	18.75 (17.92–20.50)	Southwest
23	IN	18.84 (16.99–20.09)	Plains	UT	18.72 (17.34–20.46)	Mountain
24	NC	18.77 (17.21–20.03)	Southeast	PA	18.68 (16.61–20.56)	Mideast
25	NJ	18.75 (17.14–21.72)	Mideast	CT	18.68 (15.85–21.71)	New England
26	VT	18.62 (18.51–19.22)	New England	LA	18.66 (16.87–21.08)	Southeast
27	NM	18.52 (18.36–22.24)	Southwest	TN	18.59 (17.06–22.68)	Southeast
28	KS	18.46 (17.75–20.82)	Plains	OR	18.44 (16.83–19.42)	Far West
29	CA	18.39 (16.74–20.75)	Far West	MO	18.42 (17.29–21.39)	Plains
30	OR	18.37 (17.31–18.98)	Far West	VA	18.30 (17.54–22.04)	Southeast
31	MO	18.30 (17.09–21.02)	Plains	FL	18.12 (15.32–20.71)	Southeast
32	PA	18.23 (16.00–20.85)	Mideast	SD	18.11 (15.64–21.01)	Plains
33	LA	18.23 (16.30–19.86)	Southeast	IA	18.11 (17.38–19.17)	Plains
34	NE	18.21 (18.13–18.31)	Plains	TX	18.10 (16.54–20.44)	Southwest
35	FL	18.19 (15.46–20.59)	Southeast	NE	18.10 (17.93–18.34)	Plains
36	SD	18.19 (15.90–19.51)	Plains	CA	18.04 (16.04–20.09)	Far West
37	IL	18.09 (16.68–19.84)	Great Lakes	NM	18.00 (16.96–22.02)	Southwest
38	IA	18.07 (17.82–18.97)	Plains	NC	17.89 (16.71–20.52)	Southeast
39	SC	18.02 (16.51–19.15)	Southeast	SC	17.88 (17.09–19.77)	Southeast
40	WA	18.01 (16.77–19.02)	Far West	WA	17.86 (15.89–19.17)	Far West
41	TX	17.98 (16.15–20.13)	Southwest	WI	17.86 (13.91–19.34)	Great Lakes
42	WI	17.97 (15.25–19.55)	Great Lakes	IL	17.66 (16.90–19.94)	Plains
43	MI	17.70 (15.52–18.80)	Great Lakes	MI	17.54 (15.79–18.48)	Great Lakes
44	CO	17.49 (14.26–18.44)	Mountain	CO	17.32 (13.05–18.58)	Mountain
45	AZ	14.41 (11.45–17.82)	Southwest	AZ	14.29 (10.84–17.36)	Southwest

Geographic variation in risk-standardized all-cause mortality by state

Next, we calculated median 30-day RSMR at the state level using both the SRS model and the boosted tree model (Table 3). State-level RSMRs ranged from 14% to 23% in both models. The SRS model found clusters of high RSMR states in the Southeast (West Virginia, Virginia, Arkansas, Mississippi), New England (Massachusetts, Connecticut), and individual states such as Wyoming (Mountain), Ohio (Great Lakes), North Dakota (Plains), and Alaska (Far West). (Figure 1(a)). The boosted tree found clusters of high RSMR states in the northern Mountain states of Wyoming and Montana, the Southeast (Mississippi, West Virginia, Arkansas, Kentucky), and New England (Rhode Island, Maine, and Massachusetts), and individual states such as North Dakota (Plains) and Alaska (Far West) (Figure 1(b)). West Virginia, Massachusetts, Wyoming, Ohio, North Dakota, Connecticut, Arkansas, New York, Missouri, Alaska, Georgia, Rhode Island, Montana, Utah, Maryland, Nevada, Maine, Tennessee, Minnesota, Oklahoma, Kentucky, Indiana, New Jersey, and Vermont were above the median national RSMR in both models. The lowest RSMR states were identical in both models: Arizona (Southwest), Colorado (Mountain), and Michigan (Great Lakes). Differences between crude mortality and RSMR for each state are shown in Table 4.

Fig. 1.

State-level median 30-day risk-standardized mortality rate (RSMR) of sepsis predicted by the sepsis risk prediction score (SRS) model (top) and boosted tree model (bottom) among inpatients hospitalized with sepsis in 2011, color coded by sextile. Note that data for Idaho, Alabama, Hawaii, New Hampshire, and Delaware was unavailable or incomplete in 2010–2011 and are thus colored in white.

Table 4.

Difference between crude and risk-standardized 30-day all-cause mortality for each state in 2011 using the boosted tree model.

Rank	State (Abbreviation)	Median 30-day RSMR (25th–75th %ile) %	Median 30-day crude all-cause mortality (25th–75th %ile) %	Relative difference, %
1	WV	23.19 (20.33–24.87)	17.68 (14.66–22.92)	11.57
2	WY	20.45 (18.48–22.42)	30.77 (20.00–45.45)	−32.71
3	MA	20.43 (18.32–22.09)	25.81 (21.89–30.05)	−28.56
4	OH	20.13 (18.25–22.33)	15.74 (12.44–21.16)	16.84
5	ND	20.11 (15.36–21.55)	36.06 (22.12–50.00)	−76.34
6	VA	20.08 (17.95–21.47)	20.71 (16.71–22.75)	−13.16
7	CT	20.08 (17.94–21.23)	18.12 (14.68–23.79)	3.00
8	AR	20.00 (18.71–22.76)	18.69 (14.42–33.33)	6.35
9	NY	19.96 (18.28–23.41)	23.27 (18.83–29.48)	−19.34
10	MS	19.94 (17.04–23.18)	19.46 (15.69–26.23)	4.75
11	AK	19.89 (18.68–21.48)	30.00 (10.00–50.00)	−51.51
12	GA	19.80 (19.17–20.44)	18.18 (14.89–22.78)	7.68
13	RI	19.79 (18.16–22.92)	25.83 (22.51–27.63)	−28.30
14	MT	19.71 (18.00–21.10)	17.78 (14.29–50.00)	11.45
15	UT	19.69 (17.28–21.20)	16.02 (10.00–20.00)	14.43
16	MD	19.57 (18.93–19.92)	19.49 (16.12–22.27)	2.01
17	NV	19.50 (17.27–21.67)	25.00 (18.73–27.18)	−26.83
18	ME	19.44 (16.70–22.41)	21.54 (9.09–33.33)	−7.09
19	TN	18.93 (16.68–21.05)	18.38 (13.49–24.14)	1.15
20	MN	18.91 (17.61–22.39)	16.09 (8.33–25.00)	14.53
21	OK	18.83 (18.04–21.52)	14.29 (0.00–20.00)	23.79
22	KY	18.75 (17.92–20.50)	20.00 (14.29–30.32)	−0.31
23	IN	18.72 (17.34–20.46)	17.97 (12.82–23.08)	7.56
24	NC	18.68 (16.61–20.56)	18.13 (12.09–20.00)	−1.31
25	NJ	18.68 (15.85–21.71)	24.28 (21.00–27.99)	−22.71
26	VT	18.66 (16.87–21.08)	21.01 (15.48–29.34)	−7.33
27	NM	18.59 (17.06–22.68)	21.90 (15.97–24.26)	−21.71
28	KS	18.44 (16.83–19.42)	16.33 (4.69–34.17)	13.61
29	CA	18.42 (17.29–21.39)	19.10 (15.79–24.04)	−5.88
30	OR	18.30 (17.54–22.04)	16.33 (9.09–18.88)	11.41
31	MO	18.12 (15.32–20.71)	16.97 (8.70–24.84)	7.90
32	PA	18.11 (15.64–21.01)	18.03 (14.29–23.81)	3.47
33	LA	18.11 (17.38–19.17)	18.49 (15.56–24.09)	0.91
34	NE	18.10 (16.54–20.44)	0.00 (0.00–11.11)	100.00
35	FL	18.10 (17.93–18.34)	18.04 (15.27–22.99)	0.46
36	SD	18.04 (16.04–20.09)	8.87 (0.00–25.00)	51.04
37	IL	18.00 (16.96–22.02)	17.46 (12.5–22.35)	1.11
38	IA	17.89 (16.71–20.52)	8.86 (0.00–16.67)	51.10
39	SC	17.88 (17.09–19.77)	18.30 (13.39–20.00)	−2.38
40	WA	17.86 (15.89–19.17)	15.05 (0.00–16.92)	15.77
41	TX	17.86 (13.91–19.34)	17.35 (11.43–22.12)	4.13
42	WI	17.66 (16.90–19.94)	12.50 (8.70–16.67)	30.00
43	MI	17.54 (15.79–18.48)	12.50 (9.52–20.57)	28.74
44	CO	17.32 (13.05–18.58)	15.38 (7.28–20.58)	11.16
45	AZ	14.29 (10.84–17.36)	11.18 (7.25–16.58)	21.82

Hospital volume and risk-standardized all-cause mortality

To determine if RSMR was affected by hospital volume, we divided hospitals into quintiles by their number of sepsis admissions per year. Hospitals in the lowest and highest quintiles had a median of 3 (IQR: 1–4) and 547 (IQR: 547) sepsis admissions per year, respectively (eFigure 7). The median 30-day RSMR differed significantly across the 5 quintiles (p < 0.001), but this association was not linear (eTable 2, eFigure 5).

Discussion

In this study of 1,739,033 inpatients with sepsis drawn from a nationally-representative database, we constructed the first state-level estimates of risk-standardized all-cause mortality rates for sepsis patients in the United States. We employed a traditional logistic SRS model as the reference and constructed a supervised ML model to increase the accuracy of risk prediction, the latter of which demonstrated higher model discrimination and calibration. We further built an online dashboard to illustrate time-dependent RSMR changes by state.¹⁸ We found significant inter-state variation in RSMR, suggesting that quality of sepsis care is not uniform across the country. These risk-standardized findings suggest that there is opportunity to improve sepsis-related survival in many states, and can inform policymaking.

Because sepsis outcomes are highly dependent on patient characteristics and comorbidities, it is important to develop accurate estimates of mortality risk with which to standardize observed mortality, in order to differentiate between whether differences in sepsis-related mortality are due to differences in patient distributions or differences in quality of care. Investment to improve sepsis care should be prioritized in states such as Wyoming, North Dakota, Mississippi, Rhode Island, Maine, Massachusetts, and Montana, which are in the top sextile of RSMR. At the extremes, Wyoming had an RMSR of 23.2% while Arizona had an RSMR of 14.3%, which were 4.8% above and 4.1% below the national median RSMR, respectively. Arizona, Colorado, Michigan, Illinois, Wisconsin, Washington, and South Carolina, in the bottom sextile of RSMR, should be studied to identify success factors and processes that can be replicated at hospitals in other states.

Our work geographically refines RSMR estimates by Hatfield,⁸ who coded facilities at the four-region level (e.g. Midwest, Northeast, Southwest, and West), and consequently does not identify individual high-performing and low-performing states. Wang's¹⁹ state-level analysis of mortality was age-standardized but not risk-standardized to comorbidities or in-hospital factors, and identified a cluster of states in the Southeast and Mid-Atlantic (Arkansas, Louisiana, Mississippi, Alabama, Georgia, Tennessee, North Carolina, Virginia, Maryland, Delaware, and New Jersey) with the highest age-standardized sepsis mortality (ASMR). These states were all found to have RSMR above the national median in our analysis (except Alabama for which we had no data). Moore's²⁰ county-level analysis of ASMR found three hotspots with the highest ASMR: Central Appalachia (Kentucky, Tennessee, Virginia), Middle Georgia, and Mississippi Valley (Mississippi, Arkansas, Louisiana). These hotspots, termed the “sepsis mortality belt,”²¹ were also identified in our analysis as areas of high RSMR. We additionally identified individual states in the Northwest, including Montana, Wyoming, and North Dakota, with high RSMR that were not identified in Wang's or Moore's ASMR analyses.

The variation of care quality between states and hospitals is an important issue for policymakers to address for reducing health disparities. The advent of standardized measurement, required public reporting, Joint Commission accreditation requirements, Medicare payment incentives, and the threat of Medicare payment penalties, have dramatically improved mortality in AMI, HF, and PNA.²² Specific hospital-based initiatives have been shown to be associated with lower AMI RSMR²³ There is increasing support for reporting HF readmission rates in the context of RSMR.²⁴ There is reason to be optimistic that focusing RSMR as the outcome indicator of sepsis care quality will improve patient health. In New York, the implementation of mandatory hospital reporting of adherence to sepsis protocol initiation and completion of 3 hour and 6 hour sepsis bundle requirements led to increased compliance with these metrics, and reduced RSMR from 28.8% to 24.4%.²⁵ More rapid completion of the 3 hour sepsis bundle was associated with reduced adjusted mortality.²⁶ Besides improving the accessibility and the quality of health care, policymakers should also pay attention to other factors that will affect the regional disparity of sepsis outcome, such as the community poverty rate.²⁷ Moore et al. showed that counties in Sepsis Cluster or Sepsis Belt regions had larger numbers of physicians per 100,000 persons, suggesting the existence of financial, social, cultural barriers for those people to access health care.²⁷

Although speculation about reasons for poor sepsis outcomes in specific states is beyond the scope of this study, higher poverty rate, larger uninsured population, lower education, and higher unemployment have been suggested as factors explaining high age-adjusted sepsis mortality rate in poor-performing counties.²⁰ Uninsured critically-ill patients are less likely to receive critical care procedures and post-acute care compared to insured critically ill patients, and thus have higher mortality rates.²⁸ Genetics and environmental exposures, not included in our risk standardization, may alter sepsis outcomes.^29,30 Although protocoled resuscitation strategies have improved mortality in sepsis,^3,26,31–33 differential adherence to these protocols may account for variation in RSMR. Our finding that RMSR differed across quintiles of hospital volume should not necessarily be interpreted as higher volumes corresponding to improved RSMR, because the relationship was not monotonously decreasing. Some groups have shown that hospitals with higher case volumes had improved adjusted sepsis survival,³⁴ but regional inter-hospital transfer of sepsis patients from low to high sepsis case volume hospitals has not improved outcomes, possibly due to delays in treatment and adverse events from care transitions.³⁵ Meanwhile, interventions focusing on improving coordination of existing hospital responses to suspected severe sepsis cases have shown promise.³⁶ Implementation of evidence-based sepsis processes of care was more likely at hospitals with higher sepsis case volumes, but did not strongly mediate case volume-mortality associations.³⁷

Our results should be interpreted in light of several limitations. First, the NIS does not allow data linkage between hospitals, rendering information about subsequent mortality unavailable for transferred patients. The NIS did not provide state-specific data after 2011, limiting our ability to provide current estimates. Nevertheless, our study features the largest sample size to date in estimating sepsis RSMR. Second, because the data was administrative, variation in coding practices among hospitals may limit accuracy. Moreover, we did not employ advanced deep learning method in both the recognition of sepsis and the prediction of mortality as described in previous studies.^38–42 However, standardized clinical criteria has been shown to have good reliability for identifying sepsis in the EMR^43,44 and trade-off between data complexity and model interpretation also exists in deep learning algorithm.^45,46 Third, in-hospital mortality could be biased by hospital discharge practice and length of hospital stay,⁴⁷ and may not necessarily reflect the quality of care.^48,49 In what has been termed the “early post-discharge mortality” phenomenon, 50% of older patients who died from PNA within 30 days of admission did not die in hospital but after discharge.⁵⁰ This can be addressed by examining a different mortality time frame or examining the risk adjusted readmission rate,⁴ which are areas of future investigation in our group. Fourth, although we utilized a robust risk-adjustment model, we cannot assure that differences in RSMRs between states are purely owing to quality differences, as we cannot identify, for example, patients who are admitted expressly for hospice care. Our estimates are strengthened by the use of complementary models to estimate predicted sepsis mortality. Compared to generalized logistic regression, boosted trees have more freedom because they do not assume any a priori hypothesis about the feature-prediction relationship; instead, they deduce that relationship from the data. To make our final result simpler and interpretable for policy makers, we have made our tool publicly available at https://sepsismap.shinyapps.io/index2/.¹⁸

Conclusion

In conclusion, high RSMRs persist for sepsis. The highest RSMRs were found in Wyoming, North Dakota, and Mississippi, while the lowest RSMRs were found in Arizona, Colorado, and Michigan. To our knowledge, this is the first description of state-level variation in RSMR of sepsis patients. Most importantly, we constructed a national map of sepsis RSMR, using a dashboard which we are making freely available to any investigator or clinician. We hope that identification of these high-RMSR hotspots will facilitate investment and innovation in sepsis mortality-reduction strategies, improving sepsis mortality and healthcare expenditures.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076211072400 - Supplemental material for Risk-standardized sepsis mortality map of the United States

Supplemental material, sj-docx-1-dhj-10.1177_20552076211072400 for Risk-standardized sepsis mortality map of the United States by Jiun-Ruey Hu, Chia-Hung Yo, Hsin-Ying Lee, Chin-Hua Su, Ming-Yang Su, Amy Huaishiuan Huang, Ye Liu, Wan-Ting Hsu, Matthew Lee, Yee-Chun Chen and Chien-Chang Lee in Digital Health

Footnotes

Acknowledgments

We thank the staff of the Core Labs, the Department of Medical Research, and National Taiwan University Hospital for technical support, and Medical Wizdom, LLC, USA for technical assistance in statistical analysis. We also thank Shih-Chuan Chou, MD, MPH at the Department of Emergency Medicine, Brigham and Women's Hospital, Boston, MA, USA for reviewing the article and his insightful comments.

Contributorship

C-CL has full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis, concept and design, critical revision of the manuscript for important intellectual content, obtaining funding and supervision. J-RH and C-HY were responsible for drafting the manuscript, interpretation of the data, and critical revision of the manuscript for important intellectual content. H-YL was responsible for drafting the manuscript and interpretation of the data. C-HS was responsible for statistical analysis. Y-CC, ML, MYS, AHH, YL were responsible for critical revision of the manuscript for important intellectual content. W-TH was responsible for drafting the manuscript, statistical analysis, interpretation of the data and critical revision of the manuscript. All authors read and approved the final manuscript.

Conflict of interest

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Not applicable, because this article does not contain any studies with human or animal subjects.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work were supported by the Taiwan Ministry of Science and Technology (Grant 105-2811-B-002-031) and Far Eastern Memorial Hospital, Taiwan (Grant FEMH-2021-C-031).

Supplemental material

Supplemental material for this article is available online.

ORCID iD

Chien-Chang Lee

References

Levy

Evans

Rhodes

. The surviving sepsis campaign bundle: 2018 update. Intensive Care Med 2018; 44: 925–928.

Gallagher

Blackwell

Thomas

, et al. Successful prospective quality improvement programme for the identification and management of patients at risk of sepsis in hospital. BMJ Open Qual 2019; 8: e000369.

Gatewood

Wemple

Greco

, et al. A quality improvement project to improve early sepsis care in the emergency department. BMJ Qual Saf 2015; 24: 787–795.

Suter

S-X

Grady

, et al. National patterns of risk-standardized mortality and readmission after hospitalization for acute myocardial infarction, heart failure, and pneumonia: update on publicly reported outcomes measures based on the 2013 release. J Gen Intern Med 2014; 29: 1333–1340.

Bernheim

Grady

Lin

, et al. National patterns of risk-standardized mortality and readmission for acute myocardial infarction and heart failure: update on publicly reported outcomes measures based on the 2010 release. Circ Cardiovasc Qual Outcomes 2010; 3: 459–467.

Krumholz

Merrill

Schone

, et al. Patterns of hospital performance in acute myocardial infarction and heart failure 30-Day mortality and readmission. Circ Cardiovasc Qual Outcomes 2009; 2: 407–413.

Lindenauer

Bernheim

Grady

, et al. The performance of US hospitals as reflected in risk-standardized 30-day mortality and readmission rates for medicare beneficiaries with pneumonia. J Hosp Med 2010; 5: E12–E18.

Hatfield

Dantes

Baggs

, et al. Assessing variability in hospital-level mortality among U.S. medicare beneficiaries With hospitalizations for severe sepsis and septic shock*. Crit Care Med 2018; 46: 1753–1760.

Wang

Donnelly

Shapiro

, et al. Hospital variations in severe sepsis mortality. Am J Med Qual 2015; 30: 328–336.

10.

Darby

Davis

Barbash

, et al. An administrative model for benchmarking hospitals on their 30-day sepsis mortality. BMC Health Serv Res 2019; 19: 21.

11.

Walkey

Shieh

M-S

Liu

, et al. Mortality measures to profile hospital performance for patients with septic shock*. Crit Care Med 2018; 46: 1247–1254.

12.

Singer

Deutschman

Seymour

, et al. The third international consensus definitions for sepsis and septic shock (sepsis-3). JAMA 2016; 315: 01.

13.

Martin

Moss

. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med 2003; 9: 1546–1554.

14.

Gagne

Glynn

Avorn

, et al. A combined comorbidity score predicted mortality in elderly patients better than existing scores. J Clin Epidemiol 2011; 64: 749–759.

15.

Ford

Goodwin

Simpson

, et al. A severe sepsis mortality prediction model and score for Use With administrative data. Crit Care Med 2016; 44: 319–327.

16.

Friedman

. Greedy function approximation: a gradient boosting machine. Ann Statist 1999; 29): 1189–1232.

17.

Chen

Guestrin

. XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’16 2016:785–94. DOI: 10.1145/2939672.2939785.

18.

Lee

M-T

Lin

F-C

Chen

S-T

, et al. Web-based dashboard for the interactive visualization and analysis of national risk-standardized mortality rates of sepsis in the US. J Med Syst 2020; 44: 54.

19.

Wang

Devereaux

Yealy

, et al. National variation in United States sepsis mortality: a descriptive study. Int J Health Geogr 2010; 9: 9.

20.

Moore

Donnelly

Griffin

, et al. Defining sepsis mortality clusters in the United States. Crit Care Med 2016; 44: 1380–1387.

21.

Kopterides

Mayr

Yende

. Understanding the sepsis mortality belt: time to buckle down!. Ann Transl Med 2016; 4: 319. DOI: 10.21037/atm.2016.08.01

22.

Chassin

Loeb

Schmaltz

, et al. Accountability measures — using measurement to promote quality improvement. N Engl J Med 2010; 363: 683–688.

23.

Bradley

. Hospital strategies for reducing risk-standardized mortality rates in acute myocardial infarction. Ann Intern Med 2012; 156: 18.

24.

Psotka

Fonarow

Allen

, et al. The hospital readmissions reduction program. JACC: Heart Fail 2020; 8: 1–11.

25.

Levy

Gesten

Phillips

, et al. Mortality changes associated with mandated public reporting for sepsis. The results of the New York state initiative. Am J Respir Crit Care Med 2018; 198: 1406–1412.

26.

Seymour

Gesten

Prescott

, et al. Time to treatment and mortality during mandated emergency care for sepsis. N Engl J Med 2017; 376: 2235–2244.

27.

Moore

Donnelly

Griffin

, et al. Community characteristics and regional variations in sepsis. Int J Epidemiol 2017; 46: 1607–1617.

28.

Lane-Fall

Iwashyna

Cooke

, et al. Insurance and racial differences in long-term acute care utilization after critical illness*. Read Online: Crit Care Med | Soc Crit Care Med 2012; 40: 1143–1149.

29.

Sarmiento

Moore

McClure

, et al. Fine particulate matter pollution and risk of community-acquired sepsis. Int J Environ Res Public Health 2018; 15: 818. DOI: 10.3390/ijerph15040818

30.

Scherag

Schöneweck

Kesselmeier

, et al. Genetic factors of the disease course after sepsis: a genome-wide study for 28 Day mortality. EBioMedicine 2016; 12: 239–246.

31.

Emanuel

Bryant

Suzanne

, et al. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med 2001; 10: 1368–1377.

32.

Liu

Fielding-Singh

Greene

, et al. The timing of early antibiotics and hospital mortality in sepsis. Am J Respir Crit Care Med 2017; 196: 856–863.

33.

Miller

Dong

Nelson

, et al. Multicenter implementation of a severe sepsis and septic shock treatment bundle. Am J Respir Crit Care Med 2013; 188: 77–82.

34.

Gaieski

Edwards

Kallan

, et al. The relationship between hospital volume and mortality in severe sepsis. Am J Respir Crit Care Med 2014; 190: 665–674.

35.

Rush

Tyler

Stone

, et al. Outcomes of ventilated patients with sepsis who undergo inter-hospital transfer: a nationwide linked analysis. Crit Care Med 2018; 46: e81–e86.

36.

Rosenqvist

Bengtsson-Toni

Tham

, et al. Improved outcomes after regional implementation of sepsis alert: a novel triage model*. Read Online: Crit Care Med | Soc Crit Care Med 2020; 48: 484–490.

37.

Fawzy

Walkey

. Association between hospital case volume of sepsis, adherence to evidence-based processes of care and patient outcomes. Crit Care Med 2017; 45: 980–988.

38.

Nesaragi

Patidar

Thangaraj

. A correlation matrix-based tensor decomposition method for early prediction of sepsis from clinical data. Biocybern Biomed Eng 2021; 41: 1013–1024.

39.

Nesaragi

Patidar

Aggarwal

. Tensor learning of pointwise mutual information from EHR data for early prediction of sepsis. Comput Biol Med 2021; 134: 104430.

40.

Nesaragi

Patidar

. An explainable machine learning model for early prediction of sepsis using ICU data. In: Neri

Huang

(eds) Infections and sepsis development. London: IntechOpen, 2021; DOI: 10.5772/intechopen.98957.

41.

Nesaragi

Patidar

. Early prediction of sepsis from clinical data using ratio and power-based features. Crit Care Med 2020; 48: e1343–9.

42.

Anda Du

Sadr

de Chazal

. Automated prediction of sepsis onset using gradient boosted decision trees, 2019. https://doi.org/10.22489/CinC.2019.423.

43.

Iwashyna

Odden

Rohde

, et al. Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis. Med Care 2014; 52: e39–e43.

44.

Rhee

Dantes

Epstein

, et al. Incidence and trends of sepsis in US hospitals using clinical vs claims data, 2009-2014. JAMA 2017; 318: 1241.

45.

Sanchez-Pinto

Luo

Churpek

. Big data and data science in critical care. Chest 2018; 154: 1239–1248.

46.

Beam

Kohane

. Big data and machine learning in health care. JAMA 2018; 319: 1317.

47.

Pouw

Peelen

Moons

KGM

, et al. Including post-discharge mortality in calculation of hospital standardised mortality ratios: retrospective analysis of hospital episode statistics. Br Med J 2013; 347: f5913–f5913.

48.

Pitches

Mohammed

Lilford

. What is the empirical evidence that hospitals with higher-risk adjusted mortality rates provide poorer quality care? A systematic review of the literature. BMC Health Serv Res 2007; 7: 91.

49.

Shahian

Wolf

Iezzoni

, et al. Variability in the measurement of hospital-wide mortality rates. N Engl J Med 2010; 363: 2530–2539.

50.

Metersky

Waterer

Nsa

, et al. Predictors of in-hospital vs postdischarge mortality in pneumonia. Chest 2012; 142: 476–481.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.64 MB