Abstract
Background and Objectives:
Colon surgical site infections (SSIs) impose clinical and financial burden. To guide reimbursement, the Centers for Medicare and Medicaid Services utilize the National Healthcare Safety Network’s (NSHN) Complex 30-Day SSI Model for Colon Procedures, which is used to compare the observed number of SSIs to the expected number of SSIs for all colon surgeries, regardless of underlying etiology. This study assessed the predictive accuracy of the Complex 30-day SSI Model for inpatients undergoing emergent colon operations for trauma.
Patients and Methods:
The Trauma Quality Program Participant Use File was used to identify all patients who underwent emergent colon surgery because of trauma between 2017 and 2021 (excluding 2020). The predicted probability of colon SSI was calculated using the NSHN Complex 30-day Model using the following variables: diabetes mellitus, gender, age, body mass index, closure technique, American Society of Anesthesiology Score, and oncology hospital status. Receiver operating characteristic (ROC) curves were generated, and the area under the curve (AUC) was calculated to evaluate the model’s predictive accuracy.
Results:
In total, 292,688 patients were included. Among all patients, an SSI occurred in 0.33% (n = 1,097). For patients with complete data, the rate of SSI was 0.34%. The ROC curve for complete case analysis showed an AUC of 0.523 (95% confidence interval 0.506–0.541).
Conclusion:
The predictive accuracy of the NSHN Complex 30-Day SSI Model after colon injury is poor. The current model may not adequately capture the complexities and risk associated with trauma. An improved model is needed for more accurate risk assessment to improve patient care and accuracy of hospital reimbursement.
Introduction
Surgical site infection (SSI) can cause devastating consequences for patients and the healthcare system. The U.S. Health and Human Services Agency for Healthcare Research and Quality considered healthcare-associated infection (HAI) the most common serious complication in 2008.1,2 Of the various types of HAI, SSI is the most common. A large retrospective study using the National Inpatient Sample, which consisted of over 35 million patients admitted to the hospital in 2016, found that 1.8% of all admissions were related to hospital acquired infection. Of these, 47% were SSI. 3 In 2016 alone, SSI was responsible for $3.7 billion dollars in U.S. healthcare costs; however, when considering estimated additional costs associated with SSI, this increases to $7.2–$14.9 billion dollars. 4
In addition to the potential harm to the patient and the staggering cost to the national healthcare system, SSI reporting has significant implications for hospital management. The Hospital Inpatient Quality Reporting Program (HIQRP) was originally mandated in 2003 as part of the Medicare Prescription Drug, Improvement, and Modernization Act. 5 For receipt of full reimbursement by the Centers for Medicare and Medicaid Services (CMS), acute-care hospitals must report SSI outcomes as part of the Inpatient Prospective Payment System. The American Recovery and Reinvestment Act and the Affordable Care Act modified this policy as to reduce reimbursement by up to 25% if HIQRP requirements are not met. 5
The Centers for Disease Control and Prevention’s National Healthcare Safety Network (NHSN) is the reporting system used by CMS for hospitals to report and track data. HAIs are reported as a standardized infection ratio (SIR). The SIR is calculated by comparing the observed number of HAIs with the expected number of HAIs based on a predictive model. 6 Of these models, the Colon Procedure, Complex 30-Day Model is the model used for NHSN reporting for all colon surgeries. This study evaluates the accuracy of the Complex 30-Day SSI Model for Colon Procedures to predict SSI in the setting of traumatic colon surgery. We hypothesized that this model is a poor predictor of SSI and thus poses significant implications for quality reporting for hospitals treating trauma patients. A secondary aim of this study was to determine factors associated with an increased risk of SSI in trauma patients undergoing colon surgery.
Patients and Methods
Patient population and variables
The American College of Surgeons Trauma Quality Programs (TQP) Participant Use File (PUF) was utilized from 2017 to 2021 (excluding 2020) to obtain variables including demographics (age, gender, and race), injury type (blunt vs. penetrating), mechanism of injury, volume (mL) of packed red blood cells (PRBCs) received within 4 hours of emergency department (ED) or hospital arrival, and hospital (HLOS) and intensive care unit length of stay (ICU LOS). Age, PRBCs, and LOS variables are reported as mean ± standard deviation. TQP PUF was also queried to obtain variables associated with colon SSI prediction using the NSHN Complex 30-Day SSI Model. Predictive variables included diabetes mellitus (yes vs. no), gender (male vs. female), age (per 10-year increment), body mass index (BMI) (≥30 kg/m2 vs. <30 kg/m2), closure technique (non-primary vs. primary), American Society of Anesthesiologists Physical Status Classification System (ASA) score (ranging from 1 to 6), and oncology hospital status (yes vs. no).
Definitions
Because TQP data did not include closure technique, our institutional non-primary closure technique rate (0.265) was utilized for model prediction. ASA score, a variable also not collected in TQP, was designated as a “4” for the purposes of this investigation, which would infer severe trauma and/or shock in this population of patients requiring operative intervention for their injury. The designation of “Yes” for oncology hospital was chosen because of its association with a higher SSI rate according to the model.
Statistical analysis
For each patient, the predicted probability of an SSI (deep or organ/space) was calculated using the NSHN Complex 30-day model. Receiver operating characteristic (ROC) curves were then generated, and the area under the curve (AUC) was calculated to evaluate the model’s predictive accuracy. Initially, the data were analyzed using listwise depletion (complete case analysis). Subsequently, multiple imputation by fully conditional specification methods was employed to handle missing BMI and gender data. Furthermore, to assess the sensitivity of the results, closure technique was randomly assigned to each patient using a Bernoulli distribution, assuming a probability of non-primary closure of 0.265. Statistical analysis was done using SAS (SAS Institute Inc., Cary, NC, USA). The MIANALYZE procedure in SAS was utilized to combine AUC estimates from multiple imputations. Significance was defined as p <0.05.
Results
A total of 333,139 patients met the inclusion criteria with 292,688 patients (87.9%) having complete data (Table 1). Overall, the study population had a mean age of 36.4 (±15.0) years and 79.5% were male. White (44.4%) and Black (39.3%) patients were the predominant racial groups. Penetrating injury (59.7%) was more common than blunt traumatic injury (39.3%). Of those with a penetrating mechanism, a firearm was used in 170,117 patients (85.6%). Nearly one-third of the patients (31.5%) had a BMI >30 mg/kg2 and 4.0% of the patients had a diagnosis of diabetes mellitus. Among the total number of patients, SSI occurred in 0.33% (n = 1,097). For patients with complete data, the rate of colon SSI was 0.34%.
Study Population Summary and Demographics
BMI, body mass index; MCC, motorcycle crash; MVC, motor vehicle crash; SD, standard deviation.
When comparing those in the study population who developed a colon SSI with those who did not, characteristics associated with colon SSI include penetrating mechanism, use of a firearm, and BMI ≥30 kg/m2 (Table 2). There was no association in the development of a colon SSI with gender, race, diabetes mellitus diagnosis, or the volume of PRBCs transfused in the first 4 hours after ED/hospital arrival. Although the association with younger age and colon SSI was statistically significant, the differences in age were not clinically relevant (35.2 ± 13.1 yrs vs. 36.4 ± 15.0 yrs). Patients who developed a colon SSI had longer ICU LOS (17.0 ± 16.6 d vs. 8.2 ± 10.5 d; p <0.001) and HLOS (36.7 ± 28.4 d vs. 15.3 ± 16.0 d; p <0.001).
Variables Predictive of Surgical Site Infection
*p <0.05.
BMI, body mass index; SD, standard deviation.
In analyzing the ability of the Complex 30-Day SSI Model for Colon Procedures to predict SSI in the study population with complete data (292,688 patients), the predicted SSI rate was 9.9% in comparison with the observed SSI rate of 0.34%. The ROC curve for this analysis shows an AUC of 0.523 (95% confidence interval [CI] = 0.506–0.541; Fig. 1). Similar results for the total study population (n = 333,139) were obtained with imputation for non-primary closure technique (AUC = 0.524, 95% CI = 0.507–0.542) and multiple imputations for the missing gender and BMI data (mean AUC = 0.523; 95% CI = 0.506–0.539). Although the total number of SSI observed was low (0.33% of total cases), the accuracy of the analysis of predicting usefulness of the Complex 30-Day Model for Colon Procedures is unaffected by quantity of cases.

Receiver operating characteristic (ROC) curve.
Discussion
The results of this study suggest that the ability of the Complex 30-Day SSI Model for Colon Procedures to predict colon SSI in trauma patients is poor. Profoundly negative financial consequences may result with the use of a model having such inaccurate predictive ability. When a hospital has more observed colon SSIs than the number predicted by the Complex 30-Day SSI Model for Colon procedures, that hospital is at risk of having payments by CMS for Medicare beneficiaries reduced. 7 According to Definitive Healthcare, 775 hospitals received Hospital-Acquired Condition (HAC) Reduction Program penalties in 2021 with the top 25 revenue losses ranging from $1,254,963 to $4,043,719. For the 775 hospitals that faced HAC penalties in 2021, there was $254 million in estimated loss revenue with an average loss of $328,800. 8
In addition to the severe financial penalties that may be rendered because of the HAC Reduction Program, the information generated by the Complex 30-Day SSI Model for Colon Procedures is publicly reported on sites such as CMS’s Hospital Compare. 7 Quality indicators such as infection rates are reported in the form of star ratings and report cards. Healthcare consumers utilize this information to choose high-quality providers over low-quality providers. That is not to say that efforts to prevent SSI are not critically important financially and clinically.1,4,9 Previous studies have reported an increased hospital LOS and increased readmission rates secondary to SSI.1,10 Our study similarly found a staggering increase in ICU and hospital LOS, that being 1 and 2 weeks longer, respectively. Accurate risk prediction of SSI is paramount, however, given the potential financial and reputational impact on healthcare centers because of using imprecise models. 11
There are several reasons that may explain the poor predictive ability of the Complex 30-Day SSI Model for Colon Procedures in trauma patients. Although the model is risk adjusted based on variables found to be statistically significant predictors of SSI, those factors that are specific to trauma are excluded. 12 Trauma patients frequently present in extremis, with coagulopathy, and/or with concomitant injuries.13–15 One retrospective study reported factors highly prevalent in the trauma setting that significantly increased risk of SSI: penetrating mechanism, injury severity, lactic acidosis, coagulopathy, and damage control surgery. 16 Additionally, there is high potential for fecal contamination in destructive colonic injuries, which may require colostomy creation, thus increasing the risk for SSI.13,15,17,18 Penetrating injury and use of a firearm were statistically significant risk factors for SSI in this study, thereby demonstrating the increased likelihood of the high-risk factors described above. Although the volume of PRBC transfusion within 4 hours of ED/hospital arrival was not statistically significant in our analysis, other studies have found blood transfusion to be an independent risk factor for SSI.16,19,20
Trauma patients are substantially different than patients undergoing elective colon surgery, particularly since the widespread adoption of SSI prevention bundles and enhanced recovery after surgery protocols, resulting in a significant reduction in SSI for elective colon surgery patients.21,22 Patients undergoing elective surgery have the opportunity to adhere to evidence-based guidelines such as smoking cessation, glucose control, nutritional optimization, and bowel preparation. 23 Although proved to be effective in the elective setting, these measures are frequently not feasible in the emergent trauma setting. These factors raise the question of whether the rate of SSI in trauma colon surgeries should be judged similarly to colon procedures that occur in an elective setting with data-proved preventive measures such as preoperative bowel preparation.
With further investigation into specific risk factors, a more accurate prediction model may assist with the development of mitigating factors to prevent SSI in the trauma patient. For example, Wei et al. generated a new predictive model that includes age, gender, race, BMI, mechanism of injury (penetrating vs. blunt), first intra-operative base excess, first intra-operative temperature, units of PRBCs transfused, open small bowel injury, small bowel resection, open colon injury, colon resection, performance of damage control laparotomy, wound class, and surgery duration; this model had improved AUC of 0.78. 23 A more accurate predictive model would allow for better stratification of SSI risk in this highly prone population.11,24,25
The authors of this study initially hypothesized that the Complex 30-Day Model for Colon Procedures would underestimate the rate of SSI for the reasons described previously. The total number of SSI identified in this study was surprisingly low. This is likely because of a variety of reasons. The Complex 30-Day Model for Colon Procedures only includes deep incisional and organ space infections. These are the colon SSIs that are publicly reported by CMS. There are likely a larger number of superficial SSIs that were not accounted for in this study.
Another likely significant factor in the low number of observed infections is the exclusion of infection present at the time of surgical procedure (PATOS). Since 2017, PATOS SSI was excluded from the NHSN colon SSI SIR. However, there is currently no standardized definition for PATOS for trauma patients. There is currently debate as to how long fecal contamination must be present to be considered PATOS. Furthermore, there is room for interpretation in regard to the term “feculent peritonitis” when classifying a “dirty wound” versus “infected wound” at the time of surgical procedure. 6 There is a considerable amount of subjectivity in determining PATOS and thus omitting an SSI from reporting as a result.26,27 PATOS is not a variable that can be extracted from the TQP database, and thus, we were unable to control for this.
Lastly, the use of TQP PUF contributes to the low number of colon SSI infections realized in the current investigation. Unlike the National Surgical Quality Improvement Project, which surveys patients after discharge to capture SSIs, 28 TQP PUF is limited to data from the index hospitalization. The contributory impact of the absence of post-discharge information on the SSI rate in this study is unknown. Hübner et al. suggested in their study that the diagnosis of a colon SSI was established after discharge in 22.7% of their patients. 29
The authors recognize several limitations of this study. Primarily, the use of a large database poses limitations because of the accuracy of data reporting. Data imported into the TQP are highly dependent on manual coding and are thus subject to error. Specifically, in regard to TQP PUF, some elements used in the Complex 30-Day Model, including ASA classification and non-primary closure method, are not included in this database and had to be estimated. The exclusion of PATOS in this model is another limitation. The lack of consistency in reporting PATOS likely leads to error in SSI reporting and may bias the results. Additionally, a large database study limits the ability to draw cause and effect conclusions. Although there are strong associations with various factors, such as penetrating trauma, with SSI, we cannot definitively state that penetrating trauma causes SSI.
Although this study has known limitations, there are also many strengths of this investigation that should be noted. This study used a large sample size with nationally representative data specific to trauma patients. Furthermore, this study was able to redemonstrate the known independent risk factors proved to be associated with SSI in the trauma population. Despite the low SSI rate following traumatic colon injury revealed in this study, this rate does not affect the accuracy of the ROC curve analysis, which demonstrated an AUC of 0.523 and 0.524 for the study population with complete data and the total study population, respectively, suggesting poor discriminative ability. Our findings contribute to the sparse body of literature that exists, which discusses the issues of the current predictive model used for colon SSI in trauma patients. This study emphasizes the need for a separate model for SSI prediction to improve the equity of reimbursement for acute care and trauma hospitals.
Conclusion
On the basis of the findings of the current study, the predictive accuracy of the current NSHN Complex 30-day SSI Model after penetrating colon injury is poor. These results have significant implications for clinical practice. It suggests that the current model may not adequately capture the complexities and specific risk factors associated with trauma patients. Therefore, there is a critical need for the development of improved models that incorporate additional risk factors specific to trauma patients to enhance the predictive accuracy of colon SSI. Such advancements would have profound implications for patient care, enabling more accurate risk assessment and targeted interventions to reduce the incidence of SSIs in this vulnerable population. With the consequences of a colon SSI being so substantial, a tool to predict the occurrence of this complication needs to be more accurate than a coin flip.
Authors’ Contributions
A.B.: Visualization, writing—original draft, and writing—review and editing. W.I.: Methodology, formal analysis, investigation, and writing—review and editing. M.H.: Conceptualization, data curation, investigation, methodology, writing—original draft, and writing—review and editing. M.N.: Conceptualization, data curation, investigation, supervision, writing—original draft, writing—review and editing, and Guarantor.
Footnotes
Author Disclosure Statement
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding Information
The authors received no financial support for the research, authorship, and/or publication of this article.
Ethical Approval
The East Carolina University Institutional Review Board waived the requirements for IRB approval, as all data are publicly available.
