Abstract
Background:
Surgical site complications (SSCs) are common, yet preventable hospital-acquired conditions. Single-use negative pressure wound therapy (sNPWT) has been shown to be effective in reducing rates of these complications. In the era of value-based care, strategic allocation of sNPWT is needed to optimize both clinical and financial outcomes.
Materials and Methods:
We conducted a retrospective analysis using data from the Premier Healthcare Database (2017–2021) for 10 representative open procedures in orthopedic, abdominal, cardiovascular, cesarean delivery, and breast surgery. After separating data into training and validation sets, various machine learning algorithms were used to develop pre-operative SSC risk prediction models. Model performance was assessed using standard metrics and predictors of SSCs were identified through feature importance evaluation. Highest-performing models were used to simulate the cost-effectiveness of sNPWT at both the patient and population level.
Results:
The prediction models demonstrated good performance, with an average area under the curve of 76%. Prominent predictors across subspecialities included age, obesity, and the level of procedure urgency. Prediction models enabled a simulation analysis to assess the population-level cost-effectiveness of sNPWT, incorporating patient and surgery-specific factors, along with the established efficacy of sNPWT for each surgical procedure. The simulation models uncovered significant variability in sNPWT's cost-effectiveness across different procedural categories.
Conclusions:
This study demonstrates that machine learning models can effectively predict a patient's risk of SSC and guide strategic utilization of sNPWT. This data-driven approach allows for optimization of clinical and financial outcomes by strategically allocating sNPWT based on personalized risk assessments.
Surgical site complications (SSCs) impose a substantial strain on our healthcare system through increased morbidity, elevated healthcare costs, heightened resource utilization, and diminished patient satisfaction and trust.1–3 Despite advancements in surgical techniques and peri-operative care, SSCs remain among the most common preventable hospital-acquired conditions.4,5 Single-use negative pressure wound therapy (sNPWT) has emerged as an effective intervention at reducing SSC risk,6–12 however, this intervention is also more expensive than traditional surgical dressings. In an era in which clinical decisions are increasingly influenced by cost considerations, it is crucial to identify prevention strategies that are both clinically effective and financially viable.
Machine learning provides a promising avenue for assessing a patient's risk for SSCs. Leveraging large, diverse datasets, machine learning algorithms can discern complex patterns and relations between patient characteristics, procedural variables, and SSC risk, which may not be evident through traditional statistical methods. Although studies have demonstrated the potential of machine learning-based SSC risk prediction algorithms, their practical implementation has been hindered by limitations, including inadequate consideration of prevention strategy efficacy and cost implications.13–17
In our study, we introduce a machine learning model for pre-operative SSC risk prediction. This risk prediction model is further integrated with a financial simulation model to assess the cost-effectiveness of sNPWT across a range of surgical procedures and patient risk profiles. This comprehensive approach provides a pragmatic, data-driven framework, for enhancing pre-operative planning and the strategic utilization of sNPWT for surgical site closure.
Materials and Methods
Data sources
This study utilized the Premier Healthcare Database (PHD) to develop and validate a machine learning model for predicting SSCs and assess the clinical and financial utility of sNPWT for risk mitigation. The PHD is a large hospital-based, service-level, all-payer database that includes information from more than 700 hospitals and healthcare systems in the United States. The database contains data on approximately 20% of all hospitalizations nationwide and includes information on patient demographics, diagnoses, procedures, drug utilization, healthcare costs, and more. For this study, we analyzed data from more than three million representative patients who had open surgery between the years 2017 and 2021. Economic data on the incremental costs associated with SSCs18–21 and the clinical efficacy of negative pressure wound therapy were compiled from previously published literature.6–12,22–25 For the cost of sNPWT, we used the manufacturer reported Medicare Part B average sales price. 26
Study population and inclusion criteria
The study population consisted of patients who underwent one of 10 high-volume, open surgical procedures that were grouped across five surgical subspecialties. Patients were included in the analysis if they had sufficient demographic and clinical data available. Exclusion criteria included closed procedures, having an SSC documented on the day of the index surgery, or having two different surgical procedures within 90 days of having an SSC diagnosed.
Data Preprocessing and Feature Engineering
The raw data from the PHD were cleaned and transformed for data pre-processing. Clinical diagnoses, coded using the International Classification of Diseases 10th Revision (ICD-10-CM) coding scheme, were reduced to approximately 500 categories using the Clinical Classifications Software Refined (CCSR) categorization schema. The included features encompassed variables that were defined pre-operatively, such as patient demographics, clinical features, and general information about the type of surgery.
Model selection, development, and evaluation
Several commonly used machine learning algorithms were evaluated, including logistic regression, random forests, support vector machines, and gradient boosting machines. The top three models in each surgical category were selected for hyperparameter tuning using grid search with cross-validation. The optimized models were assessed using various performance metrics, including area under the receiver operating characteristic curve, recall, precision, and F1-score. Feature importance analysis was conducted using CatBoost's Prediction Values Change method, which shows how much, on average, the prediction changes if the feature value changes. Feature importance values were normalized such that the sum of all feature importance values was equal to 100.
Validation strategy
The dataset was split into training, validation, and testing sets. The training set was used to train the model, the validation set for hyperparameter tuning, and the testing set for evaluating the model's performance on new, unseen data. The top-performing model for each surgical category was used to calculate the probability of SSC at the patient level and the output of this model was used for the financial simulation analysis.
Assessing cost-effectiveness of sNPWT across surgical cohorts
A financial simulation was conducted by applying the SSC risk prediction model across a large sample of patients within each surgical cohort. For each patient, our machine learning model first calculated each patient's baseline probability of developing an SSC. This baseline probability was then multiplied by the established cost of SSC for the procedure type. The simulation then applied the relative risk reduction of using sNPWT, using efficacy data from the published literature. 6–12,22–25 Within each surgical cohort, the simulation was applied to a testing set that represented approximately 20% of all patients. For each patient, the net financial value of sNPWT was calculated. At the population level, the percentage of patients in whom sNPWT was found to be cost saving was reported.
With respect to the cost of SSCs, cost figures were derived from previously published literature. 3 No cost was attributed to hematomas or seromas, reflecting the assumption that the cost associated with these specific complications are generally insignificant relative to the costs associated with surgical site infections (SSI) and surgical wound dehiscence. Patients who only had a hematoma or seroma were therefore removed from the test set used for the financial simulation.
Given the lack of cost data specific to wound dehiscence, the cost of this complication was assumed to be equivalent to that of SSI. This assumption aligns with the National Institute for Healthcare Excellence report, 27 which supports the assumption of equal costs for dehiscence and SSI. Cost estimates for SSI were adjusted for inflation to 2023 values, with separate cost estimates applied to each surgical category. 3
Ethics approval and consent to participate
This study involved a retrospective analysis of de-identified data and was therefore deemed exempt from Institutional Review Board approval.
Results
Patient characteristics at index surgery
More than three million patients were included in model training, across five different surgical subspecialities. Orthopedic surgery had the largest training set, with 1.6 million patients analyzed. Cardiovascular had the smallest training set, with 37,000 patients analyzed. The incidence of SSC, as well as the underlying demographic and clinical features, varied widely across surgical subspecialities (Table 1).
Descriptive Statistics on Surgical Cohorts
This table presents demographic and clinical data for five surgical procedure categories. The datasets varied in size, with orthopedics having the most patients and cardiovascular having the least. Age ranges differed significantly, with the youngest patients occurring in cesarean deliveries (30.1 y) and the oldest for orthopedic procedures (65.6 y). Admission types varied across procedures, with abdominal surgeries having higher rates of emergency admissions (35.5%). Incidence of surgical site complications (SSCs) was highest in cardiovascular surgeries (17.9%) and lowest in orthopedic surgeries (1.0%). Comorbidities such as obesity were most prevalent in orthopedic surgeries, whereas diabetes melllitus complications were highest in cardiovascular surgeries. Essential hypertension was a common comorbidity across all surgeries, particularly in cardiovascular surgeries.
SSI = surgical site infection.
Model performance
For each surgical subspeciality, the performance of five machine learning models were evaluated. The top-performing model (CatBoost) had an average area under the curve was 76%, with a performance range from 0.73 to 0.81 across surgical subspecialities. Performance metrics are summarized in Table 2.
Performance of SSC Prediction Models
Performance metrics for CatBoost machine learning model are presented across different surgery types. The model demonstrated the highest discriminative power for cardiovascular surgeries (0.81), followed by abdominal (0.78), with cesarean delivery, orthopedic, and breast surgeries trailing. Precision was most accurate for cardiovascular (0.73) and cesarean delivery (0.69) surgeries. Recall was highest for cardiovascular (0.51) and abdominal surgeries (0.41). The highest balance between precision and recall (F1 score) was achieved in CV (0.60) and abdominal surgeries (0.45). The F2 score, which prioritizes recall, was again highest for CV (0.54) and abdominal surgeries (0.43).
Feature importance
Although each model had a unique feature importance plot (Fig. 1), all of the models were highly sensitive to age, obesity, and the urgency of the surgical procedure (i.e., elective vs. urgent vs. emergent). For breast surgery patients specifically, the strongest drivers of SSC risk were related to the presence of underlying breast malignancy. For patients undergoing open abdominal surgery, some of the strongest pre-operative predictors of SSC risk were related to underlying intestinal anomalies or obstruction. In orthopedic surgery, pre-operative risk was driven primarily by the presence of osteoarthritis, hip fracture, or a complication related to previously implanted orthopedic hardware. In cesarean delivery patients, SSC risk was largely driven by obesity, a history of pregnancy complications, and advancing maternal age.
This figure provides an overview of the top features influencing the five different surgical models. Weights are shown for the most important features in each prediction model. For each model, the sum across all features is 100%. Age consistently emerges as a key feature across all surgical types, reinforcing the role of patient age on surgical risk. The importance of other features varies by surgery type, reflecting the unique considerations across surgical procedures. For example, a prior maternal delivery significantly influences the surgical site complication (SSC) risk in cesarean delivery patients, whereas conditions such as osteoarthritis and previous implant complications are notable risk factors in orthopedic surgery. The urgency and setting (i.e., inpatient vs. outpatient) of surgery also contributes to the risk profile.
Financial simulation
The SSC risk prediction model was used to simulate the financial implications of utilizing sNPWT within each of the surgical procedure categories. The percentage of patients within each surgical category where sNPWT was found to provide a net financial benefit varied widely. Additionally, when assessing the financial benefit of sNPWT, there was a high degree of variability across procedures in terms of the relative importance of patient-specific versus surgery-specific variables (Table 3).
Simulation of Cost-Effectiveness of sNPWT Across Surgical Populations
Results of the financial simulation model are presented. Each patient's SSC risk and the associated costs were computed in the test set, considering the expense of sNPWT against its expected reduction in SSC risk. For each surgical cohort, the table reports the maximum sNPWT utilization rate at which the intervention retains its cost-saving benefit for the patient population. For example, in the breast surgery cohort, sNPWT is cost-saving in up to 18% of patients, emphasizing the importance of targeted, risk-based application. Conversely, in the cardiovascular surgery cohort, sNPWT maintains its cost-saving potential when applied to 100% of the population, indicative of the high baseline SSC risk and associated costs in this group. The table also quantifies the relative influence of patient-specific risk factors on the cost-saving analysis, normalized to a scale of 1–100. In surgical cohorts with lower baseline SSC risk and SSC cost, such as breast surgery, the cost-saving potential of sNPWT is predominantly determined by an analysis of patient-specific risks. As the baseline SSC risk and cost associated with a type of surgery decrease, the necessity of conducting a personalized patient risk assessment correspondingly escalates.
sNPWT = single-use negative-pressure wound therapy; SSC = surgical site complication.
Across the range of surgical subspecialties analyzed, the maximum sNPWT utilization percentages that yielded a net financial benefit ranged from 18% to 100%. The financial benefit of sNPWT was strongest in cardiovascular surgery patients, where the baseline risk of SSC is high and the associated cost of treating an SSC is also high. In this high-risk surgical population, the simulation found that sNPWT continued to be overall cost saving at a population utilization rate of 100%. However, in breast surgery, the simulation found that sNPWT only remained cost saving up to utilization rates of 18%. The results of the simulation also illustrated that in surgical populations in which the financial benefits of sNPWT are more constrained (i.e., breast surgery), the strategic allocation of sNPWT requires more sensitivity to patient-specific variables.
Discussion
This study reinforces the utility of machine learning models in accurately predicting a patient's risk of SSCs and underscores their potential in powering clinical decision support tools that strategically guide the use of certain prevention measures. Our top SSC risk prediction models demonstrated robust performance across a wide range of surgical procedures, highlighting the versatility of these predictive tools. Notably, the features that were most important to the model's performance varied by surgical type. Although factors such as age, obesity, and the urgency of the procedure were universally important across models, each surgical subspecialty presented its own set of uniquely impactful features. For example, a pre-operative diagnosis of intestinal obstruction or ileus emerged as a predictor of SSC risk in abdominal procedures, but this was not found to be a risk factor for any of the other surgery procedures. The presence and importance of procedure-specific risk factors underscores the need to tailor SSC risk assessments toward each surgical procedure.
Our prediction models were designed specifically to concentrate on pre-operative variables, to enhance surgical planning. A pre-operative prediction model can identify SSC risk prior to surgical incision, thereby providing enough time to ensure that risk mitigation devices such as sNPWT are available and ready for surgical site closure.
Although future iterations of the model could incorporate intra-operative variables (i.e., surgical time, blood loss, patient vital signs, etc.), the implementation of a dynamic, real-time SSC risk prediction model will require workflow and logistical considerations.
Our financial simulation model revealed that the cost-effectiveness of sNPWT varies based on both patient-specific and surgery-specific variables. In high-risk and high-cost procedures, such as cardiac surgery, our financial simulation demonstrated that sNPWT could be cost-saving even at a 100% population utilization rate. Conversely, in lower-risk and cost procedures, such as breast surgery, the simulation revealed that a more selective, patient-specific sNPWT utilization strategy is needed to ensure cost savings at the population level. For instance, in breast surgery, the model suggested that exceeding a utilization rate of 18% might not be financially justifiable, underlining the importance of more strategic allocation of sNPWT. Therefore, in surgical procedures in which the baseline risk and associated costs of SSCs are comparatively lower, optimizing sNPWT utilization through personalized patient risk profiling is essential. Our model offers a framework for such targeted allocation, emphasizing the variability and importance of both patient and surgical factors in determining the cost-effectiveness of sNPWT.
It should be noted that our simulation model was based on a large sample of patients across the healthcare spectrum. Although the result from this large population analysis can provide general guidance, the true financial implications of sNPWT will depend on local variables at the surgeon, hospital, and health system level. The financial benefits of sNPWT will ultimately depend on each population's unique case mix, patient complexity, and local cost structures.
Clinical decision support tools that integrate SSC risk prediction models with financial simulations can provide care teams with a more nuanced, data-driven approach to managing SSC risk. By considering both the clinical implications and financial benefits of sNPWT, these tools enable a more strategic deployment of preventive measures. This method of allocating SSC prevention measures not only optimizes clinical outcomes but does so in a financially sensitive manner. Such an approach is particularly useful when considering the high variability in the cost-effectiveness of sNPWT across surgical procedures.
Although our study provides valuable insights, limitations should be considered, including the retrospective nature of the data and the reliance on claims data. Notably, research has shown that the incidence of surgical complications reported in claims databases may not accurately reflect the true rates observed in clinical settings.28,29 This discrepancy is because of a variety of factors, including potential underreporting or misclassification of complications within these databases. Consequently, although our algorithm offers valuable predictive insights, it is crucial to interpret its outputs with an understanding of these inherent limitations.
Another limitation of our study is the lack of detailed information on specific intra-operative SSC prevention measures within the patient claims data utilized by our model. We assumed a low utilization rate of sNPWT for surgical wound in our study population, thereby assuming the impact on our model's predictive performance to be minimal. However, this assumption may not reflect the varied practices and utilization rates of SSC preventive measures accurately in different clinical settings. Therefore, although our model provides a strong predictive framework, the absence of data on intra-operative interventions is an important consideration when interpreting its applicability and potential impact on real-world clinical outcomes.
It is also important to acknowledge that our financial simulation considered only directly measurable costs associated with SSCs, such as increased hospital length of stay (LOS) and re-admission rates. However, these direct costs represent just a portion of the total financial impact of SSCs. Our model did not account for indirect costs such as fines, litigation, or reimbursement penalties, which can be substantial.30,31 Furthermore, the cost attributed to SSCs in our analysis is an average figure, and it is important to recognize that financial implications may vary across different hospital types because of varying cost structures. Future research would benefit from a broader cost parameter inclusion to capture these additional costs, offering a more comprehensive understanding of the true financial burden of SSCs. This expansion would allow for a more accurate assessment of the economic impact of SSCs, tailored to the specific cost dynamics of individual healthcare settings.
Future research should also focus on prospective validation of machine learning models, iterative model refinement, and integration with electronic medical records for continuous learning and improved performance. Additionally, incorporating other prevention strategies into this type of analysis could help guide deployment of more comprehensive SSC prevention bundles.
Conclusions
In conclusion, our study illustrates that a predictive model focusing on pre-operative variables can effectively guide the strategic utilization of sNPWT, balancing both clinical and cost considerations. By providing a nuanced understanding of the cost-saving potential of sNPWT across different surgical cohorts and emphasizing the importance of personalized risk assessments, particularly in procedures with lower baseline risks and costs, our approach enhances the potential for improved clinical outcomes and optimized resource allocation. The integration of personalized SSC risk prediction models with financial simulation data allows for more targeted and economically viable SSC prevention protocols to be adopted.
Footnotes
Authors' Contributions
Conceptualization (equal): Larson, Netravali. Writing—original draft (lead): Larson. Data curation (equal): Roakes, Yurick. Investigation (equal): Roakes, Yurick. Methodology (equal): Roakes, Yurick. Formal analysis (equal): Roakes, Yurick. Validation (equal): Roakes, Yurick. Visualization (equal): Roakes, Yurick.
Project administration (lead): Netravali. Resources (lead): Netravali. Supervision (lead): Netravali.
Funding Information
This research received no external funding or grants.
Author Disclosure Statement
All authors are employees of Smith+Nephew, Inc. and may own shares of Smith+Nephew.