Paper 19: Evidence-Based Machine Learning Algorithm to Predict Failure Following Cartilage Preservation Procedures in the Knee

Abstract

Objectives:

To develop machine learning algorithms to predict failure of surgical procedures that address cartilage defects of the knee and detect the most valuable variables associated with failure.

Methods:

A single institution prospectively collected database of cartilage procedures was queried for procedures performed between 2000 and 2018. Failure was defined as revision cartilage surgery and/or knee arthroplasty. One hundred and one preoperative and intraoperative features were evaluated as potential predictors. The dataset was randomly divided into training (70%) and independent testing (30%) sets. Four machine learning algorithms were trained and internally validated. Algorithm performance was assessed using area under curve (AUC) and the Brier score. Local Interpretable Model-agnostic Explanations (LIME) was utilized to assess the optimized algorithm fidelity.

Results:

A total of 1091 patients who underwent surgical procedures addressing cartilage defects in the knee with a minimum of 2-years of follow-up were included. The mean follow-up was 3.5 ± 2.8 years. The mean age was 40.5 ± 15 years. There were 205 (18.8%) patients who failed at final follow-up. The Random Forest algorithm was found to be the best performing algorithm, with an AUC of 0.765 and a Brier score of 0.135. The 10 most important features for predicting failure following surgical procedures addressing cartilage defects of the knee were: symptom duration, age, body mass index (BMI), lesion grade, total lesion area (sum of all lesion areas), number of previous surgeries, number of lesions in the knee, gender, athletic level, and traumatic etiology. LIME analysis allowed for assessment of the optimized algorithm fidelity, as well as provided a patient-specific comparison for the risk of failure of an individual patient being assigned various types of cartilage procedures.

Conclusions:

Machine learning algorithms were accurate in predicting the risk of failure following cartilage procedures of the knee, with the most important features being symptom duration, age, BMI, lesion grade, and total lesion area. Machine learning algorithms may be used to compare the risk of failure of specific patient-procedure combinations in the treatment of cartilage defects of the knee. Integrated human and machine learning decision-making may improve patient selection and bring about the new era of patient-tailored evidence-based clinical care.

Table 1.

Demographic and Clinical Characteristics.

Table 2.

Prior surgical procudures.

Figure 1.

Global feature importance plot displaying (in a descending order) the most important predictors of failure following surgical procedures addressing cartilage defects of the knee.

Figure 2.

A-D. The Corresponding receiver operating characteristic (ROC) curve analysis of the four algorithms. A. Elastic-net penalized linear regression (ENPLR, AUC=0.725); B. Random Forest (AUC=0.765); C. Neural Network (AUC=0.742); D. XG Boost (AUC=0.757).