Abstract
Study Design:
Retrospective cohort study.
Objectives:
Despite pyogenic spondylodiscitis potentially conferring significant morbidity, there is no consensus on optimal treatment. The Brighton Spondylodiscitis Score (BSDS) was developed to identify patients who would likely fail conservative management and therefore benefit from earlier surgical intervention. In this study, we attempt external validation of the BSDS.
Methods:
We carried out a retrospective review of all patients treated at our institution, 2010-2016, for pyogenic spondylodiscitis. 91 met inclusion criteria and 40 progressed to require surgical intervention. The BSDS was calculated for each patient allowing stratification into low-, moderate- and high-risk groups. Calibration and discrimination was assessed with ROC curve analysis and calibration plot.
Results:
Area under the curve (AUC) was 0.469 (0.22-0.71) in our external validation, compared with AUC 0.83 and 0.71 (CI 0.50-0.88) in the original study and test populations respectively. Only 60% of patients in the high-risk group required surgery, 50% in the moderate, and 38% of the low indicating poor calibration and predictive accuracy. Operative intervention was not higher overall in our cohort (44% vs. 32%, p = 0.14). We found greater rates of bacteraemia, more distal infection, and more advanced MRI findings in our cohort. The incidence of spondylodiscitis in our region is higher (4/100 000/year).
Conclusion:
We failed to externally validate the BSDS in our population which is likely a result of unique population characteristics and the inherently variable pathology associated with spondylodiscitis. Clinicians must be cautious in adopting treatment algorithms developed in other health care systems that may comprise significantly different patient and pathogen characteristics.
Introduction
Pyogenic spondylodiscitis represents an infection of the vertebral disc and/or adjacent vertebrae that can result in severe long-term morbidity and in some cases life-threatening illness. 1 The prevalence has risen by up to 150% over the last 15 years - this may be attributed to increasing detection due to availability of advanced imaging modalities, as well as increasing intravenous drug use, use of immunosuppressant agents and increased population age. 2 Patients may present with a range of symptoms, some non-specific, resulting in diagnostic difficulty and therefore treatment delay. 3 Once identified, the treatment regimen for spondylodiscitis may include intravenous/oral antibiotic therapy with or without surgical intervention with the goal of eliminating sepsis, limiting neurologic damage and restoring function. 4
The optimal management algorithm for spondylodiscitis remains controversial and poorly defined at best. Previous classification systems intended to guide clinicians, such as those developed by Pola et al., are based on clinical and/or radiologic criteria but do not always identify those at risk of failing antibiotic therapy.1,4 Appalanaidu et al. recently developed the Brighton Spondylodiscitis Score (BSDS) to help identify patients with spondylodiscitis who would fail non-operative management and therefore benefit from earlier surgical intervention. 1 The logistic regression model in this study compromised 65 patients and proposed 6 predictors: distant site infection; medical comorbidity; immunocompromise; magnetic resonance imaging (MRI) characteristics; anatomical location, and; neurological impairment. This allowed stratification of patients in to 3 risk categories from low- to high-risk of failing non-operative treatment.
The clinical utility of any predictive algorithm depends on external validation. 1 Geographic variation in population characteristics and healthcare systems means that disease behavior and response to treatment may vary across the globe. Scoring systems, particularly those that have been developed from a single centre, may not be universally applicable. 5 For example, previous research showed that algorithms to predict behaviour of paediatric septic arthritis developed in the Northern Hemisphere did not perform well in our region. 6 The aim of our study was to externally validate the BSDS on a larger and geographically separate cohort of patients.
Materials and Methods
The Clinical Audit Support Unit granted institutional approval for collection and analysis of this retrospective patient information, deeming it exempt from informed consent (Ref: 3472). Using hospital coding we identified all patients who were admitted with a diagnosis of pyogenic spondylodiscitis January 2010 - December 2015 inclusive. The years were selected to be certain digital radiographic records would be available to allow confirmation of diagnosis and minimum of 2-years’ follow up. Medical and radiographic records were reviewed to ensure the diagnosis of spondylodiscitis was accurate as the potential inaccuracy of hospital coding is commonly acknowledged.
Inclusion and exclusion criteria from the original paper have been used: only adult patients (age >18 years) presenting with acute bacterial spondylodiscitis were eligible, excluding those with spondylodiscitis as a result of post-operative infection. There were 16 patients excluded due to age <18. 35 patients were excluded due to insufficient clinical record or loss of follow up. The remaining 91 patients >18 years of age were suitable for scoring. All cases were followed for a minimum of 2 years.
Demographic data, laboratory results, clinical examination findings, and MRI analysis have been used to derive a BSDS for each patient. Comorbidities of interest were diabetes mellitus, immunocompromise, intravenous drug use, and smoking status. We defined microbiological etiologies as gram negative, gram positive or, polymicrobial +/- MRSA following the categorization of Appalanaidu et al. 1 Anatomical locations for an infection were grouped as cervical, thoracolumbar, and lumbosacral with either single or multiple levels affected. Time to diagnosis in days and outcome measures such as final mobility status and relapse/recurrence rates have been recorded.
The 6 categories suggested as predictors include distant site infection, medical comorbidity, immunocompromise, MRI characteristics, anatomical location and neurological impairment. Within each of the 6 categories identified by the BSDS, there is a relative score for severity from a minimum of 6 points and maximum of 35 points. Table 1 outlines allocation of points in the BSDS. Scores are grouped into low risk (6-14), moderate risk (15-20), or high risk (>21).
The Brighton Scoring System.
Legend: Odds ratios and P-values have been taken from the logistic regression originally published by Appalanaidu eta al.
For example, distant site infection such as pneumonia or urinary tract infection scores 3 points, endocarditis scores 5 points, and multifocal sepsis scores 6 points. Similarly, MRI is scored with rising severity starting with non-specific fluid collection (2 points) increasing to abscess formation (6 points). Scores were calculated in the following manner, for example: a neurologically intact patient presenting with a cervical abscess, no comorbidities or immunocompromise, and abscess formation on MRI would score 9 points putting them into the “low risk” group for surgical intervention.
The protocol for conservative management in both our institution and the original research is a minimum of 6 weeks of antibiotics, intravenous and/or oral, with support and guidance from infectious disease physicians. Surgical intervention included abscess drainage, decompression/laminectomy, and stabilization in some cases.
To externally validate the BSDS in our population, we developed a ROC curve and calibration plot with the probability cutoffs as per the original article. Characteristics of the original study patients were compared with our cohort using simple statistics (significance accepted throughout if p =< 0.05). All data analysis was performed on Excel using StatPlus.
Results
91 patients with spondylodiscitis meeting study criteria were identified. Our study population was primarily male (82%) with an average age of 66.2 years (range 41-93, SD ± 12.3) (Table 2). Our region has a catchment area of approximately 900 000 indicating a minimum incidence of 4/100 000/year over the study period. The mean duration of symptoms prior to imaging was 4.4 days (SD ± 16.4). On arrival to hospital, 35% had a motor or sensory deficit. The majority of cases were lumbosacral (51%) followed by thoracolumbar (32%). Most patients had a single level infection (77%). Patients with concurrent spinal epidural abscess and vertebral column osteomyelitis have been included in this analysis as well as in the algorithm’s development. An epidural abscess was present in 35% of cases and osteomyelitis in 68%.
Descriptive Statistics.
* Detailed neurology recorded for a total of 32 patients * Abbreviations: MRSA (Methicillin-resistant Staphylococcus Aureus).
Comorbidities included diabetes mellitus (19%), malignancy (14%) and immunosuppression secondary to steroids, human immunodeficiency virus or organ transplant (3%). Significant lifestyle factors included history of smoking (50%) or intravenous drug abuse (6%), both possibly under-reported. Concurrent infections were found in 35 patients, usually in the lung or urinary tract (46%). The mean length of hospital stay was 29 days (SD ± 23.5).
80% of patients had a positive microbiological result from blood culture or intra-operative sampling with a majority yielding gram positive organisms (52%). Specific microbiologic data is presented in Table 3.
Microbiology From Positive Cultures.
MRI findings as per the BSDS classifications were non-specific fluid collection (40%), vertebral collapse (defined as >30% of anterior height loss per the original study by Appalanaidu et al.) (16%), and abscess formation (34%). Treatment was surgical in 44% with a combination of decompression (47%) and/or stabilization (75%). The non-operative treatment protocol was 6 weeks of IV antibiotics.
Mobility status was documented for 40 patients – of these, 82% were independently mobile on arrival to hospital and 60% retained full independent mobility on discharge, some requiring further rehabilitation. 14% of patients had a recurrent episode of spondylodiscitis or readmission for symptomatic relapse within 2 years. Five of these cases were relapses due to chronic infection, 4 patients failed antibiotic therapy and were re-admitted for surgery, 4 patients were re-admitted and trialed on a different antibiotic therapy, and 2 patients were admitted for pain management. Neurological improvement was seen in 90% of those with a presenting motor or sensory deficit. Presenting and final Frankel Grades have been derived from clinical records where available (Table 2).
The mean BSDS was 14.1 (range 7-25, SD ± 4.1) points across the population, suggesting few should require surgery. Distribution was mostly in the low risk group (55%), with 40% of patients in the moderate group and only 5% meeting criteria for high surgical risk. This is very similar to the distribution of patients in Appalanaidu et als. cohort. However, the surgical intervention rates differ significantly when separated by risk allocation (Table 4). Forty patients in our cohort proceeded for surgical management. The reason for surgical intervention was indicated most frequently by a deterioration in neurology (52%), by radiological progression including those with instability or spinal deformity (23%) or deteriorating clinical picture with ongoing fevers/sepsis (25%). The reasons for selecting surgical or conservative management for the Brighton cohort were not explicitly reported by Appalanaidu et al.
Patients Grouped by Surgical Risk.
Although the overall surgical intervention rate was not higher in our population (32% vs. 44%, p = 0.14), 38% of our low risk patients underwent surgery compared with only 10% in Appalanaidu et als. cohort (p = 0.001). In the moderate and high-risk groups, 50-60% of our patients underwent surgery compared with 100% of Appalanaidu et als. cohort (Table 4).
This indicates that increasing BSDS score did not correlate with increased surgical intervention. This is further demonstrated by the ROC curve which gives an AUC of 0.47 (CI 0.22-0.71) compared with an AUC of 0.83 and 0.71 (CI 0.50-0.88) for Appalanaidu et als. primary and validation cohorts respectively (Figure 1). A ROC curve assesses the capability of a rising score to distinguish between outcomes, whereby an AUC less than 0.5 is generally considered inadequate for discrimination.
ROC curve.
Taking a moderate or high score to be evident of “positive” result, comparing the BSDS to actual rates of surgical intervention in our cohort gave a sensitivity of 52% (CI 36-68.4%) and specificity of 61% (CI 46-74%). The positive and negative predictive values were 51% (CI 40-62%) and 62% (CI 52-74%) respectively. The positive likelihood ratio was 1.34 (CI 0.85-2.1) and the negative likelihood ratio 0.78 (CI 0.53 -1.16) suggesting the BSDS does not predict the need for surgery in our clinical setting. Finally, to assess the predictive accuracy of the BSDS, a calibration plot was constructed with an R2 value of 0.06 (Figure 2). This calibration confirms a poor relationship between predicted and observed outcomes.
Calibration plot.
Failure to treat patients with a higher BSDS surgically did not result in poor clinical outcome. For example, 18 patients with a ‘moderate’ BSDS treated conservatively. Of these, one patient experienced a recurrence of infection that was treated with further IV antibiotics within the follow-up period but did not require operative intervention or stabilization. Two “high-risk” Brighton score patients were treated conservatively, and neither experienced subsequent deterioration.
Following this, we examined the 2 populations for relevant differences that might be affecting algorithm performance. Compared with Appalanaidu et als. cohort, our patients were less likely to have insulin dependent diabetes mellitus or end-stage renal failure requiring dialysis (60% vs 3.3%, p = 0.00 001) (Table 5). Our population had higher rates of lumbosacral disease (32% vs 9%, p = 0.0008). MRI findings also differed between cohorts: Appalanaidu et als. had higher rates of vertebral collapse (35% vs 17%, p = 0.007) and disease spanning multiple levels (15% vs. 6%, p = 0.03).
Comparative Statistics: Brighton Cohort vs. Validation Cohort
Although in our cohort there were higher rates of gram-positive blood culture (35% vs. 52%, p = 0.03), stratifying patients by culture result does not alter the distribution of scoring or net rate of surgical intervention between the 2 populations.
Discussion
The aim of this paper was to externally validate the Brighton Spondylodiscitis Score however, we failed to find similar predictive capabilities in a geographically remote population. The trend for algorithm development and predictive scoring should be balanced by robust external validation to demonstrate generalizability of scoring systems. 7 Despite the high AUC values in the original paper and internal validation sample (0.83, 0.71) a rising BSDS did not translate to increased risk of surgical intervention in our cohort. This is best demonstrated by our ROC curve of 0.47, which shows inadequate discrimination between patients with lower risk scores and those with higher.
Poor performance of the BSDS may be a consequence of the algorithm’s development. The BSDS has been derived from logistic regression using a large number of variables (6) and subcategories (26) despite a relatively small patient cohort (n = 65). Using odds ratios and coefficients from Appalanaidu et als. report, it is unclear how moderate or high-risk patients were defined. The methodology for assigning points in the BSDS from odds ratios is not reported. For example, the odds ratio for vertebral collapse (+1.86) is more than twice that for malignancy (+0.71) but both these factors give a score of 4 points. Only the coefficient for distant site infection has a statistically significant p-value (0.03) in the original paper, with all other coefficients ranging from p = 0.106 – p = 0.490. It may be that these variables included in the algorithm are associated with increased risk for requiring surgery. However, an escalating BSDS does not translate to a clinical difference in our population.
The clinical applicability of a rising BSDS is difficult to establish. All patients in the moderate and high-risk groups underwent surgical intervention in the original paper, therefore exact definition of ‘moderate’ risk remains unclear. In our patients, the rate of intervention was only 51% for the same scores. This was not due to a statistically lower rate of surgical intervention overall. Conservatively managed patients with a “moderate” or “high” Brighton score in our population did not experience significant recurrence or relapse during our follow-up period.
Urrutia et al. recently attempted validation of the BSDS on a heterogenous cohort of pyogenic spinal column infection. 8 Sixty random cases were selected to represent the 3 regions of the spinal column. Similarly, they found that the BSDS was poor at stratifying patients into a group that accurately reflected the need for surgery. By modifying the BSDS and adding greater weight to cervical infection accuracy was improved but still with the acknowledgement that wider assessment of the score is need before routine use. In our cohort we did not find that spinal region was a significant predictor of surgical intervention.
Some of the heterogeneity between our population and the population from the original paper may contribute to differences in algorithm performance, such as the seemingly reduced severity of MRI findings in our cohort (lower rates of vertebral collapse in particular). However, despite these differences the overall rate of surgical intervention between the cohorts was similar, indicative of the burden that spondylodiscitis confers. The total number of patients used in our external validation was higher than in the original paper and in the external validation by Urrutia et al. 8 This gives a broader range of clinical presentation and disease severity, reflecting the diverse nature of spinal infection as a whole. Our local incidence of spondylodiscitis seems to be higher: 4/100 000 per year. For comparison, there is an estimated rate of 0.4-2.4/100 000/year in most Western populations. 3 This disparity is possibly related to an increased rate of bone and joint infection for the wider population. 9 Skin colonization with S. aureus and overcrowding have previously been cited as possible contributors. 10
These findings demonstrate the challenges with the utility of scoring systems to predict surgical decision-making. Surgical intervention for spondylodiscitis does not have universally agreed criteria. Therefore, any analysis to predict the risk of intervention in a cohort of patients with spondylodiscitis rather reflects the decision making by the clinicians rather than a well-defined endpoint. Scoring systems to predict clinical decision-making are subject to significant confounding. This is an inevitable consequence of variability in surgeon experience and protocol between regions, even for areas within the Organisation for Economic Co-operation and Development (OECD). There are significant ethnic and socioeconomic disparities for patients in our catchment area which may lead to inequitable access to healthcare and, at times, delayed and more severe presentations for treatable conditions. 11
Differences in MRI findings between our cohort and Appalanaidu et als. adds further diversity to analysis but is considered representative of the variable behaviour of spinal infection. . The BSDS gives a binary variable for “vertebral collapse”, which may be due to their original study having smaller numbers. This binary variable does not reflect the spectrum of instability that we have experienced in clinical practice. Greater extent of deformation and severity of instability may result in increased levels of surgical invasiveness. Once the absolute indications for immediate surgery are established, a larger, multicentre cohort would help refine the levels of risk associated with worse radiographic findings. 12
Future efforts should therefore firstly aim to define clearly the absolute indications for surgical intervention. This could be better understood by examining outcomes following conservative management such as categorical deterioration in neurology, subsequent segmental instability, health related quality-of-life scores or mortality.
Conclusion
We were unable to externally validate the Brighton Spondylodiscitis Score and provide support for its use in our population. In particular, there was little discrimination between moderate and high-risk groups. Poor algorithm behaviour may be exacerbated by local population characteristics, such as higher overall rates of spondylodiscitis when compared with other regions. A larger cohort, or multicentric study, across multiple geographic regions with defined surgical indications, would better aid in developing a severity score with global applicability.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
