Decompression Alone vs Decompression and Fusion: Spin in Abstracts of Systematic Reviews and Meta-Analysis

Abstract

Study Design

Cross-sectional.

Objectives

Spin, a prevalent bias, can distort outcomes in well-validated research. Treatment of lumbar stenosis with spondylolisthesis through decompression alone (DA) vs decompression and fusion (DF) remains a controversial topic. We aimed to identify the prevalence of spin in meta-analyses and systematic reviews regarding DA vs DF in the treatment of spinal stenosis with concomitant degenerative spondylolisthesis.

Methods

MEDLINE, Web of Science, and Google Scholar were searched for meta-analyses and systematic reviews comparing DA vs DF treatment of lumbar spinal stenosis accompanied by degenerative spondylolisthesis. Each study was evaluated for the nine most severe spin types. We also explored the association between spin and methodological quality of a systematic review using the revised A Measurement Tool to Assess Systematic Reviews appraisal tool.

Results

The search yielded 1506 articles, of which 13 met inclusion. It was found that 46.2% (6/13) of the articles contained spin within the abstract. Of the nine most severe types, type 5 was found to be the most prevalent (4/13, 30.8%), followed by types 9 (2/13, 15.4%), 3 (1/13, 7.7%), 4 (1/13, 7.7%), and 6 (1/13, 7.7%). Spin types 1, 2, 7 and 8 were not found. According to AMSTAR-2, 53.8% (7/13) of the studies were appraised as “critically low” quality and 46.2% (6/13) as “low” quality.

Conclusion

This study demonstrated the presence of spin in 46.2% of abstracts pertaining to the treatment for spinal stenosis with degenerative spondylolisthesis. Spine surgeons should learn to recognize spin as they review articles before implementing them into practice.

Introduction

Spinal stenosis with degenerative spondylolisthesis (DS) is a common cause of low back pain, difficulty walking, and even neurogenic claudication. It is characterized anatomically by the narrowing of the spinal canal by spondylotic changes and the displacement of one vertebral body over the subjacent vertebral body without disruption of the vertebral ring. The result is compression of central and occasionally foraminal neural tissue. With life expectancy continuing to rise, a substantial portion of the senior population will develop DS. Hence, it is critical to assess the effectiveness of laminectomy and determine the populations that would benefit more from the additional fusion.

Decompression through laminectomy has been a standard surgical treatment for spinal stenosis and spondylolisthesis for many decades. However, laminectomy alone has raised concerns regarding instability caused by several factors related to the compromised integrity of the posterior spinal column. In the early 1990s, two significant studies investigated the benefit of fusion and concluded that patients reported better outcomes after decompression and fusion.^1,2 In the decades that followed, there was a dramatic increase in the rate of fusion surgeries.

There are clear benefits to both decompression alone and decompression with instrumentation. Multiple previous studies have determined that decompression alone demonstrates decreased operative time and blood loss but is associated with increased instability.^3-5 In contrast, decompression and fusion with instrumentation demonstrated a decrease in the likelihood of reoperation due to instability, however, leads to increased operative time, blood loss, and possible complications related to pedicle screw placement. For these reasons, controversy continues to exist regarding the most optimal treatment for spondylolisthesis.

As spine surgeons place emphasis on practicing evidence-based medicine (EBM), continuous awareness of current literature and the shift in standard of care is necessary. However, due to the rapidly increasing body of literature, a large majority of physicians have become increasingly reliant on meta-analyses and systematic reviews due to their ability to summarize evidence. While meta-analyses and systematic reviews are the highest level of evidence, they can still be subject to bias, or “spin.” “Spin” has been defined in biomedical research as a specific way of intentionally or unintentionally implying the beneficial effect of the experimental treatment, in terms of efficacy or safety, is greater than that shown by the results. ⁶ Thus, it is critical to be aware of spin and its prevalence in spinal surgery meta-analyses and systematic reviews.

The primary objective of this cross-sectional study is to examine the characteristics and prevalence of different types of spin found in the abstracts of systematic reviews and meta-analyses that seek to determine the prognostic difference between decompression alone vs decompression and fusion. The spin classification system developed by Yavchitz et al is used as the evaluation tool in this study. ⁷ The secondary objective of this study includes grading manuscripts with the AMSTAR 2 criteria proposed by Shea et al, which assesses the quality of conduct of systematic reviews, and evaluating the correlation between the presence of spin and various other article characteristics such as the year of publication, number of times cited, and journal impact factor. ⁸ Given that previous studies have identified spin in the abstracts of other types of orthopaedic research, we hypothesize that spin would be present in ≥ 30% of abstracts of systematic reviews and meta-analyses.^9,10

Methods

Training

Training was provided to data extractors and study reviewers (TT and HA) by MG. Instructions included article screening, the process of data extraction, and screening of spin types to ensure calibrated results between the two reviewers. Subsequently, a search was conducted by TT and HA in February of 2024 using MEDLINE, Web of Science, and Google Scholar with the keywords search: (lumbar) and (systematic review OR Meta-analysis) and (fusion OR decompression).

Inclusion Criteria

All systematic reviews and meta-analyses comparing the effectiveness of decompression alone vs decompression and fusion were reviewed. For an article to be included, it must have met the following criteria: 1) a systematic review with or without a meta-analysis; 2) degenerative etiology, 3) conducted only on human subjects; 4) available in English. There was no exclusion based on the date of publication or country of origin.

Review Process

Two independent authors, TT and HA, reviewed all articles provided by the keyword search. After applying the exclusion criteria, a consensus was reached on the final 13 articles. Each of the 13 manuscripts was independently reviewed by TT and HA for the nine most common forms of spin found in abstracts, as defined by Yavchitz et al. While Yavchitz et al discussed and ranked the 21 types of spin in systematic reviews, the decision was made to focus on the nine most severe forms of spin for two main reasons. Firstly, the nine most severe forms of spin are more readily verifiable, while less severe forms are less clearly defined. Secondly, we believe that focusing on the more severe forms of spin will allow for a greater focus on addressing the more relevant and potentially harmful forms of spin as opposed to more minor forms. In addition to spin, A Measurement Tool to Assess Systematic Reviews (AMSTAR 2), a survey comprising 16 questions aimed at critically examining the methodology of systematic reviews, was used to analyze each of the 13 manuscripts. Through the AMSTAR 2 tool, a rating was obtained from critically low (>1 critical flaw) to high (0 or only one non-critical weakness). Furthermore, the year of publication, the number of yearly citations, and the journal’s impact factor were recorded in March 2024. This data was collected independently by TT and HA. Any discrepancies within this collection process were resolved through discussion amongst the authors, after which a consensus was reached.

Statistical Analysis

Data was collected and organized into an Excel spreadsheet. Linear regression modeling was utilized to independently examine the presence or absence of an association of AMSTAR classification, number of citations per year, and journal impact factor with the amount of spin present in abstracts. Additionally, logistic regression modeling was used to assess whether AMSTAR classification, journal impact factor, and/or number of citations per year were associated with the presence of spin. Lastly, multiple regression modeling was performed to assess if AMSTAR classification, journal impact factor, and number of yearly citations combined could predict the presence or absence of spin.

Results

The electronic database search yielded 1506 articles, as illustrated in Figure 1. Of these, 13 met our inclusion criteria and were included in the final analysis.^11-23 The year of publication of the included articles ranged from 2007 to 2023. The total number of citations ranged from 0 to 293 with an average of 47 citations. The yearly citations ranged from 0 to 19.3 with an average of 6.57 citations per year. The impact factor for the journals included ranged from .7 to 15.1. Our analysis revealed no significant association between the presence of spin in the abstract of the articles and the AMSTAR 2 grade, number of citations, impact factor of the journal, or the year of publication.

Figure 1.

Flow diagram of study selection.

AMSTAR 2 Rating

The AMSTAR-2 survey resulted in a quality rating of the included manuscripts ranging from ‘critically low’ to ‘low’. Of the included articles, 53.8% (7/13) were rated as ‘critically low’ and 46.2% (6/13) were rated as ‘low’. Of note, none of the articles included satisfied the AMSTAR-2 criteria #10 criteria “Did the review authors report on the sources of funding for the studies included in the review?”. Conversely, all included manuscripts satisfied criteria #1 “Did the research questions and inclusion criteria for the review include the components of PICO?”

Spin Rating

After reviewing the 13 manuscripts, 46.2% (6/13) (Table 1) of the included systematic reviews contained spin within the abstract. The most prevalent type of spin found, out of the nine most severe types, was the type 5 spin (the conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies), which was found in 30.8% (4/13) of the articles. The second most common type of spin was type 9 (conclusion claims the beneficial effects of the experimental treatment despite reporting bias), found in 15.4% (2/13). Spin types 1, 2, 7, and 8 were not found. According to AMSTAR-2, 53.8% (7/13) of the studies were appraised as “critically low” quality and 46.2% (6/13) as “low” quality.

Table 1.

Inclusion and Exclusion Criteria.

Inclusion Criteria	Exclusion Criteria
Systematic review & meta-analysis	Non-systematic review or meta-analysis
Degenerative etiology	Traumatic, infectious, oncological etiology
Spondylolisthesis	Non-spondylolisthesis degenerative pathology (eg, spinal stenosis)
Adult patients	Adolescent & pediatric patients
	Non-English literature

Discussion

As the medical field continues to strive towards an evidence-based model, improving the quality and accuracy of biomedical research becomes vital. The exploration of abstract spin remains a novel focus within spine research. In this study, the authors employed a similar method in the review process as previously published articles on spin bias in orthopaedic research, which have demonstrated strong inter-rater reliability.^9,10,24 Through this examination, the authors aim to bolster the credibility and clarity of conclusions presented in abstracts. This is particularly important in today’s aging population, where decompression alone and decompression with fusion procedures are increasingly more common.

Our current study found the presence of one of the nine most severe forms of spin in 46.2% (6/13) of the papers included in our analysis (Table 2). To the author’s knowledge, our study is the first to evaluate the presence of spin in meta-analyses and systematic reviews on the topic of decompression alone vs decompression and fusion of lumbar stenosis accompanied by degenerative lumbar spondylolisthesis. Our study expands upon the efforts to determine the incidence of spin in spine surgery research. Published research evaluating spin in orthopaedic surgery research has shown vastly ranging and alarming incidence of spin, from 28% to 93.1%.^9,10,24-26

Table 2.

Nine Most Severe Types of Spin Per Yavchitz et al. ⁷

Nine Most Severe Types of Spin	Abstracts With Spin; n (%)
1. Conclusion contains recommendations for clinical practice not supported by the findings	0 (0%)
2. Title claims or suggests a beneficial effect of the experimental intervention not supported by the findings	0 (0%)
3. Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention	1 (7.7%)
4. Conclusion claims safety based on non-statistically significant results with a wide confidence interval	1 (7.7%)
5. Conclusion claims the beneficial effect of the experimental treatment despite high risk of bias in the primary studies	4 (30.7%)
6. Selective reporting of or overemphasis on harm outcomes or analysis favoring the safety of the experimental intervention	1 (7.7%)
7. Conclusion extrapolates the review’s findings to a different intervention	0 (0%)
8. Conclusion extrapolates the review’s findings from a surrogate marker of a specific outcome to the global improvement of the disease	0 (0%)
9. Conclusion claims the beneficial effects of the experimental treatment despite reporting bias	2 (15.3%)

The most common level of spin identified in our study was type 5, which is when the “conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies.” An example of this was a study by Dijkerman et al, which concluded that decompression alone is the preferred method in patients with low-grade spondylolisthesis. ²¹ However, the primary studies analyzed in the review contained significant bias, with the majority of studies having a moderate risk of bias. ²¹ Additionally, all studies included in the systematic review demonstrated a low level of evidence.

The second most prevalent level of spin found in our study was type 9, which is defined by the “conclusion claims the beneficial effects of the experimental treatment despite reporting bias.” An example of this is an article by Wei et al. which deemed the clinical effectiveness of decompression alone comparable to that of decompression with fusion for degenerative lumbar spondylolisthesis despite reporting bias and heterogeneity in the primary studies secondary to the different definitions, tools, and follow-up periods that the included studies utilize. ¹⁴ The highlighting of this conclusion in the abstract of the study without reference to the limitations of the included studies, possibly due to abstract word count limitations, can be misleading to readers resulting in a type 9 spin.

The AMSTAR 2 survey is a tool used to critically appraise systematic reviews. Of the articles included in our analysis, 53.8% (7/13) were rated as ‘critically low’ and 46.2% (6/13) were rated as ‘low’ (Table 3). This suggests that the low rating may be either due to inadequacies in the designs of spine studies on this topic or due to the exceedingly high standards set by the AMSTAR 2 survey. Another possible explanation is the subjective nature of scoring systematic reviews using this survey, although the authors demonstrated acceptable inter-rater reliability using the AMSTAR 2 survey. Notably, all included studies had a critical flaw due to failure in satisfying the 7th question of the AMSTAR survey: “Did the review authors provide a list of excluded studies and justify the exclusions?” which led to either a “low” or “critically low” designation. Of note, the authors did not observe a correlation between the AMSTAR 2 quality rating and the presence of spin, likely attributable to all included studies being designated as “low” or “critically low”.

Table 3.

Characteristics of Studies Which Did and Did Not Contain Spin.

	No Spin	Spin
AMSTAR-2 rating
High	0	0
Moderate	0	0
Low	2	4
Critically low	5	2
Impact factor	2.46 ± 1.08	4.21 ± 5.41
Yearly citations	6.73 ± 6.60	6.4 ± 4.78

Although the mechanisms behind spin bias have yet to be elucidated, multiple theories attempt to explain why this form of bias is so prevalent. Research scientists are under immense pressure to produce high volumes of research that will get published in high-impact journals. Therefore, they can be prone to reporting statistically significant differences when in reality there are none. ²⁷ Furthermore, while more journals require adherence to the PRISMA guidelines, previous studies show no significant effect on the prevalence of spin. Additionally, most journals do not set guidelines to directly minimize the prevalence or discourage the use of spin in studies.^27-29 Despite the underlying mechanisms, the presence of spin bias significantly clouds study outcomes. When these studies inform clinical decisions, the repercussions can be substantial. This is especially true when physicians are planning surgical approaches, as in the case of decompression alone vs decompression with fusion.

Fortunately, some authors have proposed mechanisms to help prevent spin bias in biomedical research. One such example is for journals to set forth strict guidelines, such as PRISMA, which is already in effect. Although our study did not demonstrate a correlation between adherence to PRISMA qualifications and the presence of spin, prior research has highlighted such a connection. ²⁹ Therefore, increased adoption of PRISMA guidelines, along with the potential integration of spin-related measures into the guidelines, could potentially mitigate spin rates. Furthermore, granting journal access to the statistical analysis and raw data of the study can also help prevent spin. ²⁷ Additionally, journals often set limitations on the abstract word or character count, leading authors to struggle to adequately highlight their work while abiding by journal limitations. While difficult to analyze, allowing longer abstracts may help authors more comprehensively address their results thus decreasing the prevalence of spin.

Ultimately, increasing awareness among researchers about spin bias and its implications will allow for greater recognition of the bias by readers and encourage responsible reporting of results. Instituting these measures not only fortifies the integrity of orthopaedic surgery and neurosurgical research but also fosters a culture of transparency and accountability, ultimately advancing the pursuit of reliable and impactful research.