Abstract
Study Design
Systematic review.
Objectives
Spin in scientific literature is defined as bias that overstates efficacy and/or underestimates harms of procedures undergoing review. While lumbar microdiscectomies (MD) are considered the gold standard for treating lumbar disc herniations (LDH), outcomes of novel procedures are being weighed against open MD. This study identifies the quantity and type of spin in systematic reviews and meta-analyses of LDH interventions.
Methods
A search was conducted on the PubMed, Scopus, and SPORTDiscus databases for systematic reviews and meta-analyses evaluating the outcomes of MD against other LDH interventions. Each included study’s abstract was assessed for the presence of the 15 most common types of spin, with full texts reviewed during cases of disagreement or for clarification. Full texts were used in the assessment of study quality per AMSTAR 2.
Results
All 34 included studies were observed to have at least 1 form of spin, in either the abstract or full text. The most common type of spin identified was type 5 (“The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies”), which was observed in ten studies (10/34, 29.4%). There was a statistically significant association between studies not registered with PROSPERO and the failure to satisfy AMSTAR type 2 (P < .0001).
Conclusion
Misleading reporting is the most common category of spin in literature related to LDH. Spin overwhelmingly tends to go in the positive direction, with results inappropriately favoring the efficacy or safety of an experimental intervention.
Introduction
Lumbar microdiscectomies (MD) are the gold standard for treating herniated lumbar discs with concomitant symptoms of low back pain and radiculopathy refractory to nonoperative treatment. 1 Outcomes after MD have previously been compared to the efficacy of conservative management, the line of treatment that is administered and encouraged until symptoms of radiculopathy demonstrate no significant improvement or signs of resolution. Traditional MD, pioneered by Yasargil and Caspar in 1977 involves the removal of disc fragments through an open transflaval approach by laminotomy. 1 However, other novel minimally invasive spine procedures (MIS) have been developed 1 including percutaneous endoscopic lumbar discectomy (PELD), tubular microdiscectomy (TMD) and sequestrectomy.
In PELD, a lateral, full-endoscopic approach is utilized in which the disc fragments are removed through the neuroforamen.2–5 Finally, TMD avoids direct cutting through the muscles through progressive dilation of the soft tissues, thus expecting to minimize the area of exposure required for microdiscectomy, compared to traditional open approaches.6,7
Although MIS procedures demonstrate potential to improve safety and reduce complication rates, there is no consensus in literature as to which procedures are most efficacious and safest for patients.5,8 Nonetheless, all aforementioned procedures are characterized as minimally invasive, which explains the large number of comparative studies between the open microdiscectomy and all other developing MIS procedures. These alternative procedures and their outcomes are frequently compared to MD and have been evaluated in the form of systematic reviews and meta-analyses.
Systematic reviews and meta-analyses collate data across many studies that investigate similar subject matter and are therefore considered to be research studies of higher level of evidence. Despite such investigations, systematic reviews and meta-analyses are subject to spin, otherwise known as bias in scientific literature, due to included studies that contain small sample sizes. Furthermore, these reviews may be at even higher risk of spin simply due to the experimental nature of the more novel MIS procedures and its less expansive reach into clinical practices than when compared to traditional MD. Because spine procedures and their safety and efficacy are also highly debated, investigating bias within current literature is critical.5,8 Therefore, it is imperative that incidences of bias are quantitatively and qualitatively evaluated so that clinicians and researchers may have an increased awareness to identify spin before conclusively deciding on any given technique or procedure.
The primary aim of this study was to identify the quantity and type of spin in systematic reviews and meta-analyses of MD and novel MIS techniques. The secondary aim was to classify the studies in which incidences of spin were identified to determine how and when distinct patterns of spin are presented.
Methods
This study was conducted per Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines using a predetermined protocol. Search criteria was determined a priori. A single author (MA) conducted a search of the PubMed, Scopus, and SPORTDiscus databases using ‘‘lumbar microdiscectomy’’ AND ‘‘systematic review’’ OR ‘‘meta-analysis’’. The search results were aggregated and de-duplicated in EndNote X9 (Clarivate, Philadelphia, PA, USA). Two authors (MA and AT) independently screened the identified studies for inclusion. A third author (OG) reconciled any disagreements while evaluating spin.
Eligibility
Systematic reviews and/or meta-analyses on the topic of MD alone or in conjunction with another procedure were eligible. Databases were queried from inception to September 13, 2022, on which the searches were conducted. Exclusion criteria were studies that were not peer-reviewed, not published in English, not systematic reviews or meta-analyses, did not discuss outcomes, retracted or withdrawn, included nonhuman or cadaver subjects, published without an abstract, or did not have full text available.
Training
Full Assessment of Spin in the Abstracts of Included Articles.
Data Extraction
Data was extracted from the included articles independently by 2 authors (MA and AT) and any disagreements were reconciled by a third author (OG). The data collected included study title, authors, publication year, journal, level of evidence, study design, funding source, adherence to PRISMA guidelines, journal impact factor, preregistration of study protocol with PROSPERO, primary outcomes, and if available, secondary outcomes. Data on the 15 criteria of spin in abstract were collected. If there was uncertainty in a given criteria, the full text was referred to and compared to the abstract to obtain clarity and identify any discrepancies in a study’s reported results. Once the spin of an abstract was evaluated, the authors utilized the 16-point AMSTAR 2 criteria to evaluate spin of the full text.
Results
Our systematic search identified 102 eligible studies, of which 44 were removed as duplicates. 17 studies were excluded during the title and abstract screening process for failing to meet the inclusion criteria (Figure 1). During the full text screening, four studies were excluded because no microdiscectomy outcomes were directly measured, or only measured the outcomes of a MIS technique with no reference to standard microdiscectomy. Three studies were excluded because the full text did not contain a discussion, analysis, or results. Ultimately, 34 studies published in 16 unique journals were included in this review. Of the 34 included articles, 25 included meta-analysis (25/34, 74%). 11 articles (11/34, 32%) reported having received external funding for the study. 17 of the included studies reported adherence to the PRISMA guidelines (17/34, 50%). Only six studies (6/34, 18%) registered with PROSPERO (University of York, York, UK). The mean 2021 Clarivate Impact Factor of the included studies was 4.421 (range: 1.033-18.473). The mean Scopus CiteScores of the included studies was 5.219 (1.5-21.3). The preferred reporting items for systematic review and meta-analysis (PRISMA) flow diagram.
Frequency of Spin and Analysis
At least 1 form of spin was observed in the abstracts of 21 out of 34 studies (61.8%). The median number of spin categories identified per study was 1 (range: 0-6). The most common type of spin identified was type 5 (“The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies”), which was observed in ten studies (10/34, 29.4%). The next most common types of spin were type 3 (“Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention”) and type 14 (“Failure to report a wide confidence interval of estimates”), both of which were observed in seven studies (7/34, 20.6%). A full assessment of spin within abstracts is shown in Table 1.
Complete AMSTAR 2 Assessment.
aonly applicable to studies conducting meta-analysis.
RoB = risk of bias.
Included Studies with Spin Types Cited in Abstract and Full Text.
Discussion
The aim of this study was to identify the quantity and type of spin in systematic reviews and meta-analyses of MD. The secondary aim was to classify the studies in which incidences of spin were identified to determine how and when distinct patterns of spin are presented. At least 1 form of spin was observed in 21 out of 34 studies (61.8%). Among the 21 studies that were reported to have spin, 18 studies had at least one indication of positive spin (18/21, 85.7%). Only three studies indicated at least one form of negative spin (3/21, 14.3%). This finding highlights the general direction of spin, where positive spin inappropriately favors the experimental intervention. Furthermore, none of the studies included in our review were classified as “high quality” according to AMSTAR criteria. This high prevalence of spin highlights the widespread nature of bias in research literature and its correlation with studies that are lower quality. Of all types of spin, the most common type identified was type 5 (“The conclusion claims the beneficial effect of the experimental treatment despite a high risk of bias in primary studies”), which was observed in ten studies in our review (10/34, 29.4%). This finding agrees with other literature, where type 5 was the most common spin type in Kim et al, reporting on superior capsular reconstruction (11/17, 65%). 37 The prevalence of this spin type aligns with our finding of no “high quality” studies, and that in such studies, results often claim beneficial effect of the experiment despite intrusion of bias and its tendency to distort the accuracy of results.
Another important finding is the statistically significant association between studies not registered with PROSPERO and the failure to satisfy AMSTAR type 2 spin (P < .0001). A study must have preregistered their research protocol or declared that research methods were established a priori in order to have fulfilled AMSTAR type 2. Failure to discuss how and when review methods were established often coincides with failure to register with PROSPERO, a database of prospectively registered systematic reviews that functions to help avoid duplication and reduce opportunity for reporting bias. 38 We found no other significant associations between a study’s incidences of spin and other study characteristics, including year of publication, journal impact factor (ScopusCiteScore, Clarivate Impact Factor), funding, and adherence to PRISMA. This presence of heterogeneity is a compelling illustration of the fact that spin is not impervious to journals with higher impact factors or in articles with higher levels of evidence. Our findings align with other investigations, which found no significant associations between funding, impact factor, adherence to PRISMA, and year of publication. 39 This serves as a reminder that bias is pervasive throughout the literature, and that readers must diligently assess and evaluate findings related to the topic of interest.
Several studies have shown a high prevalence of spin in systematic reviews in orthopedic research.37,39,40 An example of type 5 spin in Wang et al 34 is present in the conclusion of the abstract which states “the results of this meta-analysis demonstrate that interlaminar minimally invasive discectomy (ILMI) and MD are both safe and effective surgical procedures for treating LDH. Compared with MD, ILMI can shorten days in hospital, decrease the amounts of blood loss during surgery”. 34 However, the study reported that among the eleven included studies, ten had high risk of bias, calling into question the validity of the conclusions made in the study. While they acknowledge the overall Grading of Recommendations, Assessment, Development and Evaluations (GRADE) quality as low, they claim that the results of ILMI are superior to those of MD in terms of safety. Consequently, readers may interpret such conclusions without appropriately accounting for the impact of bias. This example can be further characterized as positive spin. By inappropriately favoring the experimental intervention when compared to the control intervention, positive spin is presented. In this case, the findings reported for ILMI were inappropriately favored compared to MD, as the risk of bias in this study was evaluated as high.
Another example that illustrates positive spin, and type 1 spin, is found in Alvi et al’s 11 systematic review and meta-analysis of different surgical interventions for treating LDH. The abstract’s results state “TD [tubular discectomy] was found to be associated with a greater rate of overall complications (odds ratio [OR] 1.49, P = .002), greater incidence of dural tears (OR 1.72 P = .04), and recurrent herniation (OR 2.09, P = .0007). Finally, [open discectomy] OD/MD was associated with significantly lower incidence of revision surgery (OR .53, P = .0007).” While the results illustrate that TD is less safe compared to OD/MD based on the outcomes measured in their study, their conclusion nonetheless affirms that “tubular-discectomy and percutaneous-endoscopic-discectomy, the most commonly employed MIS techniques for discectomy, can be used as safe alternatives for open discectomy depending on the preference of the operating surgeon.” By stating that the MIS investigated can be used as safe alternatives, despite the results illustrating otherwise, a clear indication of positive spin is presented. Drawing such a conclusion without correct reflection of the results may misguide clinicians in their evaluation of a procedure’s safety and their overall decision-making.
Alternatively, negative spin is when a study underemphasizes the efficacy or safety of the experimental intervention or inappropriately favors the control intervention. In our review, we found one example of negative spin. Zhang et al. 8 compared microdiscectomy against sequestrectomy and the results state “meta-analysis showed that microdiscectomy resulted in higher low back pain VAS [visual analog scale] score.” However, the conclusion affirms the opposite finding stating, “both microdiscectomy and sequestrectomy had good curative results in the treatment of LDH. In low back pain VAS score, the former [microdiscectomy] was better than the latter.” This contrary conclusion may lead the reader to believe that MD performed more favorably than sequestrectomy in the metric of pain, when the statistically significant results stated otherwise. Such a statement may lead clinicians to dismiss a potentially more effective procedure when selecting an intervention for LDH treatment. Nonetheless, positive spin continues to be more prevalent than negative spin in literature, as studies tend to highlight findings that confirm their hypothesis, often one that favors the efficacy and safety of the experimental intervention. We found that among the 21 studies that were reported to have at least one type of spin, 18 studies had at least one indication of positive spin (18/21, 85.7%). Only three studies indicated at least one form of negative spin (3/21, 14.3%), illustrating how authors scarcely underemphasize the safety or efficacy of their experimental intervention in order to confirm the present hypothesis or expected results.
Another important and frequently identified spin type falling under misleading reporting is type 3 spin (“Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention”), the second most common type of spin identified in our review. This was also found to be the second most common type after spin type 5 in Kim et al, 37 suggesting that these spin types often coincide. In a study investigating spin of systematic reviews and meta-analyses of Achilles tendon ruptures, type 3 spin was the most common, occurring in 53.5% (23/43) abstracts. 39 In Wei et al’s 1 meta-analysis on MD, they investigate the efficacy and safety of eight surgical interventions for lumbar disc herniation, including MD. However, the abstract’s conclusion states “the results of this study provided evidence that PELD and PLDD [percutaneous laser disc decompression] were with lower intraoperative and post-operative complication rates, respectively. TD, PELD, PLDD, and MED were the safest procedures for LDH according to complications, reoperation, operation time, and blood loss.” The conclusion entirely omits any relevant and statistically significant findings related to MD, lending classification of spin type 3, 37 For example, the discussion in the full text states that the MIS, chemonucleolysis, had statistically significant higher reoperation rates than MD and other procedures. Such an example highlights the issue of overemphasis on nonsignificant findings as well as a failure to include and discuss significant findings in the abstract. Furthermore, the findings for complications, blood loss, and operation time reached no statistical significance between any 2 interventions, yet are reported as significant in the abstract. Including findings without referring to their significance or lack thereof is inherently misleading in that readers are not given comprehensive context or an opportunity to assess clinical significance.
The primary limitation of our study is the subjective nature of identifying and evaluating spin in literature. We attempted to mitigate any bias by screening and evaluating spin independently, and then reconciling any disagreements using a third author. Scoring spin of the abstract was performed by comparing the data found in abstract to data found in full text, helping to identify any disagreements and score spin. In addition to AMSTAR 2 evaluation which evaluated spin of full text, we chose to appraise spin of abstracts as clinicians often base their clinical recommendations only on the findings of an abstract.9,37 Lastly, five of the included articles were published before the PRISMA statement was published in 2009 and PROSPERO registration was established in 2011. Because these studies could not have utilized the guidelines or registered their research methods through these now widely used research protocols, this may have distorted our analysis related to PROSPERO registration. It is important to mention that while spin is critical to assess, bias found specifically in abstracts may be inadvertent in nature if subject to the restrictions and limitations of a given journal. One reason for unintentional spin may be restrictions on word count for abstracts, which may lead authors to cover compelling, but not necessarily significant findings. Additionally, while important for providing context to the validity of results, authors may not deem discussion of risk of bias as important if limited to a word count.
Evaluating and identifying patterns of spin in literature of MD is critical for promoting a general understanding of a procedure’s efficacy and safety compared to emerging minimally invasive techniques. Quantifying spin in literature is helpful for building higher quality designs for future studies, as researchers can find ways to mitigate forms of bias and thus improve the quality of studies. 9 Our goal in conducting such an investigation was to illuminate the prevalence of spin in literature so that clinicians and researchers are more well-informed about diverse presentations of bias and are more prepared to critically evaluate literature before making clinical recommendations.
Conclusion
This study identified that bias found within the scientific literature, otherwise known as spin, is highly prevalent in the abstracts of systematic reviews and meta-analyses that investigate the outcomes of surgical interventions for treating symptomatic LDH. Misleading reporting is the most common category of spin in literature related to MD. More specifically, spin overwhelmingly tends to go in the positive direction, with results inappropriately favoring the beneficial effect or safety of an experimental intervention. Studies absent of registration with PROSPERO had a higher incidence of spin. Future efforts should be directed towards guidelines that highlight the importance and need for increased representation of abstract results that are otherwise discussed in full text. Additionally, future research should also focus on the effects of spin on clinical recommendations and practices related to treatment of lumbar disc herniation.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Frank A. Petrigliano has received consulting fees from Stryker, Exactech, Micromed, Smith and Nephew, Zimmer Biomet, Flexion Therapeutics, and Depuy Synthes. Joseph N. Liu has received consulting fees from Stryker, Smith and Nephew, Mertz North America Inc., and Neuracrine Biosciences. Liu also serves as a board member to American Shoulder and Elbow Surgeons and Arthroscopy Association of North America. Ram K. Alluri receives stock options for MiRus and HIA and is a consultant for MiRus and prior consultant for Eccential. Alluri is also a board member for HIA and is deputy editor of Global Spine Journal.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
