Abstract
Study Design
Systematic Review.
Objectives
Systematic reviews and meta-analyses that aggregate data on the intrawound application of vancomycin for surgical site infection (SSI) prophylaxis in spine surgeries are at an increased risk of spin, the overstating of beneficial effects of an intervention. The purpose of this study was to identify studies that coalesce outcomes of intrawound vancomycin SSI prophylaxis in spine surgeries and define the prevalence and types of spin in the identified literature. Secondarily, this study aimed to identify patterns within study characteristics that were associated with certain spin types.
Methods
This study was conducted using the Preferred Reporting Items for Systemic Reviews and Meta-Analyses guidelines. A search was performed in 4 databases (PubMed, SCOPUS, Medline and Cochrane) for systematic reviews and meta-analyses on intrawound vancomycin use for SSI prophylaxis in spine surgeries. Two authors independently assessed studies for inclusion criteria and then aggregated study characteristics: titles, publication journal and year, authors, level of evidence, etc. Each study was subsequently evaluated for the presence of 15 different spin types. Statistical analysis was performed for patterns between spin prevalence and study characteristics.
Results
From the database search, 18 studies met the inclusion criteria. In total, 13 studies (72%) were found to have spin. The most common types of spin identified were spin type 9 (“Conclusion claims the beneficial effect of the experimental treatment despite reporting bias”), found in 9 (50.00%) studies and type 3 (“Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention”), found in 7 (38.89%) studies.
Conclusions
There is a high prevalence of spin in systematic reviews and meta-analyses on the intrawound application of vancomycin for SSI prophylaxis in spine surgery. Our analysis demonstrated that studies tended to selectively report positive findings while minimizing negative outcomes within abstracts.
Introduction
Spine surgery has become an increasingly mainstay mode of intervention to treat patients with spinal pathologies. Trends in surgery in the United States show that yearly there are over 1 000 000 spinal surgeries performed, with that number projected to increase.1,2 As with any surgical modality, surgical site infection (SSI) is a potential complication of spine surgery, of which
Current practice is non-standardized for the prophylactic use of antibiotics for spine surgery patients in preventing SSI. Despite the use of prophylactic antibiotics, the overall rate of SSI reported by the North American Spine Society ranges between 0.7% and 10%. However, the rate of infection differs when accounting for comorbidities, ranging from 0.7% to 4.3% for patients without comorbidities and 2.0% to 10% for patients with comorbidities. It is important to note that amongst pooled studies, different antibiotic regiments were used, varying in which antibiotics and whether they were given preoperatively, postoperatively, or both. 6
Prophylactic perioperative use of vancomycin at the surgical site wound is a debated strategy that has been touted to decrease SSI. Suggested guidelines include the application of local powdered vancomycin at the surgical wound before closure in addition to intravenous antibiotics.7,8 The application of powdered vancomycin aims to provide adequate antibiotic concentrations at the surgical site while minimizing possible systemic adverse effects. Some studies have shown that intrawound vancomycin powder does reduce overall SSI rates, while further analyses demonstrate that its application does not reduce overall SSI rates but rather just those of
Clinical studies have demonstrated conflicting evidence regarding postoperative SSI rates, and grouped analyses of these studies are prone to spin. In scientific literature, spin refers to the practice of distorting results by reporting positive claims while minimizing negative findings. 10 Spin is particularly evident in abstracts where study synopses purport favorable results while neglecting negative findings. Physicians often refer to abstracts of studies to inform patient care due to limited time for reading, making spin a potentially hazardous practice. 11 Thus, it is necessary to minimize spin during the writing of research studies and during the peer-review process to reduce its potentially detrimental effect on clinical practices.
The goal of this study was to identify the prevalence of spin in abstracts of systematic reviews and meta-analyses on the use of prophylactic intrawound vancomycin for SSI prevention in spine surgery. Further, this study sought to define and categorize spin types, as outlined by Yachitz et al., within these studies. 10 The secondary objective was to identify any potential associations of spin with study characteristics.
Methods
This study was conducted following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. 12 A search was conducted on PubMed, Scopus, Medline, and Cochrane databases on January 22, 2024, using combinations of key search terms, “vancomycin”, “powered”, “spine”, and “surgery”, as well as specific searches for the phrases: “vancomycin spine surgery”, “vancomycin spine operation”, “vancomycin spine procedure”, “vancomycin powder spine surgery”, and “vancomycin spine surgery organism”. Primary journal articles, systematic reviews, and meta-analyses that reported patient outcomes following spine surgery where powdered vancomycin was used for intra-wound SSI prophylaxis published in peer-reviewed journals were eligible for analysis. References were excluded if they were not published in a peer-reviewed journal, in English, were cadaveric or biomechanical studies, or where the primary outcome was not centered on patient-reported outcomes. Two independent reviewers (ST and MP) screened studies for inclusion in 2 phases; first by title and abstract, and next by full text. If any conflicts arose, a third author (AF) would make a final decision on inclusion or exclusion.
Upon identifying references, 2 independent reviewers (ST and MP) extracted data including study title, year of publication, journal of publication, authors, level of evidence, study design, adherence to PRISMA guidelines, preregistration of study guidelines to PROSPERO, and funding sources. Adherence to PRISMA guidelines or preregistration to PROSPERO was determined if a study stated it clearly within their manuscript. In addition, reviewers noted 2022 Scopus CiteScores for references that were Scopus indexed and Clarivate Impact Factor 2022 for the journals in which studies were published. Studies were graded based on their Level of evidence (LOE) per the Journal of Bone & Joint Surgery guidelines and using AMSTAR grading. 13 Individual AMSTAR criteria were categorized as either a yes (1) or a no (0), with each criterion weighing equally and final aggregate scores collected.14,15
Reviewers were trained to identify the most common types of spin as outlined by Yavchitz et al.
10
Screening results were aggregated, and any discrepancies were resolved by an independent author. Aggregated data on spin type was analyzed for frequency. Reference characteristics and frequency of spin presence underwent statistical analysis using the chi-squared test or the Fisher exact test as deemed appropriate. Statistical significance was set at
Results
An initial search of PubMed, Scopus, Medline, and Cochrane databases led to 2646 studies, of which 1656 were removed as duplicates (Figure 1). Studies were then screened for titles and abstracts, of which 18 met our inclusion criteria for analysis.5,7,16-31 These 18 studies were sourced from 16 unique journals (Table 1). None of the references were primary journal articles, 15 (83.33%) were meta-analyses, and 3 (16.68%) were systematic reviews. The average year of publication of the 18 studies was found to be 2018, with all being published in the past decade. In terms of LOE, only 1 (5.56%) study was classified with a LOE of I, 8 (44.44%) with a LOE of II, 9 (50.00%) with a LOE of III, and none with a LOE of IV. Explicit statement of use of PRISMA guidelines was found in 12 (66.67%) studies and a statement of PROSPERO registration was found in 2 (11.11%) studies. The average Scopus CiteScore was 5.02 (σ = 1.52) and the average 2022 Clarivate Impact Factor was 2.5 (σ = 1.13). Using AMSTAR criteria, there was an average AMSTAR aggregate score of 10.06 (σ = 2.67). Study and journal characteristics can be found in Table 2. PRISMA diagram of study identification process. Study Characteristics of Included Studies. Study Characteristics, Quality Grading, and Journal Characteristics of Included Studies.
Spin Characteristics
Frequency of Spin Types and Categories.
AMSTAR 2 Rating of Reviewed Studies.
PICO, patient, intervention, comparison, outcome.
*RoB: Risk of Bias.
Discussion
Our study found that 72% of abstracts of systematic reviews and meta-analyses on prophylactic intra-wound vancomycin use in spine surgery contained spin. On average, these studies had more than 1 spin type per abstract, demonstrating not only a high prevalence but also high frequency in the use of spin within the literature on intrawound vancomycin use in spine surgery. Spin type 9 (“Conclusion claims the beneficial effect of the experimental treatment despite reporting bias”) and type 3 (“Selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention”) were found to be the 2 most prevalent, appearing in 50.00% and 38.88% of studies, respectively. “Misleading reporting” was the most common spin category. Given the high prevalence found within our study, there is a need to reduce spin within published scientific works during both the writing and review process. Furthermore, it is essential to make clinicians aware of the high prevalence and frequency of spin, so they can evaluate studies more critically to better inform their conclusions about intrawound vancomycin use for spine surgery SSI.
The use of spin within abstracts is a selective reporting of positive findings while not accurately representing possible limitations or biases that help inform the true value of a study’s conclusions. Dodson et al. demonstrate how spin type 9 and spin type 3 within abstracts lead to selective reporting of positive findings while downplaying limitations to external validity. In Dodson et al, authors pooled data from 21 studies and concluded in their abstract that “intrawound vancomycin powder is protective against SSI”. 30 However, within the manuscript, Dodson et al. describe confounding factors such as intrawound irrigation, predisposition to infection, and control group variability among study results. By not disclosing the liberty that the authors had when pooling study parameter outcomes, the initial conclusion within the abstract appears more scientifically sound than what is later revealed in the full text. 30 The presence of spin within Dodson et al. serves to highlight the potency of intrawound powdered vancomycin to reduce SSI while not divulging limitations to this claim. 30 Spin could have been avoided in the preparation of this manuscript by including these limitations within their abstract rather than solely highlighting the efficacy of intrawound vancomycin use. While later acknowledged, the lack of full disclosure within the abstract could prove to misinform physicians, as omitting pertinent data considerations in the abstract may overstate the external validity of the author’s conclusions. 27
The high incidence of spin amongst studies that reported patient outcomes on the use of prophylactic intrawound vancomycin for SSI prevention is not surprising, as there exists a growing body of literature identifying high rates of spin within orthopaedic studies.32-37 Previous work performed by Arthur et al. reported spin in 44% of abstracts in orthopaedic randomized control trials, with the most common being nonsignificant data being represented as significant. 32 More recent work published by Reddy et al. demonstrated in 2 studies that 36.4% of studies on rotator cuff tear treatment and 28.4% of studies on platelet-rich plasma contained spin.33,34 Reddy et al., in a similar manner to our study, found no association between study characteristics such as date of publication or impact factor ratings.33,34
A variety of factors may explain the prevalence of spin in orthopaedic studies, especially when evaluating meta-analyses and systematic reviews. Many of the studies reviewed drew from non-homogenous literature when performing their analyses. While pooling is used to strengthen the statistical power and generalizability of results in meta-analyses and systematic reviews, it comes with an increased risk of methodological flaws that can affect the validity of conclusions. Failure to account for the variability between studies’ uses of diagnoses, treatments, and outcome measures can lead to misleading conclusions that can both over- and underestimate effect sizes, over-generalize findings, and produce inconsistent findings. 38 As such, it is difficult for authors to glean overarching conclusions when pooling data from various sources, where definitions of outcomes, control and treatment groups, and treatment protocol can differ. Thus, it is an imperfect endeavor to group data into a tangible measure for clinical use. Furthermore, abstracts inherently have word limits, leading authors to focus on the positive aspects of their study. For systematic reviews and meta-analyses, this limits the ability to fully divulge how they handled heterogeneity or what limitations affect their findings. Both practices can lead to a misrepresentation of data which can impact patient care, especially as abstracts are commonly the primary section read by clinicians and abstracts alone have been shown to alter physician decision-making.11,39 With spin at such a high prevalence among these studies, it raises the possibility of misleading clinicians and affecting clinical decision-making. Therefore, it is essential that the orthopaedic research community work to reduce the presence of spin by being scrutinous in the review process about conclusions reported in abstracts and encouragement of increased transparency.40,41 This will facilitate improvement quality of care within spine surgery when it comes to prevention of SSI and whether or not intrawound vancomycin use should become standard of care.
This study is not without limitations. While efforts were made to conduct a thorough search of studies on intrawound use of vancomycin for SSI prophylaxis in spine surgery, it is possible that our search parameters did not capture all eligible studies. Additionally, we did not assess spin in non-English publications. Non-English literature is also likely used by clinicians when making decisions and our search did not capture this population of studies. Furthermore, the review process is inherently subjective in determining the presence of spin. To reduce this, we had 2 authors independently review papers for spin, with a third clarifying any disagreements. Our study was also limited by the low number of studies that were ultimately yielded by our database search. Thus, we lacked enough statistical power to analyze associations between all spin types and study characteristics and could only do so for the most prevalent spin types.
Conclusion
Spin is highly prevalent in the systematic review and meta-analysis literature of prophylactic intrawound vancomycin use for spine surgery, identified in 72% of abstracts. Physicians should be mindful of this high prevalence and evaluate abstracts critically, turning to the full text when necessary, when using the literature to inform their clinical decision-making on the use of intrawound vancomycin to prevent SSIs in spine surgery.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
