Abstract
Background:
Systematic reviews on the use of platelet-rich plasma (PRP) in orthopaedic surgery are abundant in current published literature. However, a beautification of results (referred to as spin) has been noted in abstracts across various aspects of medicine.
Purpose:
To determine the prevalence of spin in systematic reviews of PRP-related orthopaedic surgery abstracts.
Study Design:
Cross-sectional study.
Methods:
Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and Murad and Wang guidelines, we conducted a search in Medline, Embase, and the Cochrane Database for reviews on PRP-related orthopaedic surgery. The search included studies published from inception until June 30, 2021. Included were systematic reviews written in English that involved the use of PRP in the treatment of orthopaedic injuries in human participants. The abstracts of the included reviews were evaluated for the top 9 types of spin as described by Yavchitz et al in 2016. We determined the relationship between spin and study characteristics using odds ratios.
Results:
Of an initial 1560 studies, 176 were included. We found that 50 studies (28.4%) contained at least 1 form of spin. The 2 most common forms of spin found in our sample were type 5 (“Conclusion claims the beneficial effect of treatment despite high risk of bias”; n = 27 [15.3%]) and type 3 (“Selective reporting or overemphasis of efficacy in outcomes favoring beneficial effect of intervention”; n = 18 [10.2%]). No statistical significance was found between study characteristics and the presence of spin.
Conclusion:
Spin was present in 28% of the systematic reviews that covered PRP-related orthopaedic treatments. Spin was not associated with general study characteristics, including adherence to PRISMA guidelines or funding. Journals and authors should be aware of spin in articles and avoid its usage.
Platelet-rich plasma (PRP) and other biologics are currently topics of high interest in orthopaedic surgery, providing a nonoperative option to degenerative disease. 1,8,19 There is tremendous variability in the results of PRP studies showing no significant benefit and the contrary. 3,22 As a result, the literature is constantly being updated with varying levels of evidence. Since there is a constant influx of new literature on the topic, multiple recent systematic reviews have been published on the topic. 10,12,20 As PRP injections are expensive, systematic reviews can help health care providers understand whether a procedure is cost-effective for the patient. 2
When clinicians are interpreting the results of systematic reviews, they need to be aware of a form of misrepresentation called spin. Spin manifests as beautification of results, whether intentional or unintentional, that can occur for a variety of reasons, ranging from funding entity pressures to improving the publishability of the manuscript. 9,13,14 This is problematic, as spin can prevent clinicians from fully interpreting the results of a study. Boutron et al 4 found that physicians interpreting results of a statistically nonsignificant study that contained spin were more likely to find it beneficial and less rigorous. This showed that spin in abstracts influenced clinical interpretation of study results.
Spin has been shown to be prevalent in the systematic reviews of various medical treatments and interventions. 7,11,15,18,23,24,26 In a 2016 study, Yavchitz et al 30 created a classification system for the presence of spin in the abstracts of systematic reviews. These investigators determined the top 9 types of spin based on severity (Table 1). Evaluating for spin is essential, as high-level systematic reviews are the cornerstone for recommendations within the American Academy of Orthopaedic Surgeons clinical practice guidelines, which aid in clinical decision making.
Top 9 Types of Spin According to Yavchitz et al 30
The purpose of the current study was to evaluate the presence of spin within the abstracts of systematic reviews on PRP-related orthopaedic treatments. We hypothesized that >25% of the included abstracts would have some type of spin.
Methods
Search Strategy
When starting this cross-sectional analysis, we searched Medline, Embase, and Cochrane Database for systematic reviews on PRP-related orthopaedic surgery. The search was performed on June 30, 2021, and included all studies until that date. Our search string is contained in Figure 1. The study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and the Murad and Wang 21 guidelines for meta-epidemiological studies.
Search string used in the database queries.
Inclusion Criteria
Studies meeting the following criteria were included: (1) systematic review design, (2) use of PRP in the treatment of orthopaedic injury, (3) publication in English, and (4) only human participants. Our search yielded 1560 studies, of which 608 were duplicates. Of the 952 articles that remained, 209 were included in the full-text screen. Our final sample comprised 176 systematic reviews related to PRP use in orthopaedics. Figure 2 summarizes the study screening process.
Flowchart of study selection. PRP, platelet-rich plasma.
Data Extraction
Two investigators (K.L. and A.W.) extracted the data for the study using a pilot-tested Google form. Both investigators are published authors on the topic of spin and well versed in identifying spin in systematic reviews. 18,23 The following characteristics were extracted for each study: (1) date of review search, (2) type of intervention/treatment (area of the body), (3) adherence to PRISMA guidelines, (4) funding source, (5) continent of origin of the first author, and (6) impact factor of the publishing journal
In addition, the investigators determined the presence of spin in the abstract of each study. Specifically, the presence of spin was determined after evaluating the full text, based on previously published literature in various aspects of medicine, 7,11,15,18,23,26 and the type of spin was determined according to the classification by Yavchitz et al. 30 Any disagreements between the investigators were resolved by the senior authors (M.V. and M.H.).
Statistical Analysis
This study used descriptive statistics to explain the results of the data extraction. One investigator (M.H.) statistically analyzed the data using frequency counts and percentages. We also calculated unadjusted odds ratios with 95% CIs to determine whether there was a relationship between the types of spin in the abstract and the study characteristics extracted. Statistical significance was defined as 95% CI ranges that did not overlap 1. All analyses were conducted using Stata 16.1 (StataCorp).
An a priori power analysis using GPower 3.1.9.7 indicated that a sample of 185 systematic reviews would be needed to have sufficient power.
Results
Of the 176 studies, 129 (73.3%) reported adherence to PRISMA guidelines. The largest funding source for our sample was public funding (n = 17; 9.7%); a majority of the studies either did not mention funding (n = 75; 42.9%) or had no funding at all (n = 52; 29.7%). A summary of the study characteristics is shown in Table 2.
General Characteristics of the Included Studies (N = 176) and Results of Logistic Regression Analysis a
a Dashes indicate not applicable. OR, odds ratio; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
The abstracts of 50 studies (28.4%) contained at least 1 form of spin. The 2 most common forms of spin found in our sample were type 5 (“Conclusion claims the beneficial effect of treatment despite high risk of bias”; n = 27 [15.3%]) and type 3 (“Selective reporting or overemphasis of efficacy in outcomes favoring beneficial effect of intervention”; n = 18 [10.2%]). Spin types 1, 4, 7, and 8 were not found within our sample (Table 3). Our analysis of unadjusted odds ratios between spin and the study characteristics indicated no statistically significant association for any characteristic (Table 2). We did not perform multivariable logistic regression because our sample size of 176 did not have enough power.
Frequency of Spin According to Type
a Safety was not mentioned in the full text of 133 articles (75.57%).
b Three articles (1.70%) assessed but did not discussed bias within the full text, and 37 (21.02%) did not assess bias within the full text.
c Safety was not assessed in the full text of 90 (51.11%) articles.
d Bias was not reported in the full text of 79 (44.89%) articles.
Discussion
The study findings indicated that, after analysis of the full text, 28% of abstracts within our sample contained at least 1 form of spin. The most common forms of spin found within the studies were type 5 (conclusions are drawn despite high risk of bias) and type 3 (selective reporting of or overemphasis on outcomes favoring beneficial effects) at 15% and 10%, respectively. These findings are similar to other studies that examined spin in randomized controlled trials of medical interventions and treatments. 16,25,29 Within orthopaedics, a study evaluating spin in randomized controlled trials for lower extremity joints identified almost 60% spin in abstracts. 6 In addition, our findings revealed that spin was not associated with any of the characteristics gathered from each study, indicating that all systematic reviews related to orthopaedic use of PRP are susceptible to spin.
Type 5 was the most common type of spin within our sample. Misinterpretation can occur from this form of spin as a result of readers misinterpreting the strength of the study by not knowing the risk of bias. 28 An example of this is the study by Tang et al. 27 Their abstract states that “PRP was associated with more improvement in pain intensity and function in the long term than were the comparators.” However, a review of the full text indicated that the authors failed to mention in their abstract that there was a high or unclear risk of bias in 18 (90%) of the 20 included studies. Not placing the information in the abstract within the context of the risk of bias limits the reader’s ability to make sound judgments regarding the nature of the overall results.
An example of spin type 3 is in the study by Li et al, 17 who conducted a systematic review and meta-analysis on patient outcomes of PRP after rotator cuff repair. The abstract states, “The current evidence shows that using PRP in arthroscopic rotator cuff repair can improve pain levels and functional outcome scores while reducing the retear rate after surgery.” However, a review of the full text indicated that there were statistically nonsignificant results for the Disabilities of the Arm, Shoulder and Hand questionnaire score, Simple Shoulder Test score, and range of motion. Thus, spin type 3, in which information is omitted from the abstract, can be misleading for readers and distort important perspectives for clinical practice.
The study by Yavchitz et al 30 helps with identifying the different types of spin found within the abstracts of the literature. After that publication, multiple studies were published on the presence of spin within systematic reviews for different treatment options, including orthopaedics. 7,11,15,18,23,26 Carr et al 5 published an article in Foot & Ankle Orthopaedics looking at the presence of spin within the systematic reviews of treatment options for Achilles tendon ruptures. The authors identified the presence of spin in 65% of their studies. Carr et al determined that the 2 most common forms of spin in their sample were types 3 and 5, similar to our results. In a study published by The Journal of Shoulder and Elbow Surgery, Jones et al 15 evaluated spin within systematic reviews on the treatment of proximal humeral fractures. Their study revealed a spin rate of 34%, with types 3 and 5 also being the most common. In addition to our study, spin studies related to orthopaedics are showing that there needs to be a push toward identifying types 3 and 5 within abstracts to prevent a large portion of spin from happening there.
We have multiple recommendations to help reduce spin within abstracts, especially types 3 and 5. First, journal editors, reviewers, and authors need to be educated on the subject of spin. By understanding spin, editors and reviewers can identify it within abstracts while screening through submitted manuscripts and ask authors to remove it from abstracts during the revise-and-resubmit process, before acceptance and publication. In addition, authors can help with limiting spin by being conscientious and not adding it to their abstracts. Second, journals can help with the process of limiting spin by adding a high word count to their journal abstracts. By allowing more words in the abstract, there will be fewer limitations to what authors can state within it, permitting greater transparency and reducing spin.
Additionally, journals can add restrictions to their author guidelines targeting spin-type language—specifically types 3 and 5, the most commonly seen in the orthopaedic literature. For instance, journals can require authors to state all main outcomes within their abstracts and thereby avoid spin type 3, which occurs by selecting only statistically significant outcomes. To avoid spin type 5, journals can require authors to state the high risk of bias within their literature whenever stating beneficial outcomes. Finally, one of the findings of this study was that there was no correlation between our study characteristics and the presence of spin; thus, authors and journals must be aware that any study can be susceptible to spin bias within abstracts. By following our recommendations, researchers and journals can identify spin at an early stage and be able to eliminate it by the time an article is published. 5,15,18
Limitations
Limitations to our study include spin being a subjective topic. Although we have published studies related to spin and undergone training to identify spin, the topic is inherently open for interpretation. Also, our study lacked power to conduct a logistic regression. Finally, our results are not generalizable given the cross-sectional design of the study.
Conclusion
The findings of this study indicate that that spin was present in 28% of the systematic reviews that cover PRP-related orthopaedic treatment. We also found that spin was not associated with general characteristics, including adherence to PRISMA or funding. We recommend that journals and authors be aware of spin when reviewing articles and avoid its usage.
Footnotes
Acknowledgment
A special thanks goes to Dr Brent Norris for allowing the primary author to do his research year under the Oklahoma State University Orthopaedic Surgery Program.
Final revision submitted August 22, 2022; accepted August 30, 2022.
One or more of the authors has declared the following potential conflict of interest or source of funding: J.S. has received education payments from Pylant Medical. C.D.H. has received education payments from Pylant Medical and hospitality payments from Stryker. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval was not sought for the present study.