Abstract
Study Design
Narrative Review
Objectives
The objective of this study was to compare publication status of clinical trials in adult spine surgery registered on ClinicalTrials.gov by funding source as well as to identify other trends in clinical trials in adult spine surgery.
Methods
All prospective, comparative, therapeutic (intervention-based) trials of adult spinal disease that were registered on ClinicalTrials.gov with a start date of January 1, 2000 and completion date before December 17, 2018 were included. Primary outcome was publication status of published or unpublished. A bivariate analysis was used to compare publication status to funding source of industry vs non-industry.
Results
Our search identified 107 clinical trials. The most common source of funding was industry (62 trials, 57.9% of total), followed by University funding (26 trials, 24.3%). The results of 76 trials (71.0%) were published, with industry-funded trials less likely to be published compared to non–industry-funded trials (62.9% compared to 82.2%, P = .03). Of the 31 unpublished studies, 13 did not report any results on ClinicalTrials.gov, and of those with reported results, none was a positive trial.
Conclusions
Clinician researchers in adult spine surgery should be aware that industry-funded trials are less likely to go on to publication compared to non–industry-funded trials, and that negative trials are frequently not published. Future opportunities include improvement in result reporting and in publishing negative studies.
Introduction
Surgery for the adult spine has the potential to alleviate pain and improve quality of life in appropriately selected patients. Given an aging population as well as advances in surgical technique and implant selection, the number of surgeries performed each year is increasing.1,2 Concurrently, non-operative treatment options for spine pathology are also increasing in number, timing, and complexity. 3 With the increase in the number of operative and non-operative treatment options comes an increasing responsibility for the surgeon to understand the risks and benefits of each particular treatment. Ideally, this understanding is based on evidence that has proven that one clinical decision is superior to another. Such evidence is often derived from prospective clinical trials.
Within adult spine surgery, prospective clinical trials have had a profound impact on how patients are treated. 4 For instance, the Spine Patient Outcomes Research Trial (SPORT) aids clinicians in the management of patients with intervertebral disc herniation, degenerative spondylolisthesis, and stenosis.5–7 Yet not all prospective clinical trials in adult spine surgery are equal in design. There is significant variability in study parameters including randomization, blinding, and crossover. The clinical questions that a given trial aims to answer may be related to a pharmacological, procedural, device, or behavioral intervention.
The source of funding varies from study to study as well. For instance, SPORT was government-sponsored, while other influential trials have received industry support. Berenson et al. performed a prospective-randomized trial funded by Medtronic Spine. Their results suggested that balloon kyphoplasty for painful vertebral compression fractures in patients with cancer may improve self-assessment of disability compared to non-surgically treated patients. 8 In addition to trial design and funding, the results of some trials are published, while others are not. While the reason for this is multifactorial, certain studies in spine surgery have found an association between failure to publish and source of funding, with the results of industry-funded studies less likely to be published compared to the results of non–industry-funded studies. 9 Other studies suggest that there is no such association between failure to publish and source of funding. 10
In light of the complexity of interpreting prospectively collected data and the previous, the objective of this study was to compare publication status of clinical trials in adult spine surgery registered on ClinicalTrials.gov by funding source as well as to identify other trends in clinical trials in adult spine surgery. We hypothesized that studies funded by industry were less likely to be published than those with non-industry funding.
Materials and Methods
Data Source
A search of trials registered on ClincalTrials.gov was conducted with the search terms “spine” and “spinal”
Inclusion/Exclusion Criteria
Inclusion criteria were prospective, therapeutic (intervention-based) clinical trials with a start date after January 1, 2000 and a completion date before December 17, 2018 that investigated the effect of a drug, procedure, medical device, or behavioral intervention on pathology of the adult spine. Exclusion criteria were pediatric trials (patient age less than or equal to 18 years), observational studies, trials completed after December 17, 2018, and trials that were conducted outside the United States.
Data Collection and Analysis
Variables recorded were trial title, national clinical trial (NCT) identification number, year of trial start, year of completion, and, if results were published, the journal in which results were published. Outcome variables assessed were spinal pathology for which the trial was performed (lumbar degenerative disease, lumbar disc herniation, cervical degenerative disease, adult deformity, vertebral compression fracture, infection, kyphosis, pain, neuromonitoring, osteoporosis, other or mixed disease), spinal level for each procedure (cervical, lumbar, or multiple; no trial focused on thoracic pathology exclusively), trial design (randomized or not randomized, blinded or not blinded, parallel or crossover), source of funding, intervention type (drug, procedure, device, behavioral, other), procedure type (lumbar fusion, lumbar decompression, anterior cervical decompression and fusion) number of enrolled patients, trial phase, publication status (published or not published), and number of citations for original research articles that were published based on trial results. Publication status was determined by a PubMed search of trial titles and principal and/or co-principal investigator names. Number of citations was obtained by a Google Scholar search performed on March 27th, 2021. If trial results yielded more than one publication, we selected the publication with the highest number of citations. For unpublished results, we recorded whether results were available on ClinicalTrials.gov, the result type (positive trial, negative trial, no significance reported), the percent of patients starting but not completing the study, follow-up timing, and whether or not any primary or secondary outcome variable was recorded. Because we were not comparing clinical outcomes, we did not assess risk of study bias or certainty in the body of evidence.
Statistical analysis were descriptive statistics of the frequency of each outcome variable and the percentage of total that each outcome variable represented. Fisher’s exact test was utilized to determine the significance of association for categorical variables between groups. A one-way analysis of variance (ANOVA) test was used to compare means among multiple (greater than 2) groups. A P-value of less than .05 was considered statistically significant.
Results
Included Studies
Our search revealed 107 prospective, comparative, intervention-based clinical trials in adult spine surgery that were completed in the United States between January 1, 2000 and December 17, 2018. (Figure 1) The year with the most trials completed was 2012 (14 trials), and the year with the least was 2004 (zero trials). The most common source of funding was industry (62 trials, 57.9% of total), followed by intramural or University funding (26 trials, 24.3%). The NIH funded 11 trials (10.3%). There was a mean of 155.5 ± 211.5 enrolled patients per trial (range 6 to 1874). (Table 1) Flow diagram of included studies. Funding Source of Clinical Trials.
Intervention Type
Intervention Type by Funding Source.
*Chi-squared analysis, P-value of less than .05 is considered significant
Clinical Trial Design
Clinical Trial Study Characteristics.
Clinical Trial Publication Outcomes
Publication Outcome of Adult Spine Surgery Clinical Trials.
Comparison of Trials Based on Publication Status.
*P-value less than .05 is considered significant.
Result Reporting for Unpublished Clinical Trials.
The total number of unpublished trials was 31; NIH: National institute of health
The most common journal of publication was Journal of Neurosurgery: Spine (16 publications) followed by Spine (15 publications). Six trials were published in The Journal of Bone and Joint Surgery, and 4 were published in The New England Journal of Medicine.
Comparison of Number of Citations per Publication Based on Funding Source.
*P-value less than .05 is considered significant.
Abbreviation: NIH, National institute of Health; SD, Standard deviation
Procedure Type, Pathology, and Spinal Level by Publication Rate.
Abbreviation: ACDF, anterior cervical decompression and fusion.
*Chi-squared analysis. P-value less than .05 is considered significant. **No trial focused exclusively on thoracic spinal pathology.
Discussion
Our hypothesis that clinical trials funded by industry were less likely to go on to publication than trials that were not funded by industry was validated. In this current analysis, NIH-funded studies had significantly higher rates of publication compared to industry-sponsored research, which was more likely to go unpublished. Prior studies have demonstrated a significant association between funding source and publication of clinical trials for spine surgery, with trials funded by industry less likely to be published. 9 Industry-sponsored research can benefit the field of spine surgery, but it can also bias both publication of results as well as presentation of data and its interpretation. 16 In general, industry-sponsored research is associated with more favorable efficacy results, more favorable study conclusions, and lower agreement between study results and conclusions when compared to non–industry-funded research. 17 The physician tasked with interpreting and applying clinical research to treatment decisions must be aware of these associations. The significant association we found between funding source and number of citations per publication, with NIH-funded studies more likely to be cited than industry-cited studies, may suggest that clinician researchers are aware of such associations and put more faith in data from non–industry-funded studies.
We do emphasize that investigators can and do perform high-quality clinical trials regardless of funding source. SPORT for intervertebral disc herniation was government funded and has produced numerous impactful publications. It is a multi-center, prospective, randomized study of 501 patients with minimum 2-year follow-up and a primary endpoint of 2 commonly used patient-reported outcome measures (the 36-item Short-Form Health Survey bodily pain and function scales, and the Oswestry Disability Index). 18 The most frequently cited industry-sponsored study, the Cancer Patient Fracture Evaluation (CAFE) study, cited 426 times, is also a prospective-randomized trial, with 134 patients, and a follow up of 1 month, which is an adequate timeframe for cancer patients for whom pain relief is important. 8 Another example of a quality industry-funded trial is the Prestige Cervical Disc Study, which enrolled 541 patients at 32 investigational cites and reported results at 60 months. It is the second most-commonly cited industry-funded study. 19
Our data also showed that failure to publish trial results was common (71.0% of trials published, 29.0% of trials unpublished). Other orthopedic subspecialties including adult arthroplasty and orthopedic trauma have reported similar publication rates for trials registered at ClinicalTrials.gov.20,21 Low publication rates raise a concern for publication bias, as studies with positive results are more commonly published than studies with negative results, with 1 analysis showing that positive studies had a 3.3 times higher rate of being published when compared to negative studies.16,22 Our data reflects this propensity to withhold negative results. Of the 31 clinical trials that were unpublished, 13 did not report any results, none were a positive study, eight were negative, and 10 reported results but did not report whether the study was positive or negative (Table 6). When studies do reach publication, they tended to be published in journals with high impact factors (Table 4). This may indicate that the studies we identified are in general higher quality than other studies that reach publication.
Not only is publication bias a problem from an ethical standpoint, but it also affects future research, as meta-analyses adopt the biases of the data that they analyze.
23
In turn, physicians may interpret data incorrectly and make treatment decisions that data does not in fact justify. Finally, if results from negative studies are not published, other investigators may waste valuable resources recreating research that has already been performed. There are numerous potential methods to increase the publication rate of negative trials. Based on our data, one potential solution is to separate study sponsorship (funding) from implant, device, or drug companies, which would allow investigators to collect, analyze, and publish data without influence of industry. Other methods are for leaders in the field of spine surgery research to publish opinion pieces or letters to the editor stressing the importance of publishing negative trials. Journal editors could publish special issues (supplemental issue, for example) dedicated to negative trials. Finally, a task force or new branch of the NIH could be formed as a watchdog group that ensures negative results from trials registered on clinicaltrials.gov are published
The current study identified trends in design, methodology, and result reporting of prospective clinical trials that investigate treatment for adult spinal pathology. These trends help clinician-researchers appreciate the state of evidence in adult spine surgery and anticipate future development in it. While analyses of trends in prospective clinical trials have helped investigators in other specialties understand their respective fields, our study is the first to do so in a comprehensive manner for the field of adult spine surgery.11,24–28 Other investigations in the field of spine surgery have been more limited in scope. For example, Ohnmeiss et al. analyzed 72 spine-related trials from ClinicalTrials.gov to determine their fate with regard to publication, but did not determine important aspects including trial design, randomization status, blinding, and industry influence on results. 9 Son et al. investigated publication bias for spine-related studies registered on ClinicalTrials.gov, but likewise did not describe important elements of trial design. 29 Our data shows a high rate of randomization (77.6% of trials), a low rate of blinding (68.2% of trials), and trials designed more commonly as parallel as opposed to single group or crossover studies (75.7% compared to 21.5% and 2.8%, respectively).
One weakness of this current study is the heterogeneity of our data. For example, the number of spinal pathologies present within our study was quite variable such that the most common pathology, lumbar spinal stenosis, was studied in only 32 trials (30.0%) and there were no studies that focused exclusively on thoracic spinal pathology. Moreover, within a certain pathology or within a certain intervention category, there was heterogeneity of data. Within the behavioral intervention category alone, treatments ranged from different iterations of physical therapy to yoga to chewing gum. Over the timeframe that we investigated, surgeries and procedures were more commonly investigated than behavioral interventions and drugs. We believe the heterogeneity revealed by our data is useful in its own right, as it shows the broad spectrum of spinal pathology and the even broader spectrum of attempts to address it. Furthermore, this study was limited to adult patients only in an effort to maintain sample homogeneity and may not be applicable to a pediatric population. Another weakness of our study is that data from ClinicalTrials.gov may not be completely accurate, as the National Library of Medicine does not have an established means of policing accuracy of data entry, which is dependent on the principal investigator. We associated trials that went on to publication with publication titles on PubMed, which helped mitigate this weakness. Another weakness is our reliance on PubMed to identify trials that went on to publication, as this may fail to capture studies not indexed in this registry. However, PubMed is one of the most used and dependable libraries of published medical literature. 17 Finally, we did not have data regarding the amount of funding received per trial. Future research may be directed toward identifying reasons for not publishing data as well as limitations in study design, such as barriers to randomization and other aspects of trial design that can improve the quality of a given study. Future opportunities may be in fields that are less well-represented in the current landscape of prospective trials, such as adult spinal deformity.
Conclusion
Clinician researchers in adult spine surgery should be aware that industry-funded trials are less likely to go on to publication compared to non–industry-funded trials, and that negative trials are frequently not published. Future opportunities include improvement in result reporting and in publishing negative studies.
Supplemental Material
sj-pdf-1-gsj-10.1177_21925682211073313 – Supplemental Material for Clinical Trial Quality Assessment in Adult Spinal Surgery: What Do Publication Status, Funding Source, and Result Reporting Tell Us?
Supplemental Material, sj-pdf-1-gsj-10.1177_21925682211073313 for Clinical Trial Quality Assessment in Adult Spinal Surgery: What Do Publication Status, Funding Source, and Result Reporting Tell Us? by Nicholas C. Danford, Venkat Boddapati, Matthew E. Simhon, Nathan J. Lee, Justin Mathew, Joseph M. Lombardi, Zeeshan M. Sardar, Lawrence G. Lenke and Ronald A. Lehman in Global Spine Journal
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
IRB Statement
This study utilized national, de-identified data and is exempt from IRB review.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
