Abstract
Background
Studies on factors affecting editorial decisions of scientific journals are scarce. In this study, we focused on case reports submitted to oncology journals and analyzed whether their nature or other relevant variables affected the chances of their acceptance.
Methods
We analyzed case reports submitted to 2 oncology journals: Tumori Journal and The International Journal of Biological Markers, and split them into 3 predefined groups: those (a) describing rare or unusual presentation of diseases, (b) describing the side effects of an intervention or (c) describing the success of a novel intervention. Publication status was retrospectively retrieved from the submission system, and acceptance rates were calculated taking into account other variables including geographic location of corresponding author.
Results
A total of 326 case reports were suitable for analysis. The acceptance rate was 35.4% for group (a), 27.9% for group (b), 19.6% for group (c) (p = 0.01). After correcting for other variables, the odds ratio (OR) of being accepted for group (c) was 0.58 (95% CI, 0.33-1.00) compared with the other groups combined. There was a highly significant difference of acceptance rates between manuscripts with authors coming from developed vs. developing countries that remained significant (OR = 5.94; 95% CI, 3.05-10.09) after correcting for multiple variables.
Conclusions
The nature of a case report in oncology may affect acceptance rate, with case reports describing successful approaches or side effects of treatment being accepted with a higher frequency then case reports describing a rare clinical or diagnostic scenario. Also, works coming from developed countries are accepted significantly more frequently than case reports coming from developing countries.
Introduction
Despite its ups and downs (1, 2), editorial evaluation and peer reviewing of manuscripts remains the backbone of scientific journals. Several studies over the past several years have evaluated whether there is a bias in peer review based on the positive or negative outcome of randomized control trials (3), the behavior of US vs. non-US peer reviewers (4), the type of industry involvement (5) and others, but evidence remains relatively scarce when case reports are concerned.
Therefore, in this study we approached the issue from a different angle and focused our attention on case reports, which represent a relatively homogenous type of article, generally do not have industry support at variance with clinical trials and can be broadly categorized into different groups.
We tested the hypothesis that the country of affiliation of the corresponding authors and/or the nature of the case reports could per se influence the final outcome of the editorial decision (acceptance or rejection) of case reports.
Methods
Study Protocol
The full text of manuscripts labeled as “case reports,” received by the Editorial Offices of the Tumori Journal (TJ) and The International Journal of Biological Markers (IJBM) for which a decision had been made between January 1, 2015, and June 30, 2016, were selected for the study.
This interval was chosen as it represented a time when there were no changes in the Editorial Board of either journal and particularly in the editors of the sections of reference for the case reports submitted.
Countries of the corresponding author were collected, and cases were classified by at least 2 independent evaluators into 3 broad categories and a total of 9 subcategories, according to the following classification:
1a. Report of a rare disease;
1b. Report of a rare presentation of a disease or of an elusive diagnosis or of an unusual biomarker(s) phenotype/genotype;
2a. Report of successful diagnosis or successful predictive use of markers;
2b. Report of successful treatment (drug);
2c. Report of successful treatment (device, radiotherapy, other);
2d. Report of successful treatment (surgery);
3a. Report of a side effect of a treatment (drug);
3b. Report of a side effect of a treatment (device, radiotherapy, other);
3c. Report of a side effect of a treatment (surgery).
When the report was dealing with different scenarios (e.g., successful novel treatment of a relatively rare disease), the evaluators allocated the case to the prevalent category. In the few cases of discrepancy between the 2 evaluators regarding the classification, the case would be evaluated by a third evaluator.
Countries of corresponding authors where clustered into 2 predefined categories: Developed Countries (United States, Canada, Western European countries, Japan, Australia and New Zealand) and Developing Countries (Mexico and Latin America, Eastern European countries, other Asia Pacific countries), to evaluate whether the country of the corresponding author could influence the acceptance rate.
Data Analysis
Categorical data were presented as numbers and percentages, and comparisons between groups were performed using contingency table analysis with the chi-square test. The odds ratios (ORs) of acceptance of case reports and the corresponding 95% confidence intervals (CIs) were derived using unconditional multiple logistic regression models. Multivariate models included terms for country (developing vs. developed), topic of case reports (rare diseases or elusive diagnoses vs. other topics) and calendar period (2015 vs. 2016).
All tests were 2-sided, and a p value of less than 0.05 was considered as statistically significant. Data analyses were conducted using SAS version 9.2 (SAS Institute, Cary, NC, USA) statistical software.
Results
A total of 327 manuscript labeled as case reports were received by the Editorial Offices in the period taken into consideration. One was clearly mislabeled and sent back to authors for correction. Out of the remaining 326, a total of 83 (25.5%) were accepted for publication either in regular issues (6-7-8-9-10-11-12-13-14-15-16-17-18-19-20-21-22-23-24-25-26-27-28-29-30-31-32-33-34-35-36-37-38-39-40-41-42-43-44-45-46-47-48-49-50), or in a dedicated case reports supplement (51).
The acceptance rate was significantly different when the case reports were clustered by topic, with successful diagnosis or treatment case reports having an acceptance rate of 35/99 (35.4%) followed by report of an adverse event (12/43, 27.9%) and report of a rare disease (36/184, 19.6%). Acceptance rate declined throughout the study period, with acceptance of 65/224 (29%) during 2015 and 18/102 (17.6%) during the first 6 months of 2016 (p = 0.03).
When comparing developing and developed countries of affiliation of the corresponding authors, a significant difference was observed in acceptance rates: 66/162 (40.7%) vs. 17/164 (10.4%; p<0.001).
Multivariate models including terms for country (developing vs. developed), topic of case reports (rare diseases vs. other topics) and calendar period (2015 vs. 2016) showed that the difference between developed and developing countries remained highly significant. The difference of topic remained of borderline significance with an adjusted OR of 0.58 (95% CI, 0.33-1.00), and the temporal trend was also significant (Tab. I).
Association between selected factors and acceptance of case reports submitted to TJ and IJBM accepted between January 1, 2015, and June 30, 2016
CI = confidence interval; IJBM = The International Journal of Biological Markers; OR = odds ratio; TJ = Tumori Journal.
Model adjusted for topic and country.
Model adjusted for all variables reported in the table.
Interestingly, although several manuscripts received by the Editorial Office had a title explicitly referring to a “literature review” as a complement to the description of the case report, a careful analysis of the full text indicated that a thorough review of the literature was indeed present in only a very few of those manuscripts (n = 3) thus preventing any meaningful analysis of its role in affecting acceptance of the manuscript.
Discussion
Editorial scrutiny and peer review of submitted manuscripts remains the main pillar of scientific literature. Despite the criticism inherent in the process itself (52), the development of alternative methods of evaluating the merits of scientific manuscripts, such as straight-away publication followed by post reviews and others, the vast majority of manuscripts ultimately published in the scientific literature, still go through the standard peer reviewing procedure, with an initial screening by the Editor in Chief (EIC) or the Associate Editors, and then review by experts in the field leading to the EIC's final decision on the manuscript. Given the ubiquity of this process, it is surprising that data about what happens at peer review, and regarding which variables might affect acceptance or rejection of the manuscript are relatively scarce. Link (4) has evaluated whether US vs. non-US peer reviewers for JAMA would be more inclined to accept results obtained by US vs. non-US colleagues, thus focusing on the attitude of peer reviewers toward studies coming from the United States. Link found that US reviewers showed a preference for US papers. Van Lent et al (3) evaluated whether the reporting of positive results might facilitate acceptance but did not show any significant difference in acceptance rates compared with manuscripts reporting negative results. In a more recent study, the same group (5) investigated whether industry sponsorship might affect acceptance and found that reviewers tended to identify fewer shortcomings in the design and statistical analysis of industry-related trials; however, the data showed that the research question and the methodological soundness were till the major drivers of acceptance.
In trying to quantify and model potential editorial bias, Wang et al (53) have proposed an agent-based model in which the process of peer review is guided mainly by the social interactions among authors, editors and reviewers, respectively. They applied the model to analyze a number of editorial behaviors such as decision strategy, number of reviewers and editorial bias in peer review and found that peer review outcomes were significantly sensitive to different editorial behaviors. While the simulation analysis was insightful, it is still unclear how this would apply to the day-to-day peer review process in any attempt to decrease bias (52).
In our study, we focused our analysis on the final outcome of the peer review process (rejection or acceptance), and we took into consideration the potential influence of 2 main variables: (a) the country of affiliation of the corresponding author and (b) the topic of the case reports. Because a progressive trend toward a lower acceptance rate was highlighted by comparing the acceptance rates of 2015 vs. those of the first 6 months of 2016, the analysis of the data gathered was corrected in a multivariate analysis to evaluate the time-independent influence of those variables on outcomes.
The results were particularly striking for the country of affiliation of the corresponding author, with case reports authored by researchers working in “developed” countries having at least a fivefold increase in acceptance rates compared with authors working in developing countries. This difference remained highly significant even adjusting for topic and time, as shown in Table I. It can be argued that the highly significant difference in the acceptance rate was due to a different novelty, and in the quality of the description of the cases themselves, with the authors affiliated with institutions from developed countries perhaps describing more interesting or more thorough or better-written case reports, and that there was no bias toward where the work came from. But the finding remains that the authors affiliated with institutions coming from developed countries have a significantly higher chance of acceptance, and the data seem consistent with those reported by Link (4) where US peer reviewers tended to have a more positive approach toward US papers.
Perhaps more intriguing is the finding that the type of case report itself per se may independently influence the acceptance rate for the article. Indeed, in our study case reports describing a report of a rare disease, a report of a rare presentation of a disease or describing an elusive diagnosis or an unusual biomarker(s) phenotype/genotype had an overall much lower acceptance rate (19.6%) compared with either the description of a severe side effect of a treatment (i.e., a drug, a device, radiotherapy, surgery), or other (27.9%), and the report of successful diagnosis or successful predictive use of markers or of successful treatment (35.3%). This suggests that reviewers might be more prone to accept case reports where the diagnostic approach or medical or surgical intervention made a difference in the natural history of the disease. This observation needs confirmation in other case report series either in the same therapeutic areas or others.
Footnotes
Acknowledgment
We thank Elena Colombo for useful insights and comments on the manuscript.
Financial support: No grants or funding have been received for this study.
Conflict of interest: The authors declare they have no conflict of interest.