Status of retraction notices for biomedical publications associated with research misconduct

Abstract

In order to assess the status of retraction notices for publications involving research misconduct, we collected and analyzed information from the Office of Research Integrity website. This site lists confirmed instances of misconduct in research supported by the National Institutes of Health. Over a 10-year period, 200 publications derived from misconduct were identified. For 20.5% of those papers, no retraction notice was published. We found that the majority of these cases were from investigations concluded at least two years before our analysis, and thus are unlikely to be explainable by timing considerations. These findings demonstrate that retraction notices for papers associated with misconduct are often not published and suggest that clear, adherent policies are needed in this circumstance to correct the scientific record.

Retractions of scientific papers are on the rise; in the decade spanning 2000 to 2009, there was a 10-fold increase in the number of published retractions (Wager and Williams, 2011: 567–570). According to the website Retraction Watch (McCook, 2016) there was a total of 500 retractions in the scientific literature in 2014 and 684 retractions in 2015. In the biomedical sciences, the percent increase in retractions outpaced the increase in the total number of overall publications (McCook, 2016).

Broadly speaking, there are two main reasons for paper retraction: research misconduct or unintentional errors in methods, data collection, or analysis. Instances of research misconduct include articles in which authors volitionally engaged in fabrication or falsification of data, or plagiarism (Steen, 2011: 249–253). Just as is the case with retractions in general, retractions due to scientific misconduct are also considered to be on the rise (Fang et al., 2012: 17028–17033).

In studying the relationship between retractions and misconduct, previous work focused on already published notices, retrospectively analyzing the percent where misconduct was cited as a cause (Damineni et al., 2015: 19–23; Fang et al., 2012: 17028–17033; Grieneisen and Zhang, 2012: e44118; Steen, 2011: 249–263). On the other hand, to what extent papers known to involve research misconduct are subsequently retracted is not clear. Gaining a clearer understanding in this context requires prospectively following the fate of clearly identified publications documented to involve misconduct.

To address this issue, we gathered and analyzed data gleaned from the Office of Research Integrity (ORI) website (https://ori.hhs.gov/). This office, situated in the Department of Health and Human Services, investigates allegations of research misconduct in work supported by National Institutes of Health (NIH). Summaries of cases where research misconduct occurred along with the titles of publications that resulted from this misconduct are posted and freely available to the public. This approach allowed us to identify a defined population of publications associated with research misconduct and to then assess the status of their retraction notices.

Methods

Identification of articles involved in research misconduct

The ORI maintains a website (https://ori.hhs.gov/) that lists case summaries with names of investigators who engaged in documented research misconduct in studies supported by NIH funding. The outcomes of investigations into these cases along with the titles of associated published articles that contain data obtained by research misconduct are detailed. For some investigations, case summaries were published in a newsletter also available at the website. All such listed published articles for a 10-year period (2007–2017) formed the basis of this investigation.

These articles were searched on PubMed to determine whether there were published retractions. In addition, the impact factor of the journal was recorded. Impact factors were obtained from the Web of Science from the year in which the article was retracted, except in one case where the article was retracted after the original journal ceased publication. In this case, the most recent available impact factor was used. If there was no retraction, the impact factor of the year in which the ORI published its misconduct findings was used.

Statistical analysis

Data were analyzed using SAS Studio software, version 3.71 (Cary, NC, USA). We used descriptive statistics to assess for association between journal impact factor and whether a retraction notice was published using a Fisher’s exact test. A two-sided alpha <0.05 was considered the threshold for statistical significance.

Results

We analyzed all case summaries of research misconduct published at the ORI website from 2007 to 2017. Based on this analysis, we identified the titles and authors of 200 papers that contained data derived from documented misconduct. Of those papers, 41/200 (20.5%) did not have a published retraction notice as of November 2017. The 41 unretracted papers include 11 instances for which a so-called correction was published, one expression of concern, and two withdrawals with no retraction notices.

For the 200 papers identified in this study, there were a total of 65 authors who were found to engage in research misconduct. Twenty-four individuals authored papers that were not retracted. One author was responsible for 10 unretracted papers, while the rest of the 31 unretracted papers were distributed among 23 different authors.

Publication dates for the 41 unretracted papers ranged from March 1993 to March 2016. For these papers, 38 were published prior to 2015. The dates on which the ORI concluded its research misconduct investigations that encompassed these 41 papers were distributed as follows: eight prior to 2010, 27 during 2010–2015, and six on or after 2016. Notably, for only two of the 41 unretracted papers were investigations concluded in 2017.

Journal impact factors for papers identified in our study ranged from 0.38 to 53.3. The median impact factor was 4.73 (interquartile range (IQR) = 4.02); the mean was 7.46 (SD=7.71). Among the 41 unretracted papers, the median impact factor was 4.61 (IQR=3.03); the mean was 6.14 (SD=5.7) with no significant association between impact factor and the absence of a published retraction.

Discussion

In order to examine the status of retraction notices from papers associated with documented research misconduct, we took advantage of information available at the ORI website. We found that 20.5% of such papers have not been retracted at the time this study was conducted. Importantly, 35/41 of unretracted papers were from ORI investigations that concluded prior to 2016. While there are no established guidelines for a timeline for retraction submission, the average time span between publication to retraction, no matter what the cause, is just under two years (Steen et al., 2013). In our data set, there were 83 notices that were published after conclusion of the investigation with an average time from completion of the investigation to publication of 237 days. In 76 cases, retraction notices were published before the completion of the investigation. The lack of a retraction notice is thus unlikely to be explained by the timing of the original publication or subsequent investigation.

The publications we analyzed were all cited by PubMed; therefore any subsequent retraction should be found in this database since, as a matter of policy, PubMed indexes the entire published contents of its vast number of participating journals. Even if published, however, an analysis of retraction notices from papers at the ORI website from 1993 to 2003 showed that only 40.6% revealed misconduct as the cause (Abritis, 2015).

We recognize that the relatively limited scope of this study and the fact that it included only data from NIH funded biomedical research may limit its generalizability. It would be important to clarify what happens in other scientific fields; for example, it might be informative to examine the status of retraction notices from National Science Foundation supported work since this agency sponsors research in a wider array of science and engineering fields.

The purpose of comprehensive, timely, and transparent retraction notices is to ensure the integrity of the scientific record. To guarantee that articles containing data derived from research misconduct are retracted in a timely manner, we suggest that submission of such notices be part of the final process that concludes an investigation. One other possibility is for journals to publish links to the findings of those investigations.

We wonder, however, whether publication of an appropriate retraction notice in settings of misconduct ultimately requires greater collaboration amongst stakeholders. Offending authors, funding agencies, research institutions, and journals may have to work together to ensure submission of comprehensive and transparent retraction notices. This would help speed the process of retraction, enhance a culture of integrity and rigor, and improve scientific credibility amongst the lay public.