Abstract
Objectives
The phrase ‘Initial Clinical Experience' in a manuscript's title implies that the described technique offers promise of future clinical relevance. We assessed, using rates of subsequent citations, the actual academic relevance of such articles in comparison to articles not containing the phrase.
Design
We searched ISI database for articles that included the studied phrase in their titles between 1975 and 2009 and grouped the results by the related medical specialty. We excluded articles from journals with no available impact factor. For each identified article, we extracted number of included patients, citations/year, the average impact factor of the publishing journal over the last five years and the proportion of articles published in the same journal that garnered zero subsequent citations.
Setting
Retrospective review of a scientific database.
Participants
None
Main outcome measures
Citation rate
Results
Among a total of 982,127 articles published in 186 journals representing eight major publishing medical specialties, 531 (0.05%) were Initial Clinical Experience articles. Thirty percent of Initial Clinical Experience articles were never cited compared with 7% of the overall article volume (p < 0.0001). Citations/year for Initial Clinical Experience articles were significantly lower than the median impact factor (p < 0.0001). There was no correlation between citations and number of patients described in the Initial Clinical Experience articles (p = 0.61).
Conclusions
Initial Clinical Experience articles are cited less frequently than the average, especially in Cardiovascular, Radiology and Ophthalmology journals.
DECLARATIONS
Introduction
Authors of clinical studies frequently include modifiers to titles of articles, especially those focused on relatively new methodologies. These modifiers include words such as ‘preliminary', ‘pilot', ‘early' or ‘initial'. It is not entirely clear whether authors are trying to impart the sheen of novelty to their work or are simply trying to lower the expectations of reviewers and editors, so that the study can be given the ‘benefit of the doubt' given its (apparent) newness.
Editors typically strive to maximize the academic relevance of their respective journals. One of the most widely applied metrics of such relevance is the rate of citation of papers in a given journal. Although many measures of relative rates of citation are available, journal impact factor represents the most widely applied metric. Impact factor simply counts the number of current year citations to the source items published in that journal during the previous two years. 1
Publication of nascent technologies or methodologies introduces both potential risk and reward for journal editors focused on maximizing high impact factors. If work that is described by the authors as ‘early' subsequently enjoys widespread application, then publication of that early work likely will lead to high rates of citation and thus improves a journal's impact factor. However, if such early work fails to stimulate future studies, the journal's impact factor will suffer. As a continuation of a previous effort of studying ‘preliminary' and ‘pilot' articles, 2 the purpose of the current study was to characterize the subsequent academic relevance of papers with titles containing the phrase ‘Initial Clinical Experience'.
Methods
Data selection
In October 2011, we collected our study cohort from ISI Web of Knowledge database. 3 We searched between 1975, the earliest date provided by ISI, and 2009, to allow a minimum of two years for articles to be cited. We used the search phrase ‘Initial Clinical Experience' in the ‘Title' search bar to identify articles of interest and, accordingly, the publishing journal. Articles from journals that had no reported impact factor on ISI were excluded from our studied population. The ISI database provides not only the number of citations of the studied article and the impact factor of the publishing journal in the last five years, but also categorizes articles by medical specialty. Articles were categorized into one of the following groups: Anesthesiology, Cardiovascular system, Neurosciences, Oncology, Ophthalmology, Radiology, Surgery, Urology and other specialties with less frequent Initial Clinical Experience publications (i.e. Gastro-enterology, Hepatology, Dermatology). We then extracted the number of citations of each included article, the publishing year, the five most recent impact factors of the publishing journals and the number of patients of each study. The five most recent values of each journal's impact factor were averaged. We also examined all articles that were published by the included journals over the same study period (1975–2009) and calculated the percentage that were never cited in each group.
Given the cumulative nature of number of citations, older articles are more likely to have more citations than newer articles. To normalize older and newer articles, we calculated the ‘citations per year' for each article as the number of citations garnered divided by the number of years since publication.
Statistical analysis
All statistical analyses were performed using the software packages Microsoft Excel 97 (Microsoft Corporation, Redmond, WA, USA) and JMP (SAS Institute, Cary, NC, USA). Statistical significance was assigned to P values of <0.05. Continuous variables were presented as median with interquartile ranges due to deviations from normal distributions as calculated using the Shapiro–Wilk test. Correlation between number of patients and citations per year was determined using linear regression. Two-tailed Fisher's exact test was used to analyse differences in the never-cited proportion among specialties. Wilcoxon rank-sum test was used to compare median citations per year with the average impact factor and to compare the median number of patients among specialties.
Results
Our search identified 581 articles from 201 journals that contained the phrase Initial Clinical Experience in their titles. In total, 50 (9%) of these articles from 15 journals were excluded as the journals did not report an impact factor. As such, 531 articles from 186 different journals were included in our study.
Journal and article demographics
Interquartile range
Comparison of citation rates between Initial Clinical Experience articles and overall journal publications
Interquartile range
Excluding zero citation articles
Discussion
Our findings suggest that articles with the phrase Initial Clinical Experience in their title are less likely to be cited than other articles published in the same journals. This indicates that the volume of subsequent dependent research work does not reach the expected degree that is promised from the title. Thus, the lower average of citations would likely have a negative effect on the publishing journal's impact factor. Our results are comparable to an earlier study showing that only 27% of ‘preliminary' or ‘pilot' articles were subsequently followed by a more definitive publication. 2
Several studies have examined the impact of an article's title on subsequent citations. Howard et al. 4 suggested that methodological and review articles tend to be cited more frequently than other article categories in Drug and Alcohol journals. Montori et al. 5 and Bhandari et al. 6 separately further specified that systematic reviews are more cited than narrative ones. Jacques et al. 7 concluded that number of citations was positively correlated with the length of the title and the presence of specific words like ‘colon' or an acronym. They, however, found that reference to a specific country makes the article poorly cited. Our study of Initial Clinical Experience articles expands upon the effect of an article's category on its chances of success.
The current study has several limitations. Due to conducting an automated search, we may have missed related articles that do not literally mention our search phrase. We also calculated the average impact factor of each journal over the last five years even if the article was published earlier. Finally, we were forced to exclude (9%) of initially identified Initial Clinical Experience articles that were published in journals that did not list impact factors.
Conclusions
Articles with the phrase Initial Clinical Experience in their titles are cited less frequently than average articles published in the same journals, especially in Cardiovascular, Radiology, and Ophthalmology journals. These articles likely negatively affect the impact factor of the publishing journal.
Footnotes
Acknowledgements
None