Abstract
A popular, ‘fun’ challenge entails patting one's head while simultaneously rubbing one's abdomen in a circular motion. The task requires considerable concentration, and even then can prove difficult to perform optimally. Usually, a person will perform one action well, but ignore or pay little attention to the other. The twin role facing scholarly journals in relation to the articles they publish is similar, and therefore, periodically, a journal should take time to acknowledge some of the fine articles that have appeared in its pages, while continuing to define those it seeks to publish.
In this Editorial we look back at some of the key papers in the 45 year history of the Australian and New Zealand Journal of Psychiatry (ANZJP), as well as anticipate changes with the appointment of the new Editor (G.M.). In recent editorials [1,2], the Editor has outlined the direction in which he wishes to take the Journal and the types of quality articles that he seeks to attract and publish. Along with comprehensive and informative reviews, original research will continue to be prioritized, especially that which is novel. However, the content of the Journal will tend towards greater diversity. For example, debate and discussion regarding clinical and research matters will be encouraged because in addition to providing a window on science the ANZJP has to be the critical voice of psychiatric professionals. This discourse can take the form of editorials, commentary or correspondence to the Journal. The only essential ingredient is that of quality. Previous editors, of this [3] and other [4] journals, have grappled with what constitutes a quality paper. The viewpoints have been strongly expressed and argued that one cannot draw conclusions about the quality of a paper solely on the basis of how many times it has been cited in other articles and must utilize, in addition, other methods [5].
As the new Editor seeks to make his mark on the ANZJP and attract papers conforming to his vision, it is timely to reflect on those articles that have been judged of high quality and/or highly cited. Such examination permits assessment of whether papers that have been acclaimed on the basis of peer opinion or some other means are more regularly cited over a number years compared to other articles.
In 2002, the Editorial Board launched an experiment to select, in their view, the best articles featuring in the ANZJP in 2001 [6]. From 2001 to 2004, each article was selected by an international panel of judges using the following criteria: (i) the article adds consequentially to the field through original, innovative research findings; (ii) the article expands or challenges current knowledge; (iii) the article opens additional areas for new research or a pathway to advance knowledge; (iv) the article integrates discoveries obtained by different approaches and/or disciplines through creative synthesis bringing new insights to bear on original research; or (v) the article reflects critically on research findings to guide the direction of further research [6–9].
From 2005 to 2007, different criteria were used in the selection process. For articles published in 2005, Associate Editors and the Editor selected the top ten articles from twenty most frequently downloaded articles (two from each of the ten issues) [10]. For articles appearing in 2006, the top nine papers were selected based on the number of citations they received in the first year of publication [11] and for 2007, these were chosen from the perspective of enhancing outcomes [12].
We used two databases (Web of Science and MEDLINE) to examine the Journal's performance according to various bibliometric indicators as of March 2011. The findings suggest that the Journal has grown substantially over the last four and a half decades based on the increased number of citations it has attracted. For example, Figure 1 shows the number of citations to all articles in 5-year periods; the graph depicts the exponential growth of citations received.
Number of citations listed in the Web of Science for each 5-year period for all articles appearing in the Journal from 1970 to 2009 (as of 10 March 2011). Note that some of the growth can be attributed to more articles being published after 2000 (from 6 issues per year to 12) and also the expansion of the WoS database itself (more journals added to the database).
To gain a better picture of a journal's performance, it is necessary to obtain more comprehensive data than just citation number, as this does not take into consideration the increase in the number of articles published each year, or the other factors. Table 1 lists the number of articles published in the Journal for each of the last four decades, the number of citations these have received, and average citations per items according to the Web of Science (WoS) database. The next column lists the articles that have received the most citations for each decade [13–16]. The next two columns show the number of articles that have received at least 100 citations and 50 citations up to March 2011. Compared to higher ranked psychiatry journals [17], the ANZJP has not published as many articles that attract extremely high citations; for example, only 15 papers have 100 or more citations in the history of the Journal. However, when interpreting the data in Table 1, it should be noted that articles published in the previous decade (2000–2009) have not reached their full citation potential because it normally takes 10 to 15 years for this to happen. Thus, the number of articles that have already obtained at least 50 citations (n = 36) will likely increase over the next decade, as will the average number of citations per item.
Bibliometric data from MEDLINE and Web of Science of articles published in the ANZJP, 1970–2009 (as of 7 March 2011)
†Citable items included articles, reviews and proceeding papers.
Table 2 lists the top ten articles from the ANZJP that have received the most number of citations to date. Interestingly, all four decades are represented: 1970s (2), 1980s (3), 1990s (2), 2000 (3), and this list is somewhat different from the top 10 list derived from a similar search in 2006 [18]. Although six of the items appear on both lists, there are four new entries, and all except the highest ranked article [14] are ranked differently compared to the 2006 list. Thus, the data summarized in this article will also likely change over time as new articles are added to the mix and more citations are added annually.
Top ten articles in ANZJP receiving the most citations (Web of Science, as of 7 March 2011)
Table 3 provides an annual citation analysis of the ANZJP from 2001 to 2010 and those selected as top papers. Judging what papers would be ‘winners’ between 2001–2007 was somewhat ‘hit and miss’ when compared against citation counts to date. Over the seven year ‘experiment’, five of the selected papers were the most highly cited item to date (71% hit rate) and the average number of citations per item was higher for the ‘winners’, compared to the annual journal average. However, the majority of the ‘winners’ did not appear in the top ten most cited list of each year, so using this criterion, only 11% to 44% of the articles were stand out winners. Interestingly, there was a great deal of variability in the number of citations received for a given year (see column 5, min/max).
Citation analysis of the ANZJP 2001–2010 and ‘winners’ selected by Editors, 2001–2007
‘Winners’ were selected by various means by the Editor (see text). †Indicates most highly cited article appeared in the selected winner's list for that year. Source: Web of Science and Journal Citation Reports.
Of the 62 items selected between 2001 and 2007, 35% (22/62) received ten or fewer citations, 39% (24/62) received 11 to 30 citations, and 26% (16/62) received more than 30 citations. Thus, 16 articles can be considered as having been ‘well chosen’, but what about the others? Why did they not attract similar attention in the citation stakes? We can speculate that some articles selected by peers (and by other means) are important, even though they do not receive large numbers of citations. It should also be noted here that an article may be heavily cited because it is of high quality, or for potentially other reasons; for example, it provides the first description of a research instrument, is in a readily accessible journal, or even because it is widely seen to be problematic such that citing authors wish to draw attention to its limitations [3].
Discussion
It is apparent from this brief excursion into the publishing statistics of the Journal that assessing the performance of papers in the ANZJP is a task fraught with challenges. Judging the quality of individual papers, and in turn gauging the reputation of the Journal according to a handful of parameters, is inevitably subject to bias and distortion. Therefore the data presented in this Editorial is open to interpretation.
Synthesizing the findings presented in Tables 1, 2 and 3, it is clear that not many papers within the ANZJP achieve extraordinarily high citation rates possibly because, until more recently, its readership has been somewhat parochial. The Internet has perhaps widened the Journal's appeal, and extended its reach, but over the past three decades the average number of citations per item (approximately 11) has remained relatively constant. The few articles that have been outstanding, as reflected in the top ten with most citations, predictably comprise surveys and epidemiological studies, measures and scales, a randomized trial and controversial topics such as stimulant prescription to children and suicide. However, though the content of popular papers is in keeping with expectations, the identification of articles that will be highly cited in the future is notably difficult. There is no crystal ball.
It is likely that the variably defined ‘winners’ are articles that hold much broader appeal than that of citability alone. Further, the mismatch between annual ‘winners’ and their rates of citation is an indication that citations are a narrow measure, and that ANZJP articles per se serve a much broader range of functions.
This is an important lesson, and it is essential that the ANZJP does not lose ‘sight’ of it. But, at the same time, the Journal must strive to achieve impact where possible. Therefore quality and excellence are critical, and these can be best achieved by adhering to sound publishing principles and prioritizing psychiatric science.
Conclusion
This Editorial is the first foray into this complex field of publishing statistics. It provides a number of valuable insights and serves as a prelude for future editorials that will discuss topics such as the h-index and impact factor. The aim is to identify what constitutes quality and determine where the Journal's priorities should lie. If we get this right, we will all be able to pat our heads and rub our tummies to our heart's content.