Is citation normalization realistic?

Abstract

One method for assessing quality of research outputs across different technical disciplines is comparing citations received by the research output documents. However, crossdiscipline citation comparison studies require discipline normalization, in order to eliminate discipline differences in cultural citation practices and discipline differences in the number of active researchers available to cite. The ‘definition’ of, and number of documents used to represent, a discipline become critical. This study attempted to determine whether the citation characteristics (average, median) of a discipline’s domain stabilized as the domain’s size was decreased. A sample of papers (classified asresearch articles only, not review articles, by the Institute for Scientific Information) published in the journalOncogene in 1999 was clustered hierarchically, and the citation averages and medians were computed for each cluster at different cluster hierarchical levels. The citation characteristics became increasingly stratified as the clusters were reduced in size, raising serious questions about the credibility of a selected denominator for normalization studies. An interesting side result occurred when all the retrieved articles were sorted by number of citations. Thirteen of the fifty most highly cited research articles had 100 or more references, whereas zero of the fifty least citedresearch articles had 100 or more references.