Objective: Bibliometrics is a method of examining scholarly communications. Concerns regarding the use of bibliometrics in general, and the impact factor score (IFS) in particular, have been discussed across disciplines including social work. Although there are frequent mentions in the literature of the IFS as an indicator of the impact or quality of scholars' work, little empirical work has been published regarding the validity of such use. Method: A proportionate, stratified, random sample, of n=323 articles was selected from 17 Web of Science listed social work journals published during the 1992 to 1994 period. Results: The relationship between journals'IFSs and the actual impact of articles published in those journals (predictive validity) was r= .41 (short term) and r = .42 (long term).Conclusion: The practice of using the IFS as a proxy indicator of article impact merits significant concern as well as further empirical investigation.
Abbasi, K. (2004). Let's dump impact factors. British Medical Journal, 329-329. Retrieved October 16, 2004, from http://bmj.bmjjournals.com/cgi/content/full/329/7471/0-h
2.
Adam, D. (2002). Citation analysis: The counting house. Nature, 415, 726-729.
3.
Aronowitz, S. (2001). The knowledge factory: Dismantling the corporate university and creating true higher learning. Boston: Beacon.
4.
Baker, D. R. (1990). Citation analysis: A methodological review. Social Work Research & Abstracts, 26, 3-10.
5.
Baker, D. R. (1991). On-line bibliometric analysis for researchers and educators. Journal of Social Work Education, 27, 41-47.
6.
Balaram, P. (2000). The impact of publication lists. Current Science, 78, 1177-1178.
7.
Barker, K. (1995). Contingent work: Research issues and the lens of moral exclusion. In L. Tetrick & J. Barling (Eds.), Changing employment relations: Behavioral and social perspectives (pp. 31-60). Washington, DC: American Psychological Association.
8.
Barker, K. (1998). Toiling for piece-rates and accumulating deficits: Contingent work in higher education. In K. Barker & K. Christensen (Eds.), Contingent work: American employment relations in transition (pp. 195-220). Cornell, NY: Cornell University Press.
9.
Barker, K. (2001). Standard and nonstandard employment in higher education: Findings from the 1990's. In Commission on Professionals in Science and Technology (Ed.), Changing career paths in science and engineering: Report of a CPST workshop (pp. 12-30). Washington, DC: AAAS, Commission on Professionals in Science and Technology. Retrieved April 15, 2006, from http://www.cpst.org/Changing.pdf
10.
Barker, K. (2003). Contingent work in the United States. In Supplement to the 19th Edition of the Encyclopedia of Social Work (pp. 9-19). Washington, DC: National Association of Social Workers.
11.
Barker, K., & Christensen, K. (1998a). Introduction: Controversy and challenges raised by contingent work arrangements. In K. Barker & K. Christensen (Eds.), Contingent work: American employment relations in transition(pp. 1-18). Cornell, NY: Cornell University Press.
12.
Barker, K., & Christensen, K. (1998b). Charting future research. In K. Barker & K. Christensen (Eds.), Contingent work: American employment relations in transition (pp. 306-320). Cornell, NY: Cornell University Press.
13.
Barker, K., & Kohout, J. (2003). Contemporary employment in psychology and future trends. In M. J. Prinstein & M. Patterson (Eds.), The portable mentor: Expert guide to a successful career in psychology (pp. 309-332). New York: Kluwer Academic/Plenum.
14.
Beck, U. (2000). The brave new world of work: The political economy of insecurity. Cambridge, UK: Cambridge University Press.
15.
Borgman, C. L., & Furner, J. (2002). Scholarly communication and bibliometrics. Annual Review of Information Science and Technology, 36, 3-72.
16.
Braxton, J. M., & Del Favero, M. (2002). Evaluating scholarship performance: Traditional and emergent assessment templates. New Directions for Institutional Research, 114, 19-31.
17.
Burgan, M. (1998, March/April). What we talk about when we talk about tenure. Academe, 84(2), 2-2.
18.
Campling, J. (1992). Social-policy digest 81. Journal of Social Policy, 21, 99-118.
19.
Campling, J. (1993). Social-policy digest 85. Journal of Social Policy, 22, 99-118.
20.
Christenson, J. A., & Sigelman, L. (1985). Accrediting knowledge: Journal stature and citation impact in social science. Social Science Quarterly, 66, 964-975.
21.
Cnaan, R. A., Caputo, R. K., & Shmuely, Y. (1994). Senior faculty perceptions of social work journals. Journal of Social Work Education, 30, 185-199.
22.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
23.
Cole, J., & Cole, S. (1971). Measuring the quality of sociological research: Problems in the use of the Science Citation Index. American Sociologist, 6, 23-29.
24.
Cole, J. R. (2000). A short history of the use of citations as a measure of the impact of scientific and scholarly work. In B. Cronin & H. B. Atkins (Eds.), The web of knowledge: A festschrift in honor of Eugene Garfield(pp. 281-300). Medford, NY: Information Today.
25.
Colquhoun, D. (2003). Challenging the tyranny of impact factors. Nature, 423, 479-479.
26.
Dahwan, S. M., & Gupta, B. M. (2004). Comparative evaluation of Indian physics research: Impact factor vs. citations frequency. Current Science, 86, 1194-1195.
27.
Decker, O., Beutel, M. E., & Brahler, E. (2004). Deep impact— Evaluation in the sciences. Sozial Und Praventivmedizin, 49, 10-14.
28.
Fassoulaki,A., Sarantopoulos, C., Papilas, K., Patris, K., & Melemeni,A. (2001). Academic anesthesiologists' views on the importance of the impact factor of scientific journals: A North American and European survey. Canadian Journal of Anesthesia, 48, 953-957.
29.
Fava, G. A., & Ottolini, F. (2000). Impact factors versus actual citations. Psychotherapy and Psychosomatics, 69, 285-286.
30.
Finkelstein, M. J., Seal, R. K., & Schuster, J. H. (1998). The new academic generation: A profession in transformation. Baltimore, MD: Johns Hopkins University Press.
31.
Frank, M. (2003). Impact factors: Arbiter of excellence. Journal of the American Medical Library Association, 91, 4-6.
32.
Furr, A. L. (1995). The relative influence of social work journals: Impact factors vs. core influence. Journal of Social Work Education, 31, 38-45.
33.
Garfield, E. (1983). How to use citation analysis for faculty evaluations, and when is it relevant? Part 2. Current Comments, 45, 5-14.
34.
Garfield, E. (1996). How can impact factors be improved?British Medical Journal, 313, 411-413.
35.
Garfield, E. (1997). Validation of citation analysis. Journal of the American Society for Information Science, 48, 962-964.
36.
Garfield, E. (1999). Journal impact factor: A brief review. Canadian Medical Association Journal, 161, 979-980.
37.
Garfield, E. (2003). The meaning of the impact factor. International Journal of Clinical and Health Psychology, 3, 363-369.
38.
Gastel, B. (2001). Assessing the impact of investigators' work: Beyond “impact” factors. Canadian Journal of Anesthesia, 48, 941-945.
39.
Glanzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53, 171-193.
40.
Hansen, H. B., & Henriksen, J. H. (1997). How well does journal “impact” work in the assessment of papers on clinical physiology and nuclear medicine. Clinical Physiology, 17, 409-418.
41.
Holden, G., Rosenberg, G., & Barker, K. (2005a). Tracing thought through time and space: A selective review of bibliometrics in social work. Social Work in Health Care, 41(3/4), 1-34.
42.
Holden, G., Rosenberg, G., & Barker, K. (2005b). Shallow science or meta-cognitive insights: A few thoughts on bibliometrics. Social Work in Health Care, 41(3/4), 129-148.
43.
Holden, G., Rosenberg, G., & Barker, K. (2005c). Bibliometrics: A potential decision making aid in hiring, reappointment, tenure and promotion decisions. Social Work in Health Care, 41(3/4), 67-92.
44.
Howard, M. O., & Howard, D. A. (1992). Citation analysis of 541 articles published in drug and alcohol journals: 1984-1988. Journal of Studies on Alcohol, 53, 427-434.
45.
Hutcheson, P. (1997). The corrosion of tenure: A bibliography. NEA Higher Education Journal, Fall, 89-106.
46.
ISI. (2003). Journal Citation Reports. Retrieved November 12, 2004, from http://www.isinet.com/
47.
ISI. (2004). Journal Citation Reports. Retrieved November 12, 2004, from http://www.isinet.com/
48.
Katz, J. S. (1999). Bibliometric indicators and the social sciences. Retrieved July 27, 2003, from http://www.sussex.ac.uk/Users/sylvank/pubs/ESRC.pdf
49.
Kirk, S. A. (1984). Methodological issues in the comparative study of schools of social work. Journal of Social Service Research, 7(3), 59-73.
50.
Kostoff, R. N. (1998). The use and misuse of citation analysis in research evaluation. Scientometrics, 43 (1), 27-43.
51.
Kostoff, R. N. (2002). Citation analysis of research performer quality. Scientometrics, 53, 49-71.
52.
Kreuger, L. W. (1993). Should there be a moratorium on articles that rank schools of social work based on faculty publications? Yes!Journal of Social Work Education, 29, 240-246.
53.
Kurmis, A. P. (2003). Understanding the limitations of the journal impact factor. Journal of Bone and Joint Surgery, 85A, 2449-2454.
54.
Linardi, P. M., Coelho, P. M. Z., & Costa, H. M. A. (1996). The “impact factor” as a criterion for the quality of scientific production is a relative, not absolute, measure. Brazilian Journal of Medical and Biological Research, 29, 555-561.
55.
Lindsey, D. (1976). Distinction, achievement and editorial board membership. American Psychologist, 31, 799-804.
56.
Lindsey, D. (1978). The scientific publication system in social science. San Francisco: Jossey-Bass.
57.
Lindsey, D. (1980). Production and citation measures in the sociology of science: The problem of multiple authorship. Social Studies of Science, 10, 145-162.
58.
Lindsey, D. (1982). Further evidence for adjusting for multiple authorship. Scientometrics, 4, 389-395.
59.
Lindsey, D. (1989). Using citation counts as a measure of quality in science: Measuring what's measurable rather than what's valid. Scientometrics, 15, 189-203.
60.
Lindsey, D., & Kirk, S. A. (1992). The role of social work journals in the development of a knowledge base for the profession. Social Service Review, 66, 295-310.
61.
MacRoberts, M. H., & MacRoberts, B. R. (1989). Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40, 342-349.
62.
MacRoberts, M. H., & MacRoberts, B. R. (1997). Citation content analysis of a botany journal. Journal of the American Society for Information Science, 48, 274-275.
63.
Meittunen, J., & Nieminen, P. (2003). The effect of statistical methods and study reporting characteristics on the number of citations: A study of four general psychiatric journals. Scientometrics, 57, 377-388.
64.
Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht, Netherlands: Springer.
65.
Monastersky, R. (2005a). Impact factors run into competition. Chronicle of Higher Education, 52(8), A17-A17.
66.
Monastersky, R. (2005b). The number that's devouring science. Chronicle of Higher Education, 52(8), A12-A12.
67.
Oromaner, M. (1974). Collaboration and impact: The career of multiauthored publications. Social Science Information, 14, 147-155.
68.
Ouwehand, A. (2002). Power calculator. Retrieved March 31, 2005, from http://calculators.stat.ucla.edu/powercalc/
69.
Pardeck, J. T. (1992). Are social work journal editorial boards competent? Some disquieting data with implications for research on social work practice. Research on Social Work Practice, 2, 487-496.
70.
Pardeck, J. T., Arndt, B. J., Light, D. B., Mosley, G. F., Thomas, S. D., Werner, M. A., et al. (1991). Distinction and achievement levels of editorial board members of psychology and social work journals. Psychological Reports, 68, 523-527.
71.
Pardeck, J. T., Chung, W. S., & Murphy, J. W. (1995). An examination of the scholarly productivity of social work journal editorial board members and guest reviewers. Research on Social Work Practice, 5, 223-234.
72.
Pardeck, J. T., & Meinart, R. G. (1999). Scholarly achievements of the Social Work editorial board and consulting editors: A commentary. Research on Social Work Practice, 9, 86-91.
73.
Phelan, T. J. (1999). A compendium of issues for citation analysis. Scientometrics, 45, 117-136.
74.
Plomp, R. (1990). The significance of the number of highly cited papers as an indicator of scientific prolificacy. Scientometrics, 19, 185-197.
75.
Porter, A. L. (1977). Citation analysis: Queries and caveats. Social Studies of Science, 7, 257-267.
76.
Rhoades, G. (1998). Managed professionals: Unionized faculty and restructuring academic labor. Albany: State University of New York Press.
77.
Rhoads, R. A., & Rhoades, G. (2005). Graduate employee unionization as symbol of and challenge to the corporatization of U.S. research universities. Journal of Higher Education, 76 (3), 243-275.
78.
Rosenberg, G., Holden, G., & Barker, K. (2005). What happens to our ideas? A bibliometric analysis of articles in Social Work in Health Care in the 1990s. Social Work in Health Care, 41, 35-66.
79.
Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality?Journal of the Medical Library Association, 91, 42-46.
80.
SAS Institute Inc. (2003). SAS 9.1 for Windows [Computer program]. Cary, NC: SAS Institute.
81.
Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
82.
Seglen, P. O. (1992). The skewness of science. Journal of the American Society for Information Science, 43 (9), 628-638.
83.
Seglen, P. O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45 (1), 1-11.
84.
Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. British Medical Journal, 314, 498-502. Retrieved February 17, 2003, from http://bmj.com/cgi/content/full/314/7079/497
85.
Seipel, M. M. O. (2003). Assessing publication for tenure. Journal of Social Work Education, 39, 79-88.
86.
Sellers, S. L., Mathiesen, S. G., Perry, R., & Smith, T. (2004). Evaluation of social work journal quality: Citation versus reputation approaches. Journal of Social Work Education, 40, 143-160.
87.
Sevinc, A. (2004). Manipulating impact factor: An unethical issue or an editor's choice. Swiss Medical Weekly, 134, 410-410.
88.
Slaughter, S., & Leslie, L. L. (1997). Academic capitalism: Politics, policies, and the entrepreneurial university. Baltimore: Johns Hopkins University Press.
89.
Smith, R. (1997). Journal accused of manipulating impact factor. British Medical Journal, 314, 461-461.
90.
Smith, R. (1998). Unscientific practice flourishes in science. British Medical Journal, 316, 1036-1040.
91.
Snijders, T. A. B., & Bosker, R. J. (1994). Modeled variance in two-level models. Sociological Methods & Research, 22, 342-363.
92.
Snijders, T. A. B., & Bosker, R. J. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage.
93.
Staff. (1999). Citations reveal concentrated influence: Some fields have it, but what does it mean?Science Watch, 10(1), 1-2.
94.
Staff. (2004b). The undue importance of the impact factor. European Journal of Cancer, 40, 919-919.
95.
Statistical Package for the Social Sciences. (2004). SPSS for Windows, 13.0. Chicago: Author.
96.
Tenure Modifications. (2005, September). Copy editor. New York University: Faculty Senators Council, p. 3-3.
97.
Thyer, B. A. (2004). Evaluating social work journal publication systems. Journal of Social Work Education, 40, 361-363.
98.
Urbaniak, G. C., & Plous, S. (2005). Research randomizer. Retrieved July 31, 2005, from http://www.randomizer.org/
99.
Weale, A. R., Bailey, M., & Lear, P. A. (2004). The level of non-citation of articles within a journal as a measure of quality: A comparison to the impact factor. BMC Medical Research Methodology, p. 4-4. Retrieved July 15, 2005, from http://www.biomedcentral.com/1471-2288/4/14
100.
Weigel, R. J., Dracon, G., Radhakrishnan, R., Rho, Y., Sevgen, F., & Dafoe, D. C. (2004). Incentive systems for academic productivity in a depart of surgery. Journal of the American College of Surgeons, 199, 300-307.
101.
Whitehouse, G. H. (2001). Citation rates and impact factors: Should they matter?British Journal of Radiology, 74, 1-3.
102.
Zwahlen, M., Junker, C., & Egger, M. (2004). Commentary II. The journal impact factor in the evaluation of research quality: Villain, scapegoat or innocent bystander?Sozial Und Praventivmedizin, 49, 1, 19-22.
