Abstract
In this paper we raise concerns over the methodological approach employed by Knudson (2013a) in attempting to identify core journals in kinesiology. Philosophical concerns about the nature of his results and their meaning are also brought to bear. The authors argue for more consistent, explicit, and inclusive methodology in identifying the journals and the associated measures of impact and prestige, and raise questions about the nature and purpose of scholarship in kinesiology that must be considered more fully. Our intention in raising these concerns is meant to answer the call of Knudson, Morrow, and Thomas (2014), who observed that scholars in kinesiology rarely challenge one another through letters to the editor, replication, or by other means. In the spirit of such scholarly discourse, we offer several observations that we feel should be considered, particularly prior to individuals or institutions attempting to operationalize and/or apply his study results.
Scholarship in kinesiology has a long history (Park, 1980, 2005), with increased recognition and acceptance following the release of the Surgeon General's Report on Physical Activity and Health in 1996 (U.S. Department of Health and Human Services, 1996; Cardinal, 2014). As but one example, there were 12 journals listed within the “Sport Sciences” category of the SCI® Journal Citation Reports in 1995. By 2012, there were 84 such journals. Even with that, many kinesiology journals remain unlisted in this indexing system (Cardinal & Thomas, 2005), and those that are listed do not appear exclusively within the “Sports Sciences” category. For example, some are listed or cross-listed in other categories (e.g., “Psychology,” “Public, Environmental and Occupational Health”), and others appear in the Social Science Citation Index within categories such as “Education and Educational Research,” “Ethics,” “Hospitality, Leisure, Sport and Tourism,” and “Sociology.”
Kinesiology scholars also publish outside the realm of “kinesiology journals.” For example, Pate, Pratt, Blair, Haskell, Macera, Bouchard, et al.'s (1995) paper with more than 3,700 citations as of July 7, 2014 was published in the Journal of the American Medical Association, and other scholars in kinesiology publish in a range of scientific areas too. Likewise, scholars from outside of kinesiology publish within kinesiology journals. For example, with 644 citations as of July 7, 2014, the most highly cited article in the Research Quarterly for Exercise and Sport's history came from a group of psychologists (Marcus, Selby, Niaura, & Rossi, 1992). This is representative of the diverse nature of kinesiology and the broad importance of and interest in the subject matter (Cardinal & Lee, 2013).
With a long-standing interest in the organizational structure of kinesiology, narrowing the research to practice gap, and communicating the relevance of research, among other more specialized topics, Knudson is clearly knowledgeable about these issues (Knudson, 2005; Ives & Knudson, 2007; Knudson, Elliott, & Ackland, 2012). In recent years he has devoted considerable effort toward the study of the journals of the field, especially those related to the sub-disciplinary area of biomechanics (Knudson & Chow, 2008; Knudson, 2012, 2013b). In one of his more recent articles, he sought to identify core journals in kinesiology (Knudson, 2013a). He did so by creating impact and/or prestige rankings for 82 “kinesiology-related journals” using bibliometric variables extracted from three sources (i.e., Thomas-Reuters' Journal Citation Reports for 2011, SCImago Journal and Country Rank using Elsevier's Scopus for 2011, and Eigenfactor.org as of June 17, 2013), and applying these metrics to selected journals identified in previous studies. Three of the four previous studies were conducted by Knudson himself (Knudson & Chow, 2008; Knudson, 2013b, in press) and one came from the field of library studies more than 20 years ago (Miranda & Mongeau, 1991). It is unclear which journals came from which source(s). Methodologically, this creates problems, as Knudson indicated in the “Note” accompanying Table 1: “Journals with missing data (–) are due to lack of indexing in databases so there are no data available for both bibliometric variables needed to calculate a mean z score.” By including journals that do not meet the study's implied inclusion criteria, a misleading impression of each journal's stature may result (i.e., people may mistakenly dismiss them).
Six of the journals identified in Table 1 of Knudson's (2013a) study have no values listed for prestige or impact, whereas one journal, the International Journal of Sport Nutrition and Exercise Metabolism, had enough information available to create a prestige ranking, but not enough to create an impact ranking. For one of the six journals identified as having no data available for either variable, the Journal of Philosophy of Sport [sic], is listed. However, the lack of available data appears to be due to two possible factors, one of which is human error (i.e., the journal's name is listed incorrectly). The proper name of the journal is the Journal of the Philosophy of Sport. Using that name, the necessary information for computing prestige and impact scores in the manner described by Knudson would have been available. Specifically, in 2012 its SCImago 10-year h index was 10, its five-year Eigenfactor was .276, its 2-year impact factor was .324, and its 3-year SCImago Journal Rank would vary depending on which subject category was selected (i.e., “Social Sciences (Miscellaneous)” = 222 out of 409 or “Health (Social Sciences)” = 73 out of 131).
Second, while we wholeheartedly agree that the Journal of the Philosophy of Sport falls within the domain of kinesiology, it is not listed under the subject headings of “Sport Sciences” or “Hospitality, Leisure, Sport and Tourism” within the Journal Citation Reports. The aforementioned subject headings correspond with the “Science Edition” and the “Social Science Edition,” respectively, of the Journal Citation Reports. As far as we can tell, these are the closest inclusive subject headings to kinesiology in the database. However, as noted previously, this is not an absolute as “kinesiology-related” journal titles appear under multiple subject category headings, and in some cases more than one subject category heading. Kinesiology-related articles also appear in journals that encompass a wide spectrum of subject categories.
Since “Kinesiology” is not a subject heading in and of itself in the database, it is important to know what exclusion and inclusion criteria were employed in selecting the journals used in Knudson's (2013a) study. For example, the Journal of the Philosophy of Sport is listed under the subject heading of “Ethics” in the Journal Citation Reports (“Social Science Edition”). Other journals that were included in the ranking, such as Physical Therapy, fall under the subject headings of “Orthopedics” and “Rehabilitation,” but do not appear under the subject headings of “Sport Sciences” or “Hospitality, Leisure, Sport and Tourism.” What qualifies these as kinesiology journals, per se? In a separate study Knudson (2013b) noted,
…a combination of expert ratings and bibliometric measures are needed to document the core journals and their influence in the field of kinesiology. Documenting the core journals and perception of their influence in the field by kinesiology scholars would also better define kinesiology to other scholars and bibliometric databases. (p. 24)
Given the diverse nature of the discipline, as well as its continued expansion, agreement on what journals to exclude and what journals to include may be difficult to achieve. Scholars may be most familiar with the journals affiliated with their own sub-disciplinary area(s), but unfamiliar or less familiar with journals outside of their area(s). For the journals they are not familiar with, they may turn to indices such as the journal impact factor—a practice not encouraged (Alford, 2012; Cardinal, 2013a, 2013b)—to obtain information about the journals. One may further question the results because some of the sub-disciplines within kinesiology are quite small, whereas others are very large (Spirduso & Reeve, 2011). As such, comparisons of this type are discouraged (Garfield, 2006) because the end result may be a value-laden ranking that establishes and reinforces scientific hegemony within kinesiology (e.g., theoretical over applied, quantitative over qualitative, traditional over emerging; Andrews, 2008). In the discussion section of his paper, Knudson (2013a) does caution readers against using measures such as those employed in his study across disciplines and/or sub-disciplines. Others have warned against this too (Baethge, 2012).
Knudson (2013a) also uses the phrases “core journals in kinesiology” and “kinesiology-related journals” in his study somewhat interchangeably. It is unclear what these phrases mean, or how the phrases should be differentiated from one-another. This is more than mere semantics. Thirty journals—more than one-third of the sample—reviewed by Knudson (2013a) do not appear in the Journal Citation Reports–Science Citation Edition under the “Sport Sciences” subject heading or the Journal Citation Reports–Social Science Citation Edition under the “Hospitality, Leisure, Sport and Tourism” subject heading. Why were these journals included? Moreover, there are 31 and 25 unique journals, respectively, listed in the 2012 “Sport Sciences” and “Hospitality, Leisure, Sport and Tourism” subject heading categories that were not examined by Knudson. In other words, 56 potential journals were excluded from consideration, and there are likely others as these are only two possible subject headings that kinesiology-related journals appear under in the Journal Citation Reports. A sampling of journals not included in his study but listed in the Journal Citation Reports are: Archives of Physical Medicine and Rehabilitation, Exercise Immunology Review, International Journal of Sport History, International Journal of Sport Science and Coaching, International Review for the Sociology of Sport, Journal of Exercise Science and Fitness, Journal of Human Kinetics, Journal of Sport Economics, Journal of Sport and Social Issues, and Kinesiology.
This further underscores what we see as the critical problem with Knudson's (2013a) study: certain journals were excluded and other journals were included, but we don't have a clear sense of how, or if, subjectivity informed these selections. With a multitude of potential journal metrics to choose from, it is also not absolutely clear whether the variables he elected to assess are the right ones (Baethge, 2012; West & Rich, 2012; Wilhite & Fong, 2012; Cardinal, 2013a, 2013b). Perhaps this combination of factors partially explains why only 25 journals in Knudson's (2013a) list have a > 0.00 prestige rating and only 27 have a > 0.00 impact rating. Of course, there may be other factors at play here too.
As noted, Pate, et al. (1995) published a kinesiology-related paper in the Journal of the American Medical Association. The paper is a citation classic. Would the paper have achieved the same stature had it been published in one of kinesiology's core journals or a kinesiology-related journal? Or, should the fact that a kinesiology scholar published a paper on a kinesiology-related topic qualify the journal as a kinesiology-related journal?
The most highly cited paper in the Research Quarterly for Exercise and Sport—the oldest continuously published research journal in kinesiology (Park, 1980, 2005; Cardinal & Thomas, 2005)—was published by a group of psychologists not affiliated with a kinesiology-type department or program (Marcus, et al., 1992). In terms of citation counts, Marcus, et al.'s paper is a citation classic, comparing favorably with highly cited papers in Psychological Bulletin (Sternberg, 1992), which was the highest ranked journal in the subject category of “Psychology” in the 2012 Journal Citation Reports.
As Garfield (2006) has cautioned, citation analysis is convenient and it does have a certain degree of utility, but to truly understand the scholarly value of any piece of scholarship, the individual article must be read and evaluated on its own merits. To illustrate the importance of this point, we consider Henry's (1964) paper. Henry's paper was published in one of the journal's that had no bibliometric measure of impact or prestige (as measured by Knudson, 2013a). Yet, the paper is often credited with the spawning of the scientific movement and sub-disciplines within kinesiology, and it has been at the forefront of discussion about the organization of kinesiology as an academic unit and scientific discipline for half a century. As of July 8, 2014, Henry's paper had accumulated 237 times in Google Scholar, another more inclusive indexing system (Harzing & van der Wal, 2008), including seven or eight citations per year for each year of the past four years (i.e., 2010–2013) and already garnering two citations in 2014 (i.e., 31 citations between 2010 and July 8, 2014).
Another one of Knudson's (2013a) conclusions, i.e., that knowing bibliometric impact and prestige scores can help scholars select the most appropriate journal for their research, is also worrisome. This suggestion minimizes the importance of a journal's mission and audience. It may also encourage scholars to publish in journals that will enhance their academic careers over those that benefit the discipline of kinesiology and its associated professions in other meaningful ways, something that may be difficult to combat without widespread and systematic reform (Cooper, 2013).
This latter point touches upon something that Knudson (2005) and others (e.g., Alford, 2012) have written about: the devaluing of applied research in the field. Unfortunately, studies such as his may unintentionally reinforce a hierarchy of scholarship and reflect a linear functionality between research and practice. Research and practice do not have to be mutually exclusive. Rather, they can and should inform one another in deeply meaningful ways (e.g., community-based participatory research). We believe this is consistent with Knudson's overarching body of work (Knudson, 2005; Ives & Knudson, 2007; Knudson, et al., 2012), and we only emphasize the point here in an attempt to make certain that it is not lost or overshadowed by “empirical evidence” that might be used to suggest something different.
Knudson (2013a) also made the choice to only consider English-language journals in his analysis, and most of those journals were from the United States. This reinforces the one-way mirror approach to science, whereby the contributions of international scholars are obscured, which has been cautioned against for years, particularly among those in humanities and social and behavioral science-based areas of inquiry (Valentine, Allison, & Schneider, 1999; Cardinal, Powell, & Lee, 2009). For example, there are 56 kinesiology-related scholarly journals in China and those journals publish 10,000 articles annually (Zhang, 2012). That's a remarkable wealth of information to discount, particularly given the cost of science (both direct and indirect). Systematically excluding such literature is exclusionary, hampers disciplinary and professional progress, and underrepresents the value of kinesiology journals within the broader scientific community.
Lastly, the term “kinesiology” is most prominently used in North America and especially the United States (Čustonja, Milanović, & Sporiš, 2009). While we have no problem with the term or its use, it may not fully reflect a consensual worldview of the field.
Our intention in raising these concerns is meant to answer the call of Knudson, et al. (2014), who observed that scholars in kinesiology rarely challenge one another. In this spirit, we have offered several observations that should be considered, particularly before attempts to operationalize and/or apply Knudson's (2013a) study results occur. Hopefully, this will result in increased clarity about scholarship in kinesiology.