Evaluating Reliability of Google Scholar,Scopus,and Web of Science: A Study of Faculty in U.S. Advertising and Public Relations Programs

Abstract

Reliability of Google Scholar (GS), Scopus, and Web of Science (WoS) is examined using publications and citations of 186 scholars in 14 U.S. advertising and public relations (ADPR) programs. Career duration is controlled, and an integrated impact (II) index is proposed as a practical solution. Results suggest there are trade-offs between the uncertainty of GS’s search parameters for more inclusive coverage and the curated collections of Scopus and WoS that might undercount some influential authors or works. To further demonstrate the discipline’s value, we must pay more attention to rigor and accuracy of methods that will lead to improved outcomes.

Keywords

academic impact database performance analysis advertising and public relations research impact

Get full access to this article

View all access options for this article.

References

Antell

Foote

J. S.

Foote

J. B.

(2016). Scholarly publishing’s evolving landscape: Impact metrics, electronic-only journals, and open access in journalism and communication research. Journalism & Mass Communication Educator, 71(3), 309–328.

Bakkalbasi

Bauer

Glover

Wang

(2006). Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical Digital Libraries, 3(7), 1–8.

Barry

T. E.

Howard

D. J.

(1990). A review and critique of the hierarchy of effects in advertising. International Journal of Advertising, 9(2), 121–135.

Brown

T. A.

(2015). Confirmatory factor analysis for applied research. Guilford Press.

Carnegie Classification of Institutions of Higher Education. (2018). Basic classification description. http://carnegieclassifications.iu.edu/classification_descriptions/basic.php

Chan Terri

Tse

C. H.

Hung

. (2017). Productivity and impact in advertising research since the Millennium: A profiling and investigation of drivers of impact. International Journal of Advertising, 36(1), 11–37.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

Clarivate. (n.d.). Web of science: Social science citation index. http://wokinfo.com/products_tools/multidisciplinary/webofscience/ssci/

Diamond

S. J.

Thomas

C. R.

Desai

Holliday

E. B.

Jagsi

Schmitt

Enestvedt

B. K.

(2016). Gender differences in publication productivity, academic rank, and career duration among US academic gastroenterology faculty. Academic Medicine, 91(8), 1158–1163.

10.

Falagas

M. E.

Pitsouni

E. I.

Malietzis

G. A.

Pappas

(2008). Comparison of PubMed, Scopus, Web of Science, and Google Scholar: Strengths and weaknesses. The FASEB Journal, 22(2), 338–342.

11.

Foote

J. B.

Foote

J. S.

(2011). A tipping point for electronic journals? A comparison of highly cited journals in six disciplines. Journalism & Mass Communication Educator, 66(3), 231–242.

12.

Ford

J. B.

Merchant

(2008). A ten-year retrospective of advertising research productivity, 1997-2006. Journal of Advertising, 37(3), 69–94.

13.

Gasparyan

A. Y.

Ayvazyan

Kitas

G. D.

(2013). Multidisciplinary bibliographic databases. Journal of Korean Medical Science, 28(9), 1270–1275.

14.

Google Scholar. (n.d.). Google Scholar citation. https://scholar.google.com/intl/en/scholar/citations.html

15.

Harzing Anne-Wil

Van der Wal

. (2008). Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8(1), 61–73.

16.

Henthorne

T. L.

LaTour

M. S.

Loraas

(1998). Publication productivity in the three leading US advertising journals: 1989 through 1996. Journal of Advertising, 27(2), 53–63.

17.

Holliday

E. B.

Jagsi

Wilson

L. D.

Choi

Thomas

C. R.

Jr Fuller

C. D.

(2014). Gender differences in publication productivity, academic position, career duration and funding among US academic radiation oncology faculty. Academic Medicine, 89(5), 767–773.

18.

E. J.

Pasadeos

Ertem-Eray

(2019). Growth of public relations research networks: A bibliometric analysis. Journal of Public Relations Research, 31(1–2), 5–31.

19.

E. J.

Pasadeos

Ertem-Eray

(2021). The structure and evolution of global public relations: A citation and Co-citation analysis 1983–2019. Public Relations Review, 47(1), 1–15.

20.

Kousha

Thelwall

(2009). Google book search: Citation analysis for social science and the humanities. Journal of the American Society for Information Science and Technology, 60(8), 1537–1549.

21.

Krasnova

Veltri

N. F.

Kummer

T.-F.

Schäfer

Saunders

(2013, December). Publication trade-offs for junior scholars in IS: Conjoint analysis of preferences for quality, first authorship, collaboration and time [Conference session]. International Conference on Information Systems (ICIS 2013), Association for information systems, Milan, Italy.

22.

Levine

T. R.

(2010). Rankings and trends in citation patterns of communication journals. Communication Education, 59(1), 41–51.

23.

Leydesdorff

Bornmann

Comins

J. A.

Milojević

(2016). Citations: Indicators of quality? The impact fallacy. Frontiers in Research Metrics and Analytics, 1, Article 1.

24.

Meho

L. I.

Yang

(2006). A new era in citation and bibliometric analyses: Web of Science, Scopus, and Google Scholar. arXiv preprint cs/0612132, 1–49.

25.

Mounce

(2015). Dark research: Information content in many modern research papers is not easily discoverable online. PeerJ PrePrints, 3, Article e773v1. https://doi.org/10.7287/peerj.preprints.773v1

26.

Murnane

R. J.

Willett

J. B.

(2010). Methods matter: Improving causal inference in educational and social science research. Oxford University Press.

27.

Nosek

B. A.

Graham

Lindner

N. M.

Kesebir

Hawkins

C. B.

Hahn

. . .Tenney

E. R.

(2010). Cumulative and career-stage citation impact of social-personality psychology programs and their members. Personality and Social Psychology Bulletin, 36(10), 1283–1300.

28.

Pasadeos

Phelps

Edison

(2008). Searching for our “own theory” in advertising: An update of research networks. Journalism & Mass Communication Quarterly, 85(4), 785–806.

29.

Petersen

A. M.

(2015). Quantifying the impact of weak, strong, and super ties in scientific careers. Proceedings of the National Academy of Sciences, 112(34), E4671–E4680.

30.

Pinheiro

Bates

DebRoy

Sarkar

R Core Team. (2021). nlme: Linear and nonlinear mixed effects models. R package version 3.1-152. https://CRAN.R-project.org/package=nlme

31.

Polonsky

Carlson

(2009). Is there global inclusion of authors in the five leading advertising journals? A regional comparison 1998–2007. International Journal of Advertising, 28(4), 691–714.

32.

Rodgers

(2018a). Why we need better measures of research impact in advertising: Considerations for best practices to expand research’s reach. Journal of Advertising Research, 58(4), 385–389. https://doi.org/10.2501/JAR-2018-045

33.

Rodgers

(2018b). What’s in a name? The difference is strategic communication. Journal of Advertising Education, 22(2), 120–132.

34.

Rosseel

(2012). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA). Journal of Statistical Software, 48(2), 1–36.

35.

Russell

J. T.

Martin

C. H.

(1976). Sources of scholarly publications in marketing, advertising, and public relations. Journal of Advertising, 5(3), 29–34.

36.

Soley

L. C.

Reid

L. N.

(1983). Advertising article productivity of the US academic community. Journalism Quarterly, 60(3), 464–542.

37.

Soley

L. C.

Reid

L. N.

(1988). Advertising article productivity updated. Journalism Quarterly, 65(1), 157–164.

38.

Wang

Rodgers

Wang

Thorson

(2016). A seventeen-year study of graduate student authorship in advertising journals. Journalism & Mass Communication Educator, 71(1), 69–83.

39.

West

(2007). Directions in marketing communications research: An analysis of the International Journal of Advertising. International Journal of Advertising, 26(4), 543–554.

40.

Zou

(2005). Contributions to international advertising research: An assessment of the literature between 1990 and 2002. Journal of Advertising, 34(1), 99–110.