Book impact assessment: A quantitative and text-based exploratory analysis

Abstract

Books are an important source of knowledge to disseminate information. Researchers and academicians write books to propagate their innovative research or teachings amongst academic as well as non-academic audience. The number of books written every year is increasing rapidly. According to International Publisher Association (IPA) annual report 2015–2016, around 150 million different books were published worldwide in 2014–2015. Many e-commerce websites are also involved in selling books. A recent addition to book publishing world is e-books, which have really made it very simple to publish. While, availability of large number of books is good for readers, at the same time it is challenging to find a good book, particularly in scholarly settings. Researchers in the area of Scientometrics have attempted to view assessment of goodness of a scholarly book by measuring citations that a book receive. However, citations alone are not a true measure of a book’s impact. Many a times people use the knowledge in a book without actually citing it. Also use of books in classroom settings or for general reading often is not reflected in terms of citations. Therefore, it is important to obtain users’s opinion about a book from other forms of data. Fortunately, we have now some data of this sort available in form of reviews, downloads and social media mentions etc. Amazon and Goodreads, both of which provide the readers’ views about a book, are two good examples. This paper presents an exploratory research work on using these non-traditional data about books to assess impact of a book. A set of Scopus-indexed computer science books with good citations as well as some other popular books in computer science domain are used for analysis. The reviews of books have been crawled in an automated fashion from Amazon and Goodreads. Thereafter sentiment analysis is carried out the text of reviews. Results of sentiment analysis are compared and correlated with traditional impact assessment metrics. The experimental analysis does not show a coherent relationship between citation and online reviews. Also, majority of the online reviews are found to be positive for large number of books in the dataset. As a related exercise, the Scopus citation data and Google scholar citation data for books are also compared. A high value of correlation is observed in these two. Overall the exploratory analysis provides a useful insight into the problem of book impact assessment.

Keywords

Altmetrics book impact citation impact review mining sentiment analysis

Get full access to this article

View all access options for this article.

References

Small

, The Value of the Humanities, Oct. 2013.

Kousha

and Thelwall

, Web indica-tors for research evaluation. Part 3: Books and nonstandard outputs, El Profesional de la In-formaci’on24(6) (2015), 724.

Garfield

, Citation Analysis as a Tool in Journal Evaluation: Journals can be ranked by frequency and impact of citations for science policy studies, Science178(4060) (1972), 471–479.

Stremersch

, Verniers

and Verhoef

P.C.

, The quest for citations: Drivers of article impact, Journal of Marketing71(3) (2007), 171–193.

Torres-Salinas

, Robinson-García

, Jiménez-Contreras

and Delgado

, López-Cózar, Towards a ‘Book Publishers Citation Reports’. First approach using the ‘Book Citation Index,’ola de Documentación Científica, Revista espa 35(4) (2012), 615–624.

Bar-Ilan

, Citations to the ‘Introduction to informetrics’ indexed by WOS, Scopus and Google Scholar, Scientometrics82(3) (2010), 495–506.

Kousha

, Thelwall

and Rezaie

, Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus, Journal of the American Society for Information Science and Technology62(11) (2011), 2147–2164.

Chubin

D.E.

and Moitra

S.D.

, Content analysis of references: Adjunct or alternative to citation counting?Social Studies of Science5(4) (1975), 423–441.

MacRoberts

M.H.

and MacRoberts

B.R.

, Problems of citation analysis, Scientometrics36(3) (1996), 435–444.

10.

Almind

T.C.

and Ingwersen

, Informetric analyses on the World Wide Web: Methodological approaches to ‘webometrics,’, Journal of Documentation53(4) (1997), 404–426.

11.

Cronin

, Snyder

and Atkins

, Comparative citation rankings of authors in monographic and journal literature: A study of sociology, Journal of Documentation53(3) (1997), 263–273.

12.

Bollen

and Van de Sompel

, Smith

J.A.

and Luce

, Toward alternative metrics of journal impact: A comparison of download and citation data, Information Processing & Management41(6) (2005), 1419–1440.

13.

Kousha

and Thelwall

, Assessing the impact of disciplinary research on teaching: An automatic analysis of online syllabuses, Journal of the American Society for Information Science and Technology59(13) (2008), 2060–2069.

14.

Shema

, Bar-Ilan

and Thelwall

, Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics, Journal of the Association for Information Science and Technology65(5) (2014), 1018–1027.

15.

Poria

, Cambria

and Gelbukh

, Aspect extraction for opinion mining with a deep convolutional neural network, Knowledge-Based Systems108 (2016), 42–49.

16.

Piryani

, Gupta

and Singh

V.K.

, Generating aspect-based extractive opinion summary: Drawing inferences from social media texts, Computación y Sistemas.

17.

Piryani

, Gupta

and Singh

V.K.

, Movie prism: A novel system for aspect level sentiment profiling of movies, Journal of Intelligent & Fuzzy Systems32(5) (2017), 3297–3311.

18.

Nicolaisen

, The scholarliness of published peer reviews: A bibliometric study of book reviews in selected social science fields, Research Evaluation11(3) (2002), 129–140.

19.

Shaw

, An Analysis of the Relationship between Book Reviews and Fiction Holdings in OCLC, Library and Information Science Research13(2) (1991), 147–154.

20.

Chevalier

J.A.

and Mayzlin

, The effect of word of mouth on sales: Online book reviews, Journal of Marketing Research43(3) (2006), 345–354.

21.

Kousha

and Thelwall

, Can Amazon.com reviews help to assess the wider impacts of books?Journal of the Association for Information Science and Technology67(3) (2015), 566–581.

22.

Bar-Ilan

, Citations to the ‘Introduction to informetrics’ indexed by WOS, Scopus and Google Scholar, Scientometrics82(3) (2010), 495–506.

23.

Zuccala

and Guns

, Comparing book citations in humanities journals to library holdings: Scholarly use versus ‘perceived cultural benefit’ (RIP), Paper presented at the Proceedings of ISSI 2013 - 14th International Society of Scientometrics and Informetrics Conference2013353–360.

24.

Piryani

, Devaraj

and Singh

V.K.

, Analytical mapping of opinion mining and sentiment analysis research during 2000-2015, Information Processing and Management53(1) (2017), 122–150.

25.

Devaraj

, Piryani

and Singh

V.K.

, Lexicon ensemble and lexicon pooling for sentiment polarity detection, IETE Technical Review33(03) (2015), 332–340.

26.

Uddin

and Kumar

, Marisha and V.K. Singh, A sciento-text framework to characterize research strength of institutions at fine-grained thematic area level, Scientometrics106(3) (2016), 1135–1150.