Assessing the effectiveness of academic library services: A review on the formulation of a feasible conceptual framework

Abstract

With the rapid development of internet technology and the gradually accelerating rate of information technology, library systems are progressively transforming into hybrid library systems that provide services from traditional and electronic means. For insightful planning toward a strong service to the community, there is a need to continuously evaluate the effectiveness of library services. Based on this requirement, the purpose of this study is to determine the criteria and their measuring indicators that can be used to evaluate the effectiveness of modern academic library services by reviewing related studies found in the literature, and then build a conceptual framework. A comprehensive literature review related to library service effectiveness was conducted to identify, determine and suggest scientific evaluation criteria for evaluating the effectiveness of academic library services. The study revealed that User Satisfaction, Collection Development, Management and Staff Contributions, Service Efficiency, and Accessibility, were the top key evaluation criteria used or cited in the related works. Three or more indicator categories for each criterion have been identified, and the measurable indicators consisted of Likert-type questions on a five-point rating scale that can be used to measure indicator categories. Finally, the conceptual framework for assessing academic library service effectiveness has been developed based on these criteria and their measurement indicators. The study has presented a framework of service effectiveness, including assessment criteria and measurement indicators that can be used to evaluate modern hybrid academic libraries.

Keywords

assessment criteria measurement indicators service effectiveness conceptual framework hybrid academic library

Get full access to this article

View all access options for this article.

References

Abbas

Faiz

(2013) Usefulness of digital and traditional libraries in higher education. International Journal of Services Technology and Management 19(1/2/3): 149. DOI: 10.1504/IJSTM.2013.054203

Abdallah

Bilal

(2015) Exploring the effectiveness of library services and resources in academic libraries in Lebanon from users’ perspectives. Proceedings of the IFLA WLIC 2015, Cape Town, South Africa, pp. 1–13.

Adam

(2017) Assessment of Library Service Quality and User Satisfaction among Undergraduate Students of Yusuf Maitama Sule University (YMSU) Library.

Agosto

Rozaklis

MacDonald

, et al. (2011) A model of the reference and information service process. Reference & User Services Quarterly, 50(3), 235–244, Available at: https://journals.ala.org/rusq/article/view/3964

Allen

(2005) Hybrid librarians in the 21st century library: A collaborative service-staffing model. Proceedings of the 12th National Conference, Association of College & Research Libraries, pp. 291–301, Available at: https://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/pdf/allen05.pdf

Altman

Hernon

(1998) Service quality and customer satisfaction do matter. American Libraries 29: 53–54.

Appleton

(2017) Libraries and key Performance Indicators. USA: Chandos Publishing, pp. 1–23. Available at: https://linkinghub.elsevier.com/retrieve/pii/C20140036969

Ayob

(2011) An assessment of the effectiveness of library resources and services in supporting researchers’ Information Needs. Available at: http://eprints.usm.my/26120/1/SKPA_(ARAY).pdf

Ball

Lewis

(2016) Implementing the balanced scorecard as a strategic planning and performance management tool. In: Quality and the Academic Library. USA: Chandos Publishing, pp. 259–265. Available at: https://linkinghub.elsevier.com/retrieve/pii/B9780128021057000257

10.

Bamigboye

(2007) Evaluation of library services delivery in Olabisi Onabanjo University, Ago-Iwoye, Nigeria. Library Management 28(3): 152–162, Available at: https://www.emerald.com/insight/content/doi/10.1108/01435120710727992/full/html

11.

Bawden

Rowlands

(1999) Digital libraries: Developing a conceptual framework. New Review of Information Networking 5(1): 71–89, Available at: https://www.tandfonline.com/doi/abs/10.1080/13614579909516937

12.

Bawden

Petuchovaite

Vilar

(2005) Are we effective? How would we know?: Approaches to the evaluation of library services in Lithuania, Slovenia and the United Kingdom. New Library World 106(9/10): 454–463. DOI: 10.1108/03074800510623137

13.

Blixrud

(2003) Assessing Library Performance: New Measures, Methods, and Models. Proceedings of the IATUL Conferences. Paper 9. Available at: https://docs.lib.purdue.edu/iatul/2003/papers/9

14.

Breaks

(2002) Building the hybrid library: A review of UK activities. Learned Publishing 15(2): 99–107, Available at: https://onlinelibrary.wiley.com/doi/abs/10.1087/09531510252848854

15.

Brophy

(2001a) Electronic library performance indicators: the EQUINOX Project. Serials: The Journal for the Serials Community 14(1): 5–9. DOI: 10.1629/145.

16.

Brophy

(2001b) The library in the twenty-first century: new services for the information age. London: Library Association Publication. DOI: 10.1629/145

17.

Calvert

(2008) Assessing the effectiveness and quality of libraries. Doctor of Philosophy. Available at: https://core.ac.uk/download/pdf/41336397.pdf

18.

Carlo Bertot

McClure

Ryan

(2000) Developing national network statistics and performance measures for US public libraries: Issues, findings and recommendations. Performance Measurement and Metrics 1(1): 15–42. DOI: 10.1108/EUM0000000007225

19.

Cendon

Almeida Souza

(2017) A method for measuring satisfaction of users of digital libraries: A case study with engineering faculty. Qualitative And Quantitative Methods In Libraries 4(1): 11–19. Available at: http://www.qqml-journal.net/index.php/qqml/article/view/221

20.

Chandrashekara

(2014) Effectiveness of library and information services to the researchers of Visvesvaraya Technological University a study. Doctor of Philosophy (Ph.D.), Mysore. Available at: https://shodhganga.inflibnet.ac.in/handle/10603/72317

21.

Chiou

Perng

Wang

, et al. (2008) A conceptual framework of library reader service from customer relationship management perspective. 2008 Second International Conference on Future Generation Communication and Networking Symposia, Available at: https://ieeexplore.ieee.org/document/4813489

22.

Cox

Jantti

(2012) Discovering the impact of library use and student performance. Deputy Vice-Chancellor (Academic) – Papers , pp. 1–9.

23.

Cullen

(2001) Perspectives on User Satisfaction Surveys. Library Trends 49(4): 2–24.

24.

Cullen

Calvert

(1996) New Zealand University libraries effectivenss project: Dimensions and concepts of organizational effectiveness. Library & Information Science Research 18(2): 99–119. DOI: 10.1016/S0740-8188(96)90014-6

25.

de Jager

Nassimbeni

Daniels

, et al. (2018) The use of academic libraries in turbulent times: Student library behaviour and academic performance at the University of Cape Town. Performance Measurement and Metrics 19(1): 40–52.

26.

Deo

(2016) An assessment of user satisfaction in academic libraries: a case study of the Fiji National University Library. SSRN Electronic Journal: 1–17. DOI: 10.2139/ssrn.2805283.

27.

Du Mont

(1982) Assessing the effectiveness of library service. Champaign, Ill.: University of Illinois, Graduate School of Library and Information Science. Available at: https://www.ideals.illinois.edu/bitstream/handle/2142/3830/gslisoccasionalpv00000i00152.pdf?sequence=1

28.

Einasto

(2009) Using service quality monitoring to support library management decisions: A case study from Estonia. The International Information & Library Review 41(1): 12–20. DOI: 10.1016/j.iilr.2008.08.001

29.

Einasto

(2014) Investigating e-service quality criteria for university library: A focus group study. New Library World 115(1/2): 4–14. DOI: 10.1108/NLW-09-2013-0067

30.

Evelyn

Lydia

(2019) Library value through user satisfaction: The case of academic libraries in Ghana. International Journal of Library and Information Science 11(5): 44–57, Available at: https://academicjournals.org/journal/IJLIS/article-abstract/098C63F61639

31.

Fabunmi

(2004) Planning the university libraries for effective customer services in Nigeria. In: Technology for Information Management and Service: Modern Libraries and Information Centers on Developing Countries. Ibadan: Evi-Coleman Publication, pp. 147–158.

32.

Fernández Marcial

Costa

González-Solar

(2016) Top universities, top libraries: Do research services in academic libraries contribute to university output?. In: Proceedings of the IATUL Conferences, pp. 1–10, Available at: https://ruc.udc.es/dspace/handle/2183/19153

33.

Glorieux

Kuppens

Vandebroeck

(2007) Mind the gap: Societal limits to public library effectiveness. Library & Information Science Research 29(2): 188–208. DOI: 10.1016/j.lisr.2007.03.003

34.

Goodall

Pattern

(2011) Academic library non/low use and undergraduate student achievement: A preliminary report of research in progress. Library Management 32(3): 159–170. DOI: 10.1108/01435121111112871

35.

Gyau

Liu

Kwakye

(2021) Evaluation of user satisfaction with academic libraries services based on students’ perspectives. OALib 08(08): 1–17, Available at: https://www.scirp.org/pdf/oalibj_2021081715455223.pdf

36.

Handbook of Research on Managing Intellectual Property in Digital Libraries (2018) Advances in Library and Information Science. United States: IGI Global. Available at: https://doi.org/10.4018/978-1-5225-3093-0

37.

Hariri

Norouzi

(2011) Determining evaluation criteria for digital libraries’ user interface: A review. The Electronic Library 29(5): 698–722. DOI: 10.1108/02640471111177116

38.

Hobohm

(1996) The impact of new technology on libraries: An introductory note. Inspel 30(4): 303–307.

39.

Hsieh

Chin

(2004) The performance indicators of university e-library in Taiwan. The Electronic Library 22(4): 325–330. DOI: 10.1108/02640470410552956

40.

Hussien

FRM

Mokhtar

WNHW

(2018) The effectiveness of reference services and users’ satisfaction in the academic library. International Journal of Academic Research in Progressive Education and Development 7(3): 327–337.

41.

Ifijeh

(2011) Assessing faculty use of university library collection and services in Nigeria: a case of Covenant University, Ota. Library Philosophy and Practice: 1–9. Available at: https://digitalcommons.unl.edu/libphilprac/574.

42.

Jeng J (2005) Usability assessment of academic digital libraries: effectiveness, efficiency, satisfaction, and learnability. Libri 55(2-3): 96–121. DOI: 10.1515/LIBR.2005.96.

43.

Johnson

(2014) Evaluating the quality of services in academic libraries. Research Journal of Library Sciences 2(5): 1–7.

44.

Joo

Yeon Lee

(2011) Measuring the usability of academic digital libraries. The Electronic Library 29(4): 523–537, Available at: https://www.emerald.com/insight/content/doi/10.1108/02640471111156777/full/html

45.

(2001) Discussion of experiences in establishing an electronic library: The example of National Chiao Tung University. Shu Fan Quarterly 47: 10–33.

46.

Khan

(2013) IPR in India and USA: Its impact on library services. Doctor of Philosophy, Aligarh.

47.

Kiran

(2010) Service quality and customer satisfaction in academic libraries. Library Review 59(4): 261–273. DOI: 10.1108/00242531011038578

48.

Kumar

(2012) User satisfaction and service quality of the university libraries in Kerala. International Journal of Information Dissemination and Technology 2(1): 24.

49.

Kyrillidou

(2002) From input and output measures to quality and outcome measures, or, from the user in the life of the library to the library in the life of the user. The Journal of Academic Librarianship 28(1-2): 42–46. DOI: 10.1016/S0099-1333(01)00299-3

50.

Lai

C-F

Chiu

P-S

Huang

Y-M

, et al. (2014) An evaluation model for digital libraries’ user interfaces using fuzzy AHP. The Electronic Library 32(1): 83–95. DOI: 10.1108/EL-05-2012-0046

51.

Levi

Conrad

(2008) Usability testing of world wide web sites. U.S. Bureau of Labor Statistics Office of Survey Methods Research. Available at: https://www.bls.gov/osmr/research-papers/1996/html/st960150.htm

52.

Levine-Clark

Carter

(2013) Academic library. In: ALA glossary of Library and Information Science (4th ed.). Chicago: ALA editions, an imprint of the American Library Association, p. 2.

53.

(2006) Leveraging quality web-based library user services in the digital age. Library Management 27(6/7): 390–400. DOI: 10.1108/01435120610702387

54.

Liu

(2019) Information resource, interface, and tasks as user interaction components for digital library evaluation. Information Processing & Management 56(3): 704–720, Available at: https://www.sciencedirect.com/science/article/abs/pii/S0306457317305241

55.

Majid

Anwar

Eisenschitz

(2001) User perceptions of library effectiveness in Malaysian agricultural libraries. Library Review 50(4): 176–186. DOI: 10.1108/00242530110390451

56.

Marilou

(2019) Measurement of academic library outcomes towards achieving institutional outcomes of state universities and colleges (SUCs) in the Philippines. IFLA WLIC 2019. University of Piraeus, Greece, pp. 22–23. Available at: http://library.ifla.org/id/eprint/2724

57.

Miller

Schmidt

(2002) E-Metrics: Measures for electronic resources. In: Advances in Library Administration and Organization. Bingley: Emerald publishing, pp. 203–212. Available at: https://www.emerald.com/insight/content/doi/10.1016/S0732-0671(02)20009-7/full/html

58.

Montenegro

Clasing

Kelly

, et al. (2016) Library resources and students’ learning outcomes: Do all the resources have the same impact on learning? The Journal of Academic Librarianship 42(5): 551–556. DOI: 10.1016/j.acalib.2016.06.020

59.

Nejati

(2008) Service quality at University of Tehran Central Library. Library Management. 29(6/7): 571–582. DOI: 10.1108/01435120810894563

60.

Nicholas

(1996) Assessing Information Needs. London: Aslib.

61.

Nicholson

(2005) A conceptual framework for the holistic measurement and cumulative evaluation of library services. Proceedings of the American Society for Information Science and Technology 41(1): 496–506, Available at: https://asistdl.onlinelibrary.wiley.com/doi/full/10.1002/meet.1450410158

62.

Nimon

(2001) The role of academic libraries in the development of the information literate student: The interface between librarian, academic and other stakeholders. Australian Academic & Research Libraries 32(1): 43–52, Available at: https://www.tandfonline.com/doi/abs/10.1080/00048623.2001.10755142

63.

Nitecki

(1996) Changing the concept and measure of service quality in academic libraries. The Journal of Academic Librarianship 22(3): 181–190, Available at: https://www.sciencedirect.com/science/article/pii/S0099133396900567

64.

Osman Z, Goon CA and Hajrah Wan Aris W (1998) Quality services: Policies and practices in Malaysia. Library Management 19(7): 426–433. DOI: 10.1108/01435129810232054

65.

Pauline

(2011) User satisfaction with academic libraries services: Academic staff and students perspectives. International Journal of Library and Information Science 3(10): 209–216. DOI: 10.5897/IJLIS11.045

66.

Poll

Boekhorst

Mundt

(2007) Measuring Quality. Performance Measurement. Berlin, New York: IFLA Publications. Available at: https://www.degruyter.com/view/books/9783598440281/9783598440281/9783598440281.xml

67.

Pritchard

(1996) Determining quality in academic libraries. Library Trends 44(3): 572–594.

68.

Ramanayaka

Chen

Shi

(2018) UNSCALE: A fuzzy-based multi-criteria usability evaluation framework for measuring and evaluating library websites. IETE Technical Review 36(4): 412–431. DOI: 10.1080/02564602.2018.1498032

69.

Ranganathan

Sivaswamy Aiyer

Sayers

WCB

, et al. (2006) The five laws of library science. New Delhi [India]; Bangalore, India: Ess Publications. Available at: http://books.google.com/books?id=lcbgAAAAMAAJ

70.

Saracevic

(2000) Digital library evaluation: Toward evolution of concepts. Library Trends 49: 350–369.

71.

Saravanan

Rao

KSP

(2007) Measurement of service quality from the customer’s perspective – an empirical study. Total Quality Management & Business Excellence 18(4): 435–449. DOI: 10.1080/14783360701231872

72.

Somaratna

Peiris

Jayasundara

(2010) User expectation verses user perception of service quality in university libraries: A case study. Colombo, pp. 1–12.

73.

Song

(2009) Designing library services based on user needs: new opportunities to reposition the library. Milan, Italy: World Library and Information Congress, pp. 1–18.

74.

Soria

Fransen

Nackerud

(2013) Library use and undergraduate student outcomes: New evidence for students’ retention and academic success. Portal: Libraries and the Academy 13(2): 147–164, Available at: https://muse.jhu.edu/article/504593

75.

Stone

Ramsden

(2013) Library impact data project: Looking for the link between library usage and student attainment. College & Research Libraries 74(6): 546–559. DOI: 10.5860/crl12-406

76.

Suseela

Uma

(2015) UCD Approach for the management of user services in university libraries. International Journal of Advanced Library and Information Science 3(1): 274–290. DOI: 10.23953/cloud.ijalis.255

77.

Tenopir

(2011) Beyond usage: Measuring library outcomes and value. Library Management 33(1/2): 5–13, Available at: https://www.emerald.com/insight/content/doi/10.1108/01435121211203275/full/html

78.

Thakuria

(2007) Concept of quality in library services: an overview. INFLIBNET's Convention Proceedings.

79.

Thong

JYL

Yap

C-S

(1996) Information systems effectiveness: A user satisfaction approach. Information Processing & Management 32(5): 601–610. DOI: 10.1016/0306-4573(96)00004-0

80.

Wang

Shieh

C-J

(2006) The relationship between service quality and customer satisfaction: The example of CJCU library. Journal of Information and Optimization Sciences 27(1): 193–209. DOI: 10.1080/02522667.2006.10699686

81.

Wong

SHR

Webb

(2011) Uncovering meaningful correlation between student academic performance and library material usage. College & Research Libraries 72(4): 361–370. DOI: 10.5860/crl-129

82.

Xie

(2008) Users’ evaluation of digital libraries (DLs): Their uses, their criteria, and their assessment. Information Processing & Management 44(3): 1346–1373. DOI: 10.1016/j.ipm.2007.10.003

83.

Xie

Joo

Matusiak

(2020) Digital library evaluation measures in academic settings: Perspectives from scholars and practitioners. Journal of Librarianship and Information Science 53(1): 130–152, Available at: https://journals.sagepub.com/doi/10.1177/0961000620935505

84.

Yang

Dawson

(2018) Altmetrics and their potential as an assessment tool for digital libraries. 2018 5th International Symposium on Emerging Trends and Technologies in Libraries and Information Services (ETTLIS), Available at: https://ieeexplore.ieee.org/document/8485273

85.

Zhao

Bai

Hui

(2002) An empirical assessment and application of SERVQUAL in a Mainland Chinese department store. Total Quality Management 13(2): 241–254, Available at: https://www.tandfonline.com/doi/abs/10.1080/09544120120102478