Sage Journals: Discover world-class research

Abstract

This paper examines the impact of Whisper, an open-source AI automated speech recognition (ASR) software, on captioning work at Emory Libraries and the staff who conduct the work. Captions are essential for searchability, discoverability, and user accessibility; however, providing captions has typically been challenging due to the resources required. Whisper shows potential for enabling proactive captioning of digitized content but also raises questions about the impact of ASR-generated captions on the staff engaged in this and similar work. With data collected from a grant-funded captioning project and an ongoing oral history program, this paper examines this impact and observes that while Whisper reduces the most labor-intensive phase of captioning, this does not necessarily entail a reduction in staff time, due to changing demand and the ongoing need for human review in providing accurate, high-quality captions and transcripts. Whisper's overall impact may be on transforming human labor rather than replacing it.

Keywords

Access to Knowledge / Access to Information Audio Visual and Multimedia Serving Disabled Populations Personnel Assessment and Evaluation of Service

Get full access to this article

View all access options for this article.

References

Andersdotter

(2023) Artificial intelligence skills and knowledge in libraries: Experiences and critical impressions from a learning circle. Journal of Information Literacy 17(2): 108–130.

Ballard

(2023) Most American adults under 30 prefer watching TV with subtitles – Even when they know the language. Available at: https://today.yougov.com/entertainment/articles/45987-american-adults-under-30-watching-tv-subtitles (accessed 15 November 2024).

Deng

(2023) AI, cataloging & metadata. Available at: https://stars.library.ucf.edu/ucfscholar/1251 (accessed 25 November 2024).

Described and Captioned Media Program (n.d.) Captioning key. Available at: https://dcmp.org/learn/captioningkey (accessed 25 November 2024).

Emory University (n.d.) Emory University facts. Available at: https://apply.emory.edu/discover/facts-stats/emory.html (accessed 15 November 2024).

Frey

Osborne

(2017) The future of employment: How susceptible are jobs to computerisation? Technological Forecasting and Social Change 114: 254–280. https://doi.org/10.1016/j.techfore.2016.08.019 .

GBH (n.d.) CADET – Caption and description editing tool. Available at: https://www.wgbh.org/foundation/services/ncam/cadet (accessed 15 November 2024).

Halaychik

(2024) Embrace AI in libraries: Freeing staff for meaningful work while preserving human touch. In: IFLA Academic and Research Libraries Section blog. Available at: https://blogs.ifla.org/arl/2024/07/24/embrace-ai-in-libraries-freeing-staff-for-meaningful-work-while-preserving-human-touch (accessed 15 November 2024).

Koenecke

Choi

ASG

Mei

, et al. (2024) Careless Whisper: Speech-to-text hallucination harms. In: Proceedings of the 2024 ACM conference on fairness, accountability, and transparency, Rio de Janeiro, Brazil, 3–6 June 2024, pp. 1672–1681. ACM. Available at: https://doi.org/10.1145/3630106.3658996.

10.

Lee

(2024) Investigating AI chatbot integration in academic libraries: A case study . PhD Thesis, University of Oklahoma, Norman, USA. Available at: https://hdl.handle.net/11244/340414 .

11.

(2024) Transforming academic librarianship through AI reskilling: Insights from the GPT-4 Exploration Program. The Journal of Academic Librarianship 50(3): Article 102883. https://doi.org/10.1016/j.acalib.2024.102883 .

12.

Marcus

(2022) AI platforms like ChatGPT are easy to use but also potentially dangerous. Scientific American, 19 December. Available at: https://www.scientificamerican.com/article/ai-platforms-like-chatgpt-are-easy-to-use-but-also-potentially-dangerous/.

13.

Mykhalevych

(2024) Survey: Why America is obsessed with subtitles. In: Preply blog. Available at: https://preply.com/en/blog/americas-subtitles-use (accessed 15 November 2024).

14.

Pence

(2022) Future of artificial intelligence in libraries. Reference Librarian 63(4): 133–143.

15.

Sun

Feng

, et al. (2024) Breaking the silence: Whisper-driven emotion recognition in AI mental support models. In: 2024 IEEE conference on artificial intelligence (CAI), Singapore, Singapore, 25–27 June 2024, pp. 290–291. IEEE. Available at: https://doi.org/10.1109/CAI59869.2024.00063.

16.

Rao

(2023) Transcribing educational videos using Whisper: A preliminary study on using AI for transcribing educational videos. Available at: https://arxiv.org/abs/2307.03200 (accessed 15 November 2024).

17.

Rodriguez

Brown

(2023) Comparative analysis of automated speech recognition technologies for enhanced audiovisual accessibility. Code4Lib Journal (58). https://journal.code4lib.org/articles/17820 .

18.

Sanji

Behzadi

Gomroki

(2022) Chatbot: An intelligent tool for libraries. Library Hi Tech News 39(3): 17–20. https://doi.org/10.1108/LHTN-01-2021-0002.

19.

StageTEXT (2023) Research shows 42% of people say “concentration” is a reason to use subtitles. Available at: https://www.stagetext.org/news/research-shows-42-of-people-say-concentration-is-a-reason-to-use-subtitles (accessed 15 November 2024).

20.

Zhao