Human voices narrate most audiobooks, but the fast development of speech synthesis technology has enabled the possibility of using artificial voices. This raises the question of whether the listeners’ cognitive processing is the same when listening to a synthetic or a human voice telling a story. This research aims to compare the listeners’ perception, creation of mental images, narrative engagement, physiological response, and recognition of information when listening to stories conveyed by human and synthetic voices. The results showed that listeners enjoyed stories narrated by a human voice more than a synthetic one. Also, they created more mental images, were more engaged, paid more attention, had a more positive emotional response, and remembered more information. Speech synthesis has experienced considerable progress. However, there are still significant differences versus human voices, so that using them to narrate long stories, such as audiobooks do, is difficult.
Audio Publishers Association (APA) (2021). The Voice of the Industry. Available at: https://www.audiopub.org/ (accessed 23 May 2021).
2.
AssmannPNearyTDemblingS (2006) Effects of frequency shifts on perceived naturalness and gender information in speech. In: Proceedings of the 9th international conference on spoken language processing, Pittsburgh, PA, 17–21 September.
3.
BabinLABurnsAC (1998) A modified scale for the measurement of communication-evoked mental imagery. Psychology & Marketing15(3): 261–278.
4.
BarkerP (2015) With one voice: disambiguating sung and spoken voices through a composer’s experience. In: ThomaidisKMacphersonB (eds) Voice Studies: Critical Approaches to Process, Performance and Experience. London: Routledge, pp. 16–26.
5.
BarthesR (1985) The Responsibility of Forms: Critical Essays on Music, Art, and Representation. New York: Hill and Wang.
6.
BollsPDLangA (2003) I saw it on the radio: the allocation of attention to high-imagery radio advertisements. Media Psychology5(1): 33–55.
7.
BollsPDMuehlingDD (2007) The effects of dual-dask processing on consumers’ responses to high- and low-imagery radio advertisements. Journal of Advertising36(4): 35–34.
8.
BonePFEllenPS (1992) The generation and consequences of communication-evoked Imagery. Journal of Consumer Research19: 93–103.
9.
BradleyMMLangPJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry25(1): 49–59.
10.
BusselleRVBilandzicH (2008) Transportation and transportability in the cultivation of genre-consistent attitudes and estimates. Journal of Communication58(3): 508–529.
11.
CavareroA (2005) FoR More Than One Voice: Toward a Philosophy of Vocal Expression. Palo Alto, CA: Stanford University Press.
12.
ChenF (2006) Designing Human Interface in Speech Technology. Boston, MA: Springer.
13.
ChionM (1994) Audio-vision: Sound on Screen. New York: Columbia University Press.
14.
ConnorS (2000) Dumbstruck: A Cultural History of Ventriloquism. Oxford: Oxford University Press.
15.
CraigSDSchroederNL (2017) Reconsidering the voice effect when learning from a virtual human. Computers and Education114: 193–205.
16.
CraigSDSchroederNL (2019) Text-to-Speech software and learning: investigating the relevancy of the voice effect. Journal of Educational Computing Research57(6): 1534–1548.
17.
DeloguCConteSSementinaC (1998) Cognitive factors in the evaluation of synthetic speech. Speech Communication24(2): 153–168.
18.
Di MatteoP (2015) Performing the Entre-Deux: the capture of speech in (dis)embodied voices. In: ThomaidisKMacphersonB (eds) Voice Studies: Critical Approaches to Process, Performance and Experience. London: Routledge, pp. 104–119.
GoosensG (1994) Enactive imagery: information processing, emotional responses, and behavioral intentions. Journal of Mental Imagery18(3/4): 119–150.
23.
GreenMCBrockTC (2000) The role of transportation in the persuasiveness of public narratives. Journal of Personality and Social Psychology79(5): 701–721.
24.
GreenMCBrockTCKaufmanGE (2004) Understanding media enjoyment: the role of transportation into narrative worlds. Communication Theory14: 311–327.
25.
HaveIPedersenBS (2020) The audiobook circuit in digital publishing: voicing the silent revolution. new media & society22(3): 409–428.
26.
HumphryJChesherC (2020) Preparing for smart voice assistants: cultural histories and media innovations. new media & society. Epub ahead of print 22 May. DOI: 10.1177/1461444820923679.
27.
JenkinsJJFranklinLD (1982) Recall of passages of synthetic speech. Bulletin of the Psychonomic Society20(4): 203–206.
28.
KosslynSM (1994) Image and Brain: The Resolution of the Imagery Debate. Cambridge, MA: The MIT Press
29.
KosslynSMGanisGThompsonWL (2001) Neural foundations of imagery. Nature Reviews Neuroscience2: 635–642.
30.
LaiJWoodDConsidineM (2000) The effect of task conditions on the comprehensibility of synthetic speech. In: Proceedings of the SIGCHI conference on human factors in computing systems, The Hague, 1–6 April, pp. 321–328.
31.
LangAGaoYPotterRF, et al. (2015) Conceptualizing audio message complexity as available processing resources. Communication Research42: 759–778.
32.
LucePA (1981) Comprehension of fluent synthetic speech produced by rule. Research on Speech Perception Progress, Report No. 7, pp. 229-242. Bloomington, IN: Speech Research Laboratory, Indiana University.
33.
LucePAFeustelTCPisoniDB (1983) Capacity demands in short-term memory for synthetic and natural speech. Human Factors: The Journal of the Human Factors and Ergonomics Society25(1): 17–32.
34.
MarRAOatleyK (2008) The function of fiction is the abstraction and simulation of social experience. Perspectives in Psychological Science3: 173–192.
35.
MayerRE (2014) Principles based on social cues in multimedia learning: personalization, voice, image, and embodiment principles. The Cambridge Handbook of Multimedia Learning16: 345–370.
36.
MillerDWMarksLJ (1997) The effects of imagery evoking radio advertising strategies on affective responses. Psychology and Marketing14: 337–361.
37.
MulacAGilesH (1996) ‘You’re only as old as you sound’: perceived vocal age and social meanings. Health Communication8: 199–215.
38.
MullennixJWSternSEWilsonSJ, et al. (2003) Social perception of male and female computer synthesized speech. Computers in Human Behavior19(4): 407–424.
39.
PaivioA (1991) Dual coding theory: retrospect and current status. Canadian Journal of Psychology/revue Canadienne De Psychologie45(3): 255.
40.
ParisCRThomasMHGilsonRD, et al. (2000) Linguistic cues and memory for synthetic and natural speech. Human Factors: The Journal of the Human Factors and Ergonomics Society42(3): 421–431.
PisoniDBHunnicuttS (1980) Perceptual evaluation of MITalk: the MIT unrestricted text-to-speech system. In: IEEE International Conference on Acoustics, Speech and Signal Processing, Denver, CO, 9–11 April, pp. 572–575. New York: IEEE.
43.
PotterRFBollsP (2012) Psychophysiological Measurement and Meaning: Cognitive and Emotional Processing of Media. London: Routledge.
44.
RoderoE (2012) See it on a radio story: sound effects and shots to evoked imagery and attention on audio fiction. Communication Research39(4): 458–479.
45.
RoderoE (2016) Influence of speech rate and information density on recognition: the moderate dynamic mechanism. Media Psychology19(2): 224–242.
46.
RoderoE (2017) Effectiveness, attention, and recall of human and artificial voices in an advertising story. Prosody influence and functions of voices. Computers in Human Behavior77: 336–346.
47.
RoderoE (2020) A voice you can’t see. The Unesco, Courier1: 18–19.
48.
RoderoEPotterRFPrietoP (2017) Pitch range variations improve cognitive processing of audio messages. Human Communication Research43(3): 397–413.
49.
RoringRWHinesFGCharnessN (2007) Age difference in identifying words in synthetic speech. Human Factors: The Journal of the Human Factors and Ergonomics Society49: 25–31.
50.
SandermanAACollierR (1997) Prosodic phrasing and comprehension. Language and Speech40: 391–409.
51.
SternSEMullennixJWDysonCL, et al. (1999) The persuasiveness of synthetic speech versus human speech. Human Factors: The Journal of the Human Factors and Ergonomics Society41(4): 588–595.
van DommelenWAMoxnessBH (1995) Acoustic parameters in speaker height and weight identification: sex-specific behaviour. Language and Speech38: 267–287.
WoltersMKJohnsonCCampbellPE, et al. (2014) Can older people remember medication reminders presented using synthetic speech?Journal of the American Medical Informatics Association22: 35–42.
59.
XuK (2019) First encounter with robot Alpha: how individual differences interact with vocal and kinetic cues in users’ social responses. new media & society21(11–12): 2522–2547.
